G
Guidepointvia muse
Senior Software Engineer - Search & AI
Toronto, ON, CAPosted 3d ago
Data EngineerSeniorFull-time
About the Role
**Overview:**
-------------
We are hiring a Senior Software Engineer to build and run the search systems that connect Guidepoint clients with the right experts. You will join the Search team in Data Engineering and own work end to end: Elasticsearch index design and cluster operations, the inference services behind semantic search, the Python services in front of it all, and the AI features that turn a plain\-language question into a ranked list of people.
Our search sits on the critical path of the business. When a client asks for experts, the quality and speed of what comes back decides whether a project starts. This role is for someone who cares about relevance as much as uptime, and who can explain why a result ranked where it did.
**This role will be hybrid based in our Toronto office.**
**What You'll Do:**
-------------------
**Search and indexing**
* Design index mappings, analyzers, and tokenizers for expert, company, project, and lead data, including semantic\_text and dense\_vector fields
* Write and tune queries: bool, function\_score, nested, aggregations, kNN, and hybrid retrieval
* Plan and run reindexes, Painless data fixes, and alias cutovers with zero downtime (we recently retired Solr completely)
* Build and maintain ingestion: Logstash JDBC pipelines and incremental change feeds from PostgreSQL and SQL Server, including tuning the source SQL when indexing slows down
**Cluster operations and maintenance**
* Operate Elastic Cloud deployments on Azure across prod, QA, and dev: sizing, shard allocation, ILM, snapshots, and version upgrades
* Monitor cluster health in Datadog and Kibana; tune heap, circuit breakers, and fetch sizes; diagnose shard and recovery problems with the \_cat and \_cluster APIs
* Manage access: roles, service API keys and their rotation, least\-privilege accounts for every consumer
* Lead incident response for latency spikes, circuit breaker trips, and ingestion failures, then write the RCA and ship the follow\-up fixes
* Load test search under realistic traffic before major mapping, query, or capacity changes
**Inference and AI services**
* Run the inference layer behind semantic search: Elasticsearch inference endpoints, embedding models (ELSER, E5, or hosted models), ML node capacity, and cost
* Build services that turn natural\-language requests into structured search queries and filters using LLMs
* Build and harden MCP servers that let AI agents call search as a tool: auth, consistent error contracts, lean payloads
* Benchmark LLMs for accuracy, latency, and cost against fixed query batteries, and pick models on evidence
* Set up offline relevance evaluation (judgment lists, nDCG, recall@k) and run A/B tests online
**Python services and APIs**
* Build and maintain the Python APIs and workers, running on AKS (Kubernetes), that sit between clients, AI agents, and the search cluster
* Own services in production on AKS behind Azure APIM: tests, CI/CD, Docker images, Kubernetes deployments and scaling, dashboards, and alerts
* Fix findings from security scans and keep secrets out of code and images
* Review code and raise the bar on design, testing, and readability across the team
**Across the team**
* Turn ambiguous product asks into technical designs and break them into deliverable pieces
* Write design docs and runbooks that others can pick up without you in the room
* Mentor engineers on search and backend practices
**What You Have:**
------------------
* **6\+ years** of professional software engineering, much of it building and running backend systems in production
* Bachelor's Degree in Computer Science, Engineering, Information Technology or related field
* **Search engines, in depth (3\+ years hands\-on with Elasticsearch, Solr, or OpenSearch):** mappings and analyzers, Query DSL, relevance tuning, aggregations, cluster sizing and operations, reindexing strategies. You've owned a cluster, not just queried one.
* **Python:** strong, idiomatic Pytho
-------------
We are hiring a Senior Software Engineer to build and run the search systems that connect Guidepoint clients with the right experts. You will join the Search team in Data Engineering and own work end to end: Elasticsearch index design and cluster operations, the inference services behind semantic search, the Python services in front of it all, and the AI features that turn a plain\-language question into a ranked list of people.
Our search sits on the critical path of the business. When a client asks for experts, the quality and speed of what comes back decides whether a project starts. This role is for someone who cares about relevance as much as uptime, and who can explain why a result ranked where it did.
**This role will be hybrid based in our Toronto office.**
**What You'll Do:**
-------------------
**Search and indexing**
* Design index mappings, analyzers, and tokenizers for expert, company, project, and lead data, including semantic\_text and dense\_vector fields
* Write and tune queries: bool, function\_score, nested, aggregations, kNN, and hybrid retrieval
* Plan and run reindexes, Painless data fixes, and alias cutovers with zero downtime (we recently retired Solr completely)
* Build and maintain ingestion: Logstash JDBC pipelines and incremental change feeds from PostgreSQL and SQL Server, including tuning the source SQL when indexing slows down
**Cluster operations and maintenance**
* Operate Elastic Cloud deployments on Azure across prod, QA, and dev: sizing, shard allocation, ILM, snapshots, and version upgrades
* Monitor cluster health in Datadog and Kibana; tune heap, circuit breakers, and fetch sizes; diagnose shard and recovery problems with the \_cat and \_cluster APIs
* Manage access: roles, service API keys and their rotation, least\-privilege accounts for every consumer
* Lead incident response for latency spikes, circuit breaker trips, and ingestion failures, then write the RCA and ship the follow\-up fixes
* Load test search under realistic traffic before major mapping, query, or capacity changes
**Inference and AI services**
* Run the inference layer behind semantic search: Elasticsearch inference endpoints, embedding models (ELSER, E5, or hosted models), ML node capacity, and cost
* Build services that turn natural\-language requests into structured search queries and filters using LLMs
* Build and harden MCP servers that let AI agents call search as a tool: auth, consistent error contracts, lean payloads
* Benchmark LLMs for accuracy, latency, and cost against fixed query batteries, and pick models on evidence
* Set up offline relevance evaluation (judgment lists, nDCG, recall@k) and run A/B tests online
**Python services and APIs**
* Build and maintain the Python APIs and workers, running on AKS (Kubernetes), that sit between clients, AI agents, and the search cluster
* Own services in production on AKS behind Azure APIM: tests, CI/CD, Docker images, Kubernetes deployments and scaling, dashboards, and alerts
* Fix findings from security scans and keep secrets out of code and images
* Review code and raise the bar on design, testing, and readability across the team
**Across the team**
* Turn ambiguous product asks into technical designs and break them into deliverable pieces
* Write design docs and runbooks that others can pick up without you in the room
* Mentor engineers on search and backend practices
**What You Have:**
------------------
* **6\+ years** of professional software engineering, much of it building and running backend systems in production
* Bachelor's Degree in Computer Science, Engineering, Information Technology or related field
* **Search engines, in depth (3\+ years hands\-on with Elasticsearch, Solr, or OpenSearch):** mappings and analyzers, Query DSL, relevance tuning, aggregations, cluster sizing and operations, reindexing strategies. You've owned a cluster, not just queried one.
* **Python:** strong, idiomatic Pytho