C
CertX AGvia muse
Senior AI Engineer
REMOTEPosted 2w ago
ML EngineerSeniorFull-time#remote
About the Role
**About us** (Version française ci\-dessous)
CertX Canada Inc., part of the SGS Group, specializes in artificial intelligence, cybersecurity, and technical advisory services. We help organizations adopt emerging technologies, address complex technical challenges, and build secure, reliable solutions. Our work combines strategic consulting with hands\-on engineering and software development, serving clients across a range of industries.
**The role**
We are looking for a **Senior AI Engineer** to design, develop and improve production AI systems, with a focus on accuracy, reliability, performance and cost efficiency. You will own AI features end\-to\-end, from architecture and implementation to deployment, monitoring and continuous improvement. You will work in a small product team alongside technical leads and domain experts, translating real\-world feedback into measurable improvements. We value engineers who take ownership, make sound technical decisions and deliver robust solutions with minimal supervision. Our technology stack includes:
* **AI:** LLMs, RAG, document processing, retrieval and agentic workflows
* **Backend:** Python APIs, SQL databases and multi\-tenant access control
* **Frontend:** TypeScript web applications
* **Cloud:** Microsoft Azure, serverless services, containers, infrastructure as code and CI/CD
**Responsibilities**
* Design, develop and optimize production LLM applications, including RAG, structured outputs, prompting and agentic workflows.
* Improve retrieval quality through document parsing, chunking, embeddings, hybrid search and reranking.
* Build evaluation datasets and automated tests to measure and improve AI accuracy, groundedness and reliability.
* Evaluate and select AI models, manage upgrades and optimize performance across providers.
* Monitor and optimize token usage, latency, rate limits and operational costs without compromising quality.
* Ensure AI outputs are explainable and traceable through source citations, clear rationale and audit trails.
* Develop, deploy and maintain Azure services, infrastructure as code and CI/CD pipelines.
* Troubleshoot production issues using logs, metrics, distributed tracing and root\-cause analysis.
* Implement secure, multi\-tenant architectures with appropriate identity, access control, secrets management and data isolation.
* Deliver end\-to\-end features across backend and frontend components when required.
* Maintain clear technical documentation.
**Required qualifications**
* **3\-5\+ years of software engineering experience**, including hands\-on experience with AI/ML or LLM systems in production.
* Strong Python development skills, including asynchronous programming, REST APIs and automated testing.
* Practical experience building RAG pipelines, including document processing, embeddings and vector/hybrid search.
* Experience evaluating LLM applications and demonstrating measurable improvements in quality.
* **Strong hands\-on Microsoft Azure experience**, including deploying, operating and troubleshooting production workloads.
* Experience with infrastructure as code, CI/CD and production monitoring.
* Understanding of application security, data privacy and multi\-tenant architectures.
* Strong communication skills and the ability to explain technical decisions and AI limitations to non\-technical stakeholders.
* Ability to work independently, take ownership and deliver production\-quality solutions.
**Nice to have**
* Full\-stack development experience with TypeScript and modern frontend frameworks.
* Experience with multiple LLM providers, open\-weight models or fine\-tuning.
* Experience with agent frameworks, tool calling and multi\-step orchestration.
* Familiarity with enterprise software, regulated environments or security\-sensitive applications.
**What we offer**
* Work on real\-world AI systems with meaningful technical challenges.
* Ownership of AI architecture, quality and production delivery.
* A small, collaborative
CertX Canada Inc., part of the SGS Group, specializes in artificial intelligence, cybersecurity, and technical advisory services. We help organizations adopt emerging technologies, address complex technical challenges, and build secure, reliable solutions. Our work combines strategic consulting with hands\-on engineering and software development, serving clients across a range of industries.
**The role**
We are looking for a **Senior AI Engineer** to design, develop and improve production AI systems, with a focus on accuracy, reliability, performance and cost efficiency. You will own AI features end\-to\-end, from architecture and implementation to deployment, monitoring and continuous improvement. You will work in a small product team alongside technical leads and domain experts, translating real\-world feedback into measurable improvements. We value engineers who take ownership, make sound technical decisions and deliver robust solutions with minimal supervision. Our technology stack includes:
* **AI:** LLMs, RAG, document processing, retrieval and agentic workflows
* **Backend:** Python APIs, SQL databases and multi\-tenant access control
* **Frontend:** TypeScript web applications
* **Cloud:** Microsoft Azure, serverless services, containers, infrastructure as code and CI/CD
**Responsibilities**
* Design, develop and optimize production LLM applications, including RAG, structured outputs, prompting and agentic workflows.
* Improve retrieval quality through document parsing, chunking, embeddings, hybrid search and reranking.
* Build evaluation datasets and automated tests to measure and improve AI accuracy, groundedness and reliability.
* Evaluate and select AI models, manage upgrades and optimize performance across providers.
* Monitor and optimize token usage, latency, rate limits and operational costs without compromising quality.
* Ensure AI outputs are explainable and traceable through source citations, clear rationale and audit trails.
* Develop, deploy and maintain Azure services, infrastructure as code and CI/CD pipelines.
* Troubleshoot production issues using logs, metrics, distributed tracing and root\-cause analysis.
* Implement secure, multi\-tenant architectures with appropriate identity, access control, secrets management and data isolation.
* Deliver end\-to\-end features across backend and frontend components when required.
* Maintain clear technical documentation.
**Required qualifications**
* **3\-5\+ years of software engineering experience**, including hands\-on experience with AI/ML or LLM systems in production.
* Strong Python development skills, including asynchronous programming, REST APIs and automated testing.
* Practical experience building RAG pipelines, including document processing, embeddings and vector/hybrid search.
* Experience evaluating LLM applications and demonstrating measurable improvements in quality.
* **Strong hands\-on Microsoft Azure experience**, including deploying, operating and troubleshooting production workloads.
* Experience with infrastructure as code, CI/CD and production monitoring.
* Understanding of application security, data privacy and multi\-tenant architectures.
* Strong communication skills and the ability to explain technical decisions and AI limitations to non\-technical stakeholders.
* Ability to work independently, take ownership and deliver production\-quality solutions.
**Nice to have**
* Full\-stack development experience with TypeScript and modern frontend frameworks.
* Experience with multiple LLM providers, open\-weight models or fine\-tuning.
* Experience with agent frameworks, tool calling and multi\-step orchestration.
* Familiarity with enterprise software, regulated environments or security\-sensitive applications.
**What we offer**
* Work on real\-world AI systems with meaningful technical challenges.
* Ownership of AI architecture, quality and production delivery.
* A small, collaborative
Similar Jobs
C
Capital One• 1mo ago
Machine Learning Engineer 4 (IC)
McLean, VAFull-time
W
Waystone• 6mo ago
AI Engineer
Leeds, West YorkshireFull-time
A
Aderant• 1mo ago
AI Engineer – Center of Excellence
Atlanta, GAFull-time
S
Signifyd• 1mo ago
Senior Machine Learning Engineer I // II
Chicago, ILFull-time