E
Equinixvia muse
AI Engineer, Trust & Evaluation
Toronto, ON, CAPosted 4d ago
ML EngineerMid LevelFull-time
About the Role
### **AI Engineer, Trust \& Evaluation**
* JR\-164650
* Hybrid
* Toronto
* Technology
* Full time
**Who are we?**
Equinix is the world’s digital infrastructure company®, shortening the path to connectivity to enable the innovations that enrich our work, life and planet.
A place where tech thinkers and future builders turn bold ideas into breakthrough experiences, we welcome your unique perspective.
Help us challenge assumptions, uncover bias, and remove barriers—because progress starts with fresh ideas. You’ll find belonging, purpose, and a team that welcomes you—because when you feel valued, you’re empowered to do your best work.
**Job Summary**
Designs, develops and deploys the evaluation and trust capabilities of Equinix's GenAI platform. Builds platform features that let teams evaluate their agents at scale: golden datasets, synthetic test data, scoring with LLM\-as\-judge, and monitoring of agent quality in production. Builds and tests the trust controls that agents run on, including agent identity, access, guardrails and data boundaries. Traces agent failures to their root cause and fixes them in the platform. Works in a team with senior engineers and with product, platform and security partners to turn defined problems into robust, secure, observable and performant solutions.
Knowledge and Application
Applies solid knowledge of software engineering practice and of LLM and agent concepts (prompting, retrieval, tool and function calling, orchestration, MCP). Understands evaluation methods (code\-based checks, LLM\-as\-judge, human review, golden datasets, regression suites) and evaluation statistics (sampling, variance across runs, agreement between reviewers). Understands AI security risks such as prompt injection and data leakage, and identity concepts such as OAuth and per\-user access. Follows established standards and suggests improvements.
Complexity and Problem Solving
Works on problems of defined scope where analysis requires evaluation of identifiable factors. Traces an agent failure to its cause (retrieval, tool call, prompt, model, data or infrastructure) using logs and traces, and follows it until it is fixed. Judges whether a metric reflects real quality. Escalates unusual or high\-impact issues. Work is reviewed at defined checkpoints.
Collaboration and Interaction
Works with AI engineers, product managers, forward\-deployed engineers, platform and security teams. Turns the needs of agent builders into platform features. Explains results, defects and risks to partners who are not familiar with the subject. Asks for guidance early and shares knowledge with the team.
**Responsibilities**
Evaluation platform (primary focus)
* Build platform features for evaluation: dataset creation and versioning, golden datasets, synthetic test data and reusable metric templates
* Build scoring for agents: code\-based checks, LLM\-as\-judge, tool\-call accuracy and grounding checks
* Build human\-review features that compare judge scores against human reviewers, and recalibrate when agreement is low
* Run evaluations at scale: before release, on every change to a prompt, model or tool, and on agents in production
* Build evaluation playbooks so agent builders can evaluate their agents without specialist help
* Keep results traceable to the agent, model, prompt and dataset versions tested, with a record of who approved each release
Trust and safety
* Build and test the platform's trust controls: agent identity, per\-user access to connected systems, tool permissions, guardrails and data boundaries
* Run adversarial tests and red\-teaming (prompt injection, out\-of\-scope requests, data\-boundary tests), and turn findings into fixes and regression tests
* Work with security and governance teams to turn policy requirements into automated checks
Root\-cause analysis and quality
* Investigate agent failures and performance problems end to end, and follow them
* JR\-164650
* Hybrid
* Toronto
* Technology
* Full time
**Who are we?**
Equinix is the world’s digital infrastructure company®, shortening the path to connectivity to enable the innovations that enrich our work, life and planet.
A place where tech thinkers and future builders turn bold ideas into breakthrough experiences, we welcome your unique perspective.
Help us challenge assumptions, uncover bias, and remove barriers—because progress starts with fresh ideas. You’ll find belonging, purpose, and a team that welcomes you—because when you feel valued, you’re empowered to do your best work.
**Job Summary**
Designs, develops and deploys the evaluation and trust capabilities of Equinix's GenAI platform. Builds platform features that let teams evaluate their agents at scale: golden datasets, synthetic test data, scoring with LLM\-as\-judge, and monitoring of agent quality in production. Builds and tests the trust controls that agents run on, including agent identity, access, guardrails and data boundaries. Traces agent failures to their root cause and fixes them in the platform. Works in a team with senior engineers and with product, platform and security partners to turn defined problems into robust, secure, observable and performant solutions.
Knowledge and Application
Applies solid knowledge of software engineering practice and of LLM and agent concepts (prompting, retrieval, tool and function calling, orchestration, MCP). Understands evaluation methods (code\-based checks, LLM\-as\-judge, human review, golden datasets, regression suites) and evaluation statistics (sampling, variance across runs, agreement between reviewers). Understands AI security risks such as prompt injection and data leakage, and identity concepts such as OAuth and per\-user access. Follows established standards and suggests improvements.
Complexity and Problem Solving
Works on problems of defined scope where analysis requires evaluation of identifiable factors. Traces an agent failure to its cause (retrieval, tool call, prompt, model, data or infrastructure) using logs and traces, and follows it until it is fixed. Judges whether a metric reflects real quality. Escalates unusual or high\-impact issues. Work is reviewed at defined checkpoints.
Collaboration and Interaction
Works with AI engineers, product managers, forward\-deployed engineers, platform and security teams. Turns the needs of agent builders into platform features. Explains results, defects and risks to partners who are not familiar with the subject. Asks for guidance early and shares knowledge with the team.
**Responsibilities**
Evaluation platform (primary focus)
* Build platform features for evaluation: dataset creation and versioning, golden datasets, synthetic test data and reusable metric templates
* Build scoring for agents: code\-based checks, LLM\-as\-judge, tool\-call accuracy and grounding checks
* Build human\-review features that compare judge scores against human reviewers, and recalibrate when agreement is low
* Run evaluations at scale: before release, on every change to a prompt, model or tool, and on agents in production
* Build evaluation playbooks so agent builders can evaluate their agents without specialist help
* Keep results traceable to the agent, model, prompt and dataset versions tested, with a record of who approved each release
Trust and safety
* Build and test the platform's trust controls: agent identity, per\-user access to connected systems, tool permissions, guardrails and data boundaries
* Run adversarial tests and red\-teaming (prompt injection, out\-of\-scope requests, data\-boundary tests), and turn findings into fixes and regression tests
* Work with security and governance teams to turn policy requirements into automated checks
Root\-cause analysis and quality
* Investigate agent failures and performance problems end to end, and follow them
Similar Jobs
C
Capital One• 1mo ago
Machine Learning Engineer 4 (IC)
McLean, VAFull-time
W
Waystone• 6mo ago
AI Engineer
Leeds, West YorkshireFull-time
A
Aderant• 1mo ago
AI Engineer – Center of Excellence
Atlanta, GAFull-time
S
Signifyd• 1mo ago
Senior Machine Learning Engineer I // II
Chicago, ILFull-time