D
DoubleVerifyvia muse
Senior AI Engineer
REMOTE$131K - $241K/yrPosted 1w ago
ML EngineerSeniorFull-time#remote
About the Role
**Who We Are**
==============
DoubleVerify is the leading independent provider of marketing measurement software, data and analytics that authenticates the quality and effectiveness of digital media for the world's largest brands and media platforms. DV provides media transparency and accountability to deliver the highest level of impression quality for maximum advertising performance. Since 2008, DV has helped hundreds of Fortune 500 companies gain the most from their media spend by delivering best\-in\-class solutions across the digital advertising ecosystem, helping to build a better industry. Learn more at www.doubleverify.com.
**Role Overview**
=================
We are looking for a talented and driven **Senior AI Engineer** to join the Pinnacle AI (PinAI) team at DoubleVerify. In this role, you will be a core builder of **AgentOS** — our agentic AI platform that powers conversational, intelligent experiences across the Pinnacle advertising analytics suite.
You will design and implement AI agents, Model Context Protocol (MCP) servers, and LLM\-backed tools that allow Fortune 500 clients to interact with their advertising data through natural language. This is a hands\-on engineering role for someone who is passionate about applied AI, thrives in a fast\-moving environment, and wants to see their work directly impact how the world's largest advertisers make decisions.
**What You'll Do**
==================
### **Agentic AI Development**
* Design, build, and maintain AI agents within the AgentOS framework using architectures based on MCP (Model Context Protocol), LangChain, LangGraph, and related tooling
* Develop new MCP servers and tools that expose Pinnacle analytics capabilities — including metrics, dimensions, settings, and user data — to AI agents
* Implement multi\-turn conversational flows, tool\-use reasoning chains, and agent orchestration logic that handle complex, real\-world advertising analytics queries
* Build and maintain the chat UI integration layer, connecting AgentOS to the Pinnacle front\-end experience
### **LLM Engineering \& Optimization**
* Work with major LLM providers (OpenAI, Anthropic, Google, and others) to select, prompt, and integrate models appropriate for specific agent tasks
* Develop and own prompt engineering strategies, including system prompts, few\-shot examples, and chain\-of\-thought techniques
* Implement Retrieval\-Augmented Generation (RAG) pipelines to ground AI responses in Pinnacle's proprietary data and documentation
* Continuously optimize model performance through evals, latency tuning, and cost management
### **Evaluation \& Quality**
* Build and maintain evaluation (evals) frameworks that measure agent accuracy, instruction\-following, hallucination rates, and user satisfaction
* Define quality benchmarks and run regression evals on every agent change before production deployment
* Instrument agents with observability tooling (e.g., LangSmith, Weights \& Biases, or custom tracing) to support debugging and continuous improvement
### **Integration \& Platform Engineering**
* Integrate AgentOS with Pinnacle's existing platform services — including PIAM/Keycloak authentication, Settings APIs, and analytics data layers
* Collaborate with backend teams to build and consume APIs that surface data cleanly and securely to AI agents
* Participate in architecture reviews and contribute to technical design documents for new AI\-powered features
### **Cross\-Functional Collaboration**
* Partner closely with Product, UX, and Data Science to translate business requirements into agent capabilities and conversational flows
* Work with QE teams to define testing strategies for AI features, including behavioral testing and adversarial prompting
* Contribute to the team's shared AI engineering standards, best practices, and documentation
**Who You Are**
===============
**Required Qualifications**
---------------------------
### **Experience:**
* 5\+ years of professio
==============
DoubleVerify is the leading independent provider of marketing measurement software, data and analytics that authenticates the quality and effectiveness of digital media for the world's largest brands and media platforms. DV provides media transparency and accountability to deliver the highest level of impression quality for maximum advertising performance. Since 2008, DV has helped hundreds of Fortune 500 companies gain the most from their media spend by delivering best\-in\-class solutions across the digital advertising ecosystem, helping to build a better industry. Learn more at www.doubleverify.com.
**Role Overview**
=================
We are looking for a talented and driven **Senior AI Engineer** to join the Pinnacle AI (PinAI) team at DoubleVerify. In this role, you will be a core builder of **AgentOS** — our agentic AI platform that powers conversational, intelligent experiences across the Pinnacle advertising analytics suite.
You will design and implement AI agents, Model Context Protocol (MCP) servers, and LLM\-backed tools that allow Fortune 500 clients to interact with their advertising data through natural language. This is a hands\-on engineering role for someone who is passionate about applied AI, thrives in a fast\-moving environment, and wants to see their work directly impact how the world's largest advertisers make decisions.
**What You'll Do**
==================
### **Agentic AI Development**
* Design, build, and maintain AI agents within the AgentOS framework using architectures based on MCP (Model Context Protocol), LangChain, LangGraph, and related tooling
* Develop new MCP servers and tools that expose Pinnacle analytics capabilities — including metrics, dimensions, settings, and user data — to AI agents
* Implement multi\-turn conversational flows, tool\-use reasoning chains, and agent orchestration logic that handle complex, real\-world advertising analytics queries
* Build and maintain the chat UI integration layer, connecting AgentOS to the Pinnacle front\-end experience
### **LLM Engineering \& Optimization**
* Work with major LLM providers (OpenAI, Anthropic, Google, and others) to select, prompt, and integrate models appropriate for specific agent tasks
* Develop and own prompt engineering strategies, including system prompts, few\-shot examples, and chain\-of\-thought techniques
* Implement Retrieval\-Augmented Generation (RAG) pipelines to ground AI responses in Pinnacle's proprietary data and documentation
* Continuously optimize model performance through evals, latency tuning, and cost management
### **Evaluation \& Quality**
* Build and maintain evaluation (evals) frameworks that measure agent accuracy, instruction\-following, hallucination rates, and user satisfaction
* Define quality benchmarks and run regression evals on every agent change before production deployment
* Instrument agents with observability tooling (e.g., LangSmith, Weights \& Biases, or custom tracing) to support debugging and continuous improvement
### **Integration \& Platform Engineering**
* Integrate AgentOS with Pinnacle's existing platform services — including PIAM/Keycloak authentication, Settings APIs, and analytics data layers
* Collaborate with backend teams to build and consume APIs that surface data cleanly and securely to AI agents
* Participate in architecture reviews and contribute to technical design documents for new AI\-powered features
### **Cross\-Functional Collaboration**
* Partner closely with Product, UX, and Data Science to translate business requirements into agent capabilities and conversational flows
* Work with QE teams to define testing strategies for AI features, including behavioral testing and adversarial prompting
* Contribute to the team's shared AI engineering standards, best practices, and documentation
**Who You Are**
===============
**Required Qualifications**
---------------------------
### **Experience:**
* 5\+ years of professio
Similar Jobs
C
Capital One• 1mo ago
Machine Learning Engineer 4 (IC)
McLean, VAFull-time
W
Waystone• 6mo ago
AI Engineer
Leeds, West YorkshireFull-time
A
Aderant• 1mo ago
AI Engineer – Center of Excellence
Atlanta, GAFull-time
S
Signifyd• 1mo ago
Senior Machine Learning Engineer I // II
Chicago, ILFull-time