C
CACI Internationalvia muse
AI Engineer
REMOTE$115K - $252K/yrPosted 1w ago
ML EngineerMid LevelFull-time#remote
About the Role
Job Title: AI Engineer
Job Category: Information Technology
Time Type: Full time
Minimum Clearance Required to Start: Secret
Employee Type: Regular
Percentage of Travel Required: Up to 10%
Type of Travel: Local
\* \* \*
**The Opportunity:**
We are seeking an AI Engineer to build, test, and support AI workflows and agents on an enterprise AI workflow management platform supporting the Department of the Air Force. You will work directly with the AI Platform Architect to turn platform designs and government use cases into tested, production\-ready solutions in Azure IL\-5 cloud environments, integrated with ServiceNow. You will build within the frameworks, patterns, and tools the Architect establishes, including Microsoft AI Foundry and third\-party agentic platforms, and help translate customer mission needs into working prototypes and deployed workflows.
**Responsibilities:**
Workflow \& Agent Development
* Build AI workflows and agents using the platform's frameworks, tools, and reference architectures
* Develop RAG pipelines, including document ingestion, chunking, embedding, vector search, and retrieval tuning
* Build prompt chains, tool calls, and agent logic for document generation, summarization, and task automation
* Integrate AI workflows with Azure OpenAI services, ServiceNow APIs (Flow Designer, REST, scripted integrations), and internal data sources
* Build reusable components, templates, and connectors that both technical and non\-technical users can leverage
* Implement workflow orchestration and business process automation
Testing \& Quality Assurance
* Develop and execute test plans for AI workflows, including unit, integration, and user acceptance testing
* Build evaluation datasets and automated evaluation harnesses to measure the accuracy, relevance, groundedness, and reliability of LLM outputs
* Define and track success metrics and evaluation criteria for AI\-driven workflows
* Identify, document, and triage defects, hallucinations, and performance issues, and drive root\-cause analysis and remediation
* Perform regression testing when models, prompts, or platform components change
* Architect Support \& Collaboration
* Partner with the AI Platform Architect to implement designs, evaluate tools, and give feedback on platform capabilities and limitations
* Prototype solutions and proofs\-of\-concept for government use cases, and present results to customers and stakeholders
* Work with ServiceNow developers and Azure cloud engineers to troubleshoot integration issues across Azure, ServiceNow, and government networks
* Contribute to design reviews, engineering standards, and technical documentation
* Support technical screening of AI/ML talent when requested
Operations \& Monitoring
* Support monitoring, logging, and alerting for AI workflows and agents
* Manage prompt and model versioning, and support CI/CD deployments of AI services
* Monitor output quality, model drift, and data quality in production workflows, and report findings
* Track model and compute usage costs and recommend optimizations
Government \& Compliance
* Build and operate within DISA STIGs, IL\-5 requirements, and Department of the Air Force security policies
* Implement CUI handling and data classification controls in AI workflows
Provide engineering evidence and documentation to cybersecurity teams for ATO and continuous monitoring .
*
**Qualifications:**
*Required:*
* US Citizenship required.
* Ability to obtain and maintain a secret clearance.
* 15\+ Years of relevant experience (Bachelor’s Degree in relevant field may be substituted for 5 years of relevant experience).
* Hands\-on experience building LLM applications, including RAG, prompt engineering, and agentic workflows
* Proficiency in Python, plus experience with LangChain, LlamaIndex, or similar AI frameworks
* Experience with Azure cloud services (Azure OpenAI, AI Foundry, DevOps, API Management)
* Experience with containers (Docker)
Job Category: Information Technology
Time Type: Full time
Minimum Clearance Required to Start: Secret
Employee Type: Regular
Percentage of Travel Required: Up to 10%
Type of Travel: Local
\* \* \*
**The Opportunity:**
We are seeking an AI Engineer to build, test, and support AI workflows and agents on an enterprise AI workflow management platform supporting the Department of the Air Force. You will work directly with the AI Platform Architect to turn platform designs and government use cases into tested, production\-ready solutions in Azure IL\-5 cloud environments, integrated with ServiceNow. You will build within the frameworks, patterns, and tools the Architect establishes, including Microsoft AI Foundry and third\-party agentic platforms, and help translate customer mission needs into working prototypes and deployed workflows.
**Responsibilities:**
Workflow \& Agent Development
* Build AI workflows and agents using the platform's frameworks, tools, and reference architectures
* Develop RAG pipelines, including document ingestion, chunking, embedding, vector search, and retrieval tuning
* Build prompt chains, tool calls, and agent logic for document generation, summarization, and task automation
* Integrate AI workflows with Azure OpenAI services, ServiceNow APIs (Flow Designer, REST, scripted integrations), and internal data sources
* Build reusable components, templates, and connectors that both technical and non\-technical users can leverage
* Implement workflow orchestration and business process automation
Testing \& Quality Assurance
* Develop and execute test plans for AI workflows, including unit, integration, and user acceptance testing
* Build evaluation datasets and automated evaluation harnesses to measure the accuracy, relevance, groundedness, and reliability of LLM outputs
* Define and track success metrics and evaluation criteria for AI\-driven workflows
* Identify, document, and triage defects, hallucinations, and performance issues, and drive root\-cause analysis and remediation
* Perform regression testing when models, prompts, or platform components change
* Architect Support \& Collaboration
* Partner with the AI Platform Architect to implement designs, evaluate tools, and give feedback on platform capabilities and limitations
* Prototype solutions and proofs\-of\-concept for government use cases, and present results to customers and stakeholders
* Work with ServiceNow developers and Azure cloud engineers to troubleshoot integration issues across Azure, ServiceNow, and government networks
* Contribute to design reviews, engineering standards, and technical documentation
* Support technical screening of AI/ML talent when requested
Operations \& Monitoring
* Support monitoring, logging, and alerting for AI workflows and agents
* Manage prompt and model versioning, and support CI/CD deployments of AI services
* Monitor output quality, model drift, and data quality in production workflows, and report findings
* Track model and compute usage costs and recommend optimizations
Government \& Compliance
* Build and operate within DISA STIGs, IL\-5 requirements, and Department of the Air Force security policies
* Implement CUI handling and data classification controls in AI workflows
Provide engineering evidence and documentation to cybersecurity teams for ATO and continuous monitoring .
*
**Qualifications:**
*Required:*
* US Citizenship required.
* Ability to obtain and maintain a secret clearance.
* 15\+ Years of relevant experience (Bachelor’s Degree in relevant field may be substituted for 5 years of relevant experience).
* Hands\-on experience building LLM applications, including RAG, prompt engineering, and agentic workflows
* Proficiency in Python, plus experience with LangChain, LlamaIndex, or similar AI frameworks
* Experience with Azure cloud services (Azure OpenAI, AI Foundry, DevOps, API Management)
* Experience with containers (Docker)
Similar Jobs
C
Capital One• 1mo ago
Machine Learning Engineer 4 (IC)
McLean, VAFull-time
W
Waystone• 6mo ago
AI Engineer
Leeds, West YorkshireFull-time
A
Aderant• 1mo ago
AI Engineer – Center of Excellence
Atlanta, GAFull-time
S
Signifyd• 1mo ago
Senior Machine Learning Engineer I // II
Chicago, ILFull-time