M
Marathon TSvia muse
Senior Data Scientist (Generative AI)
REMOTE$125K - $140K/yrPosted 2w ago
Data ScientistSeniorContract#remote
About the Role
Senior or Principal Data Scientist (Generative AI)About the role
We are looking for a hands\-on data scientist to build and lead the quality of generative AI workflows that review large document sets and draft structured findings for expert reviewers in a regulated environment. You will write and tune the prompts, retrieval and output logic, and own how the program proves the outputs are accurate and well\-evidenced enough for experts to rely on.
What you'll do
* Design and build generative AI workflows that review large document sets against defined criteria, identify potential issues, and draft structured findings and documents.
* Build the evaluation framework: gold sets from historical reviews, and metrics for accuracy, evidence linkage, unsupported statements and consistency, run as a regression check before every prompt or model change.
* Set prompt engineering and retrieval standards: chunking, retrieval over source documents and reference standards, and structured outputs that match application contracts.
* Build document chunking and retrieval over large, mixed\-format document sets.
* Define provenance and confidence: prompt and model versioning in MLflow, a confidence score reviewers can act on, and links to the evidence behind each statement.
* Measure improvement from reviewer edits by comparing AI drafts with the final human versions, and feed the results back into prompts.
* Build monitoring of output quality and usage, and document each version and its results.
* Set guardrails for confidential data, and explain results to business leaders in plain language.
* At the Principal level, set standards across workflows and mentor other data scientists.
What you bring
* Senior: 8\+ years in data science or machine learning. Principal: 12\+ years. Both include 2\+ years building production applications on large language models.
* Python and PySpark; Databricks notebooks and MLflow.
* Proven evaluation design for generated text: gold sets, automated scoring and human review loops.
* Databricks Mosaic AI: Model Serving, Vector Search and Agent Framework.
* Document understanding: extracting content from PDF, Excel and scanned files.
* Ability to explain model behavior and limits to non\-technical domain experts.
* Experience with Azure OpenAI models.
Nice to have
* Audit, accounting or regulatory domain knowledge.
* Model\-graded evaluation and human\-in\-the\-loop review systems.
Working with us
* You will work inside a secure, private cloud environment with highly confidential data.
* You must be authorized to work in the United States and able to pass a background check.
* We are looking for people who can start within a few weeks.
Pay: $125,000\.00 \- $140,000\.00 per year
Benefits:
* 401(k)
* Dental insurance
* Health insurance
* Paid time off
* Vision insurance
Application Question(s):
* Do you have Python and PySpark;Databricks notebooks and MLflow experience?
* Do you have experience with Azure OpenAI models?
Experience:
* Data Science or Machine learning : 8 years (Required)
Work Location: Hybrid remote in Ashburn, VA
We are looking for a hands\-on data scientist to build and lead the quality of generative AI workflows that review large document sets and draft structured findings for expert reviewers in a regulated environment. You will write and tune the prompts, retrieval and output logic, and own how the program proves the outputs are accurate and well\-evidenced enough for experts to rely on.
What you'll do
* Design and build generative AI workflows that review large document sets against defined criteria, identify potential issues, and draft structured findings and documents.
* Build the evaluation framework: gold sets from historical reviews, and metrics for accuracy, evidence linkage, unsupported statements and consistency, run as a regression check before every prompt or model change.
* Set prompt engineering and retrieval standards: chunking, retrieval over source documents and reference standards, and structured outputs that match application contracts.
* Build document chunking and retrieval over large, mixed\-format document sets.
* Define provenance and confidence: prompt and model versioning in MLflow, a confidence score reviewers can act on, and links to the evidence behind each statement.
* Measure improvement from reviewer edits by comparing AI drafts with the final human versions, and feed the results back into prompts.
* Build monitoring of output quality and usage, and document each version and its results.
* Set guardrails for confidential data, and explain results to business leaders in plain language.
* At the Principal level, set standards across workflows and mentor other data scientists.
What you bring
* Senior: 8\+ years in data science or machine learning. Principal: 12\+ years. Both include 2\+ years building production applications on large language models.
* Python and PySpark; Databricks notebooks and MLflow.
* Proven evaluation design for generated text: gold sets, automated scoring and human review loops.
* Databricks Mosaic AI: Model Serving, Vector Search and Agent Framework.
* Document understanding: extracting content from PDF, Excel and scanned files.
* Ability to explain model behavior and limits to non\-technical domain experts.
* Experience with Azure OpenAI models.
Nice to have
* Audit, accounting or regulatory domain knowledge.
* Model\-graded evaluation and human\-in\-the\-loop review systems.
Working with us
* You will work inside a secure, private cloud environment with highly confidential data.
* You must be authorized to work in the United States and able to pass a background check.
* We are looking for people who can start within a few weeks.
Pay: $125,000\.00 \- $140,000\.00 per year
Benefits:
* 401(k)
* Dental insurance
* Health insurance
* Paid time off
* Vision insurance
Application Question(s):
* Do you have Python and PySpark;Databricks notebooks and MLflow experience?
* Do you have experience with Azure OpenAI models?
Experience:
* Data Science or Machine learning : 8 years (Required)
Work Location: Hybrid remote in Ashburn, VA
Similar Jobs
S
Scotiabank• 1mo ago
Software Engineer (AI Engineer)
Toronto, Ontario, CanadaFull-time
E
EY• 1mo ago
EY - GDS Consulting - AIA - Build AI - Senior
Kolkata, West Bengal, IndiaFull-time
C
CapTech• 3w ago
Lead Machine Learning / Data Science Engineer
Denver, COFull-time
C
Citi• 1mo ago
Assistant Vice President Data Science and Gen AI
Pune Division, Maharashtra, IndiaFull-time