S
Sully Aivia Ashby
Applied Research Scientist
REMOTEPosted 2mo ago
ResearchMid LevelFull-time#remote
Not sure if you're a good fit?
Upload your resume and TixelJobs AI will compare it against Applied Research Scientist at Sully Ai. Get a match score, missing keywords, and improvement tips before you apply.
Free preview · Your resume stays private
About the Role
About Us
At Sully.ai http://Sully.ai, We’re Building the Most Impactful Healthcare Company on Earth
We believe that access to a great doctor is a basic human right. Today, that’s not a reality. Delays, misdiagnoses, administrative chaos, and burnout plague the system.
Our Mission
One Human, One Doctor.
We enable our customers to staff 30% of their workforce with AI by creating a shared agent architecture for scale and efficiency. All powered by our own patented, world-class models and deployed in real-world care.
KEY RESPONSIBILITIES
- Build and scale automated evaluation pipelines (LLM-as-judge + human review) with clinical-grade benchmarks.
HARD REQUIREMENTS
- Proven experience designing agentic processes and LLM evaluation/benchmarking frameworks.
- Strong Python and ML background (PyTorch/TensorFlow, Hugging Face, LangChain/LlamaIndex).
- Demonstrated ability to design rigorous experiments and translate findings into production.
- Track record of published research or deep applied work in LLMs and agent evaluation.
- Strong communication and technical writing skills to articulate complex findings clearly.
- First-Month Focus
- Audit existing evaluation approaches for clinical and agentic tasks.
- Define initial benchmarks and build early automated pipelines.
- Partner with engineering to land first set of CI gates for accuracy, factuality, and safety.
- 90 Days
- Deliver a repeatable evaluation framework with automated pipelines in production.
- Demonstrate measurable improvements in robustness, hallucination reduction, or safety.
- Publish or present internal research findings that directly shape product reliability.
- If you’ve ever said, “I want to do work that actually matters”, this is it. Let’s build something life-changing, together.
KEY RESULTS (FIRST 90 DAYS)
- Deliver a repeatable evaluation framework with automated pipelines in production.
- Demonstrate measurable improvements in robustness, hallucination reduction, or safety.
- Publish or present internal research findings that directly shape product reliability.
Why Join Sully.ai http://Sully.ai?
🔥 Revolutionizing the antiquated $800B+ Healthcare market
🧠 60%+ Ex-founders who have built, scaled, exited. We hire A-players
⚡️ Speed matters: we operate with urgency, autonomy, and ownership
🧪 You’ll work on real, first-of-their-kind problems at the edge of AI and medicine
❤️ Your work directly unlocks doctors to reclaim their time, and patients get better, faster care
Sully.ai is an equal opportunity employer. In addition to EEO being the law, it is a policy that is fully consistent with our principles. All qualified applicants will receive consideration for employment without regard to status as a protected veteran or a qualified individual with a disability, or other protected status such as race, religion, color, national origin, sex, sexual orientation, gender identity, genetic information, pregnancy or age. Sully.ai prohibits any form of workplace harassment.
At Sully.ai http://Sully.ai, We’re Building the Most Impactful Healthcare Company on Earth
We believe that access to a great doctor is a basic human right. Today, that’s not a reality. Delays, misdiagnoses, administrative chaos, and burnout plague the system.
Our Mission
One Human, One Doctor.
We enable our customers to staff 30% of their workforce with AI by creating a shared agent architecture for scale and efficiency. All powered by our own patented, world-class models and deployed in real-world care.
KEY RESPONSIBILITIES
- Build and scale automated evaluation pipelines (LLM-as-judge + human review) with clinical-grade benchmarks.
HARD REQUIREMENTS
- Proven experience designing agentic processes and LLM evaluation/benchmarking frameworks.
- Strong Python and ML background (PyTorch/TensorFlow, Hugging Face, LangChain/LlamaIndex).
- Demonstrated ability to design rigorous experiments and translate findings into production.
- Track record of published research or deep applied work in LLMs and agent evaluation.
- Strong communication and technical writing skills to articulate complex findings clearly.
- First-Month Focus
- Audit existing evaluation approaches for clinical and agentic tasks.
- Define initial benchmarks and build early automated pipelines.
- Partner with engineering to land first set of CI gates for accuracy, factuality, and safety.
- 90 Days
- Deliver a repeatable evaluation framework with automated pipelines in production.
- Demonstrate measurable improvements in robustness, hallucination reduction, or safety.
- Publish or present internal research findings that directly shape product reliability.
- If you’ve ever said, “I want to do work that actually matters”, this is it. Let’s build something life-changing, together.
KEY RESULTS (FIRST 90 DAYS)
- Deliver a repeatable evaluation framework with automated pipelines in production.
- Demonstrate measurable improvements in robustness, hallucination reduction, or safety.
- Publish or present internal research findings that directly shape product reliability.
Why Join Sully.ai http://Sully.ai?
🔥 Revolutionizing the antiquated $800B+ Healthcare market
🧠 60%+ Ex-founders who have built, scaled, exited. We hire A-players
⚡️ Speed matters: we operate with urgency, autonomy, and ownership
🧪 You’ll work on real, first-of-their-kind problems at the edge of AI and medicine
❤️ Your work directly unlocks doctors to reclaim their time, and patients get better, faster care
Sully.ai is an equal opportunity employer. In addition to EEO being the law, it is a policy that is fully consistent with our principles. All qualified applicants will receive consideration for employment without regard to status as a protected veteran or a qualified individual with a disability, or other protected status such as race, religion, color, national origin, sex, sexual orientation, gender identity, genetic information, pregnancy or age. Sully.ai prohibits any form of workplace harassment.
Ready to apply?
This job is active. Apply now to get in early.