T
Toptalvia muse
ML Inference Engineer â Terminal-Bench Task Designer (vLLM / SGLang / TensorRT-LLM)
REMOTEPosted 1w ago
ML EngineerMid LevelFull-time#artificial-intelligence#docker#git#graphics-processing-unit#llm#remote
About the Role
Overview We are building a benchmark that tests how well AI agents handle real expert work in a terminal. We are hiring ML inference and serving experts to turn real, paid tasks from their jobs into expert-level Terminal-Bench challenges. Each task includes the environment, a working solution, and an automated check that proves the result is correct. ***THIS IS A TASK BASED COMPENSATION MODEL. PLEASE CHECK THE APPLICATION QUESTIONS FOR MORE DETAILS.*** Responsibilities ⢠Design expert-level Terminal-Bench tasks that turn real, paid ML inference work from your own experience into challenges an AI agent must solve end to end. ⢠Define the files, systems and environments each task runs on, such as model serving with streaming, so the challenge is realistic, reproducible and command-line solvable. ⢠Write a working reference solution for every task and verify it runs end to end, sized for about 60 to 75 minutes of expert effort. ⢠Build automated checks that objectively prove a result is correct, including cases where ML outputs go wrong silently, such as numerical drift or metric bugs. ⢠Package tasks with Docker and Git so they are reproducible, and document the systems, tools, data and rules involved so reviewers can validate difficulty. ⢠Iterate on tasks based on reviewer feedback and agent test runs, adjusting difficulty and fixing ambiguities so each task stays expert-level, fair and objectively gradable. Skills/Experience ⢠3+ years doing ML inference or serving work hands-on (not only studying, selling or managing it), or a PhD plus active research in the subfield ⢠Did this work in the last 3 years and can name the systems, tools, data and rules used ⢠Trained, served or evaluated models at real scale (multi-GPU or production serving), not only notebooks or API calls ⢠Hands-on with PyTorch or JAX internals and serving stacks such as vLLM, SGLang or TensorRT-LLM ⢠Can explain how ML results go wrong without errors (numerical drift, metric
Similar Jobs
C
Capital One• 1mo ago
Machine Learning Engineer 4 (IC)
McLean, VAFull-time
W
Waystone• 6mo ago
AI Engineer
Leeds, West YorkshireFull-time
A
Aderant• 1mo ago
AI Engineer – Center of Excellence
Atlanta, GAFull-time
S
Signifyd• 1mo ago
Senior Machine Learning Engineer I // II
Chicago, ILFull-time