TixelJobs
T
Toptalvia muse

ML Inference Engineer – Terminal-Bench Task Designer

REMOTEPosted 1w ago
ML EngineerMid LevelFull-time#full-stack-programming#remote

About the Role

Headquarters: Overview We are building a benchmark that tests how well AI agents handle real expert work in a terminal. We are hiring ML inference and serving experts to turn real, paid tasks from their jobs into expert-level Terminal-Bench challenges. Each task includes the environment, a working solution, and an automated check that proves the result is correct. ***THIS IS A TASK BASED COMPENSATION MODEL. PLEASE CHECK THE APPLICATION QUESTIONS FOR MORE DETAILS.*** Responsibilities • Design expert-level Terminal-Bench tasks that turn real, paid ML inference work from your own experience into challenges an AI agent must solve end to end. • Define the files, systems and environments each task runs on, such as model serving with streaming, so the challenge is realistic, reproducible and command-line solvable. • Write a working reference solution for every task and verify it runs end to end, sized for about 60 to 75 minutes of expert effort. • Build automated checks that objectively prove a result is correct, including cases where ML outputs go wrong silently, such as numerical drift or metric bugs. • Package tasks with Docker and Git so they are reproducible, and document the systems
Share