TixelJobs
N
Nebiusvia muse

Staff Machine Learning Engineer, LLM Inference Optimization

BerlinPosted 3w ago
MLOpsStaff+Full-time#ml

About the Role

About Nebius:

Nebius is leading a new era in cloud infrastructure for the global AI economy. We are building a full-stack AI cloud platform that supports developers and enterprises from data and model training through to production deployment, without the cost and complexity of building large in-house AI/ML infrastructure.

Built by engineers, for engineers. From large-scale GPU orchestration to inference optimization, we own the hard problems across compute, storage, networking and applied AI.

Listed on Nasdaq (NBIS) and headquartered in Amsterdam, we have a global footprint with R&D hubs across Europe, the UK, North America and Israel. Our team of 1,500+ includes hundreds of engineers with deep expertise across hardware, software and AI R&D.

Role Summary

Nebius Token Factory is building production-grade inference for open-source, customer, and internally optimized models. This role owns the path from model artifact to fast, reliable, cost-efficient endpoint. The work sits at the intersection of model internals, inference engines, serving architecture, benchmarking, and production deployment.

 

What You Will Do

  • Own a platform-level optimization pillar such as quantization, speculative decoding, KV-cache optimization, engine selection, or long-context serving.

  • Create reusable evaluation, benchmarking, and rollout frameworks across multiple model families.

  • Provide technical direction to MLEs, Applied Scientists, GPU kernel engineers, and infrastructure engineers on shared performance goals.

  • Optimize LLM and VLM endpoints for latency, throughput, memory efficiency, GPU utilization, quality, and cost per token.

  • Deploy, configure, benchmark, and extend inference engines such as vLLM, SGLang, TensorRT-LLM, Triton Inference Server, NVIDIA Dynamo, or similar systems.

  • Build and productionize model-compression workflows, including quantization, quantization-aware training, distillation, low-bit serving, and accuracy recovery.

  • Implement or integrate speculative decoding, draft-model approaches, KV-cache optimization, prefix caching, chunked prefill, continuous batching, and disaggregated prefill/decode serving.

  • Build reproducible benchmark harnesses for TTFT,

    Share