TixelJobs
U
United Softwvia muse

Senior ML Systems Engineer, LLM Inference Performance on AWS Trainium

REMOTEPosted 2w ago
ML EngineerSeniorContract#remote

About the Role

**Role Name: Sr. ML Systems Engineer, LLM Inference Performance on AWS Trainium**

**Location: Canada (Remote)**

**Role overview**

· We are seeking a hands\-on machine learning systems engineer to optimize production\-grade large language model inference across current and next\-generation AWS Trainium platforms, including Trainium2 and Trainium3\.

· The work focuses on decode latency, throughput under representative load, price\-performance, long\-context and multi\-chip scaling, and reproducible benchmarking against leading GPU platforms.

· As part of a small, senior engineering team, you will own substantial workstreams end to end, from model\-level optimization through measurement, analysis, and documentation. You will communicate technical findings and business implications clearly to customer engineers and leadership.

· This is a hands\-on, customer\-facing role focused on delivering measurable production outcomes.

**Key responsibilities**

· Lead decode optimization. Improve decode latency, throughput, and cost on AWS Trainium platforms, validating correctness at every step. Initial priorities include speculative decoding and native multi\-token prediction (MTP) for Qwen\-family models in vLLM on AWS Neuron.

· Benchmark and assess performance rigorously. Design and run matched, reproducible tests against leading GPU platforms, controlling prompts, concurrency, cache state, and warm\-up. Deliver latency\-throughput Pareto frontiers and price\-performance comparisons aligned with customer service\-level objectives.

· Lead cross\-generation optimization. Benchmark and migrate workloads across Trainium generations, including Trainium2 and Trainium3\. Scale serving on multi\-chip Trainium systems for long\-context workloads, including contexts of 100K tokens or more, and apply architecture\-specific optimizations.

· Extend model coverage. Onboard additional architectures, including hybrid state\-space and mixture\-of\-experts models, reusing and generalizing existing implementations where practical.

· Enable customer ownership. Create runbooks, implementation guides, and benchmark documentation, and support hands\-on workshops so customer engineers can operate, reproduce, and tune the stack independently.

**What success looks like**

· Decode latency and serving cost improve measurably while maintaining model quality.

· Performance results are reproducible, correctly scoped, and defensible across hardware platforms.

· Final benchmarks reflect representative customer workloads, not synthetic peak numbers alone.

· Customer engineers can reproduce the environment, run the benchmarks, and continue optimization independently.

**Basic qualifications**

· 5\+ years of experience in machine learning or systems engineering, including 2\+ years focused on production model inference or serving.

· Strong proficiency in Python and PyTorch, with the ability to read and modify model implementations involving attention, mixture\-of\-experts routing, and KV\-cache management.

· Hands\-on experience with an LLM serving framework such as vLLM, TensorRT\-LLM, SGLang, or Triton Inference Server.

· Deep understanding of LLM inference performance, including prefill and decode, continuous batching, and tensor and expert parallelism, and their impact on latency, throughput, and cost.

· Experience profiling and optimizing accelerator workloads at the framework, compiler, or kernel level, and designing rigorous performance experiments.

· Strong written and verbal communication skills, with the ability to explain engineering trade\-offs to technical audiences and business impact to senior stakeholders.

**Preferred qualifications**

· Experience with AWS Trainium or Inferentia and the AWS Neuron SDK, including NxD Inference, vLLM on Neuron, or Neuron Kernel Interface (NKI), or with migrating workloads across accelerator generations.

· Experience implementing or evaluating speculative decoding, such as draft\-model approaches, Medus
Share