H
Hyphenconnectvia Greenhouse
LLM Pre-training & Distributed Engineer (AI Infrastructure)
Seattle, USAPosted 6mo ago
NLP / LLMMid LevelFull-time
About the Role
We are seeking a highly skilled LLM Pre-training & Distributed Systems Engineer. This role is essential for orchestrating large-scale machine learning training runs and optimizing distributed infrastructure. The ideal candidate will have a deep understanding of GPU clusters and extensive experience in system engineering to ensure efficient and reliable training processes.
Responsibilities:
- Orchestrate distributed training runs across 1,000+ GPUs using PyTorch, DeepSpeed, or Megatron-LM.
- Optimize networking (InfiniBand/RDMA) and memory management to prevent out-of-memory errors.
- Automate checkpointing and failure recovery during month-long training runs.
Required Skills:
- Deep expertise in 3D parallelism (Data, Tensor, Pipeline).
- Experience managing SLURM or Kubernetes-based GPU clusters.
- Strong systems engineering background (C++, CUDA, Python).
Similar Jobs
Q
Quantanite• 2mo ago
Generative AI Solutions Analyst
Mumbai, Maharashtra, IndiaFull-time
M
Medpace• 1mo ago
AI Engineer
London, England, UKFull-time
R
Rayngroup• 6mo ago
ML Software Engineer - Platform LLM Training and Inference
PakistanFull-time
T
Trustana• 6mo ago
LLM Engineer
SingaporeFull-time