Member of Technical Staff - ML Performance
- Location
- Remote (US)
- Job type
- Full-time
- Level
- Mid-level
- Education
- Bachelor's degree
- Listed
- 23 days ago
Skills & highlights for this role
- PyTorch
- distributed training
- CUDA
Also useful
- Triton
- Megatron-Core
- Nsight Systems
- FlashAttention
Responsibilities
- Distributed Training Throughput: Own step time and model FLOPs utilization for multi-node video world model training, choosing the tensor, context, and expert parallelism mix in PyTorch FSDP2 and Megatron-Core rather than inheriting a default.
- Inference Throughput Optimization: Profile end-to-end latency, GPU utilization, memory pressure, and kernel efficiency with PyTorch Profiler, Nsight Systems, Nsight Compute, and torch.utils.benchmark, then improve throughput through torch.compile/Inductor, CUDA Graphs, mixed and low precision, quantization, operator fusion, and multi-GPU serving.
- Precision & Numerical Stability: Take BF16, FP8, and NVFP4 recipes from running to converging on Blackwell, chasing scaling-factor and accumulation bugs into the video tokenizer and VAE layers where the activation outliers actually live.
- Kernels & Compilation: Write and tune the CUDA and Triton kernels PyTorch does not give us, driving FlashAttention-4, FlexAttention, and torch.compile integration so quadratic attention over long video sequences stops setting step time.
- Communication & Overlap: Tune NCCL collectives and compute/communication overlap across NVLink domains and the fabric, using the NCCL flight recorder to turn a watchdog timeout into a named rank and collective, not a restart.
- Fault Diagnostics & Recovery: Build the detection layer for silent data corruption (SDC), stuck CUDA kernels, and "card-freeze" hangs, plus asynchronous and tiered checkpointing that makes an interruption cost minutes rather than a day.
Requirements
- Bachelor's degree or equivalent hands-on experience in Computer Science, Computer Engineering, or a related technical field.
- Deep hands-on experience with PyTorch and at least one large-scale parallelism stack (FSDP2, Megatron-Core, TorchTitan, or DeepSpeed) on real multi-node jobs, not single-node approximations.
- Fluency in Python and C++/CUDA with the ability to predict where a kernel will stall from its memory access pattern before profiling.
- Experience profiling live training and inference runs with Nsight Systems or the PyTorch profiler and translating traces into quantifiable step-time or MFU improvements.
- Expertise in at least one of low-precision numerics, kernel authoring, or large-run fault diagnosis, and credibility in the others.
Nice to Have
- Experience optimizing training or inference workloads across very large GPU clusters, including topology-aware placement, scaling efficiency, performance isolation, and diagnosing failures that emerge only at fleet scale.
- Experience writing Triton, CUTLASS, or CuTe-DSL kernels, or contributing to open-source kernel libraries.
- Experience implementing context or sequence parallelism for long-horizon video or high-token-count models.
- Experience running or porting large training workloads on AMD GPUs (ROCm) or Google TPUs (JAX/XLA).
- Experience building fault-tolerant training with elastic world size, dynamic node re-queueing, or asynchronous distributed checkpointing.
- Experience optimizing generative inference for interactive rollouts, including few-step samplers, distillation, and KV or latent caching.
- Publications or presentations on machine learning systems, compilers, or high-performance kernels.
About US
Veeda AI is building the next generation of multimodal foundation world models for Physical AI. We're a small, fast-moving team of engineers and researchers from leading AI labs, tackling some of the most challenging problems at the intersection of AI, robotics, and embodied intelligence. If you're excited about pushing the boundaries of what's possible with Physical AI, you'll have the opportunity to make an outsized impact from day one.
From Veeda AI's careers page, first listed on Rolev.ai on October 17, 2026. Rolev.ai is not the employer.
How Rolev.ai helps with this application
Rolev.ai is a job search and application tool for US job seekers. For this Member of Technical Staff - ML Performance role at Veeda AI, it shows how your resume fits, tailors your resume to the posting, and prepares the application for you to review.
See how you fit
Openings from employer career sites are ranked against your whole resume—and searchable inside Claude, ChatGPT, and Cursor via Rolev.ai MCP.
Tailor without inventing
Every change to your resume is checked against your original. New experience goes in only with your yes—zero fabricated claims.
Draft the full kit
Tailored and Full Match resumes, role-specific cover letters, and answers to employer screening questions are prepared in seconds.
1-Click Extension Autofill
The Chrome extension fills the employer's Greenhouse, Workday, Lever, or Ashby form in 1 click. You review it and press Submit.
Similar roles
All Machine Learning Engineer jobs- Machine Learning Engineer 4Capital One · McLean, VA
- Manager, Machine Learning Engineer (US)Clio · Remote -
- Member of Technical Staff - Machine Learning EngineerAsari AI · San Francisco, CA
- AI Infrastructure EngineerSpellbrush · San Francisco, CA
- Research Engineer, EvalsVariance · San Francisco, CA
- Autonomy EngineerRemedy Robotics · San Francisco, CA
Common questions
Where is the Member of Technical Staff - ML Performance job at Veeda AI located?
Yes, this Member of Technical Staff - ML Performance opening at Veeda AI is listed as Remote (US). Always check Veeda AI's application page for specific state eligibility or relocation policies.
What qualifications are required for Member of Technical Staff - ML Performance at Veeda AI?
For the Member of Technical Staff - ML Performance role, Veeda AI highlights a bachelor's degree; experience with PyTorch, distributed training, CUDA. Rolev.ai can score your existing resume against these requirements and show any skill gaps before you apply.
How do I apply for the Member of Technical Staff - ML Performance position at Veeda AI?
This listing links directly to Veeda AI's Ashby application page. You can apply directly on Veeda AI's site, or use Rolev.ai to check your resume fit, generate a zero-hallucination tailored resume and cover letter for this posting, and autofill the Ashby application form in 1 click with the Rolev.ai Chrome extension. You always review the form and press Submit yourself.