Material Group - Inference Performance Engineer

Remote - United States$206k - $206k+ Equity3mo ago

Remote Mid NA Artificial Intelligence Test Engineer Rust C++Go Python Transformers

Requirements

• 3+ years writing production systems code in Rust, C++, Go, or performance-oriented Python • Solid fundamentals in concurrency, memory, and tail latency • Familiarity with modern LLM inference: transformers, attention, KV cache, batching, speculative decoding, quantization • Comfortable optimizing with a profiler in hand • Be on the on-call rotation for what you ship • Experience with vLLM, TGI, TensorRT-LLM, SGLang, or llama.cpp • CUDA, Triton, kernel-level work, or experience with non-NVIDIA accelerators • Built or operated an OpenAI-compatible API at scale

Responsibilities

• Build and improve the inference runtime that serves our ASIC hardware • Own scheduling, continuous batching, KV cache optimization, prefill, and decode separation • Ship optimizations that move tokens/sec, TTFT, p99 latency, and cost per token • Collaborate with hardware and compiler teams to update kernels and operators • Maintain the OpenAI-compatible API surface - chat completions, streaming, tool use • Write the benchmarks and regression harnesses that catch latency and correctness drift

Benefits

• Stock Options • 100% paid Medical, Dental, and Vision insurance for Employees • Company Health Savings Account Contributions • 100% paid Short Term and Long Term Disability Insurance for Employees • Life and Voluntary Supplemental Insurance Options • Flexible Spending Account • Paid Holidays • We're a fast-paced team of optimists who believe when something is important, there is no obstacle that cannot be overcome. We work hard, are dedicated to our craft, and care about each other. • Equal Employment Opportunity