Cloud Inference Engineer

<h1>Qualifications</h1> <ul> <li>CUDA + GPU inference optimization</li> <li>vLLM, SGLang, or TensorRT-LLM experience</li> <li>KV caching, paged attention, batching, token streaming, etc.</li> <li>Distributed compute (with GPUs is a super plus)</li> <li>No degree required</li> </ul> <h1>Company</h1> <p>Luminal (YC S25) builds an AI compiler and serving stack that makes models 10x faster and production ready with one line.</p> <h1>Role</h1> <p>Founding, on site in downtown SF. Ship low latency, high throughput model serving on Luminal Cloud.</p> <p>Day to day responsibilities:</p> <ul> <li>Deploy and tune models with optimizations like KV caching, paged attention, sequence packing, etc.</li> <li>Conducting model performance reviews</li> <li>Improve scheduler, batcher, autoscaling; profile latency, cost, utilization</li> <li>Sometimes write kernels and, yes, occasional tasteful shitposting</li> </ul>