Senior Machine Learning Engineer- LLMs & Self-Hosted AI at Navan

<p>We are looking for a highly skilled <strong>Senior Machine Learning Engineer</strong> to lead our transition from on-demand, third-party LLM APIs to a fully self-hosted, scalable model ecosystem.</p> <p>Our core product is an advanced, agentic support chatbot capable of complex reasoning, API tool calling, database lookups, and orchestrating specialized Small Language Models (SLMs) for targeted NLP tasks. As we scale, our current deployment infrastructure (AWS SageMaker) is becoming unsustainable. You will be responsible for architecting, deploying, and optimizing an infrastructure capable of supporting 50 to 100 distinct models ranging from 100M to 70B parameters.</p> <p><strong>What You’ll Do:</strong></p> <ul> <li><strong>Inference Optimization:</strong> Deploy and manage large-scale models using high-performance inference engines (like vLLM) to ensure low latency and high throughput for our agentic chatbot.</li> <li><strong>Agentic Workflows:</strong> Develop and refine the chatbot's agentic capabilities, ensuring reliable tool-use, routing, and interactions between massive LLMs and specialized SLMs.</li> <li><strong>Model Fine-Tuning:</strong> Design and execute fine-tuning strategies to improve model accuracy on specific domain tasks and tool-calling execution.</li> <li><strong>Rigorous Evaluation:</strong> Build comprehensive offline and online evaluation frameworks to constantly measure model performance and business impact through structured A/B testing.</li> </ul> <p><strong>What We’re Looking For:</strong></p> <p><strong>Core Engineering &amp; AI Frameworks</strong></p> <ul> <li>Strong proficiency in <strong>Python</strong> and <strong>Bash</strong> scripting.</li> <li>Deep experience with <strong>PyTorch</strong> and the <strong>Hugging Face</strong> ecosystem.</li> <li>Experience using AI coding assistants natively in the terminal, specifically <strong>Claude Code</strong>, to accelerate development workflows.</li> </ul> <p><strong>LLMs, Inference &amp; Agents</strong></p> <ul> <li>Proven experience deploying models using <strong>vLLM</strong>, TGI, or similar high-performance inference servers.</li> <li>Strong fundamental understanding of LLM architectures, attention mechanisms, and generation parameters.</li> <li>Hands-on experience building <strong>Agentic systems</strong> (ReAct, function/tool calling, RAG).</li> <li>Expertise in <strong>fine-tuning strategies</strong> (e.g., SFT, RLHF, DPO) and parameter-efficient techniques (PEFT/LoRA).</li> </ul> <p><strong>Statistics &amp; Model Evaluation</strong></p> <ul> <li><strong>Offline Metrics:</strong> Deep understanding of classification/summarization metrics (Precision, Recall, F1, AUC) and retrieval metrics (MRR, NDCG, Precision/Recall @ k).</li> <li><strong>Online Metrics &amp; A/B Testing:</strong> Strong statistical foundation to design and analyze A/B tests safely, including the use of t-tests, Mann-Whitney U tests, and bootstrapping techniques.</li> </ul> <p><strong>Bonus Points</strong></p> <ul> <li><strong>Containerization &amp; Orchestration:</strong> Experience with Ray for orchestrating large-scale model deployments across multi-GPU clusters.</li> <li><strong>Model Quantization:</strong> Experience with memory optimization techniques like AWQ, GPTQ, GGUF, or FlashAttention to fit 70B models efficiently onto hardware.</li> <li><strong>API Development:</strong> Proficiency in building robust, asynchronous microservices using FastAPI to serve model requests.</li> </ul> <ul> <li>Knowledge of <strong>Data Engineering</strong> principles: dataset collection, cleaning, processing, and scalable storage.</li> </ul> <p><strong>Experience with core MLOps practices</strong>, including <strong>dataset versioning</strong> (e.g., DVC), <strong>experiment tracking</strong> (e.g., Weights &amp; Biases, MLflow), and <strong>model registries</strong>.</p>