NetBrain Technologies Inc. logo

NetBrain Technologies Inc.

AI Engineer at NetBrain Technologies Inc.

Toronto, ON | HybridFull-timeResearch & DevelopmentPosted 21 days ago
Apply with Pipeline

About the Role

<div class="content-intro"><p class="MsoNormal" style="margin-bottom: 12.0pt;"><span style="font-size: 10pt; font-family: 'trebuchet ms', geneva, sans-serif; color: black;">Founded in 2004, NetBrain is the leader in no-code network automation. Its ground-breaking Next-Gen platform provides IT operations teams with the ability to scale their hybrid multi-cloud connected networks by automating the processes associated with Diagnostic Troubleshooting, Outage Prevention and Protected Change Management. &nbsp;Today, over 2,500 of the world’s largest enterprises and managed services providers leverage NetBrain’s platform.</span></p></div><p><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 14pt;"><em><strong>What We Need</strong></em></span></p> <p><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">We’re looking for a Senior AI Engineer to design and build production-grade agent and RAG systems that power intelligent, reliable automation across our platform. This role combines hands-on engineering with system-level thinking—owning everything from architecture and evaluation to scalability, observability, and reliability in production. The ideal candidate thrives in ambiguity, moves quickly from prototype to production, and brings a strong focus on quality, safety, and real-world impact.</span></p> <p>&nbsp;</p> <p><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 14pt;"><em><strong>What You'll Do</strong></em></span></p> <p class="p1"><span style="font-size: 12pt;"><strong><span style="font-family: 'trebuchet ms', geneva, sans-serif;">Agent Platform Architecture</span></strong></span></p> <ul> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">Design and implement core capabilities for an enterprise-grade Agent platform, including orchestration patterns such as ReAct, Plan-and-</span></li> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">Execute, and Supervisor, as well as tool execution, context and memory management, and safety guardrails.</span></li> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">Design enterprise-grade Agent execution and governance mechanisms, including Human-in-the-Loop approval workflows, multi-tenant</span></li> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">permission isolation, policy enforcement, and secure execution controls.</span></li> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">Build reusable Agent Skills, standardized tool interfaces, and a scalable tool ecosystem deeply integrated with NetBrain platform</span></li> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">capabilities and business workflows.</span></li> </ul> <p class="p1"><span style="font-size: 12pt;"><strong><span style="font-family: 'trebuchet ms', geneva, sans-serif;">LLM and Model Optimization</span></strong></span></p> <ul> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">Design and implement LLM post-training strategies, including domain-specific Supervised Fine-Tuning (SFT), DPO/RLHF-based</span></li> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">preference alignment, and parameter-efficient fine-tuning techniques such as LoRA, to continuously improve model performance in the</span></li> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">network operations domain.</span></li> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">Build an Agent self-learning feedback loop that converts production execution traces, user feedback, and evaluation results into high-</span></li> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">quality datasets for continuously improving prompts, skills, models, and retrieval strategies.</span></li> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">Analyze and optimize LLM behavior across areas such as instruction following, tool calling, structured output generation, contextual</span></li> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">understanding, reasoning stability, and hallucination mitigation.</span></li> </ul> <p class="p1"><span style="font-size: 12pt;"><strong><span style="font-family: 'trebuchet ms', geneva, sans-serif;">Evaluation, Reliability, and Observability</span></strong></span></p> <ul> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">Build production-grade LLM and Agent evaluation frameworks and automated regression pipelines, including benchmark datasets,</span></li> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">deterministic checks, LLM-as-a-Judge, tool-call validation, retrieval-quality evaluation, and end-to-end task success metrics.</span></li> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">Establish release quality gates and hallucination-detection mechanisms for AI features to prevent significant accuracy, reliability, and</span></li> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">performance regressions from reaching production.</span></li> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">Build comprehensive AI system observability capabilities, including distributed tracing, structured logging, metrics, dashboards, and</span></li> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">alerting.</span></li> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">Rapidly diagnose and resolve production AI failures, including hallucinations, incorrect tool selection, invalid tool parameters, Agent</span></li> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">loops, retrieval-quality degradation, structured-output failures, latency regressions, and unexpected model behavior changes.</span></li> </ul> <p class="p1"><span style="font-size: 12pt;"><strong><span style="font-family: 'trebuchet ms', geneva, sans-serif;">Production Engineering and Technical Execution</span></strong></span></p> <ul> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">Design and implement highly reliable backend services for production AI and Agent workloads, including asynchronous and concurrent</span></li> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">processing, retries, timeouts, caching, rate limiting, and fault isolation.</span></li> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">Continuously optimize latency, throughput, token consumption, and infrastructure cost to meet platform SLA requirements and support</span></li> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">large-scale production workloads.</span></li> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">Independently diagnose and resolve complex AI system issues spanning prompts, models, RAG, tools, Agent workflows, backend</span></li> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">services, and infrastructure.</span></li> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">Lead technical design for critical modules and system-level capabilities, ensuring solutions align with platform architecture, security</span></li> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">requirements, engineering standards, and product requirements.</span></li> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">Drive technical improvements based on production data, evaluation results, and benchmarks, and collaborate closely with Engineering,</span></li> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">Product, QA, and other teams to deliver solutions into production.</span></li> </ul> <p class="p1"><span style="font-size: 12pt;"><strong><span style="font-family: 'trebuchet ms', geneva, sans-serif;">Applied Research and Technical Strategy</span></strong></span></p> <ul> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">Prototype, benchmark, and productionize emerging technologies such as GraphRAG, Knowledge Graphs, MCP, LLM Post-Training, and</span></li> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">Agent Self-Learning to improve grounding, multi-hop reasoning, and domain expertise.</span></li> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">Continuously evaluate Agent frameworks and supporting infrastructure, including LangChain, LangGraph, AutoGen, and LlamaIndex,</span></li> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">and provide technical recommendations for platform architecture evolution and product technology strategy.</span></li> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">Stay current with developments in LLM and Agent technologies and rapidly translate promising technologies into measurable, testable,</span></li> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">and production-ready engineering capabilities.</span></li> </ul> <p class="p2"><span style="font-family: 'trebuchet ms', geneva, sans-serif;"><em><strong><span style="font-size: 14pt;">What You Bring</span></strong></em></span></p> <ul> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">Bachelor's degree or higher in Computer Science, Artificial Intelligence, Electrical Engineering, or a related technical field. Master's or Ph.D.</span></li> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">preferred; equivalent practical experience will also be considered.</span></li> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">3+ years of experience in software engineering, machine learning, or applied AI, including 2+ years building, deploying, and operating production-</span></li> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">grade LLM or Agent applications. Must have delivered at least one LLM-powered feature end-to-end and owned its ongoing operation and</span></li> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">improvement after production launch.</span></li> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">Deep understanding of Agent architectures and LLM behavioral characteristics, including instruction following, tool-calling behavior, and context</span></li> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">sensitivity, with hands-on experience building multi-step workflows involving reasoning, tool execution, state management, structured outputs,</span></li> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">validation, and error recovery.</span></li> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">Proven ability to diagnose and resolve production LLM/Agent failures, including hallucinations, incorrect tool calls, retrieval-quality degradation,</span></li> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">Agent loops, structured-output failures, latency regressions, and regressions introduced by prompt or model changes.</span></li> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">Strong Python and distributed backend engineering skills, including API and service development, asynchronous and concurrent programming,</span></li> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">retries, timeouts, caching, rate limiting, testing, logging, and cross-service performance debugging.</span></li> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">Hands-on experience designing evaluation systems for LLM applications, including dataset construction, metric definition, regression testing, and</span></li> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">release quality gates.</span></li> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">Strong understanding of security risks associated with LLM and Agent applications, including prompt injection, data leakage, unsafe tool</span></li> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">execution, permission boundaries, and uncontrolled Agent autonomy.</span></li> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">Ability to independently design, implement, debug, deploy, and operate complex production systems.</span></li> </ul> <p class="p2"><span style="font-size: 12pt;"><strong><span style="font-family: 'trebuchet ms', geneva, sans-serif;">Preferred Qualifications</span></strong></span></p> <ul> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">Strong experience with RAG and advanced retrieval systems, including embeddings, vector and hybrid search, reranking, chunking strategies,</span></li> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">grounding and citation mechanisms, as well as multi-hop retrieval, Knowledge Graphs, or GraphRAG.</span></li> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">Familiarity with Agent frameworks such as LangGraph, LangChain, AutoGen, and LlamaIndex, as well as standardized protocols such as MCP</span></li> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">(Model Context Protocol) for connecting Agents with external tools and systems. A strong understanding of the underlying architecture is more</span></li> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">important than expertise in any specific framework.Experience designing Agent runtime mechanisms, including Human-in-the-Loop workflows such as risk classification, approval, interruption, pause</span></li> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">/resume, as well as context management for long-running Agent workflows, including state persistence, history compression, and memory</span></li> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">systems.</span></li> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">Experience with LangSmith or similar LLM observability and evaluation platforms.</span></li> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">Experience with LLM fine-tuning, including LoRA or other parameter-efficient fine-tuning techniques.</span></li> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">Experience applying LLM technologies to networking, infrastructure, cybersecurity, observability, or other complex technical domains.</span></li> <li class="p1" style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;"><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">Fluent in both English and Chinese, with strong cross-regional communication and collaboration skills.</span></li> </ul> <p>&nbsp;</p> <p><span style="font-size: 14pt; font-family: 'trebuchet ms', geneva, sans-serif;"><strong><em>What We Offer</em></strong></span></p> <p><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 12pt;">Our comprehensive compensation package is vital in how we recognize our people for the impact they make on us reaching our goals as a company.</span></p> <p><span style="font-size: 12pt; font-family: 'trebuchet ms', geneva, sans-serif;">For this role, the estimated base is CAD $130,000 - CAD $165,000 + Bonus. The actual salary may vary based on a range of factors, including market and individual qualifications objectively assessed during the interview process.</span></p> <p><span style="font-size: 12pt; font-family: 'trebuchet ms', geneva, sans-serif;">The range listed above is a guideline and may be modified. People Experience offers a comprehensive benefits package in addition to cash compensation that includes but is not limited to RRSP and medical/dental coverage. Speak with your Recruiter for more details on our Total Rewards philosophy.<br><br>#LI-BW1</span></p> <p>&nbsp;</p><div class="content-conclusion"><p><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 10pt;">NetBrain invites all interested and qualified candidates to apply for employment opportunities.</span></p> <p><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 10pt;">Qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, disability, protected veteran status, or other characteristics protected by law.</span></p> <p><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 10pt;">If you have a disability that prevents or limits your ability to use or access the site, or if you require any other accommodation in the application process due to a disability, you may request a reasonable accommodation. To make a request, please contact our People Team at:&nbsp;<a href="mailto:[email protected]">[email protected]</a>&nbsp;and we will be happy to assist you.</span></p> <p><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 10pt;">In compliance with applicable laws, NetBrain conducts holistic, individual background reviews in support of all hiring decisions.</span></p> <p><span style="font-family: 'trebuchet ms', geneva, sans-serif; font-size: 10pt;">It is unlawful in Massachusetts to require or administer a lie detector test as a condition of employment or continued employment. An employer who violates this law shall be subject to criminal penalties and civil liability.</span></p> <p>&nbsp;</p></div>