Mitratech - Senior Software Engineer - AI/ML
Requirements
• Agent Orchestration: Production experience designing multi-agent systems with tool use, memory/state management, and fault-tolerant routing. Familiarity with LangChain, LangGraph, AutoGen, or custom orchestrators. • Agent Orchestration: • RAG & Retrieval: Hands-on experience building RAG pipelines end-to-end: chunking, embedding models, vector databases, retrieval tuning, and answer synthesis at production scale. • RAG & Retrieval: • Evaluations: Strong experience defining and running evaluation pipelines for generative AI — automated scoring, human evaluation design, hallucination mitigation, and drift monitoring. LLM-as-judge patterns are a plus. • Evaluations: • LLMs & GenAI: Demonstrated experience with foundation models and GenAI providers (AWS Bedrock, OpenAI, Anthropic, Meta). Comfortable with fine-tuning, instruction tuning, and prompt engineering at scale. • LLMs & GenAI: • Traditional NLP: Solid grounding in classical NLP techniques (NER, text classification, intent detection, topic modelling) and good judgement on when to apply them alongside or instead of LLMs. • Traditional NLP: • AWS Bedrock: Hands-on experience with Amazon Bedrock: foundation model APIs, Bedrock Agents, Knowledge Bases, and Guardrails. Experience with Bedrock Model Evaluation is a plus. • AWS Bedrock: • AWS Ecosystem: Proficiency with SageMaker, Lambda, ECS/EKS, S3, OpenSearch, IAM, CloudWatch, and VPC networking. • AWS Ecosystem: • MLOps & CI/CD: Familiarity with model registries, CI/CD for ML, feature stores, canary deployments, monitoring, and rollback. • MLOps & CI/CD: • IaC: Experience with Terraform or AWS CDK for reproducible infrastructure provisioning. • Python: Production-quality Python: packaging, testing (pytest), type hints, async programming, and clean ML pipeline abstractions. • Python: • ML Frameworks: Familiarity with traditional ML frameworks and fine-tuning workflows. • ML Frameworks: • Experiment Tracking: Experience with Langfuse, Arize or Langsmith, or equivalent for tracking runs, metrics, and artefacts. • Experiment Tracking: • Ability to translate ambiguous business goals into concrete technical solutions and communicate tradeoffs to non-technical stakeholders. • Strong collaborative instincts — comfortable working across engineering, product, and data teams. • A rigorous, evidence-driven mindset: you ship with confidence because you measure, test, and monitor thoroughly. • Education: • A Master’s degree in Machine Learning, Computer Science with a preference for specialization in the NLP domain.
Responsibilities
• Design, build, and operate multi-agent workflows and tool-enabled agents, implementing orchestration logic, state management, safety guardrails, and fallback strategies for resilient production pipelines. • Architect and maintain end-to-end RAG systems, covering document ingestion, chunking, embedding, vector retrieval, reranking, and answer synthesis with a focus on quality, attribution, and latency. • Evaluate and integrate LLMs and GenAI services across cost, performance, and privacy dimensions, selecting the right mix of managed and in-house models. • Develop, version, and optimise prompting strategies; implement automated prompt testing and regression tracking to maintain output quality and reliability. • Define and own evaluation frameworks for generative outputs, including automated metrics, LLM-as-judge approaches, human evaluation protocols, hallucination detection, and drift monitoring. • Build and operate scalable, secure AI infrastructure on AWS (Bedrock, SageMaker, Lambda, OpenSearch), following well architected principles and infrastructure-as-code practices. • Own the full deployment lifecycle: CI/CD for models and agents, testing strategies, observability, and rollback procedures. • Ensure data quality through rigorous validation and augmentation, and proactively source datasets for training, fine-tuning, and evaluation.
Apply in one click
Upload My Resume
Drop here or click to browse · Tap to choose · PDF, DOCX, DOC, RTF, TXT