Advanced AI Evaluation Manager

Senior Manager, RAG/LLM Specialist

Experience: 4–7 Years

Role Overview:

Lead the design and optimization of advanced RAG pipelines and model fine tuning processes.

Bridge the gap between prototype and enterprise-scale LLM deployment.

Key Responsibilities
-
-
-
-

Pipeline Ownership: Design and manage complex, multi-stage RAG pipelines ensuring low latency and high relevance.

Model Optimization: Lead fine-tuning initiatives (PEFT/LoRA) for open-source models to improve domain-specific task performance. Advanced Evaluation: Develop automated evaluation frameworks (e.G., RAGAS) to continually measure LLM accuracy, context precision, and recall.

Vector Strategy: Architect metadata filtering and hybrid search strategies within vector databases (e.G., Pinecone, Milvus).

Team Mentorship:

Guide junior analysts in prompt engineering, chunking strategies, and code quality.

...

Ready to Apply?

Submit your application today and join our talented team at EXL.