In this role, you will shape the evaluation, validation, and enhancement of advanced AI models in production environments. You will spearhead the development of a shared evaluation platform that empowers research, engineering, and customer-facing teams to confidently measure model performance. Your efforts will establish stringent standards for benchmarking, statistical validation, data quality, and reproducibility. By combining hands-on technical leadership with people management, you will cultivate a high-performing team while remaining actively engaged in coding, experiments, and methodologies. You will evaluate predictive quality, inference latency, and cost efficiency to inform research priorities and product launches. This position offers the chance to define evaluation practices from the ground up, directly impacting the reliability and effectiveness of next-generation enterprise AI.

• Build and maintain a shared evaluation platform that supports research, engineering, and Applied AI teams, enabling independent benchmarking and investigation of model performance.
• Develop and continuously enhance benchmark suites, curated datasets, and model comparisons to evaluate predictive quality across diverse business use cases.
• Establish rigorous evaluation standards covering statistical metrics, validation strategies, data splits, leakage prevention, calibration, uncertainty estimation, and benchmark contamination.
• Design reproducible evaluation pipelines with version-controlled inputs, datasets, and artifacts, integrating automated regression checks into research and release workflows.
• Benchmark models against competing approaches using fair and consistent protocols for hyperparameter tuning, data access, computational resources, and end-to-end latency measurement.
• Evaluate predictive performance, inference latency, and cost efficiency across deployment configurations, task types, and dataset characteristics.
• Support customer proof-of-concept evaluations by providing reliable tooling, methodological guidance, and data-driven analysis to demonstrate model value.
• Translate evaluation findings into actionable research priorities, release recommendations, and evidence-based improvement plans for customer use cases.
• Recruit, lead, and develop a small team of evaluation specialists, setting priorities, providing technical coaching, and delivering constructive performance feedback.
• Remain hands-on with Python development, experimental design, statistical analysis, and evaluation methodology while fostering collaboration across research, engineering, and customer-facing teams.