Meghna Bose
Principal Test Architect · 17 years experience
Hyderabad, Telangana
[Your email] · [Your phone]
[Your LinkedIn URL] · [Your GitHub / Portfolio URL]
Professional Summary
- Seventeen-year principal engineer in test architecture, defining enterprise AI quality standards for a Fortune 500 software company with 30,000 engineers across 14 global development centers
- Designed the world's first AI Test Pyramid standard adopted as the company's engineering guideline for LLM-powered application testing across 120 AI product teams
- Architects cross-product LLM evaluation infrastructure processing 2M+ evaluation requests daily using DeepEval, RAGAS, and 8 custom-built metric engine plugins
- Established the AI Quality Standards Board governing hallucination rate, coherence, safety, and contextual relevancy evaluation criteria across 120 production AI features
- Patent holder for Distributed LLM Evaluation Orchestration enabling parallel multi-dimension LLM quality assessment at enterprise scale with pluggable evaluator architecture
- Leads a Principal Architecture team of 3 engineers responsible for AI testing innovation research, standards definition, and proof-of-concept prototyping for agentic systems
- Principal architect of the company's AI Observability platform built on OpenTelemetry with LLM-specific distributed tracing spans covering token usage, latency, and quality drift signals
- Expert in agentic test architecture: designed test harnesses for LangGraph and AutoGen-based multi-agent orchestration systems deployed in enterprise customer environments
Work Experience
Principal Test Architect
[Company name] · [Start date – End date]
17 years in software engineering with deep specialization in test architecture and AI quality systems. Currently Principal Test Architect at a Fortune 500 technology company (50,000+ employees). Defined global test architecture standards used across all engineering divisions worldwide. Managed quarterly architecture review boards. Led a 3-person Principal Architecture team and contributed directly to the company's AI strategy through the Office of the CTO.
Responsibilities
- Define enterprise AI quality architecture standards adopted across all engineering divisions globally
- Chair the AI Quality Standards Board governing LLM evaluation criteria for 120 production AI features
- Architect and continuously evolve the distributed LLM evaluation platform and AI observability infrastructure
- Lead Principal Architecture team of 3 for AI testing innovation research and next-generation prototyping
- Review architecture proposals for AI-powered products at the company's Architecture Review Board
- Partner with the Office of the CTO on AI safety strategy, responsible AI standards, and model risk governance
- Represent the company at ISO/IEC and IEEE AI quality standards bodies and industry working groups
- Publish technical guidance documents, reference architectures, and design patterns for the engineering community
- Prototype and evaluate emerging AI testing technologies including MCP tool testing and agentic evaluation
- Mentor senior engineers pursuing principal and staff engineering career paths through structured sponsorship
- Present AI quality architecture research at major industry conferences and academic institutions
Project Experience
AI Test Pyramid Standard
Authored and socialized the company's global AI Test Pyramid standard defining unit-level LLM component tests, integration-level RAG pipeline tests, and end-to-end agentic workflow tests — adopted as mandatory engineering guideline by 120 AI product teams. Technologies: DeepEval, RAGAS, Python, TypeScript, Architecture Frameworks, Confluence.
Distributed LLM Evaluation Platform
Architected a distributed LLM evaluation platform processing 2M+ evaluation calls daily with pluggable metrics, multi-model provider support, real-time quality dashboards, and automated regression scoring for model version comparisons. Technologies: Python, Kubernetes, Apache Kafka, DeepEval, Redis, OpenTelemetry, Grafana.
AI Observability Platform
Built LLM observability layer on OpenTelemetry with custom LLM spans tracking token consumption, hallucination rate, response latency, context window utilization, and model version drift across 120 production AI features. Technologies: OpenTelemetry, Python, Jaeger, Prometheus, Grafana, LangSmith.
Agentic System Test Framework
Designed enterprise-grade behavioral test harness for LangGraph and AutoGen multi-agent pipelines validating tool selection accuracy, loop prevention, goal completion rates, and inter-agent communication protocol correctness. Technologies: LangGraph, AutoGen, Python, DeepEval, Playwright, Pytest, Behavioral Testing.