Building Production RAG Systems: Citation, Hallucination Prevention & Evaluation

Detailed Description

Retrieval-Augmented Generation (RAG) systems have emerged as a transformative approach in natural language processing, combining the power of large language models with external knowledge retrieval to produce accurate, grounded, and contextually relevant responses. While basic RAG implementations can retrieve and generate text, production-grade systems require sophisticated mechanisms for citation, hallucination prevention, and comprehensive evaluation. This lesson explores the critical components needed to build reliable, trustworthy RAG systems that can be deployed in real-world applications where accuracy and accountability are paramount.

The importance of production-ready RAG systems cannot be overstated in today's AI landscape. Organizations deploying RAG solutions for customer service, medical information systems, legal research, financial analysis, or enterprise knowledge management need guarantees that responses are not only relevant but also accurate, traceable, and verifiable. A RAG system that generates plausible but incorrect information—commonly known as hallucinations—can lead to serious consequences, from damaged customer trust to legal liability. Production systems must implement robust citation mechanisms that allow users to verify claims, confidence scoring to communicate uncertainty, and hallucination detection to catch errors before they reach end users.

Several key concepts underpin production RAG systems. Source attribution involves tracking which retrieved documents contributed to specific parts of the generated response, enabling users to verify claims and understand the provenance of information. Faithfulness measures whether generated content accurately represents the retrieved context without introducing unsupported claims. Confidence scoring quantifies the system's certainty in its responses, helping users assess reliability. Context precision evaluates whether retrieved documents are genuinely relevant to the query, while answer relevance assesses whether the generated response actually addresses the user's question. Together, these concepts form a comprehensive framework for building trustworthy RAG applications.

Common use cases for production RAG systems span diverse industries. In healthcare, RAG systems help medical professionals access current research and treatment guidelines while maintaining strict accuracy standards. Legal firms use RAG to search case law and regulations, where precise citations are essential. Enterprise knowledge management systems leverage RAG to help employees find information across vast document repositories. Customer support chatbots use RAG to provide accurate product information with traceable sources. Financial institutions deploy RAG for market research and regulatory compliance, where hallucinations could have serious financial or legal consequences. Each of these scenarios demands production-grade quality assurance mechanisms.

By the end of this lesson, students will master several critical skills for production RAG development. You'll learn to implement sophisticated citation systems that trace generated text back to source documents with precise references. You'll build hallucination detection pipelines that compare generated responses against retrieved context to identify unsupported claims. You'll design comprehensive evaluation frameworks using multiple metrics to assess different dimensions of RAG quality. You'll implement confidence scoring mechanisms that quantify uncertainty in responses. Finally, you'll develop robust error handling strategies for edge cases like missing information, conflicting sources, or outdated data, ensuring your RAG system degrades gracefully under challenging conditions.

This lesson emphasizes practical, implementable solutions using modern NLP libraries and frameworks. You'll work with real code examples that demonstrate industry best practices, learn to avoid common pitfalls that plague RAG systems, and develop the skills needed to debug and improve RAG performance systematically. Whether you're building a customer-facing chatbot, an internal knowledge system, or a specialized research tool, the techniques covered in this lesson will enable you to create RAG applications that users can trust and organizations can confidently deploy.

Practical Examples

Example 1: Implementing Source Attribution with Citation Tracking

Source attribution is fundamental to production RAG systems. This example demonstrates how to track which documents and specific passages contribute to generated responses, enabling precise citation.


from dataclasses import dataclass
from typing import List, Dict, Tuple
import re
from sentence_transformers import SentenceTransformer
import numpy as np

@dataclass
class Citation:
    """Represents a citation linking generated text to source documents."""
    source_id: str
    document_title: str
    passage_text: str
    start_char: int
    end_char: int
    confidence: float

@dataclass
class Document:
    """Represents a retrieved document."""
    id: str
    title: str
    content: str
    metadata: Dict

class CitationTrackingRAG:
    """RAG system with granular citation tracking."""
    
    def __init__(self, model_name='all-MiniLM-L6-v2'):
        self.encoder = SentenceTransformer(model_name)
        
    def split_into_sentences(self, text: str) -> List[str]:
        """Split text into sentences for granular tracking."""
        # Simple sentence splitting (use spaCy for production)
        sentences = re.split(r'(?<=[.!?])\s+', text)
        return [s.strip() for s in sentences if s.strip()]
    
    def calculate_attribution_scores(self, 
                                     generated_sentence: str,
                                     source_passages: List[Tuple[str, Document]]) -> List[Tuple[float, Document, str]]:
        """Calculate similarity scores between generated sentence and source passages."""
        gen_embedding = self.encoder.encode(generated_sentence)
        
        scores = []
        for passage, doc in source_passages:
            passage_embedding = self.encoder.encode(passage)
            # Cosine similarity
            similarity = np.dot(gen_embedding, passage_embedding) / (
                np.linalg.norm(gen_embedding) * np.linalg.norm(passage_embedding)
            )
            scores.append((similarity, doc, passage))
        
        return sorted(scores, key=lambda x: x[0], reverse=True)
    
    def generate_with_citations(self,
                               generated_response: str,
                               retrieved_docs: List[Document],
                               attribution_threshold: float = 0.5) -> Tuple[str, List[Citation]]:
        """
        Generate response with citations by tracking source attribution.
        
        Args:
            generated_response: The generated text to annotate
            retrieved_docs: Documents retrieved for context
            attribution_threshold: Minimum similarity for attribution
            
        Returns:
            Annotated response and list of citations
        """
        sentences = self.split_into_sentences(generated_response)
        citations = []
        annotated_response = ""
        citation_counter = 1
        current_position = 0
        
        # Extract all passages from retrieved documents
        source_passages = []
        for doc in retrieved_docs:
            doc_sentences = self.split_into_sentences(doc.content)
            for sentence in doc_sentences:
                source_passages.append((sentence, doc))
        
        # Process each generated sentence
        for sentence in sentences:
            # Find best matching source passage
            attribution_scores = self.calculate_attribution_scores(
                sentence, source_passages
            )
            
            if attribution_scores and attribution_scores[0][0] >= attribution_threshold:
                # Attribution found
                score, source_doc, source_passage = attribution_scores[0]
                
                # Add citation marker
                citation_marker = f"[{citation_counter}]"
                annotated_response += sentence + citation_marker + " "
                
                # Record citation
                citations.append(Citation(
                    source_id=source_doc.id,
                    document_title=source_doc.title,
                    passage_text=source_passage,
                    start_char=current_position,
                    end_char=current_position + len(sentence),
                    confidence=score
                ))
                
                citation_counter += 1
            else:
                # No strong attribution found (potential hallucination)
                annotated_response += sentence + "[?] "
            
            current_position += len(sentence) + 1
        
        return annotated_response.strip(), citations
    
    def format_citations(self, citations: List[Citation]) -> str:
        """Format citations as references section."""
        if not citations:
            return "No citations available."
        
        references = "\n\nReferences:\n"
        for i, citation in enumerate(citations, 1):
            references += f"[{i}] {citation.document_title} (ID: {citation.source_id})\n"
            references += f"    \"{citation.passage_text[:100]}...\"\n"
            references += f"    Confidence: {citation.confidence:.2f}\n"
        
        return references

# Example usage
if __name__ == "__main__":
    # Sample documents
    docs = [
        Document(
            id="doc1",
            title="Python Best Practices",
            content="Type hints improve code readability. They help catch bugs early. Modern Python developers should use them consistently.",
            metadata={"author": "John Doe", "year": 2023}
        ),
        Document(
            id="doc2",
            title="Software Testing Guide",
            content="Unit tests verify individual components. Integration tests check system interactions. Both are essential for quality.",
            metadata={"author": "Jane Smith", "year": 2023}
        )
    ]
    
    # Simulated generated response
    generated = "Type hints are important for Python code quality. They help identify bugs before runtime. Testing is also crucial for software development."
    
    # Create RAG system and generate citations
    rag = CitationTrackingRAG()
    annotated_response, citations = rag.generate_with_citations(generated, docs)
    
    print("Annotated Response:")
    print(annotated_response)
    print(rag.format_citations(citations))

Explanation: This implementation tracks the provenance of each sentence in the generated response. It uses semantic similarity to match generated sentences with source passages, assigning citations when the similarity exceeds a threshold. The [?] marker indicates sentences that couldn't be strongly attributed to any source, flagging potential hallucinations. This approach provides transparency and enables users to verify claims.

Best Practices: Use higher-quality sentence encoders (like sentence-transformers or OpenAI embeddings) in production. Implement more sophisticated sentence boundary detection using spaCy or NLTK. Adjust the attribution threshold based on your use case—stricter for medical/legal applications, more lenient for general knowledge.

Example 2: Hallucination Detection Through Context Comparison

This example implements a multi-faceted hallucination detection system that compares generated responses against retrieved context to identify unsupported claims.


from typing import List, Dict, Tuple
from dataclasses import dataclass
import numpy as np
from sentence_transformers import SentenceTransformer, util
import spacy

@dataclass
class HallucinationReport:
    """Report detailing potential hallucinations in generated text."""
    has_hallucination: bool
    hallucination_score: float
    unsupported_claims: List[str]
    contradiction_score: float
    details: Dict

class HallucinationDetector:
    """Detect hallucinations by comparing generated text with retrieved context."""
    
    def __init__(self, model_name='all-MiniLM-L6-v2'):
        self.encoder = SentenceTransformer(model_name)
        try:
            self.nlp = spacy.load('en_core_web_sm')
        except:
            # Fallback if spaCy model not installed
            print("Warning: spaCy model not found. Using simplified processing.")
            self.nlp = None
    
    def extract_claims(self, text: str) -> List[str]:
        """Extract individual claims from generated text."""
        if self.nlp:
            doc = self.nlp(text)
            # Extract sentences as claims
            claims = [sent.text.strip() for sent in doc.sents]
        else:
            # Simple fallback
            claims = [s.strip() for s in text.split('.') if s.strip()]
        
        return claims
    
    def check_factual_support(self, 
                             claim: str, 
                             context_documents: List[str],
                             threshold: float = 0.6) -> Tuple[bool, float]:
        """
        Check if a claim is supported by context documents.
        
        Args:
            claim: The claim to verify
            context_documents: Retrieved documents providing context
            threshold: Minimum similarity for considering claim supported
            
        Returns:
            (is_supported, max_similarity_score)
        """
        claim_embedding = self.encoder.encode(claim, convert_to_tensor=True)
        
        max_similarity = 0.0
        for doc in context_documents:
            # Check against each document
            doc_embedding = self.encoder.encode(doc, convert_to_tensor=True)
            similarity = util.pytorch_cos_sim(claim_embedding, doc_embedding).item()
            max_similarity = max(max_similarity, similarity)
            
            if max_similarity >= threshold:
                return True, max_similarity
        
        return False, max_similarity
    
    def detect_contradictions(self,
                             claim: str,
                             context_documents: List[str]) -> Tuple[bool, float]:
        """
        Detect if claim contradicts the context.
        Uses negation detection and semantic opposition.
        """
        # Simple contradiction patterns
        negation_patterns = [
            ("is", "is not"),
            ("are", "are not"),
            ("can", "cannot"),
            ("will", "will not"),
            ("does", "does not"),
            ("has", "has not")
        ]
        
        claim_lower = claim.lower()
        contradiction_score = 0.0
        
        for doc in context_documents:
            doc_lower = doc.lower()
            
            # Check for direct negation patterns
            for positive, negative in negation_patterns:
                if positive in claim_lower and negative in doc_lower:
                    contradiction_score = max(contradiction_score, 0.8)
                elif negative in claim_lower and positive in doc_lower:
                    contradiction_score = max(contradiction_score, 0.8)
            
            # Check for semantic opposition
            claim_embedding = self.encoder.encode(claim, convert_to_tensor=True)
            doc_embedding = self.encoder.encode(doc, convert_to_tensor=True)
            
            similarity = util.pytorch_cos_sim(claim_embedding, doc_embedding).item()
            
            # High similarity with negation words suggests contradiction
            has_negation = any(word in claim_lower for word in ['not', 'no', 'never', 'none'])
            if has_negation and similarity > 0.5:
                contradiction_score = max(contradiction_score, 0.6)
        
        return contradiction_score > 0.5, contradiction_score
    
    def check_numeric_accuracy(self, 
                               claim: str, 
                               context_documents: List[str]) -> Dict:
        """Check if numbers in claim match those in context."""
        import re
        
        # Extract numbers from claim
        claim_numbers = set(re.findall(r'\b\d+(?:\.\d+)?\b', claim))
        
        # Extract numbers from context
        context_numbers = set()
        for doc in context_documents:
            context_numbers.update(re.findall(r'\b\d+(?:\.\d+)?\b', doc))
        
        # Check if all claim numbers appear in context
        unsupported_numbers = claim_numbers - context_numbers
        
        return {
            'claim_numbers': list(claim_numbers),
            'context_numbers': list(context_numbers),
            'unsupported_numbers': list(unsupported_numbers),
            'has_unsupported_numbers': len(unsupported_numbers) > 0
        }
    
    def detect_hallucinations(self,
                             generated_text: str,
                             retrieved_context: List[str],
                             support_threshold: float = 0.6) -> HallucinationReport:
        """
        Comprehensive hallucination detection.
        
        Args:
            generated_text: The generated response to check
            retrieved_context: Documents retrieved for context
            support_threshold: Minimum similarity for factual support
            
        Returns:
            HallucinationReport with detailed findings
        """
        claims = self.extract_claims(generated_text)
        unsupported_claims = []
        total_support_score = 0.0
        total_contradiction_score = 0.0
        numeric_issues = []
        
        for claim in claims:
            # Check factual support
            is_supported, support_score = self.check_factual_support(
                claim, retrieved_context, support_threshold
            )
            total_support_score += support_score
            
            if not is_supported:
                unsupported_claims.append(claim)
            
            # Check for contradictions
            is_contradiction, contradiction_score = self.detect_contradictions(
                claim, retrieved_context
            )
            total_contradiction_score += contradiction_score
            
            # Check numeric accuracy
            numeric_check = self.check_numeric_accuracy(claim, retrieved_context)
            if numeric_check['has_unsupported_numbers']:
                numeric_issues.append({
                    'claim': claim,
                    'unsupported_numbers': numeric_check['unsupported_numbers']
                })
        
        # Calculate overall hallucination score
        avg_support = total_support_score / len(claims) if claims else 0
        avg_contradiction = total_contradiction_score / len(claims) if claims else 0
        
        # Higher score = more likely hallucination
        hallucination_score = (
            0.5 * (1 - avg_support) +  # Lack of support
            0.3 * avg_contradiction +   # Contradictions
            0.2 * (len(numeric_issues) / max(len(claims), 1))  # Numeric issues
        )
        
        has_hallucination = (
            hallucination_score > 0.4 or 
            len(unsupported_claims) > len(claims) * 0.3
        )
        
        return HallucinationReport(
            has_hallucination=has_hallucination,
            hallucination_score=hallucination_score,
            unsupported_claims=unsupported_claims,
            contradiction_score=avg_contradiction,
            details={
                'total_claims': len(claims),
                'unsupported_count': len(unsupported_claims),
                'average_support_score': avg_support,
                'numeric_issues': numeric_issues
            }
        )

# Example usage
if __name__ == "__main__":
    detector = HallucinationDetector()
    
    # Example 1: Well-supported response
    context_docs = [
        "Python was created by Guido van Rossum in 1991. It emphasizes code readability.",
        "Python supports multiple programming paradigms including object-oriented and functional."
    ]
    
    good_response = "Python was created by Guido van Rossum. It focuses on code readability."
    report1 = detector.detect_hallucinations(good_response, context_docs)
    
    print("Example 1 - Well-supported response:")
    print(f"Has hallucination: {report1.has_hallucination}")
    print(f"Hallucination score: {report1.hallucination_score:.3f}")
    print(f"Unsupported claims: {report1.unsupported_claims}\n")
    
    # Example 2: Response with hallucination
    bad_response = "Python was created by James Gosling in 1995. It runs only on Windows."
    report2 = detector.detect_hallucinations(bad_response, context_docs)
    
    print("Example 2 - Response with hallucinations:")
    print(f"Has hallucination: {report2.has_hallucination}")
    print(f"Hallucination score: {report2.hallucination_score:.3f}")
    print(f"Unsupported claims: {report2.unsupported_claims}")
    print(f"Details: {report2.details}")

Explanation: This hallucination detector uses multiple strategies: semantic similarity to check if claims are supported by context, contradiction detection to identify conflicting information, and numeric verification to ensure numbers in the response appear in source documents. The weighted hallucination score combines these signals to provide an overall confidence assessment.

Common Mistakes to Avoid: Don't rely solely on semantic similarity—claims can be semantically similar but factually incorrect. Always check for contradictions explicitly. Be careful with threshold tuning; what works for one domain may not work for another. Implement domain-specific checks for critical applications (e.g., date validation for historical facts).

Example 3: Comprehensive RAG Evaluation Framework

This example implements a production-ready evaluation framework measuring multiple dimensions of RAG quality.


from dataclasses import dataclass
from typing import List, Dict, Optional
import numpy as np
from sentence_transformers import SentenceTransformer, util

@dataclass
class RAGEvaluationMetrics:
    """Container for RAG evaluation metrics."""
    answer_relevance: float
    faithfulness: float
    context_precision: float
    context_recall: float
    overall_score: float
    detailed_breakdown: Dict

class RAGEvaluator:
    """Comprehensive evaluation framework for RAG systems."""
    
    def __init__(self, model_name='all-MiniLM-L6-v2'):
        self.encoder = SentenceTransformer(model_name)
    
    def evaluate_answer_relevance(self,
                                  query: str,
                                  generated_answer: str) -> float:
        """
        Evaluate if the answer actually addresses the query.
        
        Args:
            query: User's question
            generated_answer: RAG system's response
            
        Returns:
            Relevance score (0-1)
        """
        query_embedding = self.encoder.encode(query, convert_to_tensor=True)
        answer_embedding = self.encoder.encode(generated_answer, convert_to_tensor=True)
        
        relevance = util.pytorch_cos_sim(query_embedding, answer_embedding).item()
        
        # Additional checks for common relevance issues
        penalty = 0.0
        
        # Penalize very short answers for complex questions
        if len(query.split()) > 10 and len(generated_answer.split()) < 5:
            penalty += 0.2
        
        # Penalize generic responses
        generic_phrases = ['i don\'t know', 'not sure', 'cannot answer', 'no information']
        if any(phrase in generated_answer.lower() for phrase in generic_phrases):
            penalty += 0.3
        
        return max(0.0, relevance - penalty)
    
    def evaluate_faithfulness(self,
                             generated_answer: str,
                             retrieved_contexts: List[str]) -> Dict:
        """
        Evaluate if the answer is faithful to retrieved contexts.
        Measures if claims in the answer are supported by contexts.
        
        Args:
            generated_answer: Generated response
            retrieved_contexts: Retrieved documents
            
        Returns:
            Dictionary with faithfulness metrics
        """
        # Split answer into claims (sentences)
        claims = [s.strip() for s in generated_answer.split('.') if s.strip()]
        
        supported_claims = 0
        claim_scores = []
        
        # Concatenate all contexts for embedding
        full_context = ' '.join(retrieved_contexts)
        context_embedding = self.encoder.encode(full_context, convert_to_tensor=True)
        
        for claim in claims:
            claim_embedding = self.encoder.encode(claim, convert_to_tensor=True)
            similarity = util.pytorch_cos_sim(claim_embedding, context_embedding).item()
            claim_scores.append(similarity)
            
            # Consider claim supported if similarity > threshold
            if similarity > 0.5:
                supported_claims += 1
        
        faithfulness_score = supported_claims / len(claims) if claims else 0.0
        
        return {
            'faithfulness_score': faithfulness_score,
            'total_claims': len(claims),
            'supported_claims': supported_claims,
            'unsupported_claims': len(claims) - supported_claims,
            'claim_scores': claim_scores,
            'average_claim_similarity': np.mean(claim_scores) if claim_scores else 0.0
        }
    
    def evaluate_context_precision(self,
                                   query: str,
                                   retrieved_contexts: List[str],
                                   ground_truth_answer: Optional[str] = None) -> Dict:
        """
        Evaluate precision of retrieved contexts.
        Measures what proportion of retrieved contexts are relevant.
        
        Args:
            query: User's question
            retrieved_contexts: Retrieved documents
            ground_truth_answer: Optional ground truth for better evaluation
            
        Returns:
            Dictionary with precision metrics
        """
        query_embedding = self.encoder.encode(query, convert_to_tensor=True)
        
        relevant_count = 0
        relevance_scores = []
        
        for context in retrieved_contexts:
            context_embedding = self.encoder.encode(context, convert_to_tensor=True)
            relevance = util.pytorch_cos_sim(query_embedding, context_embedding).item()
            relevance_scores.append(relevance)
            
            # Consider context relevant if similarity > threshold
            if relevance > 0.3:
                relevant_count += 1
        
        precision = relevant_count / len(retrieved_contexts) if retrieved_contexts else 0.0
        
        # If ground truth available, check if contexts contain answer information
        ground_truth_precision = None
        if ground_truth_answer:
            gt_embedding = self.encoder.encode(ground_truth_answer, convert_to_tensor=True)
            gt_relevance_scores = []
            
            for context in retrieved_contexts:
                context_embedding = self.encoder.encode(context, convert_to_tensor=True)
                relevance = util.pytorch_cos_sim(gt_embedding, context_embedding).item()
                gt_relevance_scores.append(relevance)
            
            ground_truth_precision = np.mean(gt_relevance_scores) if gt_relevance_scores else 0.0
        
        return {
            'context_precision': precision,
            'relevant_contexts': relevant_count,
            'total_contexts': len(retrieved_contexts),
            'relevance_scores': relevance_scores,
            'average_relevance': np.mean(relevance_scores) if relevance_scores else 0.0,
            'ground_truth_precision': ground_truth_precision
        }
    
    def evaluate_context_recall(self,
                               required_information: List[str],
                               retrieved_contexts: List[str]) -> Dict:
        """
        Evaluate if retrieved contexts contain all required information.
        
        Args:
            required_information: List of required facts/information
            retrieved_contexts: Retrieved documents
            
        Returns:
            Dictionary with recall metrics
        """
        full_context = ' '.join(retrieved_contexts)
        context_embedding = self.encoder.encode(full_context, convert_to_tensor=True)
        
        found_count = 0
        coverage_scores = []
        
        for required_fact in required_information:
            fact_embedding = self.encoder.encode(required_fact, convert_to_tensor=True)
            similarity = util.pytorch_cos_sim(fact_embedding, context_embedding).item()
            coverage_scores.append(similarity)
            
            # Consider information found if similarity > threshold
            if similarity > 0.5:
                found_count += 1
        
        recall = found_count / len(required_information) if required_information else 0.0
        
        return {
            'context_recall': recall,
            'found_information': found_count,
            'required_information': len(required_information),
            'missing_information': len(required_information) - found_count,
            'coverage_scores': coverage_scores,
            'average_coverage': np.mean(coverage_scores) if coverage_scores else 0.0
        }
    
    def evaluate_comprehensive(self,
                             query: str,
                             generated_answer: str,
                             retrieved_contexts: List[str],
                             required_information: Optional[List[str]] = None,
                             ground_truth_answer: Optional[str] = None) -> RAGEvaluationMetrics:
        """
        Perform comprehensive evaluation of RAG system output.
        
        Args:
            query: User's question
            generated_answer: RAG system's response
            retrieved_contexts: Retrieved documents
            required_information: Optional list of required facts
            ground_truth_answer: Optional ground truth for evaluation
            
        Returns:
            RAGEvaluationMetrics with all metrics
        """
        # Evaluate answer relevance
        relevance = self.evaluate_answer_relevance(query, generated_answer)
        
        # Evaluate faithfulness
        faithfulness_results = self.evaluate_faithfulness(
            generated_answer, retrieved_contexts
        )
        
        # Evaluate context precision
        precision_results = self.evaluate_context_precision(
            query, retrieved_contexts, ground_truth_answer
        )
        
        # Evaluate context recall (if required information provided)
        recall_results = None
        if required_information:
            recall_results = self.evaluate_context_recall(
                required_information, retrieved_contexts
            )
        
        # Calculate overall score (weighted average)
        weights = {
            'relevance': 0.3,
            'faithfulness': 0.4,
            'precision': 0.3
        }
        
        overall_score = (
            weights['relevance'] * relevance +
            weights['faithfulness'] * faithfulness_results['faithfulness_score'] +
            weights['precision'] * precision_results['context_precision']
        )
        
        # Compile detailed breakdown
        detailed_breakdown = {
            'relevance_details': {
                'score': relevance,
                'query_length': len(query.split()),
                'answer_length': len(generated_answer.split())
            },
            'faithfulness_details': faithfulness_results,
            'precision_details': precision_results,
            'recall_details': recall_results
        }
        
        return RAGEvaluationMetrics(
            answer_relevance=relevance,
            faithfulness=faithfulness_results['faithfulness_score'],
            context_precision=precision_results['context_precision'],
            context_recall=recall_results['context_recall'] if recall_results else 0.0,
            overall_score=overall_score,
            detailed_breakdown=detailed_breakdown
        )

# Example usage
if __name__ == "__main__":
    evaluator = RAGEvaluator()
    
    # Test case
    query = "What are the main features of Python programming language?"
    
    generated_answer = "Python is known for its simple syntax and readability. It supports multiple programming paradigms including object-oriented and functional programming."
    
    retrieved_contexts = [
        "Python emphasizes code readability with its clean syntax. It supports object-oriented, functional, and procedural programming.",
        "Python has a large standard library and extensive third-party packages available through PyPI.",
        "Guido van Rossum created Python in 1991 as a successor to ABC language."
    ]
    
    required_info = [
        "Python has simple syntax",
        "Python supports object-oriented programming",
        "Python has a large standard library"
    ]
    
    # Perform evaluation
    metrics = evaluator.evaluate_comprehensive(
        query=query,
        generated_answer=generated_answer,
        retrieved_contexts=retrieved_contexts,
        required_information=required_info
    )
    
    print("RAG Evaluation Results:")
    print(f"Answer Relevance: {metrics.answer_relevance:.3f}")
    print(f"Faithfulness: {metrics.faithfulness:.3f}")
    print(f"Context Precision: {metrics.context_precision:.3f}")
    print(f"Context Recall: {metrics.context_recall:.3f}")
    print(f"Overall Score: {metrics.overall_score:.3f}")
    print(f"\nDetailed Breakdown:")
    print(f"Supported Claims: {metrics.detailed_breakdown['faithfulness_details']['supported_claims']}/{metrics.detailed_breakdown['faithfulness_details']['total_claims']}")
    print(f"Relevant Contexts: {metrics.detailed_breakdown['precision_details']['relevant_contexts']}/{metrics.detailed_breakdown['precision_details']['total_contexts']}")

Explanation: This comprehensive

Natural Language Processing & Conversational AI

Module 1: NLP Fundamentals & Text Processing Foundations

Module 2: Core NLP Techniques for Conversational Understanding

Module 3: Building Rule-Based & Retrieval Chatbots

Module 4: Large Language Models & Advanced AI Integration

Module 5: Retrieval-Augmented Generation & Knowledge Systems

Module 6: Voice AI, Production Deployment & Enterprise Integration