LangSmith, Evaluation & Deployment

1. Observability & Tracing with LangSmith

When a complex agent makes 10 successive tool calls and produces an unexpected answer, debugging without tracing is nearly impossible. LangSmith provides instant, zero-code tracing of every prompt, token latency, dollar cost, and retriever document:

# Enable zero-code LangSmith tracing via environment variables:
import os

os.environ["LANGCHAIN_TRACING_V2"] = "true"
os.environ["LANGCHAIN_ENDPOINT"] = "https://api.smith.langchain.com"
os.environ["LANGCHAIN_API_KEY"] = "your-langsmith-api-key"
os.environ["LANGCHAIN_PROJECT"] = "production-rag-v1"

# Any subsequent chain or agent execution is automatically logged to your dashboard!

2. Automated Evaluation with RAGAS Metrics

Never deploy RAG pipeline changes (such as altering chunk sizes or switching embedding models) without quantifiable metrics. The RAGAS (Retrieval Augmented Generation Assessment) framework evaluates four core dimensions:

3. Production Deployment with FastAPI & Token Streaming

Deploy your LCEL chain as an asynchronous streaming HTTP endpoint using FastAPI and Server-Sent Events (SSE):

# Install: pip install fastapi uvicorn
from fastapi import FastAPI
from fastapi.responses import StreamingResponse
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser

app = FastAPI(title="LangChain Enterprise API")

chain = (
    ChatPromptTemplate.from_template("Provide expert medical analysis for: {symptoms}")
    | ChatOpenAI(model="gpt-4o", streaming=True)
    | StrOutputParser()
)

@app.get("/chat/stream")
async def chat_stream(symptoms: str):
    async def event_generator():
        async for token in chain.astream({"symptoms": symptoms}):
            yield f"data: {token}\n\n"

    return StreamingResponse(event_generator(), media_type="text/event-stream")

4. Enterprise Security & Prompt Injection Defense

Never execute raw user input directly inside SQL queries or system commands. Follow theOWASP Top 10 for LLM Applications:

• Prompt Injection Defense: Treat all user text and retrieved web data as untrusted data using delimiter fencing (e.g. <user_input>).
• Least Privilege Tool Execution: Restrict database tools to read-only replica connections. Never give an agent root terminal access.
• Output Secret Masking: Filter out API tokens, credit card numbers, and PII using regex scrubbers before emitting tokens to the client.