Day 14: Deployment & LangSmith

ying the API, and hardening the agent logic.

1. Configuring Observability with LangSmith

LangSmith integrates with LangChain and LangGraph via environment variables. This zero-code approach ensures that every execution is automatically traced without modifying the agent logic.

Implementation Steps:

1. Create a LangSmith account and generate an API key.
2. Set the required environment variables in your .env file.
3. Verify traces appear in the LangSmith dashboard after the first run.

Environment Configuration:

# .env
LANGCHAIN_TRACING_V2=true
LANGCHAIN_ENDPOINT=https://api.smith.langchain.com
LANGCHAIN_API_KEY=ls_<your_api_key>
LANGCHAIN_PROJECT=my-agent-production

Rationale: Using environment variables decouples observability from code. This allows you to enable tracing in staging and production while keeping it disabled in local development to reduce overhead. The LANGCHAIN_PROJECT variable isolates traces, making it easier to filter data by environment or feature branch.

2. Deploying the API with LangServe

LangServe wraps your LangGraph graph in a FastAPI application, automatically generating standard endpoints for invocation, streaming, and batching. It also provides a built-in Playground UI for interactive testing.

Architecture Decision: We use LangServe rather than a custom FastAPI wrapper to leverage standardized endpoint contracts and the Playground. This reduces boilerplate and ensures compatibility with LangChain client libraries.

Code Example: The following example demonstrates a modular deployment structure. Note the explicit configuration of endpoints and the use of a factory function for the graph, which supports dependency injection and testing.

# server.py
from fastapi import FastAPI
from langserve import add_routes
from src.agents.research_assistant import build_research_graph
from src.config import settings

# Initialize FastAPI application with metadata
api_app = FastAPI(
    title="Research Assistant API",
    description="LangGraph-based agent for web research and summarization",
    version="1.0.0"
)

# Instantiate the graph with configuration
agent_graph = build_research_graph(
    model_name=settings.LLM_MODEL,
    max_iterations=settings.MAX_ITERATIONS
)

# Register routes with explicit endpoint control
# Disabling /batch reduces attack surface if not required
add_routes(
    api_app,
    agent_graph,
    path="/v1/agent",
    enabled_endpoints=["invoke", "stream"],
    config_keys=["configurable"],
    playground_type="default"
)

if __name__ == "__main__":
    import uvicorn
    uvicorn.run(
        "server:api_app",
        host="0.0.0.0",
        port=8080,
        reload=settings.DEBUG_MODE
    )

Key Implementation Details:

• enabled_endpoints: Explicitly listing endpoints prevents accidental exposure of unused functionality. If batch processing is not needed, omitting /batch reduces the service footprint.
• config_keys: Exposing configurable allows clients to pass runtime configuration (e.g., temperature, system prompt overrides) without redeploying.
• Factory Pattern: Using build_research_graph allows the graph to be constructed with environment-specific parameters, supporting different models or limits per deployment.

3. Production Hardening

Before deployment, the agent must be hardened against common failure modes. This involves guardrails, memory management, and tool safety.

• System Prompt Guardrails: Define a strict system prompt that outlines behavioral constraints. For example, explicitly forbid the agent from answering questions outside its domain or generating political content. This reduces hallucination and scope creep.
• Memory Capping: Unbounded memory leads to context window overflow and increased costs. Implement WindowMemory to retain only the last N interactions, or SummaryMemory to compress history. This ensures the agent remains responsive and cost-effective over long sessions.
• Tool Safety and Human-in-the-Loop: Destructive tools (e.g., database deletions, financial transactions) require validation. Implement a human-in-the-loop mechanism where the graph pauses execution and requests user confirmation before proceeding. This prevents accidental data loss or unauthorized actions.

Pitfall Guide

The following pitfalls are derived from production experience with LangGraph deployments. Avoiding these issues is critical for system stability.

Pitfall	Explanation	Fix
Unbounded Context Growth	Failing to cap memory causes the context window to fill, leading to truncation errors or exponential cost increases.	Use WindowMemory with a fixed size or SummaryMemory to compress history. Monitor token usage in LangSmith.
Missing System Prompt	Agents without explicit guardrails may drift off-topic or generate unsafe content.	Define a comprehensive system prompt with clear constraints. Test edge cases to verify compliance.
Unsafe Tool Execution	Tools that modify state can be triggered by malicious inputs or hallucinations, causing data corruption.	Implement human-in-the-loop checks for destructive tools. Validate tool inputs and outputs rigorously.
Tracing Overhead	Enabling tracing in high-throughput environments without sampling can add latency and storage costs.	Use LangSmith's sampling configuration to trace a percentage of requests in production.
LangServe Endpoint Misconfiguration	Exposing all endpoints by default can increase the attack surface and resource usage.	Explicitly configure enabled_endpoints to only include necessary functionality.
Ignoring Latency Metrics	Failing to monitor node-level latency can hide performance bottlenecks that degrade user experience.	Use LangSmith to identify slow nodes. Optimize tool calls or model selection for high-latency steps.
Deployment Without Evaluations	Deploying agents without testing against golden sets can result in regressions and inconsistent behavior.	Create a golden set of test cases. Run evaluations before every deployment to ensure quality.

Production Bundle

Action Checklist

Use this checklist to validate your agent before deployment.

• Configure LangSmith: Set LANGCHAIN_TRACING_V2=true and API key in .env. Verify traces in dashboard.
• Define System Prompt: Write a system prompt with explicit guardrails and behavioral constraints.
• Implement Memory Capping: Add WindowMemory or SummaryMemory to prevent context overflow.
• Secure Destructive Tools: Add human-in-the-loop checks for tools that modify data or perform financial actions.
• Deploy with LangServe: Wrap the graph in FastAPI using add_routes. Configure explicit endpoints.
• Run Evaluations: Test the agent against a golden set of inputs to verify accuracy and consistency.
• Monitor Latency and Cost: Review LangSmith traces to identify bottlenecks and token usage patterns.

Decision Matrix

Select the deployment strategy based on your operational requirements.

Scenario	Recommended Approach	Why	Cost Impact
Prototype / Internal Tool	Local Script + LangSmith Tracing	Fast iteration; minimal infrastructure overhead.	Low; no hosting costs.
Production API	LangServe + LangSmith	Standardized endpoints; Playground UI; scalable.	Medium; hosting + LangSmith usage.
High-Throughput Service	LangServe + Custom Load Balancer	Handles concurrent requests; allows horizontal scaling.	High; infrastructure + LangSmith sampling.
Security-Sensitive App	LangServe + Human-in-the-Loop	Prevents unauthorized tool execution; ensures oversight.	Medium; potential latency from user waits.

Configuration Template

Copy this template to initialize your project configuration.

# .env
LANGCHAIN_TRACING_V2=true
LANGCHAIN_ENDPOINT=https://api.smith.langchain.com
LANGCHAIN_API_KEY=ls_<your_api_key>
LANGCHAIN_PROJECT=production-agent

# Application Settings
LLM_MODEL=gpt-4o
MAX_ITERATIONS=10
DEBUG_MODE=false
PORT=8080

# src/config.py
from pydantic_settings import BaseSettings

class Settings(BaseSettings):
    LLM_MODEL: str = "gpt-4o"
    MAX_ITERATIONS: int = 10
    DEBUG_MODE: bool = False
    PORT: int = 8080

    class Config:
        env_file = ".env"

settings = Settings()

Quick Start Guide

Get your agent running in under five minutes.

1. Install Dependencies: Run pip install langchain langgraph langserve fastapi uvicorn.
2. Set Environment Variables: Create a .env file with LangSmith credentials and application settings.
3. Create Server: Write server.py using the LangServe example above.
4. Start Service: Run python server.py. Access the Playground at http://localhost:8080/v1/agent/playground/.
5. Verify Tracing: Execute a test query and check LangSmith for the trace output.