LLM function calling patterns

Strict Tool Schemas Use zod to enforce runtime validation. LLMs generate JSON, but JSON is not type-safe. Runtime validation catches hallucination before execution.

import { z } from 'zod';

export const WeatherToolSchema = z.object({
  location: z.string().min(2).max(100).describe('City or coordinates'),
  unit: z.enum(['celsius', 'fahrenheit']).default('celsius'),
  include_forecast: z.boolean().default(false)
});

export const DatabaseQueryToolSchema = z.object({
  table: z.string().regex(/^[a-z_]+$/),
  columns: z.array(z.string()).min(1),
  filters: z.record(z.string(), z.unknown()).optional(),
  limit: z.number().int().min(1).max(100).default(10)
});

export type ToolSchema = typeof WeatherToolSchema | typeof DatabaseQueryToolSchema;

Step 2: Build a Typed Tool Registry

Decouple tool metadata from execution logic. This enables dynamic discovery, versioning, and observability hooks.

export interface ToolDefinition<T extends z.ZodTypeAny> {
  name: string;
  description: string;
  schema: T;
  execute: (params: z.infer<T>) => Promise<unknown>;
}

export class ToolRegistry {
  private tools = new Map<string, ToolDefinition<any>>();

  register<T extends z.ZodTypeAny>(tool: ToolDefinition<T>) {
    this.tools.set(tool.name, tool);
  }

  get(name: string): ToolDefinition<any> | undefined {
    return this.tools.get(name);
  }

  toOpenAITools() {
    return Array.from(this.tools.values()).map(tool => ({
      type: 'function' as const,
      function: {
        name: tool.name,
        description: tool.description,
        parameters: tool.schema
      }
    }));
  }
}

Step 3: Orchestrate Invocation and Validation

Isolate model response parsing from execution. Validate against schema, route to executor, and format results for continuation.

import OpenAI from 'openai';

export async function executeToolCall(
  openai: OpenAI,
  registry: ToolRegistry,
  messages: OpenAI.ChatCompletionMessageParam[]
): Promise<OpenAI.ChatCompletionMessageParam[]> {
  const completion = await openai.chat.completions.create({
    model: 'gpt-4o',
    messages,
    tools: registry.toOpenAITools(),
    tool_choice: 'auto',
    temperature: 0.1
  });

  const response = completion.choices[0].message;
  if (!response.tool_calls?.length) return [response];

  const toolResults: OpenAI.ChatCompletionToolMessageParam[] = [];

  for (const call of response.tool_calls) {
    const tool = registry.get(call.function.name);
    if (!tool) {
      toolResults.push({
        role: 'tool',
        tool_call_id: call.id,
        content: JSON.stringify({ error: 'Tool not registered' })
      });
      continue;
    }

    try {
      const parsed = tool.schema.parse(JSON.parse(call.function.arguments));
      const result = await tool.execute(parsed);
      toolResults.push({
        role: 'tool',
        tool_call_id: call.id,
        content: JSON.stringify(result)
      });
    } catch (err) {
      toolResults.push({
        role: 'tool',
        tool_call_id: call.id,
        content: JSON.stringify({ error: 'Schema validation failed', details: (err as Error).message })
      });
    }
  }

  return [response, ...toolResults];
}

Architecture Rationale

• Schema Validation at Boundary: zod parsing occurs immediately after JSON deserialization. This prevents malformed data from reaching business logic.
• Registry Pattern: Centralizes tool metadata, enabling dynamic loading, A/B testing of tool descriptions, and consistent observability tagging.
• Idempotent Execution: Each tool call is isolated. Failures do not cascade. Results are formatted as tool messages for seamless continuation.
• Low Temperature: Function calling requires deterministic output. temperature: 0.1 minimizes creative drift while preserving reasoning capability.
• Separation of Concerns: Generation, validation, routing, and execution are distinct phases. This enables independent testing, mocking, and scaling.

Pitfall Guide

1. Overcomplicating Schemas with Deep Nesting

LLMs struggle with nested objects beyond two levels. Deep nesting increases token count and parsing failure rates. Best Practice: Flatten schemas. Use string enums, explicit arrays, and top-level primitives. Reserve nesting for truly hierarchical data, and validate depth limits in CI.

2. Ignoring Tool Definition Token Limits

Verbose descriptions and excessive parameters consume context window budget. Long-running agents silently degrade when tool definitions exceed 15% of available tokens. Best Practice: Cap tool descriptions at 80 characters. Use concise parameter names. Implement dynamic tool loading based on conversation state.

3. Synchronous Execution Blocking the Event Loop

Awaiting database queries or external APIs inside the tool executor without concurrency control causes latency spikes and timeout cascades. Best Practice: Use Promise.all for independent calls. Implement circuit breakers for external dependencies. Set explicit timeouts per tool.

4. Missing Fallback and Repair Loops

When validation fails, naive implementations return errors to the LLM without guidance, causing infinite retry loops. Best Practice: Return structured error messages with correction hints. Implement a max-retry threshold (typically 2). Fall back to a clarifying prompt if repair fails.

5. Tight Coupling Between Prompt and Tool Signatures

Embedding tool usage instructions directly in system prompts creates brittle integrations. Schema changes require prompt rewrites. Best Practice: Decouple instructions from definitions. Use tool descriptions to convey usage. Maintain a versioned prompt registry separate from tool schemas.

6. No Observability or Tracing

Without request IDs, tool call IDs, and execution metrics, debugging production failures requires log scavenging. Best Practice: Attach trace_id and tool_call_id to every execution. Log schema validation results, execution duration, and error types. Export to OpenTelemetry or equivalent.

7. Assuming Deterministic Outputs

LLMs do not guarantee field order, missing optional fields, or consistent enum casing. Treating responses as contract-compliant guarantees runtime crashes. Best Practice: Always parse with a validator. Default missing optional fields explicitly. Normalize casing and whitespace before execution.

Production Bundle

Action Checklist

• Define all tool schemas using zod with explicit types, defaults, and constraints
• Implement a centralized tool registry with dynamic loading and versioning
• Add runtime validation immediately after JSON deserialization
• Set tool_choice: 'auto' and temperature ≤ 0.2 for deterministic routing
• Attach tool_call_id and trace_id to all execution logs and metrics
• Implement max-retry logic with structured error hints for LLM repair
• Cap tool definition token usage below 15% of context window
• Test schemas against adversarial inputs and edge-case parameter combinations

Decision Matrix

Scenario	Recommended Approach	Why	Cost Impact
Low-latency chatbot with single action	Single-Turn Structured Calling	Minimal overhead, fast validation, predictable routing	Low: $0.021/request
Complex data extraction with conditional logic	Multi-Turn Orchestrated Calling	Stateful reasoning, repair loops, context preservation	Medium: $0.034/request
High-volume batch processing	Batched Tool Execution with Queue	Parallelization, retry isolation, throughput scaling	Low: Amortized $0.015/request
Cost-constrained MVP	Prompt-Guided Function Calling + Schema Guard	Fastest integration, validation layer catches drift	Medium: Higher retry cost early
Multi-agent coordination	Registry-Based Tool Sharing + Event Bus	Decoupled execution, cross-agent observability	High: Infrastructure overhead

Configuration Template

// tools/config.ts
import { z } from 'zod';
import { ToolRegistry } from './registry';
import { executeToolCall } from './orchestrator';
import OpenAI from 'openai';

export function initializeToolSystem() {
  const registry = new ToolRegistry();

  registry.register({
    name: 'get_weather',
    description: 'Fetch current weather and optional forecast',
    schema: z.object({
      location: z.string().min(2),
      unit: z.enum(['celsius', 'fahrenheit']).default('celsius'),
      include_forecast: z.boolean().default(false)
    }),
    execute: async (params) => {
      // Replace with actual API call
      return { temp: 22, unit: params.unit, forecast: params.include_forecast ? 'sunny' : null };
    }
  });

  registry.register({
    name: 'query_database',
    description: 'Execute read-only database query with filters',
    schema: z.object({
      table: z.string().regex(/^[a-z_]+$/),
      columns: z.array(z.string()).min(1),
      filters: z.record(z.string(), z.unknown()).optional(),
      limit: z.number().int().min(1).max(50).default(10)
    }),
    execute: async (params) => {
      // Replace with actual DB client
      return { rows: [], count: 0 };
    }
  });

  return { registry, executeToolCall };
}

// Usage in route/controller
const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });
const { registry, executeToolCall: runTools } = initializeToolSystem();

export async function handleChatRequest(userMessage: string) {
  const messages: OpenAI.ChatCompletionMessageParam[] = [
    { role: 'system', content: 'You are a helpful assistant. Use tools when required.' },
    { role: 'user', content: userMessage }
  ];

  const updatedMessages = await runTools(openai, registry, messages);
  return updatedMessages;
}

Quick Start Guide

1. Install dependencies: npm install openai zod
2. Copy the Configuration Template into tools/config.ts and replace placeholder executors with actual API/DB clients.
3. Initialize the registry and pass it to executeToolCall alongside your OpenAI client and conversation history.
4. Validate responses by logging tool_call_id and execution duration. Add Sentry/OpenTelemetry for production tracing.
5. Deploy with temperature: 0.1 and tool_choice: 'auto'. Monitor schema validation failure rates and adjust constraints accordingly.