Read dynamic database credentials

ws a strict least-privilege flow: workload authenticates → policy evaluates → secret engine generates scoped credentials → lease binds lifetime to workload lifecycle.

Architecture Decisions and Rationale

1. Storage Backend: Use Raft (integrated storage) for most deployments. Consul adds operational overhead without meaningful benefits for single-region clusters. Raft supports built-in high availability, snapshot restores, and automatic leader election.
2. Unseal Mechanism: Cloud KMS (AWS KMS, Azure Key Vault, GCP KMS) over Shamir's secret sharing. Manual unseal creates operational drift and breaks GitOps workflows. Auto-unseal ensures cluster recovery after restarts without human intervention.
3. Authentication: Kubernetes JWT auth for containerized workloads, AppRole for CI/CD runners, and OIDC for developer access. Avoid static tokens; bind auth to workload identity.
4. Secret Engines: KV v2 for static configuration (versioning, soft delete), Database engine for dynamic credentials, Transit for envelope encryption. Avoid KV v1 in new deployments; it lacks versioning and deletion safety.

Step-by-Step Implementation

1. Provision Vault Cluster: Deploy via Helm or Vault Operator. Configure Raft storage, auto-unseal, and TLS termination.
2. Enable Auth Method: Mount Kubernetes auth, configure token review and JWT verification.
3. Configure Secret Engine: Enable database secrets engine, configure connection URL, and define roles with SQL templates.
4. Write Policies: Create least-privilege policies mapping roles to secret paths.
5. Integrate Client: Use TypeScript SDK to authenticate, request dynamic credentials, and handle lease renewal.

TypeScript Client Implementation

The official HashiCorp SDK for Node.js is node-vault. It provides async/await interfaces, automatic token renewal, and lease management.

import vault from 'node-vault';

interface VaultConfig {
  endpoint: string;
  namespace?: string;
  token?: string;
  roleId: string;
  secretId: string;
}

export class VaultSecretsClient {
  private client: ReturnType<typeof vault>;
  private leaseId: string | null = null;
  private renewalInterval: NodeJS.Timeout | null = null;

  constructor(private config: VaultConfig) {
    this.client = vault({
      endpoint: config.endpoint,
      namespace: config.namespace,
    });
  }

  async authenticate(): Promise<void> {
    const { client } = this;
    const response = await client.authAppRole({
      role_id: this.config.roleId,
      secret_id: this.config.secretId,
    });
    
    this.client.token = response.auth.client_token;
    this.startTokenRenewal(response.auth.lease_duration);
  }

  async getDynamicDbCredentials(dbRole: string): Promise<{ username: string; password: string; leaseId: string }> {
    const { client } = this;
    const secret = await client.read(`database/creds/${dbRole}`);
    
    this.leaseId = secret.lease_id;
    this.startCredentialRenewal(secret.lease_duration);
    
    return {
      username: secret.data.username,
      password: secret.data.password,
      leaseId: this.leaseId,
    };
  }

  private startTokenRenewal(duration: number): void {
    if (this.renewalInterval) clearInterval(this.renewalInterval);
    
    // Renew at 60% of lease duration to avoid expiration race conditions
    const interval = Math.floor(duration * 0.6 * 1000);
    this.renewalInterval = setInterval(async () => {
      try {
        await this.client.renewToken();
      } catch (err) {
        console.error('Token renewal failed:', err);
        // Trigger re-authentication flow
      }
    }, interval);
  }

  private startCredentialRenewal(duration: number): void {
    // Database leases are typically short-lived (1h). 
    // For production, implement exponential backoff renewal or delegate to sidecar.
  }

  destroy(): void {
    if (this.renewalInterval) clearInterval(this.renewalInterval);
    if (this.leaseId) {
      this.client.revoke(this.leaseId).catch(() => {});
    }
  }
}

Architecture Rationale in Practice

• Lease Handling: Dynamic credentials expire automatically. The client must renew or rotate before TTL. The code schedules renewal at 60% of lease duration to prevent race conditions during network latency.
• Namespace Isolation: Enterprise Vault supports multi-tenancy via namespaces. The client passes namespace to route requests correctly without cross-tenant leakage.
• Token vs Secret Renewal: Vault tokens and secret leases are separate. Token renewal maintains auth; secret lease renewal extends DB credentials. In Kubernetes, use the Vault Agent Injector to handle renewal transparently, reducing client-side complexity.
• Error Handling: Network partitions or Vault maintenance windows require fallback strategies. Implement circuit breakers and cache last-known valid credentials with TTL expiry, never indefinite caching.

Pitfall Guide

1. Root Token Proliferation: Using the initial root token in CI/CD or application code grants unlimited access. Root tokens bypass audit logging and policy evaluation. Best practice: Generate short-lived, scoped tokens via AppRole or JWT. Revoke root immediately after bootstrap.

2. Ignoring Lease TTLs: Treating dynamic credentials as static causes authentication failures when Vault revokes them. Applications must implement renewal logic or use the Vault Agent sidecar. Best practice: Set TTLs aligned with workload lifecycle (e.g., 1h for microservices, 24h for batch jobs).

3. Overly Permissive Policies: Wildcard paths (secret/data/*) defeat least-privilege. Policies should map to specific engines and roles. Best practice: Use vault policy fmt to validate syntax, test with vault token lookup, and enforce path restrictions in CI.

4. Skipping Auto-Unseal: Manual unseal breaks GitOps, causes downtime during cluster restarts, and creates operational drift. Best practice: Configure cloud KMS or Transit auto-unseal. Store unseal keys in HSM or cloud KMS, never in plaintext or version control.

5. Using KV v1 in New Deployments: KV v1 lacks versioning, soft deletion, and metadata support. Best practice: Always use KV v2. Migrate existing v1 paths using vault secrets migrate -force kv v1_path.

6. Exposing Vault API Publicly: Vault should never be internet-facing. Best practice: Place behind internal load balancers, enforce mTLS, restrict access via network policies, and use Vault Agent for local proxying.

7. Not Configuring Audit Devices: Without audit logging, you cannot track secret access, detect abuse, or meet compliance requirements. Best practice: Enable file or syslog audit devices at bootstrap. Rotate audit logs to immutable storage. Verify log integrity with checksums.

Production Bundle

Action Checklist

• Initialize Vault with auto-unseal (cloud KMS) and Raft storage
• Revoke root token immediately after bootstrap and generate scoped admin tokens
• Enable Kubernetes JWT auth or AppRole; never use static tokens in workloads
• Configure KV v2 for static config, Database engine for dynamic credentials
• Write least-privilege policies; validate with vault policy fmt and dry-run tests
• Enable audit device (file/syslog) and forward to immutable log storage
• Implement client-side lease renewal or deploy Vault Agent Injector sidecar
• Run periodic secret rotation drills and lease expiry simulations

Decision Matrix

Scenario	Recommended Approach	Why	Cost Impact
Single-region microservices cluster	Vault + Kubernetes JWT + Raft	Native identity binding, low latency, HA via Raft	Low (shared cluster, minimal egress)
Multi-cloud CI/CD pipelines	Vault + AppRole + Transit	Machine identity, envelope encryption, cross-cloud consistency	Medium (AppRole rotation overhead, Transit API calls)
Compliance-heavy workloads (SOC2/HIPAA)	Vault Enterprise + Audit + Namespace	Built-in audit trails, tenant isolation, policy versioning	High (Enterprise licensing, audit storage)
Legacy monolith with static DB creds	Vault Agent + KV v2 + Sidecar	Zero code changes, automatic rotation, lease management	Low-Medium (Agent resource overhead, migration effort)

Configuration Template

Vault Server HCL (vault.hcl)

listener "tcp" {
  address       = "0.0.0.0:8200"
  tls_cert_file = "/vault/tls/tls.crt"
  tls_key_file  = "/vault/tls/tls.key"
  tls_min_version = "tls12"
}

storage "raft" {
  path    = "/vault/data"
  node_id = "vault-1"
}

seal "awskms" {
  region     = "us-east-1"
  kms_key_id = "alias/vault-unseal-key"
}

telemetry {
  prometheus_retention_time = "24h"
  disable_hostname = true
}

ui = true

Policy HCL (app-policy.hcl)

# Read dynamic database credentials
path "database/creds/postgres-role" {
  capabilities = ["read"]
}

# Renew database lease
path "sys/leases/renew" {
  capabilities = ["update"]
}

# Revoke lease on shutdown
path "sys/leases/revoke" {
  capabilities = ["update"]
}

Kubernetes Auth Configuration

vault auth enable kubernetes
vault write auth/kubernetes/config \
  kubernetes_host="https://kubernetes.default.svc" \
  kubernetes_ca_cert=@/var/run/secrets/kubernetes.io/serviceaccount/ca.crt \
  token_reviewer_jwt=@/var/run/secrets/kubernetes.io/serviceaccount/token \
  issuer="https://kubernetes.default.svc.cluster.local"

vault write auth/kubernetes/role/app-role \
  bound_service_account_names=app-sa \
  bound_service_account_namespaces=production \
  policies=app-policy \
  ttl=1h

Quick Start Guide

1. Deploy Vault: helm install vault hashicorp/vault --set server.dev.enabled=true (for testing) or use the official Vault Operator for production.
2. Initialize & Unseal: Run vault operator init to generate unseal keys and root token. Configure cloud KMS auto-unseal before production use.
3. Enable Secrets & Auth: vault secrets enable -version=2 kv, vault auth enable kubernetes, vault secrets enable database.
4. Configure Role & Policy: Create a database role with SQL template, write least-privilege policy, bind to service account.
5. Test Client: Run the TypeScript client with AppRole or JWT auth. Verify dynamic credential issuance, lease TTL, and automatic revocation on destroy().

Vault shifts secrets management from storage to lifecycle governance. Implement lease-based access, enforce least-privilege policies, and automate renewal. The operational overhead pays for itself in reduced breach surface, faster compliance audits, and eliminated manual rotation cycles.