Skip to main content
← Back to Documentation

Semantic Caching

Dramatically reduce costs by catching repetitive prompts.

Syndicate Gate offers an advanced Semantic Caching layer powered by Redis. Instead of just exact-string matching, the gateway can compute the embedding of the incoming prompt (e.g., using text-embedding-3-small) and perform a Cosine Similarity search against previous queries.

How It Works

  • Exact Match: Standard Redis GET/SET caching. Zero latency overhead, perfect for identical system prompts.
  • Semantic Match: Uses Redis Vector Search. If a user asks "How do I reset my password?" and another asks "What is the password reset process?", the gateway identifies the high similarity and serves the cached response without hitting the underlying LLM provider.

Configuration

Caching can be enabled per API key, per workspace, or per request (via HTTP headers). You can define the similarity threshold (e.g., 0.95) and the TTL (Time-To-Live) for cached responses.