Skip to main content

​
Overview

Routing rules let you control where your AI requests go based on their context — feature, task type, provider, or model. Set a rule once in the dashboard. Every matching request is routed automatically from that moment forward. 
IF feature = "classifier" AND task = "classification"
THEN route to openai / gpt-4o-mini
Your code changes nothing. Lectr handles it at the proxy layer.

​
How it works

When a request arrives at Lectr, the routing engine evaluates your rules in priority order. 
Request arrives
      ↓
Evaluate rules in priority order (in-memory, no DB)
      ↓
First match → rewrite model + provider → forward
No match    → forward unchanged (passthrough)
Rewriting means replacing the model and/or provider before the request is sent upstream. If no rule matches — or if rule evaluation fails — the request is forwarded unchanged. Passthrough is always the fallback. Routing cannot block or degrade a request.

​
Rule conditions

Each rule can match on any combination of four conditions. All specified conditions must match — AND logic. Unset conditions are wildcards.
ConditionMatches againstHeader / field
Feature tagX-Lectr-Feature header valueYour feature name
Task typeX-Lectr-Task header valueTask type string
ProviderDetected or declared providerAuto-detected or X-Lectr-Provider
Modelmodel field in request bodyExact model name
Examples: 
IF feature = "chat"                          → matches all chat requests
IF task = "classification"                   → matches all classification tasks
IF provider = "openai" AND model = "gpt-4"  → matches specific model
IF feature = "summariser" AND task = "summarisation"  → both must match

​
Creating a rule

Go to Rules in the dashboard sidebar. Click New rule and fill in:
  • Name — a label you’ll recognise in logs (e.g. “Downgrade classifiers to mini”)
  • Conditions — one or more of: feature tag, task type, provider, model
  • Target provider — where to send matching requests
  • Target model — which model to use
Save. The rule takes effect immediately — no restart, no deploy.

​
Priority

Rules are evaluated in ascending priority order. Priority 1 is evaluated first. The first matching enabled rule wins — evaluation stops. 
Priority 1: IF feature = "chat" → gpt-4o-mini           ← evaluated first
Priority 2: IF task = "reasoning" → o1                   ← only if priority 1 didn't match
Priority 3: IF provider = "openai" → anthropic / claude  ← only if 1 and 2 didn't match
Reorder rules by dragging them in the Rules list. The new order takes effect immediately. No two enabled rules can share the same priority. If you set a priority that’s already taken, the dashboard shows an error — it won’t silently reorder.

​
Toggling rules

Rules can be enabled or disabled without deleting them. A disabled rule is never evaluated — it’s invisible to the routing engine. Useful for:
  • Testing a rule without committing to it
  • Temporarily disabling routing during an incident
  • Keeping rules around for future use without activating them

​
Seeing rules in action

Every request log shows whether a routing rule was applied and which one. In the Requests table: 
REQUEST ID    MODEL REQUESTED   MODEL ACTUAL      RULE                         STATUS
ed3f80fa...   gpt-4o            gpt-4o-mini       Downgrade classifiers         success
10405cb9...   claude-3-opus     claude-3-5-haiku  Route opus to haiku           success
3e9db15f...   gpt-4o            gpt-4o            —                             success
When model_actual differs from model_requested, the substitution is clearly visible. Routing coverage: The dashboard overview shows what percentage of your traffic is being routed versus passing through: 
Routed requests:    1,204  (62%)
Unrouted requests:    738  (38%)
Unrouted traffic indicates requests with no matching rule — which may be intentional.

​
Common rule patterns

Downgrade all classification tasks to a cheaper model 
Condition: task = "classification"
Target:    openai / gpt-4o-mini
Route all Anthropic traffic to a specific model 
Condition: provider = "anthropic"
Target:    anthropic / claude-3-5-haiku-20241022
Migrate away from a deprecated model 
Condition: model = "gpt-4"
Target:    openai / gpt-4o
Route a specific feature to a different provider 
Condition: feature = "code-review"
Target:    anthropic / claude-3-5-sonnet-20241022
Combine feature and task for precision routing 
Condition: feature = "support-bot" AND task = "classification"
Target:    groq / llama-3.1-8b-instant

​
What routing changes

When a rule matches, Lectr:
  • Rewrites the model field to the target model
  • Routes the request to the target provider
  • Normalises authentication headers automatically
Routing does not change:
  • Your prompts or messages
  • Any other request fields
  • The response format — it remains identical to provider output

​
Routing and recommendations

The recommendation engine surfaces opportunities to reduce cost. Routing rules let you apply those optimizations instantly without changing your code. The workflow: 
Recommendation: "classifier" could use gpt-4o-mini (saves 87%)
        ↓
Create rule: IF feature = "classifier" → gpt-4o-mini
        ↓
All classifier requests now use the cheaper model

​
Safety guarantees

Routing is designed to be safe for production use.
  • No matching rule → request is forwarded unchanged
  • Rule evaluation error → request is forwarded unchanged
  • Provider error → passed through unchanged
Routing only affects where the request is sent — not how it behaves.

​
Reference

Condition matching:
  • Exact string matching (provider is case-insensitive)
  • All conditions must match (AND logic)
  • Unset conditions act as wildcards
Priority:
  • Lower number = higher priority
  • Priority 1 is evaluated first
  • No duplicate priorities allowed
Cache:
  • Rules are evaluated from an in-memory cache
  • No database on the hot path
  • Cache updates immediately when rules are created, edited, toggled, or deleted
Passthrough:
  • No match = unchanged request
  • Routing never blocks execution

​
FAQ

Immediately. When you save, toggle, or delete a rule, the routing cache is reloaded server-side within milliseconds. The next matching request uses the updated rules.
The provider returns an error, which is passed through unchanged.
No. Evaluation stops at the first match. Only the highest-priority matching rule is applied.
Yes. If your client is configured for OpenAI but a rule routes to Anthropic, Lectr handles the provider switch — including auth header normalisation — automatically. Your client doesn’t need to know.
Yes. Routing works identically for streaming and non-streaming requests.
No hard limit. Most teams use fewer than 20 rules.