OpenRouter Gives You 50 Requests a Day. So I Built a 28-Model Fallback Ladder.
Building an AI agent that deploys Cloudflare Workers with LangChain.js, and the Redis circuit breaker that keeps it running after the free tier dies.
System-level engineer building reliable backend systems with a focus on performance, correctness, and real-world constraints. I work across APIs, databases, networking, and infrastructure, enjoy understanding how systems behave under load and failure, and write to break down complex backend and distributed-systems concepts through practical, real-world learnings.
This year, I joined the trend of shipping an AI feature with EdgeBalancer, which now converts a single line of plain English into a real Cloudflare Worker load balancer. You simply type your request, and it builds it for you.
Inside, I utilized LangChain.js to define the tools and manage the tool-calling loop. The model has access to seven tools: list zones, list balancers, create, update, delete, pause, and resume. It sequentially calls one tool, reads the result, and decides the next step.
Agentic AI consists of API calls and tool calls, making the model layer integral to the product's reliability.
Initially, I started with OpenRouter's free models, including NVIDIA Nemotron, Gemma, and GPT-OSS. While they were sufficient, I quickly hit the limit of 50 requests per day for the entire account. Since one agent run costs 3 to 6 model calls, this meant only about 10 test runs daily, which hindered development.
To address this, I built a ladder of 28 models in an ordered list, starting with all 13 OpenRouter free models followed by 15 Mistral models. The free quota is consumed first, while the paid quota remains in reserve. Mistral offers significantly higher limits, and I often did not encounter rate limits, which is crucial for reliable tool calling.
When OpenRouter hits its daily quota, I write a key to Redis with a 24-hour expiry. Each run checks this key before starting; if it's set, the free tier is skipped, and the run proceeds directly to Mistral. Mistral's rate limits are treated differently, with a 60-second cooldown, and if a Retry-After header is provided, that number is used.
Redis serves three functions: managing quota cooldowns across servers, pacing per model rates, and implementing a deploy lock to prevent simultaneous creations of the same balancer name.
The run streams progress over Server-Sent Events, showing which model answered, which tool is running, and what it returned, rather than just displaying a spinner.
For safety, the agent can create but not delete, pause, or update. It prepares these actions, displays the intended operations, and requires confirmation before executing them. Creation is additive and can roll back cleanly on failure, while deletion cannot.
visit: https://edge.nexoral.in