← All packages

@lacspace/keypool

v1.4.0LLM Efficiency0 deps

AI key pooling two ways. createAiChain: route-ordered multi-provider failover (Gemini, Groq, Cerebras, OpenAI, Anthropic, DeepSeek, OpenRouter, any OpenAI-compatible) that rests key+model pairs on 429 (Gemini daily quota until Pacific midnight), quarantines rejected keys (incl. Gemini's 400 'API key not valid'), cools down after N failures, shrinks Groq max_tokens to fit TPM, regrows empty reasoning replies, JSON mode with repair, per-purpose routes and caps, a call ledger hook, and a shared store (memory or Redis). createKeypool: RPM/RPD/TPM/TPD rotation and accounting. Built on @lacspace/ai.

npm i @lacspace/keypool

Usage

keypool.ts
import { createKeypool } from "@lacspace/keypool";

const pool = createKeypool({
  keys: [
    { id: "gem-1", provider: "gemini", secret: process.env.GEMINI_1! },
    { id: "gem-2", provider: "gemini", secret: process.env.GEMINI_2! },
  ],
  providerLimits: { gemini: { rpm: 15, rpd: 1500, tpm: 1_000_000 } },
});

const picked = await pool.pick("gemini", "gemini-2.0-flash", estTokens);
if (!picked) throw new Error("all keys cooling down");
const res = await callGemini(picked.secret, prompt);
await pool.report(picked.id, { ok: true, tokens: res.usage.total, model: "gemini-2.0-flash" });

Exports 2

createKeypoolkvStore

Keywords

api-keykey-rotationrate-limitkeypoolround-robin429retry-afterquotageminigroq

More in LLM Efficiency