← All packages

@lacspace/llm-cache

v1.0.0LLM Efficiency0 deps

A content-hash cache for LLM calls — key by (model family, prompt version, normalized input) so identical requests, retries after a 429 and repeated rewrites never pay twice. Pluggable async store (in-memory LRU built in, Mongo/Redis/KV via an adapter), TTL, stale-if-error and a wrap() memoizer. Zero-dependency, isomorphic.

npm i @lacspace/llm-cache

Usage

llm-cache.ts
import { createLlmCache, memoryStore } from "@lacspace/llm-cache";

const cache = createLlmCache({ store: memoryStore(), ttlMs: 86_400_000, promptVersion: "v3" });

const pack = await cache.wrap(
  { input: condensedSources, model: "gemini", variant: { lang: "ne" } },
  () => callGemini(condensedSources),   // only runs on a miss
  { staleIfError: true },               // 429? serve the last good result
);
cache.stats(); // { hits, misses, sets }

Exports 3

createLlmCachememoryStorecontentHash

Keywords

llm-cachecachememoizecontent-hashprompt-cachettllrurate-limit429openai

More in LLM Efficiency

@lacspace/condense

Extractive multi-source condenser — turn several articles on one story into a short, deduplicated, token-budgeted digest that keeps the numbers, quotes and named entities, so an LLM only rewrites a fraction of the text. BM25 sentence ranking, near-duplicate removal, per-source attribution with char offsets, Devanagari-aware (।/॥, ०-९, रु). Deterministic, isomorphic.

@lacspace/screen

A cheap lexical content screener and LLM admission gate — score text against your own weighted, multi-language term lexicons (with negation, proximity windows and a named-entity gazetteer) and get a clear / review / block decision, so obviously-clean and obviously-flagged text never reaches an expensive model. Deterministic, auditable, isomorphic.

@lacspace/keyphrase

Extractive keyphrases, tags, hashtags, named entities and category votes — a zero-dependency RAKE + TF-IDF engine with built-in English and Nepali stopwords, Devanagari-aware, so you can stop asking an LLM to generate tags/hashtags/entities. Deterministic, isomorphic.

@lacspace/keypool

AI key pooling two ways. createAiChain: route-ordered multi-provider failover (Gemini, Groq, Cerebras, OpenAI, Anthropic, DeepSeek, OpenRouter, any OpenAI-compatible) that rests key+model pairs on 429 (Gemini daily quota until Pacific midnight), quarantines rejected keys (incl. Gemini's 400 'API key not valid'), cools down after N failures, shrinks Groq max_tokens to fit TPM, regrows empty reasoning replies, JSON mode with repair, per-purpose routes and caps, a call ledger hook, and a shared store (memory or Redis). createKeypool: RPM/RPD/TPM/TPD rotation and accounting. Built on @lacspace/ai.