cost-latency-optimizer

Steps

1. Measure tokens and latency per call.
2. Enable prompt caching for stable prefixes.
3. Route simple requests to a smaller model.
4. Batch non-urgent work.
5. Stream responses to cut perceived latency.

Done When

• [ ] Quality unchanged on the golden set

Registry

Links

• Up: Gen AI Engineer skills
• Siblings: agent-tool-design, guardrails-review
• Frameworks: react, few-shot, risen, xml-structured, chain-of-thought
• Tests: testing guide