Part II · Patterns And Practice/Chapter 08
08
Frontier Leverage Patterns
The fun edge: multi sampling, debate, critique, generator verifier loops, search, synthetic data, and spending reasoning where it matters.
The gist
- "More reasoning" is expensive. Spend it where it changes the answer.
- The winning combo: cheap models route, strong models reason, code verifies, humans judge.
- Patterns to know: best of N, generator + verifier, planner + executor, router cascades.
Module 10: Frontier Leverage Patterns
This is the fun part. These are the patterns that feel like "AI turned up to 11."
Learn the patterns that feel like AI turned up to eleven:
- multi-sampling and best-of-N
- self-consistency
- debate and critique
- generator-verifier loops
- planner-executor loops
- tree search over actions
- code execution as verifier
- retrieval plus reranking
- synthetic data generation
- prompt and program optimization with DSPy
- model routing and cascades
- memory from prior runs
- sandbox agents that learn from artifacts
- human calibrated rubric loops
Build by picking one hard task and running it five ways:
- one answer
- five answers with a judge
- planner plus executor
- executor plus critic
- executor plus tests
Compare the difference.
More reasoning is expensive. Spend it where it changes the answer. The best system often follows a division of labor:
- cheap models for routing and checks
- strong models for hard reasoning
- code for verification
- humans for judgment