ax-playbook
This skill helps an LLM generate correct playbook code using @ax-llm/ax. Use when the user asks about playbook(), AxPlaybook, context playbooks, evolving context, ACE / Agentic Context Engineering, agent.playbook(), or growing/applying task knowledge offline and online with evolve() and update().
How do I install this agent skill?
npx skills add https://ax-llm.github.io --skill ax-playbookIs this agent skill safe to install?
No partner audit is available yet. Read the source before installing.
What does this agent skill do?
Playbook Codegen Rules (@ax-llm/ax)
Use this skill to generate context-playbook code. A playbook grows an evolving body of task knowledge and renders it into a program's context. The evolution engine (ACE — Agentic Context Engineering) is hidden behind playbook(...), exactly as optimize(...) hides its optimizer. Prefer the playbook(...) concept; only reach for AxACE directly when the user explicitly wants the low-level engine.
Use These Defaults
- Create with
playbook(program, { studentAI, teacherAI? }); it returns anAxPlaybookhandle. - Grow offline with
await pb.evolve(examples, metric)— returns{ bestScore, playbook }. - Grow online with
await pb.update({ example, prediction, feedback })— no metric needed. - Apply with
pb.applyTo(program)(defaults to the bound program). - Persist with
pb.toJSON()and restore withplaybook(program, opts).load(snapshot). - Inspect with
pb.render()(markdown) andpb.getState()({ playbook, artifact }). - For agents use
agent.playbook({ target: 'actor' | 'responder' }); default target is'actor'. - Use a cheaper
studentAIto run the program and an optional strongerteacherAIto reflect/curate. - Pass AI service options for the reflect/curate calls as
teacherOptions. A teacher markedisExpensive(for example OpenAIgpt-5.5-pro) is rejected unless you passteacherOptions: { useExpensiveModel: 'yes' }. The generated Python, Java, C++, Go, and Rust packages apply the same gate and acceptteacherOptionsin their option maps. - Prefer
ai(),ax(), andagent()for new code.
Critical Rules
playbook(...)binds to anAxGenprogram;evolve/updateneed that program's signature.evolve()returns only{ bestScore, playbook }. There is no Pareto front and nooptimizedProgram— that isoptimize(...)'s shape, not a playbook's.update({ example, prediction, feedback })requires the full{ example, prediction };examplemust match the program's input fields (plus any expected output). Do not pass bare input fields at the top level.update()works without a priorevolve()/load()— the handle hydrates lazily on first use.applyTo()injects a## Context Playbookblock into the program description; calling it repeatedly recomposes from the original base (no stacking).- Keep the offline
metricdeterministic and cheap, like a GEPA metric. - A playbook is plain JSON. Persist
pb.toJSON()andload(...)it into a fresh program for production. - The playbook engine, construction-time agent attachment, failure harvesting, and verified agent evolution are available in TypeScript and the generated Python, Java, C++, Go, and Rust packages. Use each package's native casing and callback types.
Offline Pattern (evolve)
import { type AxMetricFn, ai, ax, playbook } from '@ax-llm/ax';
const program = ax('review:string -> sentiment:class "positive, negative"');
const studentAI = ai({ name: 'openai', apiKey: process.env.OPENAI_APIKEY! });
const metric: AxMetricFn = ({ prediction, example }) =>
(prediction as any).sentiment === (example as any).sentiment ? 1 : 0;
const pb = playbook(program, { studentAI, maxEpochs: 2 });
const { bestScore } = await pb.evolve(train, metric);
pb.applyTo(program);
Online Pattern (update)
// After a real run, feed the outcome back so the playbook keeps learning.
await pb.update({
example: { review: 'Five stars, would buy again.' },
prediction: { sentiment: 'negative' },
feedback: 'WRONG: enthusiastic praise is positive.',
});
pb.applyTo(program);
Persist And Restore
const snapshot = pb.toJSON(); // { playbook, artifact } — plain JSON
// later, in another process / a production program instance:
playbook(prodProgram, { studentAI }).load(snapshot).applyTo(prodProgram);
Agents
a.playbook({ target }) returns an agent-aware AxAgentPlaybook (the stage AxPlaybook handle plus an agent-level evolve). The one playbook the agent renders into its prompt grows three ways:
- Continuous (trust): the construction-time
playbookoption (seeax-agent) harvests each run's failures automatically — no dataset. - On-demand (trust):
apb.update({ example, prediction, feedback }). - Batch verified (proof):
apb.evolve(dataset, options)runs the full agent over a task set, mines failure clusters, and proposes one playbook bullet per weakness; withverify(default on) it keeps a bullet only if held-in improves AND thevalidationheld-out set does not regress, else exact rollback.verify: false= trust-batch. Bullets-only.
const a = agent('ticket:string -> reply:string', { ai });
const apb = a.playbook({ target: 'actor' }); // agent-aware handle; 'actor' (default) or 'responder'
await apb.update({ example, prediction, feedback }); // online: injected into the live stage prompt
const result = await apb.evolve(
{ train, validation }, // AxAgentEvalDataset
{ metric, runsPerTask: 2 }, // verify:true by default
);
Each teacher call uses the teacherOptions set next to its model:
a.playbook({ teacherAI, teacherOptions })and the construction-timeplaybook: { teacherAI, teacherOptions }config cover the reflector and curator calls, including the onesapb.evolve(...)makes to apply a proposal.apb.evolve(dataset, { teacherAI, teacherOptions })covers the weakness miner.- Both teachers default to the agent's
judgeAI, then the student. With anisExpensivejudgeAI, setteacherOptions: { useExpensiveModel: 'yes' }in both places, plusjudgeOptionswhen the built-in judge scores the runs.
The agent-level evolve(dataset, options) is distinct from the program-level pb.evolve(examples, metric) above: it takes an AxAgentEvalDataset plus options, runs the whole pipeline, and returns baseline/final held-in & held-out with per-bullet outcomes (no { bestScore }). For full-pipeline tuning of agent instructions and demos (not the playbook) use agent.optimize(...) (GEPA).
Generated packages expose that same agent-bound loop with language-shaped APIs:
| Language | Agent-bound evolve call |
|---|---|
| Python | agent.playbook().evolve(dataset, options) |
| Java | agent.playbook(null).evolve(dataset, options) |
| C++ | agent.get_playbook()->evolve(dataset, options) |
| Go | agent.GetPlaybook().EvolveAgent(ctx, dataset, options) |
| Rust | playbook.evolve_agent(&mut agent, client, dataset, options) |
All five generated packages thread structured failureSignals through agent
evaluation predictions. The default verify gate accepts a proposed bullet only
when held-in score improves and held-out score stays within epsilon; rejection
restores the exact prior snapshot. Scoring is host-shaped: TypeScript uses its
metric, Python/Java/Go can accept a metric callback, and all generated ports can
use task score/scores values plus the agent evaluation result.
Playbook vs optimize()
playbook(...)— accumulate reusable, evolving task knowledge; the only path that also learns online viaupdate(...).optimize(...)/agent.optimize(...)— tune instruction text and few-shot demos offline to a best/Pareto result.- They are complementary; a project can use both.
Troubleshooting
- "Cannot convert undefined or null to object" from
update()→ you passed input fields at the top level; wrap them inexample: { ... }. - Empty playbook after
evolve()→ the model already scored well, so nothing was curated; use harder/ambiguous examples or a weakerstudentAIto surface lessons. - Playbook not affecting an agent's behavior → ensure
applyis notfalseand you usedagent.playbook(...)(not a bareplaybook()on an internal program). - Playbook never grows with an
isExpensiveteacher → the reflector and curator calls are rejected before any request is sent, and onlyverbose: truelogs it. SetteacherOptions: { useExpensiveModel: 'yes' }. Inapb.evolve(...), a rejected miner call is reported as aminer failedmessage throughonProgress(and the console withverbose: true).
See Also
ax-gepa-optimize(...)andAxGEPAfor instruction/demo tuning.ax-agent-context- choosing between contextMap, contextPolicy,agent.playbook(...), and recall.ax-agent-optimize-agent.optimize(...)GEPA tuning for agents.
How can the creator link this skill?
Add the canonical catalog link to the repository README so users can inspect current installs and available audits. The publishing guide covers the complete discovery path.
<a href="https://skillzs.dev/skills/ax-llm.github.io/ax-playbook">View ax-playbook on skillZs</a>