The Optimisation Flywheel
Building Self-Healing Agents with DSPy and RLAIF
The End of Static Prompts
The shift we are witnessing now is architectural. A prompt should not be static code. It should be a living system. We no longer simply write prompts; we build Optimisation Flywheels. These are self-improving structures where every interaction generates signal, every failure mode triggers adaptation, and the system compounds its own intelligence over time. The end state is an agent that removes the human from the training loop entirely, not through neglect, but through earned autonomy.
This article maps the three-phase journey from human-supervised deployment to fully autonomous self-healing agents, with particular attention to the technical frameworks that make this transition practical today.
Phase One: The Cold Start
Every self-healing system begins with dependency. In the Cold Start phase, we deploy with Human-in-the-Loop supervision as the primary quality control mechanism. This is not a weakness; it is strategic data collection disguised as quality assurance.
The architecture is straightforward. Each agent output surfaces to a human reviewer who can approve, reject, or correct. Rejection alone is insufficient signal. The critical interaction is the correction: the user provides a thumbs down alongside specific feedback. “You missed the second clause in the contract.” “The tone is too formal for this customer segment.” “You included deprecated product features.” These corrections are gold.
What distinguishes a flywheel-ready deployment from a standard HITL setup is instrumentation. Every correction must be captured as a structured tuple: the original input, the agent’s output, the human’s corrected output, and the feedback rationale. Most teams capture the first three. The fourth element, the reason why the correction was necessary, is what enables the transition to autonomous improvement.
The Cold Start phase typically runs for 30 to 90 days depending on query volume. The objective is not perfection; it is coverage. You need corrections across the full distribution of your input space. A system that only sees corrections on edge cases will only learn to handle edge cases. Deliberate sampling across routine queries ensures the flywheel has balanced training signal when it begins to turn.
Phase Two: The Golden Dataset
First, correction pairs are normalised and deduplicated. Similar corrections are clustered, and representative examples are selected. A contract review agent that received twelve corrections about missed liability clauses does not need twelve examples; it needs one exemplary correction with high coverage.
Second, corrections are indexed by failure mode. This categorical tagging enables selective retrieval. When a new query arrives, the system identifies which failure modes are most likely given the input characteristics, then retrieves few-shot examples specifically designed to prevent those failures. This is targeted inoculation, not blanket example dumping.
Third, the library implements decay. Examples that no longer correlate with improved performance are gradually downweighted. An early correction about formatting may become irrelevant after a model update changes default output structure. Without decay, the few-shot library accumulates noise.
Fourth, and most critically, the library closes the loop. New corrections continue to flow in from ongoing HITL operations, but now they augment an already-effective baseline. The system is no longer starting from zero; it is refining from competence. This is the flywheel beginning to generate momentum.
Phase Three: The Critic Agent
The transition from RLHF (Reinforcement Learning from Human Feedback) to RLAIF (Reinforcement Learning from AI Feedback) is where the flywheel achieves escape velocity. In Phase Three, we train a secondary model, the Critic Agent, whose sole purpose is to replicate human judgment at scale.
The Critic is not a general-purpose evaluator. It is trained specifically on your correction history, learning the precise criteria your human reviewers apply. If your reviewers consistently penalise outputs that exceed 200 words, the Critic learns that length threshold. If your reviewers reward outputs that acknowledge uncertainty explicitly, the Critic learns to score for epistemic humility. The Critic becomes a distilled representation of your organisation’s quality standards.
Training the Critic requires the structured tuples from Phase One. The input is the agent output; the label is the human score (derived from approve/reject/correct actions); the features include the correction rationale when available. A well-instrumented Cold Start phase yields a high-quality Critic. A poorly instrumented phase yields a Critic that captures surface patterns without understanding underlying criteria.
Once deployed, the Critic evaluates every agent output before it reaches the user. Outputs that score above threshold proceed automatically. Outputs below threshold route to human review. This creates a graduated autonomy model: the system handles routine queries independently while escalating uncertain cases. Over time, the escalation rate becomes your key performance metric. A declining escalation rate signals a flywheel that is genuinely improving.
The Critic also enables synthetic data generation. By pairing the primary agent with the Critic in a generate-and-evaluate loop, you can produce thousands of labelled examples without human involvement. The primary agent generates candidate outputs; the Critic scores them; high-scoring outputs augment the few-shot library. The flywheel turns faster.
The Technical Foundation: DSPy
The frameworks that enable this architecture are maturing rapidly. DSPy (Declarative Self-Improving Language Programs) from Stanford represents the current state of the art for automated prompt optimisation. Unlike manual prompt engineering, DSPy treats prompts as learnable parameters within a larger programme structure.
The core abstraction is the Signature: a typed specification of inputs and outputs for a language model call. Rather than writing a prompt, you declare what the model should receive and what it should return. DSPy then searches the space of possible prompts, few-shot examples, and instruction phrasings to find configurations that maximise your objective function on a validation set.
This inverts the traditional workflow. Instead of a human iterating on prompt wording, the human defines success criteria and provides labelled examples. DSPy handles the optimisation. Critically, DSPy supports automatic few-shot selection from a candidate pool, enabling direct integration with the Golden Dataset architecture described above. As your correction library grows, DSPy can automatically select the most effective examples for each query type.
The optimisation process itself uses techniques familiar from machine learning: bootstrap sampling to generate synthetic demonstrations, metric-driven search to identify high-performing configurations, and modular composition to chain multiple optimised modules into complex workflows. For teams building production agents, DSPy reduces the cost of achieving and maintaining high performance by an order of magnitude compared to manual iteration.
The Self-Healing Result
The end state is an agent that rewrites its own instructions based on failure modes. When a new edge case appears, the system does not wait for a human to diagnose and patch. The Critic identifies the failure, the correction is logged, the Golden Dataset updates, and DSPy re-optimises the prompt configuration. The next query of that type encounters an improved system.
This is not theoretical. Production deployments using this architecture demonstrate measurable improvement curves: week-over-week gains in accuracy metrics without manual intervention, declining human escalation rates, and expanding coverage of previously unhandled query types. The flywheel compounds.
The strategic implication is significant. Organisations that build optimisation flywheels accumulate a durable competitive advantage. Their agents improve with usage while competitors’ agents remain static. Every customer interaction generates signal. Every signal improves the system. Every improvement increases customer satisfaction, which drives more usage. The flywheel is not merely technical; it is commercial.
Ready to Activate Your Digital Advantage?
Interested in finding out more? Chat to Our Intelligent Assistant Now to Discover What We Can Do for You.
Discover more AI Insights and Blogs
By 2027, your biggest buyer might be an AI. How to restructure your Ecommerce APIs and product data so "Buyer Agents" can negotiate and purchase from your store automatically
Dashboards only show you what happened. We build Agentic Supply Chains that autonomously reorder stock based on predictive local trends, weather patterns, and social sentiment
Stop building static pages. Learn how we configure WordPress as a "Headless" receiver for AI agents that dynamically rewrite content and restructure layouts for every unique visitor
One agent writes, one edits, one SEO-optimizes, and one publishes. How we build autonomous content teams inside WordPress that scale your marketing without scaling your headcount
One model doesn't fit all. We break down our strategy for routing tasks between heavy reasoners (like GPT-4) and fast, local SLMs to cut business IT costs by 60%
Don't rewrite your old code. How we use Multi-Modal agents to "watch" and operate your legacy desktop apps, creating modern automations without touching the source code
You wouldn't give an intern root access to your database. Why are you giving it to ChatGPT? Our framework for "Role-Based Access Control" in Agentic Systems