Self Improving Agent

Self Improving Agent is a community skill for building AI agents that learn from their interactions and improve over time, covering feedback collection, performance tracking, prompt refinement, memory management, and evaluation loops for adaptive agent behavior.

What Is This?

Overview

Self Improving Agent provides patterns for creating AI agents that adapt and improve through operational experience. It covers feedback collection from user interactions and task outcomes, performance metric tracking across agent sessions, prompt template refinement based on measured quality trends, memory systems that accumulate knowledge from past interactions, and evaluation loops that test improvements before deploying them. The skill enables developers to build agents that become more effective over time rather than remaining static.

Who Should Use This

This skill serves developers building long-running agents that should improve with usage, teams creating customer-facing AI that adapts to feedback, and engineers designing agent architectures with built-in learning mechanisms.

Why Use It?

Problems It Solves

Static agents repeat the same mistakes across sessions without learning from corrections. User feedback is collected but never systematically applied to improve agent behavior. Prompt improvements are deployed without measuring whether they actually help. Agent memory grows unbounded without curation, eventually degrading context quality.

Core Highlights

Feedback collection captures user signals such as ratings, corrections, and task completion status. Performance tracking measures success rates and quality scores across agent sessions over time. Prompt refinement applies learnings from feedback to update prompt templates systematically. Memory curation keeps useful context while pruning outdated or irrelevant entries.

How to Use It?

Basic Usage

from dataclasses import dataclass, field
from datetime import datetime

@dataclass
class FeedbackEntry:
    session_id: str
    rating: float
    correction: str = ""
    timestamp: str = ""

    def __post_init__(self):
        if not self.timestamp:
            self.timestamp = datetime.now().isoformat()

class FeedbackStore:
    def __init__(self):
        self.entries: list[FeedbackEntry] = []

    def add(self, entry: FeedbackEntry):
        self.entries.append(entry)

    def average_rating(self) -> float:
        if not self.entries:
            return 0.0
        return sum(e.rating for e in self.entries
                   ) / len(self.entries)

    def recent(self, n: int = 10) -> list[FeedbackEntry]:
        return self.entries[-n:]

    def corrections(self) -> list[str]:
        return [e.correction for e in self.entries
                if e.correction]

Real-World Examples

from dataclasses import dataclass, field

@dataclass
class AgentMemory:
    facts: list[dict] = field(default_factory=list)
    max_entries: int = 200

    def add_fact(self, key: str, value: str, source: str):
        self.facts.append({"key": key, "value": value,
                           "source": source, "uses": 0})
        if len(self.facts) > self.max_entries:
            self._prune()

    def _prune(self):
        self.facts.sort(key=lambda f: f["uses"])
        self.facts = self.facts[
            len(self.facts) - self.max_entries:]

class SelfImprovingAgent:
    def __init__(self, base_prompt: str):
        self.base_prompt = base_prompt
        self.feedback = FeedbackStore()
        self.memory = AgentMemory()
        self.version = 1

    def refine_prompt(self) -> str:
        corrections = self.feedback.corrections()
        if not corrections:
            return self.base_prompt
        rules = "\n".join(
            f"- {c}" for c in corrections[-5:])
        self.version += 1
        return (f"{self.base_prompt}\n\n"
                f"Learned rules:\n{rules}")

    def should_improve(self) -> bool:
        recent = self.feedback.recent(10)
        if len(recent) < 5:
            return False
        avg = sum(e.rating for e in recent) / len(recent)
        return avg < 0.7

    def run_improvement_cycle(self) -> dict:
        if not self.should_improve():
            return {"action": "none",
                    "version": self.version}
        new_prompt = self.refine_prompt()
        self.base_prompt = new_prompt
        return {"action": "refined",
                "version": self.version}

Advanced Tips

Run A/B tests between the current and refined prompt versions before fully deploying improvements. Weight recent feedback more heavily than older entries when computing quality trends. Implement memory importance scoring that increases priority for facts referenced frequently across sessions.

When to Use It?

Use Cases

Build a support agent that learns from correction feedback to avoid repeating mistakes in future interactions. Create a coding assistant that accumulates project-specific knowledge and conventions across sessions. Deploy a research agent that refines its search strategies based on which results users find most useful.

Related Topics

Agent memory systems, reinforcement learning from feedback, prompt optimization, adaptive AI systems, and evaluation-driven development.

Important Notes

Requirements

A feedback collection mechanism that captures user signals after interactions. Persistent storage for agent memory and feedback history across sessions. An evaluation framework for testing prompt improvements before deployment.

Usage Recommendations

Do: validate prompt refinements against a test suite before deploying them to production. Set memory size limits and implement pruning to prevent unbounded growth. Track improvement metrics over time to verify the agent is actually getting better.

Don't: apply every piece of user feedback without filtering for quality and consistency. Allow memory to grow without bounds, which degrades context quality. Deploy refined prompts without A/B testing against the current version.

Limitations

Feedback quality varies, and noisy signals can lead to incorrect refinements. Memory-based improvements are limited to patterns the agent has encountered before. Self-improvement cycles require enough interaction volume to produce statistically meaningful feedback.