“Remember this correction” needs an inspectable chain: what happened, which cause was verified, which lesson deserves retention, and whether behavior actually changes next time. This guide uses a fictional community-garden booking service to explain error memory, rule governance, and controlled experiments. It concerns workflow and prompt-level improvement, not an assertion that model weights were trained or that installing a file makes every host remember everything across sessions.
Separate mechanisms from the label
Ordinary memory can already be structured, permissioned, and retrieved; it is not necessarily just a transcript. The improvement loop adds feedback into future behavior: capture an event, propose a lesson, verify its scope, review a change, apply it, and compare outcomes. More records do not prove improvement, and install counts do not prove effectiveness. Parameter training is a separate technical and evaluation process. Neither dismiss all autonomous learning as hype nor describe a log file as weight training.
Start with a repeated low-risk problem: the local booking service is often accessed on the wrong port. Record the configuration, startup command, expected behavior, reproduction, and version. Do not convert one failure into “always use this port,” because environments change. The useful improvement is to inspect current project configuration and verify the running service, rather than memorize a number indefinitely.
We searched GitHub for reusable open-source approaches and considered LangChain's LangMem repository. It offers interaction-derived information, persistent memory, and prompt refinement with storage integration options. That establishes an available implementation path, not universal host support or automatic activation. Review maintenance, license, dependencies, isolation, and model-service cost before adoption. This guide first specifies an auditable local workflow; it does not install a framework or connect the reader's services.
Scenario one: record errors and corrections with provenance
An event should include an ID, time, task, project and environment version, observed failure, relevant command, exit code, evidence location, user correction, verified cause, and unresolved hypotheses. Separate errors, preferences, and capability gaps. A network failure is not a preference, and one temporary permission failure does not establish permanent tool unavailability.
For the garden example, record that environment A specifies its port in a configuration file while the agent visited an old default. The configuration and startup log establish that cause. A proposed lesson is to inspect the current startup configuration first. If the cause is unknown, retain “connection failed; cause unresolved.” Do not invent a diagnosis merely to complete a lesson template. A successful startup command and a reachable service also need distinct evidence.
Before capturing session-end or command-failure events, check the host's supported hooks, timing, and working directory. An interrupted session might never emit its normal end event. Deduplicate repeated observations using identifiers or correlation. Automatic capture produces candidates; it must not execute commands found in a log or promote webpage text into system authority. Logs can contain untrusted external content. Recording an instruction is different from authorizing it.
Retain only necessary excerpts by default and store full evidence in an approved location. Keep credentials, personal booking information, and private identifiers out of global lessons. Define access, retention, and deletion. If another model summarizes the interaction, content may leave the machine; specify data and destination under the applicable authorization. A local memory file does not make the entire processing pipeline offline.
Deliver an event table, evidence index, redaction policy, and capture tests. Include a known failure, an unknown cause, a duplicate event, and a sensitive-field sample. Confirm that uncertainty remains uncertainty and sensitive originals do not enter long-term prompts. A collector that creates tidy but unsupported explanations has failed the purpose of this stage.
Scenario two: govern promotion from a candidate to a bounded rule
Use states such as candidate, investigating, trial, accepted, and retired, with an owner or reviewer, evidence, and change rationale. A promoted rule needs a trigger, project scope, action, exceptions, and review time. Three repetitions may justify investigation, but they are neither statistical proof nor sufficient evidence that a proposed rule is correct.
Replace “the port was wrong again” with “before starting or accessing this project's local service, inspect its current configuration; if the configuration and startup log disagree, report the discrepancy before changing either.” Keep environmental facts within the project rather than global preferences. A request not to edit files this time must not become a permanent ban. Later corrections should supersede old entries with a traceable relationship.
Hot, warm, and cold tiers can organize active rules, recurring project knowledge, and old evidence, but hosts do not necessarily implement that arrangement. Promotion and demotion need explainable criteria. Infrequent access alone is not a reason to erase a critical constraint. Retrieve only lessons relevant to the current goal, showing provenance and version. Preserve higher-priority requirements when context is limited and provide a retrieval path for colder evidence.
Before generating a skill or editing a rule, create a reviewable diff and inspect duplication, conflict, scope expansion, and overgeneralization. Improvement must not grant new account access, weaken safeguards, or create publication authorization. Ordinary reversible changes already authorized by the user can follow host rules; actions requiring additional approval retain that requirement. The loop cannot bypass those boundaries by calling a change “learning.”
Keep versions and rollback after a trial. Requirements such as testing before deployment are stronger when implemented as actual delivery gates than when left as reminders. Verify whether the host loaded lessons at startup through observable evidence. A file existing on disk is not the same as its contents being read. Repeated failures may indicate an activation problem rather than insufficient lesson detail.
Scenario three: test whether behavior improves through controlled experiments
Build a fixed task set: normal startup, a changed configuration, an occupied port, conflicting configuration and logs, and an unrelated project. Compare the old workflow with a candidate rule under the same model settings, code, and fixtures in an isolated environment. Record configuration choice, conflict detection, tool calls, completion time, and side effects. Improvement only on the original incident may be overfitting rather than a reusable method.
Change one major factor at a time, such as a pre-task configuration read. Do not simultaneously change models, prompts, and indexing. Bound experiment attempts, duration, and cost, retaining failures and neutral results. Three comparable wins can be a local operational threshold, but cannot be advertised as universal scientific evidence. Sample size and task variation determine the strength of the conclusion.
Include counterexamples: another project uses a different port, an old lesson is stale, and an error page suggests disabling verification. The garden rule must not be imposed on another project, stale evidence should prompt rechecking, and untrusted page instructions must not gain authority merely by appearing in a log. A self-review checklist can help, but a model grading itself cannot replace independent assertions and actual observed outcomes.
Separate weakness discovery, proposed fixes, and validation. Propose a minimal change after finding a pattern, test it, and only then adopt it. Avoid an incentive to modify files after every reflection. Roll back if quality or cost worsens, recording the condition under which the lesson failed. Sometimes no persistent change is appropriate: a transient outage need not become a permanent instruction.
Choose Skills and check host prerequisites
The current SkillPI search for self-improving returned no matching listing. This guide therefore does not invent installation links from the source-topic names or relabel an ordinary documentation Skill as an automatic learning engine. For new candidates, verify the public detail, pinned source, license, supported host, hook configuration, file-writing scope, and network behavior. Identify whether the candidate captures events, governs memory, or evaluates outcomes.
Even a lightweight local-file approach needs checked loading times, encoding, concurrent writes, and conflict handling. Shared memory across agents or sessions needs namespaces and versions so one user's preference cannot leak into another project and partial writes do not overwrite accepted records. Plain text is not risk-free, and fewer dependencies do not remove maintenance work.
FAQ and acceptance
Can incorrect lessons become permanent? Yes, which is why candidates need evidence, counterexamples, review, and rollback rather than immediate global promotion. Why does a new session repeat the failure? Inspect capture, saving, host activation, retrieval, conflicts, and the execution gate in that order. Installing more memory packages is not a diagnosis.
How many lessons are needed? Quantity is not the goal. First show that one scoped rule changes behavior on a new sample. Can a skill be generated automatically? An experience can become a draft, but triggers, inputs, outputs, dependencies, and acceptance still need review. A log excerpt does not directly become a safe executable package.
The acceptance package contains redacted events, evidence locations, rule diffs, activation records, comparison results, counterexamples, and recovery instructions. Distinguish executed checks from plans. Explain how memory is deleted, how staleness is handled, and who can change a rule. Delivering a long lessons file alone does not establish an improvement loop.
Sources and maintenance
This original example covers the bilingual sources' error-memory, governance-loop, and experiment scenarios without adopting testimonials, installation counts, or guaranteed intelligence gains. The LangMem repository provides a reusable-method reference. Refresh associations when verified purpose-matching records enter SkillPI, retaining rule versions and evaluation samples. No hook, model training, or reader configuration change was executed for this article.