Your coding agent shouldn't get to grade its own memory

Most memory systems for AI coding agents risk reinforcing errors by equating retrieval with success. This was a core challenge for skillmem, a local memory layer developed for models like Claude Code and Codex, which stores procedural notes comprising triggers, steps, outcomes, and lessons. The initial design saw a note's 'strength' increase simply upon its recall, creating a problematic feedback loop that could lead an agent to become more confident in incorrect information.

The redesigned skillmem now treats simple retrieval as 'self_report,' which refreshes a note's recency and access count, preventing it from fading but not increasing its strength. Positive reinforcement for a note's strength now requires explicit evidence, while 'Failure' events, when following a note leads to breakage, reduce strength more rapidly than mere disuse. Additionally, notes can be archived after 90 days of inactivity, and user-approved rules can be 'sealed' to prevent agent-reported failures from completely deprecating them.

Despite these improvements, a significant limitation remains: the system still relies on the agent's self-reported evidence, such as an assertion of `evidence="test_passed"`, to increase a note's strength. There is no external verification that a test actually ran or passed. This means that while the accidental reinforcement loop from mere retrieval is closed, the system does not prevent an agent from asserting false success. The developer notes that a definitive measure of success, controlled results showing that earned strength improves task completion, is not yet available.