The hard part isn't storage — it's deciding what counts as "the same" item. For web research agents, URL identity isn't sufficient (pages change, same story, different URL). Content fingerprinting on normalized text (first N chars after stripping whitespace/HTML) turns out to be more reliable than URL equality.

Also worth noting: the failure mode you described (repeating mistakes) often comes from agents not distinguishing between "I haven't seen this" and "I saw this and it failed." Storing outcome alongside identity — even just success/failure — changes the behavior significantly. Retry logic becomes explicit instead of accidental.