1. If it's not written down, it does not exist. This needs to be a ruthlessly enforced rule. I know we all love those Mel the Programmer stories, but that's a story about fragility. Similarly if you are relying on SRE or dev heroics to solve field problems, you have a gun to your head and you need to solve it yesterday.
2. The TCO of knowledge is dominated by the maintenance cost, not the creation cost. I see this mistake made a lot. Anyone can create a confluence page, Slack comment, SharePoint site or a git comment. The org must make it someone's job to grow, weed, and maintain these gardens. It does not matter what system you use, but someone needs to look after it.
You get a bonus for building information "data structures" that make it easy for people to pull data out as your org changes, but this is probably something you should accept that you'll get wrong as your org grows. AI hides this problem because it makes search so easy, sometimes, but it will come and bite you hard one day, so be ready for it.
In my experience:
1. Well meaning people try to build wikis/confluences/whatever that turn into backwaters of slowly rotting stuff
2. Beyond a certain extent that they can hold in their heads, most other people give up.
3. Some people hoard knowledge and use it for personal advantage.
I dont know what the solution is. Maybe an ai that reads every chat and email and commit - and builds some kind of browsable, queryable, executable knowledge trove.
This way, new and updated context is not lost on the next user.
i think one way to solve this is having an someone or an agent update the initial spec with any new changes as and when they are made.
I do like the idea of having a live editable wiki as the knowledge base a LLM can reference.
The problem of scattered data is to put data in one place.
Our "project knowledge" consists of artifacts of such types:
- product documentation (made by human). Intended product behavior and capabilities.
- technical documentation (maintained by human and agent who work together on feature). Implementation documentation.
- feature session journal (made by agent and human who work together on specific feature). Per feature session. Contains original problem, gaps, problems, misunderstandings and failed attempts.
- wiki and ontology articles for targeted knowledge retrieval. Built by Knowledge Keeper procedure under human supervision on demand. Consolidate the knowledge across subsystems.
- code and tests (mostly written by agents).
All these knowledge artifacts (except code and tests) are mostly markdown articles. They are stored in git - this allows versioned storage, authorship and diffs. All articles have front matter with summary, keywords, status (draft or reviewed and approved by human) and links to related articles, so we and agents know what is relevant and what the source of truth is. Wiki and ontology articles hold the semantic relationships for better retrieval.
Typical process of changes: As agent and human work on the feature, this is captured in the session journal. At any moment in this process, the engineer who works with the agent can ask the agent to hand off their work to other agents, integrate in the technical documentation or handoff to a Knowledge Keeper. Such handoff includes the message with documentation links and the link to the session journal, so one who continues starts with the context.
The Knowledge Keeper is a supervised procedure that keeps wiki and ontology aligned with the product documentation, technical documentation, known issues. Knowledge integration process is currently manually initiated by the handoff of the human who worked with agent on the feature. The Keeper prepares the integration plan which a human approves. Keeper builds the index on top of wiki and ontology after integrating the knowledge and tests the knowledge retrieval via MCP to check if integrated knowledge retrieves correctly and reports to the human maintainer. MCP allows the consolidated view of the product and its capabilities. Its answers point to the primary docs, therefore users do not need to find one person who remembers everything.