Keeping Knowledge in Sync When Source Documents Change
A practical architecture guide on incremental knowledge synchronization: managing document deletion propagation, source version tracking, change data capture, and verification pipelines without costly full reindexing.
A source document can change while its old wording still appears in search, cached answers or a curated summary. Keeping knowledge synchronized means tracking the change through every relevant stage, then checking what people and agents can actually retrieve. A successful ingestion request alone does not prove that the answer uses current evidence.
Key takeaways
- Separate change detection, durable capture, indexing and review of derived knowledge.
- Track source identity and revisions so renames, retries and delayed events do not create duplicates or restore obsolete content.
- Test deletion and permission changes across active retrieval paths, not just the document store.
- Measure freshness and failed work explicitly; incremental processing does not guarantee a particular saving or update time.
Choose a change detection method that fits the source
Git-backed storage and change data capture are not competing approaches at the same layer. One concerns how knowledge is stored; the other concerns how upstream changes are detected. A deployment can use both.
| Method | Useful when | What to verify |
|---|---|---|
| Polling an API or inventory | The source exposes listings, revisions or modified timestamps | Pagination, rate limits, deletions, permission-only changes and gaps between scans |
| Webhooks | The source publishes relevant change events | Authentication, delivery gaps, duplicates, event ordering and retry behavior |
| Database CDC | A supported database exposes a suitable change log | Snapshot-to-stream handoff, offsets, schema changes, deletes and retention of the log |
| Git revision comparison | The authoritative source is a repository | Branch/ref selection, renames, history rewrites and the meaning of source identity |
No method removes the need for reconciliation. Periodically compare the source inventory and revisions with captured state. For a concrete provider example, GitHub's webhook guidance discusses missed deliveries and delivery identifiers. Apply the actual provider's contract rather than assuming all webhooks deliver exactly once or retry automatically.
Make each change safe to replay
Keep a stable source ID separate from its display title or location. Record the source revision, content hash where useful, capture time and permission information. A matching content hash does not prove permissions are unchanged.
Use the source ID and revision to recognize repeated work. Ensure an older event cannot overwrite a newer accepted revision; when event order is unreliable, reconcile against authoritative source state. Treat a timeout after a write as an uncertain outcome and check whether the intended revision already exists before retrying.
Separate the ingestion cursor from downstream indexing progress. Advance a checkpoint only when the required change is durably captured with a recoverable processing status. Indexing can then retry independently. Failed records need a visible retry or exception queue, an owner and a way to replay them without silently skipping changes.
| Change | Expected handling |
|---|---|
| Create or edit | Capture the correct revision, update affected retrieval material, and identify dependent summaries or claims for reassessment |
| Rename or move | Preserve identity where the source supports it; update location and references without creating a false new document |
| Delete or withdraw | Suppress obsolete active retrieval paths and review derived claims; retain historical material only under applicable retention and access policies |
| Permission change | Update authorization independently of content changes and verify denied access across interfaces |
Track provenance across the update path
Distinguish the upstream source revision, captured copy, approved derivative and indexed revision. An answer citation should identify the evidence actually used, including source identity, revision and location where available. A timestamp on the answer is not a substitute for the version of its evidence.
W3C PROV-O provides concepts for derivation and responsibility. These can inform a provenance design without implying that a particular knowledge product implements the standard. Document history, provenance, audit records and backups remain different responsibilities.
When a source changes, a derived summary may require review rather than immediate replacement. Check whether its claims still hold, whether other evidence supports them, and whether the approved meaning has changed. Do not keep a withdrawn source accessible merely to preserve a citation. Restrict historical references and metadata according to the same access and retention decisions.
Estimate incremental work without promising savings
Consider an explicitly hypothetical corpus of 10,000 documents with 10 chunks each: 100,000 chunks. If 100 documents change and each still produces 10 chunks, document-level reprocessing touches 1,000 chunks before overhead, or 1% of the original chunk count.
This arithmetic is not a measured AKB result or a guaranteed 99% cost reduction. Actual work also includes detection, parsing, fixed infrastructure, deleted chunks, permission updates, retries, reconciliation and query traffic. Changes to parsers, chunk boundaries, embedding models or schemas can require broader reprocessing. Benchmark the selected pipeline using representative changes.
Verify freshness from source to answer
Measure source-to-capture lag, capture-to-search lag and total source-to-search lag using defined start and end events. Report p50 and p95 alongside pending changes, oldest pending age and failures. A low latency among completed jobs can hide a growing backlog.
Use traces, metrics and logs to follow the processing stages. OpenTelemetry describes instrumentation and telemetry signals; adopting it does not automatically establish application-specific freshness checks.
| Test | Evidence to inspect |
|---|---|
| Edit a known fact | Direct read, lexical and semantic retrieval, and answer citations use the intended revision |
| Replay or delay an event | No duplicate logical record and no regression to an older revision |
| Rename, then delete | Identity survives the rename; obsolete content disappears from active retrieval after deletion handling |
| Revoke a principal's access | No restricted content leaks through bodies, snippets, caches, relationships, provenance or derived answers in tested paths |
| Interrupt indexing and resume | Captured changes remain recoverable, failures remain visible and retries finish without lost updates |
Run these cases through a human UI/API workflow and an agent MCP client with appropriate identities. Define the acceptable content-freshness window locally before testing. Treat permission revocation as a separate acceptance requirement: after revocation succeeds, subsequent unauthorized requests must be denied. A content-index refresh window is not permission to serve restricted content. Test this across relevant access paths; passing the test does not guarantee that all future paths are safe.
Where AKB fits
The AKB public repository describes a Core knowledge store for documents, tables, files and relationships, with retrieval, document history, MCP, REST API and Web UI access. External ingestion and curation workflows must still own source synchronization and review. Core does not bundle its own knowledge gardener.
As checked on September 28, 2026, the public AKB repository describes PostgreSQL-native revisions as the default and explicit Bare Git support for legacy deployments. Verify the installed version and configuration; do not assume all data is Git-versioned. AKB's write outbox supports downstream consumers, but does not itself detect upstream SaaS changes. Revision history, provenance, audit records and tested backups remain separate responsibilities.
Frequently asked questions
Can webhooks replace reconciliation?
Not safely without evidence that the source and integration cover every required event and failure. Reconciliation helps find missed updates, deletions and permission changes.
Does a successful write mean search is current?
No. Direct reads and search indexes may become current at different times. Record and test both milestones.
Must every change trigger a full rebuild?
No. Reprocess affected material when the pipeline supports it. Broader parser, model or schema changes may require more extensive work.
Should deleting a source delete every derived summary?
Review each dependent claim and its remaining evidence. Suppress invalid or unauthorized content while applying the relevant retention policy; do not automatically preserve or purge everything.
Does AKB automatically synchronize all source systems?
No such assumption should be made. Verify connector coverage, external automation, permissions, failure handling and ownership for the actual deployment.
Build resilient knowledge workflows
Evaluate synchronization with your own sources, identities and change scenarios. Learn more.