AI Knowledge Base Software: 4 Source-Refresh Options Compared
Compare four ways to keep an AI knowledge base current, based on source-change frequency, freshness targets, and who owns each update.
If you own an AI knowledge base, choose who detects source changes, runs ingestion, approves sensitive updates, and verifies when the new information becomes searchable. This guide compares four refresh ownership models and the timestamps each owner should record.
Key takeaways
Choose refresh by who owns each update
- Use a managed refresh service when another team will own recurring source checks, ingestion runs, and failure alerts under a clear service agreement.
- Use connector sync when supported connectors cover your sources and their update schedule meets your freshness target.
- Use a manual project when the collection changes rarely and a named person can replace or revise files promptly.
- Use Dnotitia AKB with separate ingestion when you want a read/write knowledge store and your team can own or procure source-refresh automation.
Does answering questions keep sources current?
A knowledge base can return an answer from its current index while the original document has already changed. Retrieval uses information that has reached the searchable store; refresh moves new, changed, or deleted information from its source into that store.
Amazon Bedrock’s ingestion documentation lists parsing, chunking, embedding generation, and vector-store indexing as stages of ingestion. For vector stores other than Amazon Aurora (RDS), newly synced embeddings may take a few minutes after syncing finishes to become queryable.
Define end-to-end freshness as the time from a source change until the approved update is searchable. Where review is required, record review delay separately from post-approval technical delay—the time from approval until the update is searchable—so the team can distinguish review time from processing and index-visibility time.
In this illustrative timing example, the source changes at 9:00 a.m., sync starts at 3:00 p.m., approval is recorded at 4:00 p.m., and a test query confirms searchability at 4:10 p.m. End-to-end freshness is seven hours and ten minutes: six hours pass before sync starts, followed by one hour and ten minutes from sync start to confirmed searchability. The timestamps given do not show how much of that latter interval was review versus technical processing.
To attribute the delay, also record separate review-start and approval timestamps, ingestion completion, and the first time the update is searchable. Keep the source or version identifier, distinctive test-query phrase, timestamps, and query result together so another owner can repeat the check.
What four refresh models belong in the same comparison?
Compare the models using the same ownership questions: who notices source changes, who launches or runs ingestion, how a reader can see freshness, who reviews the content, and who restores service after a failed update.
| Refresh model | Change detection and ingestion owner | Review and failure owner | Strongest fit |
|---|---|---|---|
| Managed refresh service | A contracted provider or internal platform team runs recurring checks and ingestion within the agreed scope | The agreement and operating runbook assign monitoring, exceptions, recovery, and content approval | Teams that value an assigned refresh operator and can define service scope, sources, and freshness targets |
| Supported connector sync | The team checks each connector’s supported sources and documented trigger or schedule, then identifies who starts or monitors each sync | The customer assigns content owners and checks unsupported sources, failed syncs, and review requirements | Teams whose main sources and required update behavior fit documented connectors |
| Manual project or workspace | A curator notices a source change and updates the project’s files or records | The curator checks whether updates succeeded and whether users now receive the revised content | Bounded collections with infrequent change and an available owner |
| Knowledge store with separately operated ingestion | The team or its chosen integration runs source detection, ingestion, and any review workflow | The team assigns owners for the pipeline, approvals, monitoring, and recovery | Teams that want control over the knowledge store and have capacity to operate or procure refresh separately |
When does a managed refresh service fit?
Choose managed refresh when your team needs a named operator for recurring source checks and does not want to build every operational step itself. The key is the service boundary: an agreement should identify the sources, refresh trigger or schedule, expected searchable-state target, failure notification path, and the party that approves material knowledge changes.
A managed option can still require an internal content owner. The provider may run connectors and monitor jobs while your knowledge lead decides whether a changed procedure is authoritative, whether two conflicting pages need resolution, and whether a deletion should remove agent-visible content. Keep this distinction explicit when comparing internal effort: outsource repeatable transport and pipeline operation where the offer covers them, and retain decisions that depend on company policy or subject expertise.
Amazon Bedrock’s data-source sync guide and GetIngestionJob API reference illustrate the provider-specific documentation teams should check when assigning ownership of sync triggers, run monitoring, and source review.
A published service example makes cadence concrete. Ada’s content-ingestion documentation states that its website-import path runs daily syncs to detect and update changes; it also lists direct authoring and Knowledge API imports as other ingestion paths. A daily schedule does not by itself guarantee a one-day end-to-end freshness target. Include the wait until the next scheduled run, processing, required review, retries, and index visibility within the target; a shorter target may call for a faster trigger or a different workflow.
When is connector sync enough?
Connector sync is a good fit when a platform’s supported source connections cover the systems that hold your authoritative information, and its sync behavior meets your actual delay requirement.
In Amazon Bedrock’s sync guidance, adding, modifying, or removing source files requires a sync, which incrementally processes only changes since the prior run. When a document’s content or metadata changes, the service re-parses, re-chunks, re-embeds, and re-indexes it. This incremental approach avoids treating every refresh as a full reload, but the team still needs a trigger or schedule that starts each run.
Connector fit depends on source behavior as much as the connector name. An event-triggered integration may suit a rapidly changing source; a scheduled check may fit daily policy or knowledge updates. A repository of frequently edited files may need an update target measured in minutes, while a stable handbook may work with a daily refresh. Choose the cadence from the consequence of stale information, not from an assumption that every connector is real time.
When does a manual project make more sense?
Choose a manually maintained project when the knowledge collection is small enough for an owner to understand and changes slowly enough for a deliberate refresh routine. A project with a defined endpoint, such as a developer’s feature notes or a short research bundle, gives its curator a clear set of files to update. It avoids standing up recurring integration work before the source-change rate justifies it.
The curator is the change detector. When an authoritative file changes, the owner identifies the current version, replaces or revises the project copy, and tests that a representative question now returns the new information. When a file is retired, the owner removes or marks the old copy so it does not remain a competing answer. Those actions are the human equivalent of the add, modify, and delete cases in an ingestion pipeline.
A useful way to test the workload is to count the collection’s change events over a representative period and record the time needed to review, upload, and verify each one. Multiply the typical effort per event by the observed event count, then include the time needed to handle deletions and conflicting revisions. The estimate lets a team compare hands-on upkeep with the ownership and quoted cost of an automation option.
As an illustrative workload calculation, if a collection produces 12 updates per month and each takes 15 minutes to review and verify, manual upkeep uses about three hours monthly before exception handling.
When should a team pair a knowledge store with separate refresh?
Use a knowledge store with separately operated refresh when you want agents to work with shared, structured knowledge while keeping source acquisition and update timing under your team’s control. This model can suit teams with existing ingestion code, custom source systems, or strict review workflows, provided they can assign or procure the refresh work.
Dnotitia AKB is the read/write knowledge-store option in this comparison: agents can read and write documents, files, and tables through MCP. The core knowledge store and source refresh are separate responsibilities, so the team must assign or procure ingestion and curation outside the core.
AKB documents have revision history; assess table, file, and relation recovery separately. Before relying on revision history, access controls, or logs, verify the feature scope documented for the specific deployment. These records can support knowledge review, but they do not replace ownership of source refresh, approvals, monitoring, or recovery.
The current AKB README describes a PostgreSQL-native revision store and recommends Native for new installations; explicit Bare Git installations remain supported for existing deployments. Confirm history, recovery, and export against the selected backend. A change in installation defaults does not migrate an existing vault.
How should you choose for your source-change rate?
- Set the acceptable delay. Choose the slowest acceptable time from a source change to approved content becoming searchable, based on the consequences of a stale answer. Measure review delay and post-approval technical delay separately; the latter runs from approval until the content is searchable.
- Check source coverage and ownership. Confirm that the chosen approach covers each authoritative source and that a named owner can handle change detection, ingestion, review, and failures. Record each source’s location, content owner, change signal, update frequency, sensitivity, and review rule.
- Compare performance over the same period. For the same source set and freshness target, compare freshness with operating effort, including owner hours, failed or pending updates, review backlog, and service or infrastructure cost. Job-success counts alone can omit unresolved work.
What should a source-refresh acceptance checklist verify?
Before rollout, test the full path from a source edit to a correct answer. A green ingestion job is useful operational evidence, but the end-user check confirms that the changed information has reached retrieval and that an agent can use it as intended.
- Name the source and owner. Identify the authoritative location, the person responsible for its content, and the operator responsible for ingestion. Include who approves policy-sensitive changes.
- Set the freshness clock. Record the source-change timestamp and define the endpoint as updated, approved content becoming searchable. Set a target that matches the stale-answer risk and test against that target.
- Test an addition. Add a uniquely identifiable test item to an approved source. Confirm the integration detects it, ingestion processes it, the search index makes it available, and a representative query retrieves it.
- Test an edit. Change a phrase or value that affects an expected answer. Confirm the retrieved answer reflects the new value and does not rely on an earlier version.
- Test a deletion. Remove or retire an item at the source. Confirm that the old content stops appearing in normal retrieval and that the team can explain any retention or archival rule that keeps it elsewhere.
- Observe each stage. Capture detection time, job start and completion, record counts, searchable time, and any approval time.
- Exercise a failure and recovery. In a test environment, introduce a safe failure, verify that the assigned owner receives an alert, and record the recovery steps. Databricks’ pipeline monitoring guide lists update history, per-table row counts, data-quality measures, and streaming backlog as operator signals.
- Test access changes. Revoke a test user and check direct reads, search snippets, citations, and derived answers.
- Test retries and uncertain writes. Replay a source event and interrupt a batch; retain stable source IDs, destination revisions, failed items, and a durable checkpoint. Advance the checkpoint only after required work is safely recorded. Inspect committed state after an uncertain write response before retrying. These are proposed acceptance checks, not documented guarantees of every connector.
- Verify the handoff. Assign an owner for unsupported sources, rejected content, duplicates, permission changes, and conflicting documents.
- Retest after material changes. Repeat the update tests after changes to connector configuration, parsing rules, schedules, or storage settings that could affect retrieval or freshness. Keep a sample query set so teams can compare behavior across releases.
Keep a source-refresh timing record
Record the source/version identifier, change time, detection and ingestion times, required approval, the first successful query, and the responsible owner. A completed job and a verified searchable change answer different operating questions.
Related background: Keeping Knowledge in Sync When Source Documents Change
FAQ
What are knowledge base systems?
Knowledge base systems store organized information so people or software can retrieve it to answer questions or complete work. An AI knowledge base adds retrieval for an AI application; a separate refresh process brings changes from source systems into the searchable collection.
Should I build a knowledge base as a project or subscribe to something that runs continuously?
Choose a project when its source material changes infrequently and a named person can refresh it promptly. Choose a continuously operated service when your required freshness target needs recurring source checks and an identified operator; choose AKB when you want the read/write store and will arrange refresh separately.
Read the AKB documentation
Review the documented knowledge and integration workflows for your AKB deployment.
