Connectors

A connector (a "source") syncs an external system into a collection and keeps it up to date. Connector configs — URLs and API tokens — are encrypted at rest and never echoed back by the API.

The sources API

Method	Path	Purpose
GET	/sources/types	Available connector types
GET/POST	/sources	List / register sources
GET/DELETE	/sources/{name}	Source detail (doc count, last run) / remove and purge its docs
POST	/sources/{name}/sync	Queue a sync run
GET	/sources/{name}/runs	Sync run history

Incremental sync

Every sync run diffs the connector's full id listing against the document registry:

• New documents are ingested.
• Changed documents are upserted — chunks are replaced, never duplicated.
• Unchanged documents are skipped, by version stamp or content hash, so nothing is needlessly re-embedded.
• Deleted-at-source documents are removed from the index.

Run history (/sources/{name}/runs) records added / updated / deleted / unchanged / error counts per run.

The job runner

Syncs run through an in-process async queue with per-source locking and a run-history table. Set interval_minutes on a source for automatic periodic sync.

Confluence

curl -s -X POST localhost:8000/sources -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" -d '{
    "name": "eng-wiki", "type": "confluence", "source_type": "wiki",
    "interval_minutes": 30,
    "config": {"base_url": "https://yoursite.atlassian.net/wiki",
               "email": "you@company.com", "api_token": "...",
               "space_key": "ENG"}}'
curl -s -X POST localhost:8000/sources/eng-wiki/sync -H "Authorization: Bearer <token>"

Jira

Same shape, with "type": "jira":

{"base_url": "https://yoursite.atlassian.net", "email": "...",
 "api_token": "...", "jql": "project = OPS", "include_comments": true}

Jira Cloud only for now.

evo.ehs

"type": "smartplantehs" reads an evo.ehs site's Postgres read-only — permits, incidents (with full stage timelines, corrective actions, and regulatory flags), work tasks, chemical inventory (GHS hazards, SDS dates, permit/incident links), training courses (role requirements and completions with expiry), calendar events, and plants (site details, EPA compliance status, and a 14-day daily weather rollup):

The connector type is still spelled smartplantehs. That is the product's former name; the identifier is unchanged so existing tenant configs keep working, and renaming it is tracked with the rest of the product rename.

{"dsn": "postgresql://readonly@host/plantsafepermits",
 "sp_tenant_id": "<site uuid>", "schema": "app"}

One source maps to exactly one SP site. Soft-deleted records drop out of the index on the next sync. Change detection is content-hash based (several SP tables have no updated_at), so unchanged records are never re-embedded. Use a SELECT-only DB role in production.

Site auto-discovery

Customers create their own SP sites, so instead of registering each one by hand, set SP_DISCOVERY_DSN (plus optional SP_DISCOVERY_SCHEMA and SP_SYNC_INTERVAL_MINUTES) and the scheduler provisions one source per site automatically:

• new active/trialing sites are created and synced,
• suspended sites pause,
• reactivated sites resume,
• soft-deleted sites are purged from the index.

The Evo.ai tenant id is the SP site UUID, so an EvoPlatform JWT carrying that id reaches exactly its own site's data.

Layer 2 analytics

Aggregate questions — counts, rankings, durations — can't be answered from top-k chunks. For tenants with an active SP source, /query first runs a routing LLM call; a counting/ranking/duration question becomes one validated SELECT over tenant-scoped views (v_permits, v_incidents, v_tasks, and related) executed on SP_DISCOVERY_DSN.

Nothing is created in the evo.ehs database — the views are prepended to each query and the tenant is bound server-side. The SELECT is fenced by several enforcement layers:

• a SELECT-only DB role,
• a read-only transaction,
• an empty search_path (bare table names resolve to nothing),
• keyword and schema-prefix rejection,
• a row cap (ANALYTICS_MAX_ROWS, default 200),
• a statement timeout (ANALYTICS_TIMEOUT_MS, default 8000 ms).

Any planning or SQL failure falls back to normal vector retrieval and its relevance gate. Set ANALYTICS_ENABLED=false to turn the layer off entirely. The SQL and row count appear in the answer's sources as source_type: "analytics".

Adding a connector

Connectors follow one adapter pattern (app/connectors/) — the same shape as DocketMail's county adapters. A new connector type is one subclass implementing the listing and fetch methods; the sync engine, run history, and scheduling come for free.