Ingesting data

Evo.ai indexes your data into collections. You can upload files, ingest structured records, or sync from a connector. This page covers the direct-ingest paths and how document identity works.

Collections

A collection is a named group of documents with a fixed embedding configuration. Manage them under /config/collections:

• GET /config/collections — list collections
• POST /config/collections — create a collection (its embedding config is fixed at creation)
• DELETE /config/collections/{name} — drop a collection and its vectors

Embedding config is permanent per collection

Each collection records its embedding model when it is created and keeps it forever. Changing the embedding model invalidates every vector, so it is a re-embed into a new collection, not a config flip on an existing one.

If you ingest into a collection that doesn't exist yet, it is auto-created using the server's default embedding config.

Uploading files

POST /ingest/file accepts a file plus a collection and an optional stable doc_id:

curl -s -X POST localhost:8000/ingest/file -H "Authorization: Bearer <token>" \
  -F "file=@safety-manual.pdf" \
  -F "collection=handbook" \
  -F "doc_id=safety-manual"

Supported types are PDF, DOCX, CSV, TXT, MD, and JSON. An unsupported content type is rejected with 415. Uploaded text is chunked (CHUNK_SIZE / CHUNK_OVERLAP) and embedded.

Ingesting structured records

POST /ingest/records ingests structured rows into a collection — useful for data that is already tabular rather than a document:

curl -s -X POST localhost:8000/ingest/records -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{"collection": "assets", "records": [ ... ]}'

Document identity: upsert and delete

Passing a stable doc_id makes ingestion idempotent: re-ingesting the same doc_id replaces that document's chunks rather than duplicating them. This is how edits propagate cleanly.

To remove a document and all of its chunks:

curl -s -X DELETE localhost:8000/ingest/doc -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{"doc_id": "safety-manual"}'

The watched-folder agent

scripts/watch_folder.py uploads a directory tree and, unless you pass --once, keeps watching it and propagates changes:

pip install watchdog
python scripts/watch_folder.py ~/my-docs --token <token>          # watch continuously
python scripts/watch_folder.py ~/my-docs --token <token> --once   # one-shot sync

It keeps a SHA-256 manifest, so unchanged files are skipped on later runs, and it uses doc_id-based upsert/delete so file edits, moves, and deletions are reflected in the index.