Search your documents without casually exporting the source material.

Map every data hop

A “local vector database” is one component out of six, and marketing often stops there. Embedding generation receives the full text of every chunk. The final inference call receives the retrieved chunks verbatim. If either of those steps uses a hosted API, the source material has left the environment even though the index sits on your disk. The table shows where each hop runs in Glyph.

Permissions still matter

Retrieval must preserve the access boundary of the source. On a single device that is simple: your files, your index. In a team, a shared index without document-level authorisation exposes material to anyone who can ask a good question, even when the model itself runs locally. Glyph Private applies the requesting user’s permissions at retrieval time and records the request in the gateway’s usage log, so the boundary that existed in the file share survives the move into an index.

Decide who owns each source, how it synchronises, and what deletion means before you build a shared index. “Delete the file” and “delete every chunk and vector derived from it” are different operations, and only the second one is privacy.

Retrieval quality is a tuning problem

Most disappointing RAG results are retrieval failures, not model failures. Chunk size and overlap decide whether a passage arrives intact; too small loses context, too large dilutes the match. The number of chunks retrieved trades recall against the model’s attention. A capable instruction-following model that is told to answer only from the supplied context, and to say when the context does not contain the answer, produces fewer confident fabrications than a stronger model with a vague prompt.

Evaluate with questions whose answers you know are in the corpus, questions whose answers are not, and questions that require two passages at once. Keep the retrieved chunks visible while you tune; Glyph shows which sources contributed to an answer so you can tell a retrieval miss from a generation miss.

What to test before production

A useful pilot covers the whole lifecycle, and each item should have an owner and a piece of evidence.

• Ingestion: every document type you actually use parses, including scanned PDFs and spreadsheets.
• Retrieval quality on the three question types above, measured against a small answer key.
• Source updates: a changed document is re-chunked and the stale vectors are gone.
• Deletion: removing a source removes its chunks and vectors, and a query can no longer surface it.
• Interrupted indexing: a worker stopped mid-run resumes or restarts cleanly.
• Backup and restore of the index and its metadata, timed.
• Logs and metrics contain operational metadata only — never chunk text or prompts.

Choosing the generation model

The final hop is where privacy is most often lost casually: a user switches to a hosted model for a better answer and the retrieved chunks go with the question. Glyph labels the destination in the picker, and a Glyph Private deployment can restrict generation to gateway-approved models. If a hosted model is permitted for some tasks, decide which corpora may be used with it and enforce that in policy rather than in a training slide. The local LLM guide covers picking a local model that follows retrieval instructions well.