Embeddings and RAG

A language model only knows what it saw during trainingTraining Adjusting model weights from data so its predictions improve.. It has never read your internal manual, the refund policy that changed on Tuesday, or the product names your team made up last year. Retraining it every time a document changes is not practical, and pasting every document into the question does not work either: they don't fit, and even if they did, it would cost too much.

RAGRAG Retrieval-augmented generation: finding relevant passages in your documents and adding them to the prompt before the model answers. (Retrieval-Augmented Generation) takes a simpler route, much like an open-book exam. Before the model answers, the system looks up the few passages of your documents that matter for this question and hands them over, with the instruction to answer from them and say where each claim came from. The model itself never changes; only what it gets to read does.

This lesson covers the four ideas that decide whether RAG works: how the system finds passages by meaning, how documents are prepared, how to combine two kinds of search, and how to tell what went wrong when an answer is bad.

Finding text by meaning

A normal search matches words. Ask for "how long does a refund take?" and it will miss a page titled "time to get your money back", because the two share no words. RAG needs a search that understands they mean the same thing.

That is what an embeddingEmbedding A list of numbers that captures the meaning of a text, so texts with similar meanings get numbers that sit close together. provides. An embedding model reads a piece of text and turns it into a long list of numbers that works like coordinates on a map of meaning. Texts that say similar things land close together on that map, whatever words they use. To search, the system turns the question into coordinates too and picks the passages that sit nearest to it. Closeness is usually measured with cosine similarity, a score of how closely two of these lists point the same way.

The passages and their coordinates live in a vector store, a database built to answer "which of these are nearest to this point?" quickly, even across millions of entries.

Preparing the documents

Before anything can be searched, documents go through ingestion: extract the text from each file, cut it into passages called chunks, compute their coordinates and store them. How you cut matters more than it looks:

• Chunks that are too large blur several topics together and end up resembling no specific question.
• Chunks that are too small lose their context: "the window is 30 days" means little without saying which window.
• A little overlap between neighboring chunks keeps an idea from being split exactly at the cut.

Combining meaning and exact words

Meaning-based search has a blind spot: exact identifiers. A product code like QX-4471, a surname or a contract clause number carries no "meaning" for the map to capture. Classic keyword search has the opposite profile: it finds exact matches and knows nothing about synonyms. The standard keyword ranking is called BM25BM25 A classic scoring method for keyword search, often used alongside vector search so exact words and names are not missed., and databases such as Postgres and Elasticsearch provide it.

Hybrid retrieval runs both searches and merges the results. The two produce scores on different scales, so they can't simply be added. The common fix, Reciprocal Rank FusionRank fusion Merging several ranked result lists, such as vector and keyword results, into one. (RRF), ignores the scores and looks only at positions: a passage that ranks high in both lists rises to the top.

A final, optional step is rerankingReranking Re-scoring search results with a slower, more precise model before passing the best ones to the LLM.. A slower, more careful model reads the question and each of the best candidates together and reorders them. It is too slow to run over the whole collection, so it only sees the short list.

Making the model use what it found

Finding the right passages is half the job; the model still has to rely on them. The usual pattern is to number the passages in the instructions, ask the model to answer only from them and cite the numbers, and return the list of sources with the answer so a person can check.

When RAG gives a bad answer

Before blaming the model, split the problem in two: did the right passage reach the model, and did the model use it well?

The last row is easy to miss: coordinates produced by two different embedding models are not comparable, so changing that model means re-indexing the whole collection. Adapta checks for this: if the configured embedding model no longer matches the one a collection was indexed with, it stops with a clear error instead of returning quietly wrong results (Operations Runbook).

Uploaded files are also untrusted input. A compressed file can be built to explode to gigabytes, or to write outside the folder it is extracted into, and a document can hide instructions aimed at the model. That last risk, indirect prompt injectionPrompt injection An attack where hostile text, typed directly or hidden inside data the model reads, changes what the model does., is covered in Sandboxing and agent security. Adapta guards its uploaded image archives against the first two (extract_bundle in adapta/services/training.py), and document uploads have a size cap, so an oversized file is refused before it is saved.

Putting it into practice

• Measure search on its own. Keep a small set of real questions with the document each one should find. Without it, nobody can tell whether a change helped. See Evaluating models.
• Choose where the vectors live. If the team already runs Postgres, pgvector saves adding a system. A dedicated vector store earns its place through scale or features you actually need.
• Check the language. Many default embedding and reranking models were trained mostly on English. For Spanish or mixed content, pick multilingual models and measure them.
• Enforce permissions during search. If not everyone may see every document, filter them before they reach the model. An instruction in the promptPrompt Everything sent to the model in one call: instructions, conversation messages and any tool definitions. is not access control.

Go deeper

Adapta's own documentation explains the same ideas with its code as the running example:

• Learning the system builds RAG from first principles: embeddings, the vector store, hybrid retrieval and the complete cycle.
• Stack and decisions compares each retrieval piece with the options that were set aside, and says why.
• Knowledge and behavior shows RAG and fine-tuningFine-tuning Further training a pretrained model on your own examples to change how it behaves. working together in one assistant, step by step.

Key takeaways

• RAG adds knowledge without retraining: find the relevant passages, give them to the model, ask it to cite.
• Embeddings find text by meaning; keyword search finds exact terms. Combining both covers each one's blind spot.
• How documents are cut and kept up to date causes as many failures as the model does.
• When an answer is wrong, first ask whether the right passage was found at all.

Check yourself