A vector database stores text — or images, or audio — as lists of numbers called embeddings, and finds items by meaning rather than by matching keywords. If your AI feature needs to answer from your own content, you almost certainly need one. Here is what it does, where it fits, and what your real options are.
Keyword search vs meaning search
Traditional search matches strings. Search your help centre for “reset password” and it finds documents containing those words. Search for “I can’t get into my account” and it may find nothing, even though the right article is sitting there — because the words do not match.
Vector search matches meaning. It understands that “I can’t get into my account”, “credential recovery”, and “sign-in help” are all close in meaning, and returns the right passage regardless of which words the user happened to type. For an AI feature answering real questions from real people, that difference is the whole game.
What an embedding is
An embedding is a list of numbers — often around 1,536 of them — that represents a piece of text. An embedding model produces these numbers so that texts with similar meanings get similar lists. Think of each embedding as a point in a very high-dimensional space, positioned so that related ideas sit near each other and unrelated ideas sit far apart.
You run every passage of your content through the embedding model once, storing the resulting vectors. When a user asks a question, you embed the question the same way and ask: which stored vectors are closest to this one? Those nearest neighbours are the passages most relevant to the question.
What a vector database is for
Finding the nearest vectors among millions, fast, is a genuinely hard computer-science problem. A vector database is the specialised tool that does it — it indexes embeddings so that a nearest-neighbour search returns results in milliseconds rather than scanning every record. It also handles the ordinary database concerns: storing the original text alongside each vector, filtering by metadata, and updating as your content changes.
How your content gets into the vector database
The vectors do not appear by themselves. Getting your content into the database is an ingestion pipeline, and it runs in three steps. First, content is split into chunks — passages small enough to be specific but large enough to carry meaning on their own. Second, each chunk is run through an embedding model to produce its vector. Third, the vector is stored alongside the original chunk text and any metadata you want to filter on later, such as document type, date, or access level.
This pipeline runs once up front to load your existing content, and then incrementally — every time a document is added or changed, that document is re-chunked and re-embedded so the index stays current. The chunking step is the one that most affects answer quality, and it is the step teams most often get wrong: chunks that are too large bury the relevant sentence in noise, and chunks that are too small lose the context that made them meaningful.
Two further choices shape the pipeline: which embedding model to use, and whether to store a small overlap between consecutive chunks. The embedding model should suit the kind of text you have and the languages you support — a general-purpose model is fine for most cases, but it is a real decision, not a default. A small overlap helps when an answer straddles a chunk boundary. Individually these are minor; together they are much of what separates retrieval that works from retrieval that nearly works.
Where it fits in a RAG system
In a retrieval-augmented generation system, the vector database is the retrieval layer. The flow on every question: embed the user’s query, ask the vector database for the closest passages, and hand those passages to the language model as context for its answer. The model answers from what it was given. The vector database is what makes the answer about your business instead of about the world in general. It is also the part of the system most worth getting right: everything the model says downstream is built on what retrieval handed it, so a weak retrieval layer quietly caps the quality of every answer, no matter how capable the model is.
Do you need a dedicated vector database?
Postgres with pgvector
For most teams, the simplest answer is the pgvector extension on the PostgreSQL database they already run. It adds vector storage and nearest-neighbour search to your existing database — one extension, no new infrastructure, no new service to operate, and your vectors live next to the rest of your data so you can filter on both together. For the majority of AI features, this is all you need.
Dedicated vector databases
Dedicated vector databases — Pinecone, Weaviate, Qdrant, Chroma — earn their place at large scale, when you need very high query throughput, advanced filtering, or a fully managed service so your team does not operate the index. They are excellent tools. They are also a new piece of infrastructure, and most products do not need one on day one.
What good retrieval needs beyond the database
The vector database is necessary but not sufficient. Retrieval quality depends on decisions around it: how you split content into chunks; hybrid search that combines meaning-based and keyword-based matching; a reranking step that re-scores the top results for relevance; and metadata filtering so a query only searches the content it should. Most “the AI gives bad answers” problems are retrieval problems, and most retrieval problems are in these choices, not in the database itself.
Common mistakes worth avoiding
A few mistakes show up again and again when teams build their first vector-search feature. Embedding with a weak or mismatched model undermines everything downstream, because poor vectors cannot be rescued by a good database. Skipping the reranking step leaves the model working from roughly-relevant passages when precisely-relevant ones were available. Indexing content once and never refreshing it produces an AI that confidently serves outdated answers. And measuring nothing — shipping retrieval with no evaluation of whether it actually surfaces the right passages — means the first signal that retrieval is broken comes from a frustrated user.
None of these is exotic. They are the ordinary, avoidable gaps between a vector database that technically works and a retrieval layer that genuinely answers questions well. The database is the easy part; the decisions around it are what decide whether the feature is good.
Common questions
Do I need a vector database for my AI feature?
You need one if your AI feature should answer using your own content — documents, help articles, product data, internal knowledge — rather than only the model’s general training. That covers most useful AI features: support assistants, internal search, document question-answering. If your feature is purely generative with no reference to your data — drafting text from scratch, for example — you may not need one. For most teams adding AI to an existing product, a vector database is part of the build.
Is pgvector good enough, or do I need Pinecone?
For most teams, pgvector on your existing PostgreSQL database is genuinely good enough, and it is the option we recommend by default. It adds vector search to a database you already run, with no new infrastructure and your vectors stored next to your other data. A dedicated vector database like Pinecone earns its place at large scale, with very high query volumes, or when you want a fully managed index your team does not operate. Start with pgvector; move to a dedicated database when a real constraint forces it.
What is the difference between an embedding and a vector database?
An embedding is the data — a list of numbers representing a piece of text, produced by an embedding model. A vector database is the tool that stores those embeddings and finds the nearest ones to a query, fast. The embedding model and the database are separate choices: you pick an embedding model to convert text to vectors, and a vector database to index and search them. Both matter, and a poor embedding model will undermine even the best database.
Why does my AI still give bad answers if I have a vector database?
Because a vector database is only the storage and search layer — retrieval quality depends on the decisions around it. The usual culprits: chunks that are too large or too small, no reranking step to re-score the top results, the wrong embedding model, or no metadata filtering so queries search content they should not. Most bad-answer problems are retrieval problems, and most retrieval problems live in chunking, reranking, and hybrid search rather than in the database itself.
How much content do I need before a vector database is worth it?
There is no hard threshold — it depends on whether your AI feature needs to reference your content at all, not on how much you have. Even a few hundred documents benefit from meaning-based retrieval, because keyword search fails on paraphrased questions regardless of corpus size. The practical signal is the use case: if the feature should answer from your material, you want retrieval from the start. With pgvector the setup cost is low enough that there is rarely a reason to wait.
Planning an AI feature that answers from your own content?
Tell us what the AI should know. We will tell you what the retrieval layer needs — and whether pgvector is enough or a dedicated database earns its place.