Insights · 11 min read
A practical guide to building a recommendation engine: the behavior data you need, the architecture that scales, real costs, and how to avoid a system nobody uses.
By GGP Editorial
Most founders think a recommendation engine is the "customers also bought" box on a product page. That box is the visible five percent. The other ninety-five percent is the data pipeline that captures what people actually do, the model that turns that behavior into a ranked list, and the serving layer that fills the box in under a hundred milliseconds without melting your database.
I write this from the vendor side. GlobeSoft is a software development company founded in 2018, with 40-plus engineers and more than 300 delivered projects across the US, Brazil, South Africa, and Singapore. We have built recommendation and personalization features inside e-commerce platforms, marketplaces, content apps, and learning products. This guide walks through what the build actually involves, the data it depends on, and what it costs. If you are sizing the broader AI spend, start with how much AI development costs.
A recommendation engine ranks a list of items for a specific user at a specific moment. It is not search, even though the two get confused. Search answers a question the user typed. Recommendations answer a question nobody typed: what should this user see next. That difference changes the data, the model, and the way you measure success.
There are three ways to build the ranking, and most production systems mix them.
Collaborative filtering looks at what similar users liked. If people who bought a certain set of products also bought a fourth, that fourth product gets recommended. It needs no product metadata, which is why it works well for long-tail catalogs. It does need a lot of user behavior before the signal appears, which is the cold start problem.
Content-based filtering looks at the items themselves. It compares a product's attributes to what a user has engaged with before. It works from day one with a single user, but it only recommends things similar to what the user already liked, which tends toward boring, samey results.
Hybrid systems combine the two, scoring items from both methods and blending the scores. Most serious platforms are hybrid in practice.
Over the last few years a fourth approach has become the default for teams with enough data: embedding-based retrieval. You convert users and items into vectors in the same space, store the item vectors in a vector database, and find the nearest neighbors to the user's vector. It handles large catalogs well and it generalizes, but it needs real engineering around training, freshness, and evaluation. The surrounding pattern appears in RAG vs fine-tuning, which uses embeddings for a different job but shares the retrieval mechanics.
A recommendation engine is only as good as the events you capture. If you log only purchases, you are blind to everything a user does before they buy, which is most of the signal. You want views, add-to-carts, purchases, ratings, searches, and session time at a minimum. Each event needs a user ID, an item ID, and a timestamp, in a form the model can consume without a data team cleaning it every week.
The cold start problem is the part founders underestimate. A new user with no history and a new item with no interactions have nothing to model. You handle the new-user case with popularity-based fallbacks and, where you can, with onboarding signals such as category preferences. You handle the new-item case with content metadata and by showing the item to a small group to gather early signal. Neither is optional. An engine that only works for your oldest power users is not much of an engine.
There is also the question of whose data you are allowed to use. Behavioral data about users is personal data, and if you serve Europe you carry GDPR obligations around consent, purpose, and the right to erasure. In practice that means your pipeline has to be able to delete a user's history on request. Treat that as part of the data design from the start, not a note you add at the end.
The pattern that scales is a two-stage system: retrieval first, ranking second.
Retrieval pulls a few hundred candidate items from the catalog quickly. You do this with vector search over item embeddings, or with a simpler candidate generator such as "also bought" co-occurrence counts. The goal is recall: get the right items into the candidate pool without scanning the whole catalog.
Ranking takes those few hundred candidates and orders them precisely. This is where a model with richer features lives, and where business rules apply. Never show out-of-stock items. Cap how many items come from one brand. Boost promoted content. Demote items the user already bought.
Serving has to be fast. The recommendation box renders with the page, so retrieval plus ranking has to finish in tens to low hundreds of milliseconds. That means precomputing what you can. Precompute item embeddings and even per-user candidate pools in batch, then refine at request time. A system that recomputes everything live will be slow, expensive, or both.
Evaluation is where most teams go wrong. Offline metrics such as precision or NDCG tell you the model looks reasonable on historical data. They do not tell you whether people actually click or buy more. The only signal that matters is an online test: split users, serve the new model to half, and measure clicks, conversion, and revenue per session. Plan the A/B testing infrastructure as part of the build, not as a follow-up. A recommendation engine you cannot A/B test is a black box you will never trust.
Cost tracks scope more than anything else. Here is a realistic split.
| Scope | What it includes | Typical range |
|---|---|---|
| Basic engine | Collaborative or content-based ranking over a few hundred thousand items, batch updates, simple "also bought" widgets | $25,000 to $60,000 |
| Full personalization | Embedding-based retrieval, a vector database, real-time event capture, reranking with business rules, A/B testing | $60,000 to $150,000 |
| Large-scale personalization | Millions of items, real-time serving under load, recommendations on several surfaces, a data platform around it | $150,000 to $400,000+ |
These are ranges, not quotes, and they move with your catalog size, your traffic, and how fresh the recommendations have to be. The number that matters is the one from a scoped proposal, which is what our guide to estimating a custom software project walks through.
Three things push the bill up most. The first is real-time behavior. Batch recommendations that update nightly are cheap. Recommendations that react to what a user did thirty seconds ago need an event pipeline and a serving layer built for it. The second is catalog size and diversity. A million-item catalog with sparse interaction data is a different problem from a ten-thousand-item one. The third is team and location. A recommendation engine is a data-and-backend-heavy build: you need a backend engineer, a data or ML engineer, and a frontend engineer for the widgets and the A/B tooling. Where that team sits changes the bill a lot, which we break down in software development cost in China.
You have three realistic options.
Off-the-shelf personalization APIs and recommendation services get you a working widget in days. You send events, they return ranked items, you pay per request or per month. The tradeoff is that you are limited to their models and their idea of how recommendations should work, and you send your behavioral data to a third party. This is the right starting point for most e-commerce sites that want to test whether personalization lifts conversion before committing to a build.
Buying a commercial recommendation engine that you self-host gives you more control, but you are still adopting someone else's architecture and paying a license, and you still need engineers to run it.
Building custom is the right call when recommendations are core to the product, or when your catalog, your data, or your business rules do not fit an off-the-shelf option. A marketplace where sellers pay for promoted placement, or a content platform where the feed is the product, usually ends up needing control over the ranking that a vendor will not hand over. If you are weighing build versus buy for an AI feature generally, build vs buy AI software covers the same tradeoff in more depth.
Building the model before the data. The model is the easy part. The event capture, the cleaning, and the catalog pipeline are where projects stall. If you have no clean behavior data, the best model in the world recommends nothing useful.
Skipping the A/B test. A recommendation engine you cannot measure is a decoration. Wire up online testing before launch and let the numbers decide whether the engine ships.
Ignoring cold start. New users and new items are the majority on most platforms. If your engine only works for established users with long histories, it is not doing its job for the people you are trying to convert.
Treating recommendations as a one-time build. Catalogs change and user behavior drifts. A model that was good last quarter degrades. Plan for retraining, freshness, and monitoring as an ongoing cost, not a launch detail.
Keeping the logic opaque. Whatever you build or buy, you want visibility into why an item was recommended. Business rules, such as never showing out-of-stock items or respecting a merchant's exclusions, have to live somewhere you can change them without an engineering ticket.
A basic collaborative or content-based engine over a few hundred thousand items runs roughly $25,000 to $60,000. Full embedding-based personalization with real-time events and A/B testing runs $60,000 to $150,000, and large-scale systems with millions of items and heavy traffic run $150,000 to $400,000 or more. These are ranges, not quotes.
Behavioral events with a user ID, an item ID, and a timestamp, covering views, add-to-carts, purchases, ratings, searches, and session time. Item metadata helps with content-based filtering and cold start. Without clean behavior data, the engine has nothing to learn from.
A basic engine runs six to twelve weeks. A full personalization system with embedding retrieval, real-time events, and A/B testing runs three to six months, and large-scale systems run six months or more. The data pipeline work is usually the longest single part.
Yes, and it is a sensible first step for many e-commerce and content sites. You get a working widget in days and pay per request or per month. The tradeoff is less control over the ranking and sharing behavioral data with a third party.
A new user with no history and a new item with no interactions have nothing for the model to learn from. You handle new users with popularity fallbacks and onboarding signals, and new items with content metadata and early exposure to gather signal.
Not for a basic engine, which a strong backend team can ship. Embedding-based and real-time systems usually benefit from an ML engineer or data scientist on the team, or a vendor with that capability.
GlobeSoft is a software development company founded in 2018, with 40-plus engineers and more than 300 delivered projects for over 100 clients across the US, Brazil, South Africa, and Singapore. We build custom software, mobile apps, SaaS platforms, FinTech and payment systems, ERP and CRM, and AI products on a stack that includes Java, Spring Boot, Go, Python, React, Vue, and Node. If your platform already has an e-commerce or marketplace foundation, see how much it costs to build an e-commerce platform and how to build a multi-vendor marketplace.
If you are planning to add recommendations or personalization to your platform, talk to us about your project. Send us your catalog size, the behavior data you already capture, and the surface you want recommendations on, and we will help you work through the data pipeline, the architecture, and a realistic scope.
Tell us what you are building and where you are today. We typically reply within 24 hours.
