AI-generated editorial illustration of a fictional retail scenario.
A customer starts a return for a broken product. Before the retailer confirms the next step, a personalized recommendation appears for the same category. The product may be relevant to her purchase history, but the timing suggests that service and marketing are working from different versions of the customer.
This is an illustrative scenario, not a claim about a particular retailer. It captures a common design question: when a system can recommend something, what determines whether the company should offer it now?
Personalization discussions often focus on relevance. Teams improve signals, refine segments, and adjust ranking. The conversation becomes more useful when it also covers the conditions that make an action appropriate and deliverable. An open service issue, an unavailable product, or a customer setting can change that decision before any model score matters.
The operating difficulty is that those conditions may sit in different systems. A recommendation process can have purchase history without current return status. A campaign platform can hold a preference that changed elsewhere. The company may own the information while the decision being made does not have a reliable version of it.
Amazon Personalize's documentation offers a concrete technical example. It supports recommendation filters and describes different filter-update timing depending on how data arrives. That is a product-specific behavior, not a universal benchmark. It does make a useful leadership question visible: how does a material change reach the action, and what happens before it gets there?
A cautious backup option should preserve useful service. Withholding an unsolicited promotion while a damaged-item return is open does not require blocking search, a requested replacement, or the customer's access to help. Those are different actions. The rule should be narrow enough that employees and systems can apply it without making the original problem harder to solve.
Measurement needs the same care. A click can tell a team something about response to an offer, but it cannot explain the whole customer or commercial result. Microsoft Research's experimentation guidance separates overall outcomes, diagnostic metrics, checks for harm, and data quality. A personalization review should make those distinctions visible rather than presenting engagement as the entire case for expansion.
For the next review, choose one space used for recommendations and ask the team to show three paths: ordinary conditions, an unresolved service issue, and missing information. Look at the actual resulting action. Then identify who can stop the optional action, what remains available, and what information is needed before restarting it.
That small demonstration can expose a much more important issue than another discussion about model accuracy. The company is responsible for the action it delivers, including the moment when the information is incomplete.
This week's AI, Not KPI examines that decision and includes a practical planning kit for paid members. Read the complete issue on Substack.
