Catalogue20+ categories, apparel to household goods
Team40+ trained auditors
1M+product pairs audited
99%accuracy maintained
20+product categories reviewed
Client background
When an item is out of stock, a marketplace offers the shopper something else. That substitute is a recommendation the retailer is making on its own authority, and when it is wrong — a different size, an incompatible model, a single unit offered in place of a pack — the cost is not a failed sale but a shopper who stops believing the suggestions.
A global e-commerce leader running a catalogue across apparel, electronics, accessories, personal care, toys and household goods asked for its substitute pairings to be checked. StaffCloud audited over a million product pairs and recorded, for each one, whether the second item genuinely replaces the first.
Problem
Operational challenges
The client supplied pairs — an original item and a proposed substitute — and the question for each was whether a customer would accept the second in place of the first. That is a judgement, and judgements made by forty people across twenty categories drift unless the criteria are fixed first.
A catalogue with no single rule
Clothing, electronics, accessories, toys, beauty and pet supplies do not share a test for interchangeability. Apparel has to match on size and colour; electronics on specification; accessories on device compatibility; bundles on unit structure. One rule applied across all of them would be wrong in most.
Differences too small to see
Many pairs looked identical and were not: a slight variation in dimensions, a different compatible device model, another material, a different unit count in the pack. These are the cases where a fast audit produces a confident wrong answer.
Listings that could not be judged
Some products carried too little information or too few images to decide either way. Guessing on those would have put noise into the dataset, so they needed a category of their own rather than a forced call.
Solution
Custom solution executed
01
A fixed evaluation framework
Before auditing began, the decision was reduced to a checklist every auditor applied in the same order: product type, functional purpose, visual similarity, size and dimensions, material and design, technical specifications and compatibility, and unit count and packaging.
Price, ratings, reviews and availability were deliberately excluded. They move daily and they describe the offer rather than the product — a cheaper item is not thereby a substitute, and an out-of-stock one does not stop being one.
02
Reading the listing, not the thumbnail
Each audit worked from the product page itself: images, title and description, technical specifications, compatibility information, and size and dimension details. The subtle mismatches — the pack of four against the single, the case cut for a different handset — are only visible at that level.
03
Three outcomes, and a reason for the middle one
Every pair was closed as Valid Substitute, Not a Substitute, or Missing Data. Non-substitute decisions carried a reason code naming the attribute that failed — size, colour, compatibility, material or function.
The reason code is what makes the dataset usable afterwards. A flat rejection tells the client the pairing was wrong; a coded one tells them which rule their matching logic is getting wrong, and how often.
Quality control on the auditors, not just the data
Auditors were trained against the written instructions and worked examples. Random samples were pulled and reviewed a second time, and complex or uncertain pairs were escalated rather than decided alone.
With forty people on one dataset, consistency between them is the quality measure that matters — forty individually careful auditors applying forty slightly different thresholds would still produce an unusable result.
Impact
Business value delivered
Over a million product pairs audited against a single framework
99% accuracy maintained across the dataset
Valid, invalid and undecidable pairs separated, with reason codes on every rejection
20+ product categories covered by one consistent set of rules
What it changed for the client
Substitute relationships in the catalogue reflect genuine interchangeability rather than surface similarity
Shoppers comparing alternatives are shown products that actually replace the one they came for
Incorrect pairings removed, reducing the chance of a customer buying an incompatible alternative
The framework and workflow are reusable for later catalogue reviews and ongoing data-quality work
In short
Substitution is a recommendation, and a catalogue makes it at the scale of its own size — which is why the accuracy of the rule matters more than the accuracy of any single pair. The work here was less about judging a million products than about fixing one definition of ‘substitute’ and holding forty people to it, then recording why each rejection failed so the client could correct the matching upstream.