Client success story · CS32

Substitute products, audited pair by pair

Validating true interchangeability across a Fortune 500 multi-category catalogue

  1. ProblemOperational challenges
  2. SolutionWhat we executed
  3. ImpactBusiness value delivered
  • ClientFortune 500 e-commerce marketplace
  • ScopeSubstitute product pair auditing
  • Catalogue20+ categories, apparel to household goods
  • Team40+ trained auditors
  • 1M+product pairs audited
  • 99%accuracy maintained
  • 20+product categories reviewed

Client background

When an item is out of stock, a marketplace offers the shopper something else. That substitute is a recommendation the retailer is making on its own authority, and when it is wrong — a different size, an incompatible model, a single unit offered in place of a pack — the cost is not a failed sale but a shopper who stops believing the suggestions.

A global e-commerce leader running a catalogue across apparel, electronics, accessories, personal care, toys and household goods asked for its substitute pairings to be checked. StaffCloud audited over a million product pairs and recorded, for each one, whether the second item genuinely replaces the first.

Problem

Operational challenges

The client supplied pairs — an original item and a proposed substitute — and the question for each was whether a customer would accept the second in place of the first. That is a judgement, and judgements made by forty people across twenty categories drift unless the criteria are fixed first.

A catalogue with no single rule

Clothing, electronics, accessories, toys, beauty and pet supplies do not share a test for interchangeability. Apparel has to match on size and colour; electronics on specification; accessories on device compatibility; bundles on unit structure. One rule applied across all of them would be wrong in most.

Differences too small to see

Many pairs looked identical and were not: a slight variation in dimensions, a different compatible device model, another material, a different unit count in the pack. These are the cases where a fast audit produces a confident wrong answer.

Listings that could not be judged

Some products carried too little information or too few images to decide either way. Guessing on those would have put noise into the dataset, so they needed a category of their own rather than a forced call.

Solution

Custom solution executed

  1. 01

    A fixed evaluation framework

    Before auditing began, the decision was reduced to a checklist every auditor applied in the same order: product type, functional purpose, visual similarity, size and dimensions, material and design, technical specifications and compatibility, and unit count and packaging.

    Price, ratings, reviews and availability were deliberately excluded. They move daily and they describe the offer rather than the product — a cheaper item is not thereby a substitute, and an out-of-stock one does not stop being one.

  2. 02

    Reading the listing, not the thumbnail

    Each audit worked from the product page itself: images, title and description, technical specifications, compatibility information, and size and dimension details. The subtle mismatches — the pack of four against the single, the case cut for a different handset — are only visible at that level.

  3. 03

    Three outcomes, and a reason for the middle one

    Every pair was closed as Valid Substitute, Not a Substitute, or Missing Data. Non-substitute decisions carried a reason code naming the attribute that failed — size, colour, compatibility, material or function.

    The reason code is what makes the dataset usable afterwards. A flat rejection tells the client the pairing was wrong; a coded one tells them which rule their matching logic is getting wrong, and how often.

    Screenshot from the client's engagement document, shown under Three outcomes, and a reason for the middle one
    Screenshot from the client's engagement document, shown under Three outcomes, and a reason for the middle one
    Screenshot from the client's engagement document, shown under Three outcomes, and a reason for the middle one
    Screenshot from the client's engagement document, shown under Three outcomes, and a reason for the middle one
    Screenshot from the client's engagement document, shown under Three outcomes, and a reason for the middle one
  4. 04

    Quality control on the auditors, not just the data

    Auditors were trained against the written instructions and worked examples. Random samples were pulled and reviewed a second time, and complex or uncertain pairs were escalated rather than decided alone.

    With forty people on one dataset, consistency between them is the quality measure that matters — forty individually careful auditors applying forty slightly different thresholds would still produce an unusable result.

Impact

Business value delivered

  • Over a million product pairs audited against a single framework
  • 99% accuracy maintained across the dataset
  • Valid, invalid and undecidable pairs separated, with reason codes on every rejection
  • 20+ product categories covered by one consistent set of rules

What it changed for the client

  • Substitute relationships in the catalogue reflect genuine interchangeability rather than surface similarity
  • Shoppers comparing alternatives are shown products that actually replace the one they came for
  • Incorrect pairings removed, reducing the chance of a customer buying an incompatible alternative
  • The framework and workflow are reusable for later catalogue reviews and ongoing data-quality work

In short

Substitution is a recommendation, and a catalogue makes it at the scale of its own size — which is why the accuracy of the rule matters more than the accuracy of any single pair. The work here was less about judging a million products than about fixing one definition of ‘substitute’ and holding forty people to it, then recording why each rejection failed so the client could correct the matching upstream.

Services this story used

Related case studies

Have a version of this problem?

Describe it in a sentence. We will tell you what we would do about it, and whether it is something we should be doing at all.

WhatsApp