Inside the Matching Engine: How SpareCompare Validates Millions of Spare Parts

"Just automate the matching" sounds simple until you actually try to do it across tens of millions of spare parts records, sourced from different suppliers, in different formats, with different naming conventions, and zero shared identifiers half the time. At that scale, pure automation and pure manual review both fail for the same reason: one is too rigid to handle real-world messiness, the other can't possibly keep pace with the volume.
SpareCompare's matching engine runs against a database of more than 50 million items, and the way it actually handles validation, splitting the work between automation and human review rather than picking one, is worth explaining, because that split is what makes the accuracy claims behind it credible rather than aspirational.
Where the Data Actually Comes From SpareCompare While Validating Spare Parts
Before any matching happens, there's an ingestion layer built around four connected components:
The crawler. This does the scraping and crawling work, pulling in vendor data, item and specification data, pricing data, and PDF files from across the market, continuously rather than as a one-time snapshot.
The databank. Everything the crawler pulls in lands here, structured into a searchable database of vendor data, item and spec data, price data, and source documents.
The data warehouse. This is where matching actually happens. Data from the databank gets matched against a customer's own inventory data, producing three distinct outputs: the original inventory data as submitted, a match report showing exact matches enriched with vendor data, and a validation report showing non-exact matches that need to be confirmed before they're treated as reliable.
CSV, Excel, or PDF scanning. On the customer side, existing inventory data gets identified and ingested from whatever format it currently exists in, spreadsheets, exports, or scanned invoices, feeding directly into the matching process against the databank.

The 80/20 Split, and Why It's Not a Flaw
Once matching runs, roughly 80 percent of records get resolved through automatic validation, and the remaining 20 percent go through assisted validation involving human review. That's not a limitation the system is working around. It's a deliberate design choice based on what each type of match actually requires.
Automatic validation handles the majority of cases and is built around big data and volume-driven matching, synthetic models, and continuous modelling that improves as more data flows through it. This is the right approach for matches where the identifying data is clean and consistent enough that a confident, repeatable rule can resolve it: matching identifiers, consistent specifications, clear one-to-one correspondence between a customer record and a vendor listing.
Assisted validation handles the remaining share and is project-based, user-driven, and applies vertical-specific conventions that vary by industry. This is where a match is plausible but not certain enough to resolve automatically, cases where descriptions are ambiguous, where an industry-specific naming convention needs to be applied, or where the specific context of a customer's operation changes whether two records should actually be treated as equivalent.
The reason this split matters is that forcing everything through automatic validation would produce false matches at a rate that undermines trust in the whole system. And routing everything through manual review would make processing 50 million items completely impractical. The 80/20 split is what makes both speed and accuracy achievable at the same time, rather than trading one for the other.
Why This Design Reflects Real Experience With the Problem
This isn't a theoretical approach. SpareCompare was founded in 2015 by three former FL Smidth employees, industry veterans who had already lived inside the spare parts matching problem from the buyer's side before setting out to solve it. The company ran a proof of concept from 2016 to 2018 specifically in the cement industry, a category with exactly the kind of large, technical, inconsistent parts catalogs that make matching genuinely hard, before moving into broader commercialization from 2019 onward.
That background shows up directly in how the validation split is structured. A team that hadn't actually worked inside industrial parts catalogs might assume matching is a purely technical problem, solvable with enough compute. The 80/20 approach reflects a different, more accurate lesson: most of the matching problem is genuinely automatable, but a meaningful minority of cases require domain-specific judgment that no model should be trusted to make silently.
What This Looks Like From the Customer Side
For a company sending in their own spare parts data, the input side stays simple regardless of what's happening underneath: existing inventory data goes in, as CSV, Excel, or scanned PDF invoices, in whatever state it currently exists.
From there, the output comes back in a structured, usable form, organized by branch, category, and specification, feeding into the tools a customer actually works with day to day, whether that's a dashboard for reviewing match and validation reports or the SpareShop app for field-level search, comparison, and ordering.
Why the 20 Percent Matters More Than It Sounds
It's tempting to read "80 percent automatic" as the headline number and treat the remaining 20 percent as a rounding error. In practice, that 20 percent is often where the highest-value matches live: ambiguous cases involving expensive, critical, or unusual parts- exactly the records where a wrong automatic match would be most costly if it went unchecked. Keeping that slice in human hands isn't a sign the automation is incomplete. It's a sign the system knows where its own confidence runs out, which is a more useful property than blanket automation that doesn't.
The Practical Takeaway
Matching tens of millions of spare parts records well isn't a story about picking automation over manual review, or the reverse. It's a story about knowing which parts of the problem each approach is actually good at, and building a system that routes work accordingly rather than forcing a single method to handle everything. That's a less exciting pitch than "fully automated AI matching," but it's the version that actually holds up against a real catalog with real inconsistencies in it.
SpareCompare's matching engine runs your existing spare parts data against a database of more than 50 million items, automatically resolving the clear matches and flagging the ambiguous ones for review, rather than guessing on either end.



Kommentarer