A sorting app that is 'roughly right' is a liability on a live store. Here is the verification suite behind SellRank, and why every product we ship has one.
When an app rearranges the products on a store's collection pages, a subtle bug isn't a cosmetic problem — it's revenue silently going to the wrong products, on a page the merchant might not look at for days. So before SellRank had its first customer, it had a verification suite. This post explains what's in it, because we think the approach matters more than the app.
Three claims, three kinds of proof
'Accurate' means three separate things for a sorting product, and each needs its own evidence:
- Ordering correctness — given the numbers, does the engine produce exactly the order the rule describes?
- Metric accuracy — are the numbers it sorts by the real ones? Do orders and revenue match Shopify to the paisa?
- Write-path fidelity — does the computed order actually land on Shopify, position for position?
Ordering correctness: every option, verified
Every ranking option in the product's dropdown — views, add-to-carts, orders, revenue, conversion rate, profit, margin, review rating, sell-through, stock cover, price, newness — is fed a set of products with known values and checked to come out in exactly the promised order. A coverage guard fails the suite if anyone adds an option without a matching proof. The weighted blend gets its own checks: a product better on every metric must rank first, and a stale direction setting can never invert it.
The move planner: 78,240 exhaustive cases
Pushing an order to Shopify means translating 'this sequence' into a list of moves. The planner is verified exhaustively for small collections — every permutation of the live list against every permutation of every target subset, including products the target doesn't know about — and with hundreds of randomised larger cases. That exhaustive sweep is what caught a real bug before launch: positions computed on a filtered list skewed every insert once one unexpected product appeared. It's fixed, and the case that found it now runs on every build.
Live-store truth
Unit tests aren't enough on their own. The audit also runs against a real store: it performs a real sync, reads back every position from Shopify, and confirms the live collection matches the computed order exactly. Metric windows are recomputed independently from raw order data and compared to the app's own numbers.
Why we publish this
Because it's the honest version of a claim. 'Accurate' is easy to say; a suite anyone can re-run is harder to argue with. This is what consistency means at LeelaRam Technologies — the same standard on the first release and the fiftieth, checked by a machine, every time.