Key takeaways
- Score evidence, not assertions: a claim a vendor cannot demonstrate should score zero.
- Weight risk authority, data portability and licensing scope highest — they are hardest to change later.
- Run the same matrix across every vendor, including the incumbent you are considering leaving.
How to use it
Score each dimension 0–3. 0 means no answer or an assertion with no evidence. 1 means a plausible answer without demonstration. 2 means demonstrated in a controlled demo. 3 means demonstrated against your own scenarios or data.
Weight the dimensions by how expensive they are to change after signing. Anything structural — authority, portability, licensing — should carry roughly double the weight of anything cosmetic.
The matrix
| Dimension | What a 3 looks like | Weight |
|---|---|---|
| Risk authority | One system decides; a breach record shows what fired against what state, reconstructible on demand | ×3 |
| Data portability | Full export including trading and payout history, in a documented format, with a stated timeframe | ×3 |
| Market-data licensing | Explicit scope for display and non-display use, entitlement handling and your reporting duties | ×3 |
| Execution reconstruction | A specific historical fill reproduced with the inputs that produced it | ×2 |
| Operator workflows | Payout review and breach handling demonstrated at realistic volume with real evidence trails | ×2 |
| Customer identity model | One customer joins all accounts, orders, agreements and payouts | ×2 |
| Migration and acceptance | Written acceptance scenarios and a parallel-validation plan before cutover | ×2 |
| Fraud capability | Demonstrated detection of multi-account and coordinated-trading patterns with retained evidence | ×1 |
| Support architecture | Tickets attached to accounts, with escalation and ownership demonstrated | ×1 |
| Trader experience | Charting, order entry and dashboard tested by an actual trader from your program | ×1 |
| Commercial clarity | Quote states inclusions, exclusions, recurring items and what scales | ×1 |
Reading the result
The total matters less than the shape. A vendor scoring well on trader experience and poorly on authority and portability is offering an attractive product with an expensive exit — which may still be the right choice if you know that going in.
Pay particular attention to any dimension where every vendor scores 0 or 1. That usually means the question is not routinely asked in this market, which is precisely why it is worth asking.