What does accuracy mean for a system like this?
In Vantage, accuracy means: the percentage of records where the system’s tier assignment matches the tier the principal would assign to the same record.
This is the only metric that matters:
- Not the number of records scored
- Not the number of corrections submitted
- Not the processing speed
Whether the system agrees with you, on the records that matter, is the measure.
Fulcrum uses this definition explicitly in the kill criteria:
- If the system’s tier assignment does not match the principal’s judgment on at least 50% of the records scored in Batch 1, the architecture is reviewed.
- If accuracy does not reach 60% by Batch 2, the engagement stops.
---
Related: How accuracy improves over time · The correction cadence · How to know if Vantage is working