Maintain

Test breakage rate

The share of the suite that the product breaks in a year — changed screens, renamed fields, altered flows — and so needs repairing.

Also called: test maintenance rate · repair rate · false positives from product change

Unit: % of cases per year
Where you enter it: Maintain tab, per vendor.

What it means

When the application changes, some tests stop working through no fault of their own: an element moved, a step was added, a data shape changed. The breakage rate is the proportion of cases this happens to over a year. Multiplied by the total cases it gives the cases to repair; spread over the year's runs it gives the tests broken per run that show up as failures.

It is per vendor because tools differ in how resilient their cases are — self-healing tools claim a much lower rate — and it is the same number a vendor is really quoting when they claim "low maintenance".

Why it matters to the cost

Maintenance is where automation programmes die, and this rate is its engine. On 300 cases, 30% a year is 90 repairs; at 45 minutes each that is 68 hours, every year — and those 90 broken tests also cost triage on the run that found them. Reducing the rate from 30% to 10% is the single largest maintenance saving a tool can offer, which is why it deserves a proof of concept across a real application change.

Typical values

Script-based tools around 30%; homegrown frameworks around 25%; AI-native tools with self-healing around 10%; manual cases about 15% (a document edit, not a repair).

Typical values describe a category of tool — AI-native, codeless, script-based, homegrown, manual — never a named product. Why we do not price named products.