Glossary of test automation cost terms

Every term in the calculator's first column, defined once: what it means, why it moves the total, what values are typical for each category of tool, and where you enter it. The methodology shows how they combine.

Build

Total test cases: The number of test cases in the regression suite being costed — the scale every other figure multiplies against.
Test case complexity tiers: Basic, average and complex test cases — three buckets with their own build times, because a login check and a month-end close are not the same job.
Build time per test case: How long it takes one person to author, debug and get one test case passing in a given tool — the single biggest lever on build cost.
Scratch vs migration: Whether a tier's build time describes authoring cases from nothing or migrating ones that already exist in another tool.
Set up, install and configure: The one-off effort to get a tool installed, connected to your applications and environments, and ready for the first test to be written.
Training time per person: How long each person needs to become productive in the tool, multiplied by the number of people trained.
People trained: How many people will be trained on the tool — the multiplier on training time per person.
Scoping the suite: The one-off analysis to decide how many test cases the application needs and which ones — done once, whichever tool builds them.
Validate a built case: The review a newly built case gets before it is trusted: does it test the right thing, and would it fail if the feature broke?
Build effort: The total person-hours to stand the suite up: set-up, training, scoping, authoring every case, and validating it.
Build tokens: The AI model tokens an AI-assisted tool consumes while authoring the suite — a real cost line, separate from people.
Build cost: Build effort at the effective hourly rate, plus build tokens — the one-time cost of standing the suite up, landing in year one.

Run

Runs per year: How many times a year the full regression suite is executed — the multiplier that turns a per-run cost into an annual one.
Defects found per run: How many real bugs a regression cycle typically finds — a property of your product, entered once, and the only "failures" figure you are asked for.
Execution time per case: How long one test case takes to run: machine time for a tool, a tester's time for a manual approach.
Failures per run: Everything a person has to look at after a run: defects found plus tests the product broke plus flaky failures — derived, never typed.
Flaky failures per run: The share of cases that fail for no product reason — timing, environment, brittle selectors — and still have to be investigated.
Triage one failure: The time to look at one failing test and decide what it is: a real defect, a broken test, an environment problem, or a flake.
Prepare and start a run: The per-run overhead before and during execution: environment readiness, test data, kick-off and monitoring.
Write up one defect: The time to reproduce a confirmed defect, log it with steps and evidence, and hand it to development.
Report and sign off: The per-run work of summarising results against the exit criteria and giving the release a go or no-go.
Machine time per run: How long a tool takes to execute the whole suite — duration, deliberately not cost.
Run effort: The person-hours a run costs — preparation, triage, defect write-up and sign-off for a tool; execution itself for manual testing — and the same figure over a year.
Run cost per year: Run effort per year at the effective hourly rate, plus any tokens consumed during execution.
False positives per run: Test failures that are not product defects — tests the product broke plus flaky failures. The model counts both, on separate rows, because they have different owners.
Defect validation: The human work of proving that a failed test is a real product defect and getting it in front of a developer — priced as triage per failure plus write-up per defect.

Maintain

Test breakage rate: The share of the suite that the product breaks in a year — changed screens, renamed fields, altered flows — and so needs repairing.
Repair time per case: How long it takes to fix one test the product broke and get it passing again for the right reason.
Platform upkeep per year: The yearly effort to keep the tool itself working: upgrades, framework maintenance, grid and integration plumbing.
New cases added per year: Suite growth: how many new test cases the application gains each year as it acquires features.
Build a new case: The time to automate one new case once the suite exists — usually the average build time, sometimes less because the scaffolding is in place.
Self-healed cases per year: The share of broken cases an AI tool repairs on its own — which then need a person to confirm the repair was right.
Review one heal: The time to confirm an automatic repair did the right thing before trusting the test again.
Maintenance effort per year: The yearly person-hours to keep the suite alive: repairs, heal reviews, platform upkeep and new cases.
Maintenance cost per year: Maintenance effort at the effective hourly rate, plus any tokens consumed by AI-driven repair.

Licensing & Infra

Cost per licence, per year: What one seat, runner or subscription unit of the tool costs for a year — the number on the quote.
Licences needed: How many licensed units the organisation needs across every application in the workspace.
Infrastructure per year: What it costs to run the tests: execution grid, device cloud, CI minutes, environments — often the hidden line.
Licensing and infrastructure (allocated): This application's share of a vendor's licence and infrastructure cost, apportioned by its share of test cases — so nothing is counted twice.

Total Cost

Annual ongoing cost: What a vendor costs every year after the build: run, maintenance, licensing and infrastructure summed.
Year 1 total: Build cost plus one year of ongoing cost — the first-year budget line.
Analysis horizon: How many years the comparison runs over — five by default — and the total cost of ownership across them.

The model

Effective hourly rate: The loaded cost of an engineer per year divided by their productive hours — the rate every hour of effort is priced at.
Labour cost: Hours of effort multiplied by the effective hourly rate — how every effort figure in the model becomes money.
Token cost: AI model tokens consumed, divided by a million, multiplied by the price per million tokens set in Settings.
FTE capacity: The hours a week the budgeted team can give: FTEs in budget times productive hours, divided by 52.
Build duration: How many weeks the build takes with the budgeted team: build effort across every application divided by weekly FTE capacity.

See these terms at work

Every entry here is a row in the calculator. Open it with your own suite size, release cadence and hourly rate and watch which terms actually move your total.

Open the Calculator