Skip to main content
Version: Next

Test Runs

A test run is a set of test cases executed together against one environment and one or more platforms. Every run belongs to a test plan.

TestRuns

The Test Runs page​

Open TestPilot > Test Runs. By default the page lists every run across all plans. Selecting a plan, or arriving on a ?planId=<id> link, scopes the list to that plan.

Toolbar​

ControlOptions
SearchFree text over run names
PlanAll Plans, or one plan
PlatformAll Platforms, Web, Mobile, API, Desktop
EnvironmentAll Environments, or one environment
Sort byNewest first, Oldest first, Name (A-Z), Name (Z-A)

Below the toolbar, pills filter by Status (Planned, Active, Completed, Overdue) and Type (Regression, Smoke, Manual, Automation).

Every active filter appears as a removable chip, including the selected plan. Clear All resets the filters and drops planId from the address bar.

Summary bar​

FigureMeaning
totalRuns matching the current filters
passedRuns that finished with no failures
abortedRuns that were cancelled or stopped
pass rateShare of non-aborted runs that passed
aborted rateShare of loaded runs that were aborted
completion rateShare of loaded runs that reached completion

Columns​

ColumnContents
Test RunsRun name and status
Run ProfileType, platforms, and environment
ResultsResult counters and pass rate
TimelineStart and end dates

Click a row to open the run's executions. Runs are paged; use the arrows below the table to move between pages.

Run statuses​

StatusMeaning
PlannedCreated, not yet started
In ProgressCurrently being executed
CompletedFinished
CancelledStopped before completion
OverduePast its end date without completing

Creating a run​

New Test Run behaves differently depending on your current scope:

  • With a plan selected, the run is created immediately from that plan's own settings. It is named <plan name> - <today's date>, takes the plan's platform, environment, and priority, and uses the plan's dates or a one-week window if the plan has none.
  • With All Plans selected, the full run form opens and you must choose a plan.

Run form​

FieldNotes
Test PlanRequired when creating; locked when editing, so a run cannot be moved between plans here
Test Run NameRequired
TypeRegression, Smoke, Manual, Automation
PlatformMulti-select: WEB, MOBILE, API, DESKTOP
EnvironmentDEVELOPMENT, TEST, STAGING, PRODUCTION
PriorityCritical, High, Medium, Low
Start DateRequired
End DateRequired, and must be after the start date
Test CasesDual-panel picker for the run's test cases

Choosing a plan seeds the case selection with the plan's own test cases, which you can then add to or trim.

Multi-platform runs​

A run carries a list of platforms rather than a single one, so one run can cover, for example, both Web and Mobile.

The test case picker filters the repository by the platforms you selected, and says so above the list. Test cases that carry no platform information are still shown, so nothing is hidden by the filter.

note

Test plans also take a platform list, but a run created directly from a plan takes the plan's platform as a single-item list. Edit the run to add more platforms.

Run actions​

Each row has an action menu:

ActionResult
Copy URLCopies a direct link to the run's executions
EditReopens the run form
CloneCopies the run under a new name
Manage Test CasesAdds or removes the run's test cases
Delete Test RunPermanently deletes the run

From inside a plan's expanded run list you also get View Executions and Remove from Plan. See Test Plans.

Cloning a run​

Clone asks for a new name and offers four options:

  • Include execution attachments
  • Include test execution status, which keeps the current results
  • Include assignees
  • Include linked Jira issues

Leave them all clear to get a fresh copy of the run's test cases with no results.

Permissions​

ActionPermission key
Create or clone a runtestpilot:testset:create
Edit a run or manage its test casestestpilot:testset:update
Delete a runtestpilot:testset:delete

Actions you do not hold the permission for are hidden.

Next steps​