Test Runner

Execute automated tests, monitor active runs in real-time, and review detailed results with step-by-step logs and performance metrics.

What is the Test Runner?

The Test Runner is AutoRunner's execution engine interface. It handles the full run lifecycle — from starting a test suite to displaying final results. You can monitor active runs in real-time, view step-by-step execution details, and access logs, screenshots, and performance metrics for every completed run.

Starting a Test Run

From the Suite Detail Page

1. Navigate to your project and open a test suite.
2. Click the Run button in the suite toolbar.
3. Review the run configuration dialog:

Setting	Description
Driver	Browser or device driver (Chrome, Firefox, Edge, Safari, Appium)
Environment	Target application environment
Tags	Run only scenarios matching specific tags
Parallel	Enable parallel scenario execution
Retries	Number of retries for failed scenarios

4. Click Start Run to begin execution.

Quick Run from Dashboard

1. Open the Dashboard and click the Quick Run dropdown.
2. Select a project and suite.
3. Click Run Now to start with default configuration.

Run Status Lifecycle

Every test run progresses through a defined status lifecycle:

PENDING → QUEUED → RUNNING → COMPLETED
                           → FAILED
                           → STOPPED
                           → TIMEOUT

Status	Description
Pending	Run created, waiting for engine allocation
Queued	Assigned to engine, waiting in execution queue
Running	Actively executing test scenarios
Completed	All scenarios executed successfully
Failed	One or more scenarios failed during execution
Stopped	Manually stopped by a user
Timeout	Execution exceeded the configured time limit

Status Colors

Each status has a color-coded badge: Pending (gray), Queued (blue), Running (purple), Completed (green), Failed (red), Stopped (orange), Timeout (yellow).

Active Runs Drawer

The Active Runs Drawer shows all currently executing runs across your projects.

To open the drawer:

1. Click the Active Runs indicator in the AutoRunner header.
2. The drawer slides open from the right side.

Each active run entry displays:

• Project and suite name
• Current status and progress bar
• Elapsed execution time
• Pass/fail count so far
• Stop button to halt execution

Live Monitoring

While a run is in Running status, you can monitor it in real-time:

1. Click on an active run to open the run detail page.
2. The page auto-refreshes as scenarios complete.
3. View:
   • Progress bar — Percentage of scenarios completed
   • Live pass/fail count — Updated as each scenario finishes
   • Current scenario — The scenario currently being executed
   • Elapsed time — Total run duration so far

Run Detail Page

After a run completes, click on the run entry to open its detail page. The detail page provides a summary of the run outcome including pass rate, duration, framework, driver, and timestamps.

Key sections available on the run detail page:

• Summary — Pass rate, duration, framework, driver, and start/end timestamps
• Step-by-Step Viewer — Expand any scenario to see individual step results, screenshots, and errors
• Logs — Full execution log with timestamped entries and framework output
• Metrics — Performance data including average step duration and slowest scenario

For comprehensive post-run analysis including step-by-step breakdowns, log filtering, screenshot galleries, and performance metrics, see Run Details.

Parallel Execution

AutoRunner supports running multiple scenarios in parallel to reduce total execution time.

To enable parallel execution:

1. Open the suite Settings panel.
2. Enable Parallel Execution.
3. Set the Max Concurrency value (e.g., 4 scenarios simultaneously).
4. Start a run — scenarios execute concurrently up to the concurrency limit.

Parallel Considerations

Ensure your scenarios are independent and do not share mutable state. Scenarios that depend on a specific execution order should not be run in parallel.

Stopping a Run

To stop an active run:

1. Open the Active Runs drawer or navigate to the run detail page.
2. Click the Stop button.
3. Confirm the stop action in the dialog.

Stopping a run sets its status to Stopped. Completed scenarios retain their results; pending scenarios are marked as skipped.

Runs History

The Runs tab in the project or suite view displays a complete history of all test runs:

• Filter by status, date range, or driver
• Sort by date, duration, or pass rate
• Click any run to view its full detail page
• Compare pass rates across runs to identify trends

Next Steps

• Run Details — Deep-dive into completed run results with step-by-step analysis
• Test Suites — Configure and manage test suites
• Test Runners — Register and manage test execution engines
• AI Studio — Generate test scenarios with AI assistance
• Reports — Generate detailed analysis from your run results
• Scheduling — Automate recurring test runs
• CI/CD Integration — Trigger runs from your deployment pipeline