Skip to main content
Version: 1.0.4

TestPilot Quick Start Guide

Get started with test execution and orchestration in 10 minutes.

info

Estimated Time: 15-20 minutes This guide walks you through creating your first test run, importing test cases, assigning to team, executing tests, and generating reports.

Prerequisites​

Before you begin, ensure you have:

  • TestPilot access - Ensure you have TestPilot module enabled
  • User permissions - Tester, Test Lead, or Admin role required
  • Test cases ready - Have test cases in CaseWriter or import files prepared
  • Test environment - Access to QA/Staging environment for testing

Step-by-Step Guide​

Step 1: Create a Project​

Duration: 2 minutes

Organize tests by release or sprint.

Step 1: Create a Project

  1. Navigate to TestPilot module
  2. Click "Projects" in sidebar
  3. Click "New Project" button
  4. Enter project name (e.g., "Sprint 23 Testing")
  5. Set start and end dates
  6. Add description
  7. Click "Create"

Step 2: Create a Test Run​

Duration: 3 minutes

Configure test execution campaign.

Step 2: Create a Test Run

  1. Go to "Test Runs" page
  2. Click "New Test Run"
  3. Enter test run name (e.g., "Payment Module Regression")
  4. Select project from dropdown
  5. Choose environment (QA, Staging, etc.)
  6. Select test type (Regression, Smoke, etc.)
  7. Set schedule dates
  8. Click "Create"

Step 3: Import Test Cases​

Duration: 5 minutes

Add tests from CaseWriter or files.

Step 3: Import Test Cases

  1. Open your test run
  2. Click "Add Test Cases"
  3. Choose import method:
    • From CaseWriter repository
    • Import from file (Excel, CSV, JSON)
    • Create new test case
  4. Select tests or test sets
  5. Click "Add to Test Run"
  6. Verify tests added successfully

Step 4: Assign Tests to Team​

Duration: 3 minutes

Distribute workload across testers.

Step 4: Assign Tests to Team

  1. In test run, view test case list
  2. Select tests to assign
  3. Click "Bulk Actions" → "Assign"
  4. Choose assignee from dropdown
  5. Or use "Auto-Assign" for balanced distribution
  6. Enable notification to assignee
  7. Click "Assign"
  8. Team members receive notification

Step 5: Execute & Mark Results​

Duration: 2-60 minutes per test

Run tests and record outcomes.

Step 5: Execute and Mark Results

  1. Navigate to "Test Executions" page
  2. Select your project and test run
  3. Open a test case (click row)
  4. Read test steps and expected results
  5. Execute test steps
  6. Mark each step (Pass/Fail/Blocked)
  7. Set overall test status
  8. Add notes and evidence (screenshots)
  9. Log defect if test failed
  10. Save and move to next test

Step 6: Monitor & Report​

Duration: 5 minutes

Track progress and generate reports.

Step 6: Monitor and Report

  1. View Dashboard for live metrics
  2. Check pass/fail rates
  3. Monitor execution progress
  4. Review blocked tests
  5. Navigate to "Reports" page
  6. Select project and test run
  7. Choose report type (Executive, Detailed, etc.)
  8. Export to PDF or Excel
  9. Share with stakeholders

Common Workflows​

Sprint Testing Workflow​

End-to-end sprint regression testing:

  1. Create project: "Sprint 23"
  2. Create test run: "Sprint 23 Regression"
  3. Import test cases from CaseWriter
  4. Assign tests to team (round-robin)
  5. Execute tests over 1 week
  6. Log defects in real-time
  7. Retest fixed bugs
  8. Generate final report
  9. Present in retrospective

Smoke Testing Workflow​

Quick validation of critical paths:

  1. Create test run: "Daily Smoke Tests"
  2. Import 10-15 critical tests
  3. Assign to on-duty tester
  4. Execute in 30-60 minutes
  5. Report any failures immediately
  6. Block release if smoke fails

Hotfix Validation Workflow​

Rapid testing of production fixes:

  1. Create project: "Hotfix 2.2.1"
  2. Create test run: "Login Fix Validation"
  3. Add related test cases
  4. Execute impacted tests
  5. Verify bug fix
  6. Run smoke tests
  7. Approve deployment

Tips for Success​

Start Small​

Begin with 10-20 tests to familiarize yourself with the platform before scaling up.

Use Test Templates​

Create reusable test run templates for recurring testing cycles (sprints, releases).

Leverage Automation​

Integrate with AutoRunner to automate repetitive tests and focus manual efforts on exploration.

Collaborate Early​

Involve the team in test planning. Balanced assignments and clear communication drive success.

Troubleshooting​

IssueSolution
Cannot create test runEnsure you have created a project first. Test runs must belong to a project.
Tests not importingCheck file format (Excel, CSV, JSON supported). Verify column mapping matches TestPilot fields.
Cannot update test statusVerify test run is "In Progress" status. Completed or cancelled runs are read-only.
Reports not generatingEnsure at least one test is executed. Empty test runs cannot generate reports.

Next Steps​

Need Help?​

If you encounter any issues, check out the TestPilot documentation or contact your organization's RabbitQA administrator for assistance.