CI/CD Integration

Start test runs from a pipeline, poll for the outcome, and keep a repository source in step with your branch.

Overview

There is no dedicated pipeline plugin. A pipeline integrates with AutoRunner in one of three ways:

Approach	Use it for
AutoRunner API	Starting a run from a pipeline step and waiting for its result.
Repository sync	Pulling the latest scenarios from a connected source before a run.
Schedules	Recurring runs that do not need to be tied to a build.

API Base URL

AutoRunner requests go to the RabbitQA API host under the /autorunner/api/v1 gateway prefix.

https://api.rabbitqa.com/autorunner/api/v1

uyarı

app.rabbitqa.com is the web application host, not the API host. Requests sent there will not reach the AutoRunner API.

Authenticate every request with a bearer token as described in API Authentication. Send the workspace you are targeting in the X-Workspace-Id header so the request is scoped to the right workspace.

Each endpoint below requires a permission:

Permission	Grants
autorunner:run:read	Reading runs, results, summaries, and exports
autorunner:run:create	Starting runs
autorunner:run:update	Stopping runs

Starting a Run

Start one test plan:

curl -X POST https://api.rabbitqa.com/autorunner/api/v1/test-runs/start \
  -H "Authorization: Bearer $RABBITQA_TOKEN" \
  -H "X-Workspace-Id: $WORKSPACE_ID" \
  -H "Content-Type: application/json" \
  -d '{
    "testSuiteId": 42,
    "runProfileId": 7,
    "environmentProfileId": 3
  }'

testSuiteId is the ID of the test plan and is the only required field.

Field	Type	Description
testSuiteId	number	The test plan to run. Required.
scenarioIds	number[]	Run only these scenarios from the plan.
runProfileId	number	Run profile to execute with. Falls back to the plan or workspace default.
environmentProfileId	number	Environment profile to resolve variables from.
selectedJiraIssue	string	Jira issue key to associate with the run.
mode	string	CREATE_NEW_TEST_RUN or OVERWRITE_EXISTING.
maxRetryCount	number	Retry attempts for the run.
retryOnlyFailed	boolean	Retry failed scenarios only.

The response is 201 Created:

{
  "runGroupId": "5b1f...",
  "runIds": [1841]
}

A plan that targets several execution targets produces more than one run ID in the same run group.

Starting Several Plans

curl -X POST https://api.rabbitqa.com/autorunner/api/v1/test-runs/start/bulk \
  -H "Authorization: Bearer $RABBITQA_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{ "testSuiteIds": [42, 43], "runProfileId": 7 }'

Stopping a Run

curl -X POST https://api.rabbitqa.com/autorunner/api/v1/test-runs/stop \
  -H "Authorization: Bearer $RABBITQA_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{ "ids": [1841] }'

Reading the Result

Endpoint	Returns
GET /test-runs/{id}	Full run detail including per-scenario results
GET /test-runs/{id}/lite	The same run without step logs and scenario histories — intended for polling
GET /test-runs/{id}/results	The run's scenario results
GET /test-runs/summary	Aggregated counts, filtered by suit-ids or project-ids
GET /test-runs/status	The valid status and result values
GET /test-runs/export	CSV of runs, filtered and capped at 5000 records

Poll GET /test-runs/{id}/lite and read status:

status	Meaning
WAITING	Accepted, waiting for an engine
PENDING	Preparing to execute
RUNNING	Executing
FINISHED	Terminal — read result for the outcome
ABORTED	Terminal — stopped manually or cancelled by the system

When status is FINISHED, result carries the outcome:

result	Meaning
SUCCESS	Every scenario passed
FAILURE	At least one scenario failed
ERROR	At least one scenario ended in an error
PARTIAL	The run completed without covering every scenario

Useful fields on the same response:

Field	Description
passed, fail, skip, error, untested, total	Scenario counts
passRate	Pass rate as a percentage
duration	Execution time in milliseconds
qualityGateThreshold, qualityGatePassed	The workspace quality gate threshold and whether the run met it
triggeredBy	MANUAL, SCHEDULE, CI_CD, or WEBHOOK
runProfileName, environment	The profile and environment used

Quality gates are reported, not enforced

qualityGatePassed tells you whether the run met the configured pass-rate threshold. AutoRunner does not block a merge, a deployment, or another run when a gate fails. If you want a failing gate to fail your build, read the field in your pipeline and exit non-zero yourself.

Pipeline Examples

Both examples start a plan, poll until the run reaches a terminal status, and fail the job on anything other than SUCCESS.

GitHub Actions

name: AutoRunner

on:
  push:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    env:
      API: https://api.rabbitqa.com/autorunner/api/v1
    steps:
      - name: Start run
        id: start
        run: |
          RUN_ID=$(curl -sf -X POST "$API/test-runs/start" \
            -H "Authorization: Bearer ${{ secrets.RABBITQA_TOKEN }}" \
            -H "X-Workspace-Id: ${{ vars.WORKSPACE_ID }}" \
            -H "Content-Type: application/json" \
            -d '{"testSuiteId": ${{ vars.PLAN_ID }}}' | jq -r '.runIds[0]')
          echo "run_id=$RUN_ID" >> $GITHUB_OUTPUT

      - name: Wait for the run
        run: |
          for _ in $(seq 1 120); do
            BODY=$(curl -sf "$API/test-runs/${{ steps.start.outputs.run_id }}/lite" \
              -H "Authorization: Bearer ${{ secrets.RABBITQA_TOKEN }}")
            STATUS=$(echo "$BODY" | jq -r '.status')
            if [ "$STATUS" = "FINISHED" ] || [ "$STATUS" = "ABORTED" ]; then
              echo "$BODY" | jq '{status, result, passRate, passed, fail}'
              [ "$(echo "$BODY" | jq -r '.result')" = "SUCCESS" ] || exit 1
              exit 0
            fi
            sleep 15
          done
          echo "Timed out waiting for the run" && exit 1

GitLab CI

autorunner:
  stage: test
  image: alpine:latest
  variables:
    API: "https://api.rabbitqa.com/autorunner/api/v1"
  before_script:
    - apk add --no-cache curl jq
  script:
    - |
      RUN_ID=$(curl -sf -X POST "$API/test-runs/start" \
        -H "Authorization: Bearer $RABBITQA_TOKEN" \
        -H "X-Workspace-Id: $WORKSPACE_ID" \
        -H "Content-Type: application/json" \
        -d "{\"testSuiteId\": $PLAN_ID}" | jq -r '.runIds[0]')
      echo "Started run $RUN_ID"
    - |
      for _ in $(seq 1 120); do
        BODY=$(curl -sf "$API/test-runs/$RUN_ID/lite" -H "Authorization: Bearer $RABBITQA_TOKEN")
        STATUS=$(echo "$BODY" | jq -r '.status')
        case "$STATUS" in
          FINISHED|ABORTED)
            echo "$BODY" | jq '{status, result, passRate}'
            test "$(echo "$BODY" | jq -r '.result')" = "SUCCESS" || exit 1
            exit 0 ;;
        esac
        sleep 15
      done
      echo "Timed out waiting for the run"; exit 1

Keeping a Repository Source in Sync

A repository-backed source holds the scenarios your plan runs. Sync it so a run executes the current branch content.

• Sync manually from Sources with Sync Source, or from Settings > Source.
• The sync history records each sync with its trigger — Manual, Scheduled, or Webhook — and its outcome.

See Sources for connecting and syncing a source.

Rate Limits

Run-start and bulk-start requests are rate limited per user and company; CSV export is rate limited per user. Retry with a backoff rather than tightening your polling interval.

Next Steps

• Test Plans — Find the plan ID your pipeline should run
• Settings — Run profiles, environments, and the quality gate threshold
• Schedules — Recurring runs without a pipeline
• Run Detail — Investigate a failing pipeline run