Version: 1.0.2

Test Event Integration

Report test execution events from your automation framework to see live results in the Dashboard.

Overview

Device Farm provides a REST API for reporting test execution events. When you integrate your test framework with this API, you get:

Live Progress - See test results in real-time as they execute
Historical Analysis - All results are stored for reporting and trends
WebSocket Updates - Dashboard receives instant updates via WebSocket

API Reference

Endpoint

POST /api/automation/sessions/{sessionId}/test-events

Authentication

Include your API key in the X-API-Key header. This is the same key used for Appium/WebDriver endpoints.

X-API-Key: df_auto_xxxxx

Event Types

Type	When to Send	Required Fields
`TEST_STARTED`	When a test begins	`testName`
`TEST_PASSED`	When a test passes	`testName`, `duration`
`TEST_FAILED`	When a test fails	`testName`, `duration`, `error`
`TEST_SKIPPED`	When a test is skipped	`testName`

Request Body

{
  "eventId": "uuid-v4",
  "type": "TEST_PASSED",
  "testName": "testLogin",
  "suiteName": "LoginTests",
  "duration": 1250,
  "error": {
    "message": "Assertion failed",
    "stackTrace": "..."
  }
}

Integration Examples

Java (TestNG)

import io.appium.java_client.AppiumDriver;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.net.URI;
import org.testng.ITestListener;
import org.testng.ITestResult;

public class DeviceFarmReporter implements ITestListener {
    private final String baseUrl;
    private final String sessionId;
    private final String apiKey;
    private final HttpClient client = HttpClient.newHttpClient();

    public DeviceFarmReporter(String baseUrl, String sessionId, String apiKey) {
        this.baseUrl = baseUrl;
        this.sessionId = sessionId;
        this.apiKey = apiKey;
    }

    @Override
    public void onTestStart(ITestResult result) {
        sendEvent("TEST_STARTED", result.getMethod().getMethodName(),
            result.getTestClass().getName(), null, null);
    }

    @Override
    public void onTestSuccess(ITestResult result) {
        long duration = result.getEndMillis() - result.getStartMillis();
        sendEvent("TEST_PASSED", result.getMethod().getMethodName(),
            result.getTestClass().getName(), duration, null);
    }

    @Override
    public void onTestFailure(ITestResult result) {
        long duration = result.getEndMillis() - result.getStartMillis();
        String error = result.getThrowable().getMessage();
        sendEvent("TEST_FAILED", result.getMethod().getMethodName(),
            result.getTestClass().getName(), duration, error);
    }

    private void sendEvent(String type, String testName, String suiteName,
                          Long duration, String error) {
        try {
            String json = String.format(
                "{\"type\":\"%s\",\"testName\":\"%s\",\"suiteName\":\"%s\"%s%s}",
                type, testName, suiteName,
                duration != null ? ",\"duration\":" + duration : "",
                error != null ? ",\"error\":{\"message\":\"" + error + "\"}" : ""
            );

            HttpRequest request = HttpRequest.newBuilder()
                .uri(URI.create(baseUrl + "/api/automation/sessions/" + sessionId + "/test-events"))
                .header("X-API-Key", apiKey)
                .header("Content-Type", "application/json")
                .POST(HttpRequest.BodyPublishers.ofString(json))
                .build();

            client.sendAsync(request, HttpResponse.BodyHandlers.ofString());
        } catch (Exception e) {
            System.err.println("Failed to send test event: " + e.getMessage());
        }
    }
}

Python (pytest)

import requests
from datetime import datetime

class DeviceFarmReporter:
    """pytest plugin for reporting test events to Device Farm"""

    def __init__(self, base_url: str, session_id: str, api_key: str):
        self.base_url = base_url
        self.session_id = session_id
        self.api_key = api_key
        self.test_start_times = {}

    def _send_event(self, event: dict):
        try:
            requests.post(
                f"{self.base_url}/api/automation/sessions/{self.session_id}/test-events",
                headers={
                    "X-API-Key": self.api_key,
                    "Content-Type": "application/json"
                },
                json=event,
                timeout=5
            )
        except Exception as e:
            print(f"Failed to send test event: {e}")

    def pytest_runtest_setup(self, item):
        self.test_start_times[item.nodeid] = datetime.now()
        self._send_event({
            "type": "TEST_STARTED",
            "testName": item.name,
            "suiteName": item.module.__name__ if item.module else None
        })

    def pytest_runtest_makereport(self, item, call):
        if call.when != "call":
            return

        start_time = self.test_start_times.get(item.nodeid)
        duration = int((datetime.now() - start_time).total_seconds() * 1000) if start_time else None

        if call.excinfo is None:
            event = {
                "type": "TEST_PASSED",
                "testName": item.name,
                "suiteName": item.module.__name__ if item.module else None,
                "duration": duration
            }
        elif call.excinfo.typename == "Skipped":
            event = {
                "type": "TEST_SKIPPED",
                "testName": item.name,
                "suiteName": item.module.__name__ if item.module else None
            }
        else:
            event = {
                "type": "TEST_FAILED",
                "testName": item.name,
                "suiteName": item.module.__name__ if item.module else None,
                "duration": duration,
                "error": {
                    "message": str(call.excinfo.value),
                    "stackTrace": str(call.excinfo.getrepr())
                }
            }

        self._send_event(event)


# Usage in conftest.py:
def pytest_configure(config):
    reporter = DeviceFarmReporter(
        base_url="https://api.rabbitqa.com",
        session_id=config.getoption("--session-id"),
        api_key=config.getoption("--api-key")
    )
    config.pluginmanager.register(reporter, "device_farm_reporter")

JavaScript (Mocha)

const axios = require('axios');

class DeviceFarmReporter {
  constructor(runner, options) {
    const { baseUrl, sessionId, apiKey } = options.reporterOptions;
    this.baseUrl = baseUrl;
    this.sessionId = sessionId;
    this.apiKey = apiKey;
    this.testStartTimes = new Map();

    runner.on('test', (test) => this.onTestStart(test));
    runner.on('pass', (test) => this.onTestPass(test));
    runner.on('fail', (test, err) => this.onTestFail(test, err));
    runner.on('pending', (test) => this.onTestSkipped(test));
  }

  async sendEvent(event) {
    try {
      await axios.post(
        `${this.baseUrl}/api/automation/sessions/${this.sessionId}/test-events`,
        event,
        {
          headers: {
            'X-API-Key': this.apiKey,
            'Content-Type': 'application/json',
          },
          timeout: 5000,
        },
      );
    } catch (error) {
      console.error('Failed to send test event:', error.message);
    }
  }

  onTestStart(test) {
    this.testStartTimes.set(test.fullTitle(), Date.now());
    this.sendEvent({
      type: 'TEST_STARTED',
      testName: test.title,
      suiteName: test.parent?.title || null,
    });
  }

  onTestPass(test) {
    const startTime = this.testStartTimes.get(test.fullTitle());
    const duration = startTime ? Date.now() - startTime : test.duration;
    this.sendEvent({
      type: 'TEST_PASSED',
      testName: test.title,
      suiteName: test.parent?.title || null,
      duration,
    });
  }

  onTestFail(test, err) {
    const startTime = this.testStartTimes.get(test.fullTitle());
    const duration = startTime ? Date.now() - startTime : test.duration;
    this.sendEvent({
      type: 'TEST_FAILED',
      testName: test.title,
      suiteName: test.parent?.title || null,
      duration,
      error: {
        message: err.message,
        stackTrace: err.stack,
      },
    });
  }

  onTestSkipped(test) {
    this.sendEvent({
      type: 'TEST_SKIPPED',
      testName: test.title,
      suiteName: test.parent?.title || null,
    });
  }
}

module.exports = DeviceFarmReporter;

cURL Examples

# Report TEST_STARTED event
curl -X POST "https://api.rabbitqa.com/api/automation/sessions/{sessionId}/test-events" \
  -H "X-API-Key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "type": "TEST_STARTED",
    "testName": "testLogin",
    "suiteName": "LoginTests"
  }'

# Report TEST_PASSED event
curl -X POST "https://api.rabbitqa.com/api/automation/sessions/{sessionId}/test-events" \
  -H "X-API-Key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "type": "TEST_PASSED",
    "testName": "testLogin",
    "suiteName": "LoginTests",
    "duration": 1250
  }'

# Report TEST_FAILED event with error
curl -X POST "https://api.rabbitqa.com/api/automation/sessions/{sessionId}/test-events" \
  -H "X-API-Key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "type": "TEST_FAILED",
    "testName": "testPayment",
    "suiteName": "PaymentTests",
    "duration": 3500,
    "error": {
      "message": "Expected status 200 but got 500",
      "stackTrace": "at PaymentTest.testPayment(PaymentTest.java:42)..."
    }
  }'

Best Practices

tip

Use eventId for Idempotency If your test framework might retry requests, include a client-generated UUID. Duplicate events with the same ID are ignored.

tip

Always Send TEST_STARTED Send TEST_STARTED before each test runs. This enables real-time progress tracking and accurate duration calculation.

tip

Include suiteName Group tests by suite/class for better organization in the Dashboard. Use the full class name for clear categorization.

tip

Include Error Details For failed tests, include both error message and stack trace. This helps with debugging directly from the Dashboard.

Troubleshooting

Issue	Solution
Events not appearing	Check session ID and API key. Ensure session is active or within 5-min grace period.
401 Unauthorized	Invalid or missing API key. Check X-API-Key header.
422 Not Acceptable	Session is not an automated session or has ended beyond grace period.
Missing duration	Send TEST_STARTED event before completion events for accurate timing.

Overview​

API Reference​

Endpoint​

Authentication​

Event Types​

Request Body​

Integration Examples​

Java (TestNG)​

Python (pytest)​

JavaScript (Mocha)​

cURL Examples​

Best Practices​

Troubleshooting​