Bring Your Test Framework
Push results from Playwright, Cypress, Jest, Vitest, pytest, and any framework that emits JUnit XML.
Certyn ingests results from the tests you already run in CI. Two report formats cover almost every framework:
- Playwright JSON (
format=playwright): read natively, including retry and attachment detail. - JUnit XML (
format=junit): the lingua franca almost every test runner can emit.
Point your existing suite at POST /api/ci/results and each run lands next to your agent runs, with pass and fail history per test. Failing runs are triaged automatically: flaky vs regression, grouped by root cause.
No format query? Certyn sniffs it
If you omit format, Certyn detects it from the first byte: a leading < is treated as JUnit XML, otherwise Playwright JSON. Passing format explicitly is still recommended.
Framework matrix
| Framework | Format | Emit the report |
|---|---|---|
| Playwright | playwright | npx playwright test --reporter=json > report.json |
| Cypress | junit | --reporter junit (or cypress-multi-reporters + mocha-junit-reporter) |
| Jest | junit | jest-junit reporter → junit.xml |
| Vitest | junit | vitest run --reporter=junit --outputFile=junit.xml |
| Mocha | junit | mocha --reporter mocha-junit-reporter |
| pytest | junit | pytest --junitxml=junit.xml |
| PHPUnit | junit | phpunit --log-junit junit.xml |
| Go (gotestsum) | junit | gotestsum --junitfile junit.xml |
| JUnit / TestNG | junit | native XML report |
| NUnit / xUnit | junit | trx2junit, or a JUnit logger |
Anything not listed works too as long as it can produce JUnit XML.
Push a report
Store CERTYN_API_KEY as a CI secret, run your tests, then upload the report. Use continue-on-error (or || true) so a failing test suite still uploads its results.
# JUnit XML from any framework
curl -sS -X POST \
"$CERTYN_API_URL/api/ci/results?projectSlug=$PROJECT_SLUG&suite=e2e&environmentKey=staging&format=junit" \
-H "X-API-Key: $CERTYN_API_KEY" \
-H "Idempotency-Key: $CI_RUN_ID" \
--data-binary @junit.xml
# Playwright JSON
curl -sS -X POST \
"$CERTYN_API_URL/api/ci/results?projectSlug=$PROJECT_SLUG&suite=e2e&environmentKey=staging&format=playwright" \
-H "X-API-Key: $CERTYN_API_KEY" \
-H "Idempotency-Key: $CI_RUN_ID" \
--data-binary @report.json
The first upload for a suite slug auto-creates an automation process for it; later uploads append to its history.
Sharded Playwright
Running Playwright across shards produces one report per shard. Merge them into a single blob report first (npx playwright merge-reports) and upload once, so a commit maps to one run rather than one per shard.
Optional query parameters
Pass these so a result links back to the change that produced it:
repository,ref,commitSha: the source under test.event: e.g.push,pull_request,pipeline.externalUrl: a link back to the CI run.
See the CI Results API for the full schema and limits.