Stop 11 · CI/CD & DevOps for QATopics · 3/8

Test Reporting & Dashboards

In short

500 tests ran overnight. Your manager has two minutes before standup. A good report answers 'are we safe to release?' in one glance, without anyone opening a log file.

Turning raw test execution results into a clear, scannable report — pass/fail counts, trends over time, and failure details — so the team can quickly understand test health without digging through logs.

The purple blob stands on a ladder hanging green and red tiles on a wooden scoreboard.

Think of it like this

The cricket scoreboard

You don't need ball-by-ball commentary to know the match situation. One look at the scoreboard tells you runs, wickets and overs. A test dashboard is your team's scoreboard: passed, failed, flaky, and how today compares with yesterday.

Watch it happen

From raw results to a team decision

Test run

JUnit XML

Report

Dashboard

Alert

1/5The suite runs in CI. Each test ends as passed, failed, skipped or flaky.

The big ideas

What you need to know

Summary first

Top of the report: total, passed, failed, flaky, duration. Details come below for whoever needs them.

Evidence on failure

Each failure should carry a screenshot, error message and ideally a trace or video, so nobody has to rerun it to understand.

Trends beat snapshots

One red run means little. A test that fails every third night is a pattern worth fixing.

Know your reader

Developers want the stack trace. Managers want 'safe to ship or not'. Give each the view they need.

See it in code

Walk through the code

1// playwright.config.ts2import { defineConfig } from "@playwright/test";3 4export default defineConfig({5  reporter: [6    ["list"],                                 // live output in the terminal7    ["html", { open: "never" }],              // browsable report with traces8    ["junit", { outputFile: "results.xml" }], // for Jenkins and dashboards9  ],10  use: {11    screenshot: "only-on-failure",12    trace: "retain-on-failure",13  },14});

1/2Three reporters at once: terminal output, an HTML report, and machine-readable JUnit XML.

Playwright can write several reports from one run. Jenkins, GitLab and most dashboards understand the JUnit XML file.

Save yourself the pain

Common mistakes

Oops

Sharing a 3,000-line console log and calling it a report.

Do this instead

Share a summary with pass rate, failures grouped by feature, and links to evidence.

Oops

Hiding flaky tests inside the 'passed' count after a retry.

Do this instead

Show flaky as its own number. It's a warning sign the team needs to see.

Your mission

Build a one-glance report

~25 min

Quick check

Your dashboard shows 98% pass rate today. What would make you most worried?

Go deeper

1Set up a test reporting tool (e.g. Allure) to generate a readable report from your test suite.
2Learn what makes a test report genuinely useful versus just a wall of pass/fail data.
3Practice presenting test results to a team in a way that highlights what actually needs attention.