Share a Maestro mobile test report

Maestro is the pleasant part of mobile testing: flows are YAML, they read like instructions, and maestro test runs them against a simulator or a real device in seconds.

Then a flow fails in CI, and what you have is JUnit XML plus a debug directory on a runner you cannot reach.

What a run leaves behind

maestro test flows/ --format junit --output report.xml \
  --debug-output ./maestro-debug
  • report.xml — machine-readable, unreadable by humans.
  • maestro-debug/ — screenshots at each step, a video of the run, the command log. This is the useful part.

The screenshot is the artifact that actually settles arguments: a picture of the app stuck on a spinner at step 7 tells a PM more than any stack trace. And it is the artifact that is hardest to get out of CI, because it is a file in a folder that expires.

Build the page around the screenshot

<h1>Mobile smoke — iPhone 16 · build 402</h1>
<p>9 flows · 8 passed · 1 failed</p>

<h2>checkout.yaml — failed at step 7</h2>
<p><code>assertVisible: "Order confirmed"</code> timed out after 10s.</p>
<img src="…step-07.png" alt="Checkout screen stuck on spinner" />
<pre>
- launchApp
- tapOn: "Basket"
- tapOn: "Pay"
- assertVisible: "Order confirmed"   ← timed out
</pre>

The flow excerpt next to the screenshot is what makes this readable by someone who has never opened Maestro. They can see what was expected, what appeared, and where it stopped — without being handed an XML file.

Mobile failures need a picture

This is more true on mobile than anywhere else. A web test failure can often be understood from a selector and an error. A mobile failure is usually about what the screen actually looked like: a keyboard covering the button, a permission dialog nobody expected, a layout that broke on a small device.

Which is why a text summary in Slack reliably produces "can you send a screenshot?" — see stop screenshotting reports for why pasting it into the channel is not the answer either.

From CI

- run: maestro test flows/ --format junit --output report.xml \
       --debug-output ./maestro-debug
  continue-on-error: true
- run: node scripts/publish-maestro.js ./maestro-debug report.xml

One report id per branch, updated each run: the URL in the PR always shows the latest state, and the comments from the last round are still attached. See publishing from CI.

Worth knowing

  • Publish only the failing flows' screenshots. A full debug directory is hundreds of frames; the reader needs the last three.
  • Restrict the report — app screenshots usually show unreleased UI and seeded data that looks real. See security.
  • 5 MB per report body. Downscale frames before inlining.

Try it

Free — unlimited reports, commenters and revisions.

Publish from CI →

Related