Skip to main content

Argos Playwright SDK

Improve test debugging and boost your visual testing capabilities by combining Argos with your Playwright tests.

Get started​

Please refer to our Quickstart guide to get started with Argos and Playwright.

Setup Visual Testing​

Argos presents a significant advantage over traditional Playwright visual tests with its streamlined approach to managing and reviewing test results:

  • Visual Testing on CI: With Argos, there's no need to run tests locally or commit screenshots to your repository. This not only saves time but also keeps your repository clean and focused on code rather than binary assets.
  • Fluid UI for Comparison: Argos intuitive interface makes it easy to spot discrepancies and understand visual changes without the need for cumbersome manual checks.
  • Enhanced Stabilization: The Argos integration with Playwright takes visual testing to the next level by ensuring stability and consistency in the screenshots captured. Fonts, images, animations, loaders, everything is just stable.

To enable Visual Testing with Playwright, first you have to setup the Argos reporter in your Playwright config:

playwright.config.ts
import { defineConfig } from "@playwright/test";
import { createArgosReporterOptions } from "@argos-ci/playwright/reporter";

export default defineConfig({
  // ... other configurations

  // Setup Argos reporter to send screenshots and traces to Argos.
  reporter: [
    // Use "dot" reporter on CI, "list" otherwise (Playwright default).
    process.env.CI ? ["dot"] : ["list"],

    // Add Argos reporter.
    [
      "@argos-ci/playwright/reporter",
      // Upload only on CI.
      createArgosReporterOptions({ uploadToArgos: !!process.env.CI }),
    ],
  ],
});

And use argosScreenshot helper to capture stable screenshots in your E2E tests:

tests/example.spec.ts
import { test } from "@playwright/test";
import { argosScreenshot } from "@argos-ci/playwright";

test("screenshot homepage", async ({ page }) => {
  await page.goto("http://localhost:3000");
  await argosScreenshot(page, "homepage");
});

Setup Tests Debugging​

The Argos Playwright reporter automatically reports failure screenshots and playwright traces. You can access them directly in Argos UI. If you are tired to download traces from artifact and run a command to see them, it is the solution for you.

  • Failure Screenshots: View failure screenshots from your CI tests directly in Argos. No extra steps, just immediate clarity where you need it most.
  • Playwright Traces: “Time travel” through your failing tests with remote Playwright traces. Gain a complete, step-by-step visual journey to the heart of any test issue.

To enable Test Debugging, you have to add Argos reporter and to turn on Playwright test use options in your Playwright config:

playwright.config.ts
import { defineConfig } from "@playwright/test";
import { createArgosReporterOptions } from "@argos-ci/playwright/reporter";

export default defineConfig({
  // ... other configurations

  // Setup Argos reporter to send screenshots and traces to Argos.
  reporter: [
    // Use "dot" reporter on CI, "list" otherwise (Playwright default).
    process.env.CI ? ["dot"] : ["list"],

    // Add Argos reporter.
    [
      "@argos-ci/playwright/reporter",
      // Upload only on CI.
      createArgosReporterOptions({ uploadToArgos: !!process.env.CI }),
    ],
  ],

  // Setup recording option to enable test debugging features.
  use: {
    // Collect trace when retrying the failed test.
    trace: "on-first-retry",

    // Capture screenshot after each test failure.
    screenshot: "only-on-failure",
  },
});

Tests Sharding​

Argos seamlessly integrates with Playwright test sharding, enabling efficient test distribution without the need for manual configuration. Argos Sharding/Parallel mode is automatically configured for you.

Argos also supports Currents orchestration using Argos parallel + finalize.

Helper Attributes for Visual Testing​

For tailored visual testing, the data-visual-test attributes provide control over how elements appear in Argos screenshots. This can be especially useful for obscuring or modifying elements with dynamic content, like dates.

  • [data-visual-test="transparent"]: Renders the element transparent (visiblity: hidden).
  • [data-visual-test="removed"]: Removes the element from view (display: none).
  • [data-visual-test="blackout"]: Masks the element with a blackout effect.
  • [data-visual-test-no-radius]: Strips the border radius from the element.

Example: Using a helper attribute to hide a div from the captured screenshot:

<div id="clock" data-visual-test="transparent">...</div>

Create multiple Argos builds from a single suite of tests​

To generate multiple Argos builds from a single suite of Playwright tests, use a dynamic buildName. This object contains two properties:

  • values: An array of possible values returned by the get function.
  • get: A function that takes a test case as an argument and returns the build name.
playwright.config.ts
import { defineConfig } from "@playwright/test";
import { createArgosReporterOptions } from "@argos-ci/playwright/reporter";

export default defineConfig({
  // ... other configurations

  reporter: [
    // Use "dot" reporter on CI, "list" otherwise (Playwright default).
    process.env.CI ? ["dot"] : ["list"],

    // Argos reporter
    [
      "@argos-ci/playwright/reporter",
      createArgosReporterOptions({
        uploadToArgos: !!process.env.CI,
        buildName: {
          values: ["app", "website"],
          // Split Argos build based on Playwright tags
          // Learn more about Playwright tags: https://playwright.dev/docs/test-annotations#tag-tests
          get: (test) => (test.tags.includes("@website") ? "website" : "app"),
        },
      }),
    ],
  ],
});

Debug flaky tests​

To debug flaky tests, Argos supports using the Playwright --repeat-each option. This runs each test multiple times to detect discrepancies.

npm exec -- playwright test --repeat-each 5

Configure Content-Security-Policy (CSP)​

To stabilize tests, Argos injects a script before taking screenshots. This script may conflict with your CSP settings. To resolve this issue, you have two options:

1. Allow the Argos script in your CSP​

The @argos-ci/playwright package provides a getCSPScriptHash method to retrieve the hash of the script injected by Argos. Additionally, Argos requires the 'unsafe-eval' directive to function properly.

To use this method, you need to be able to inject CSP headers into your server. Here's an example in a Playwright configuration:

import { defineConfig } from "@playwright/test";
import { getCSPScriptHash } from "@argos-ci/playwright";

export default defineConfig({
  webServer: {
    command: "node my-app.js",
    port: 3000,
    env: {
      CSP_SCRIPT_SRC: `${getCSPScriptHash()},'unsafe-eval'`,
    },
  },
});

2. Configure Playwright to bypass CSP​

If you do not have the flexibility to add custom CSP headers for Playwright test execution, you can configure Playwright to bypass CSP in your Playwright configuration:

import { defineConfig } from "@playwright/test";

export default defineConfig({
  use: {
    bypassCSP: true,
  },
});

API Overview​

argosScreenshot(page, name[, options])​

  • page: Instance of the Playwright Page.
  • name: Unique name for the screenshot.
  • options: Explore Page.screenshot command options for details.
  • options.element: Use a Locator or a string selector to capture a specific element's screenshot.
  • options.viewports: Define specific viewports for capturing screenshots. More on viewports configuration.
  • options.argosCSS: Specific CSS applied during the screenshot process. More on injecting CSS
  • options.disableHover: Disable hover effects by moving the mouse to the top-left corner of the page. Default to true.
  • options.threshold: Sensitivity threshold between 0 and 1. The higher the threshold, the less sensitive the diff will be. Default to 0.5.
  • options.root: Folder where the screenshots will be saved if not using the Argos reporter. Default to ./screenshots.
  • options.stabilize: Wait for the UI to stabilize before taking the screenshot. Set to false to disable stabilization. Pass an object to customize the stabilization. Default to true.
  • options.stabilize.ariaBusy: Wait for the aria-busy attribute to be removed from the document. Default to true.
  • options.stabilize.fonts: Wait for fonts to be loaded. Default to true.
  • options.stabilize.images: Wait for images to be loaded. Default to true.

Unlike Playwright's screenshot method, set fullPage option to true by default. Feel free to override this option if you prefer partial screenshots of your pages.

getCSPScriptHash()​

Returns the Content-Security-Policy script hash used by Argos (ex: 'sha256-xaC9wWpMVRiAXSfhxhP+Wyqkw0mgO+MIrHuzmMPIxEI=').

Playwright reporter​

The Argos reporter offers extensive configuration options. Specifically, all upload parameters are available for customizing the reporter.

The createArgosReporterOptions ensures that your options are correctly typed.

playwright.config.ts
import { defineConfig } from "@playwright/test";
import { createArgosReporterOptions } from "@argos-ci/playwright/reporter";

export default defineConfig({
  // ... other configurations

  reporter: [
    // Use "dot" reporter on CI, "list" otherwise (Playwright default).
    process.env.CI ? ["dot"] : ["list"],

    // Argos reporter
    [
      "@argos-ci/playwright/reporter",
      createArgosReporterOptions({
        uploadToArgos: !!process.env.CI,
        buildName: "custom-build-name",
      }),
    ],
  ],
});

Additional Resources​