Announcing Microsoft Playwright Testing: Scalable end-to-end testing for modern web apps

We are excited to announce the preview of Microsoft Playwright Testing, a new service for running Playwright tests easily at scale. Playwright, a fast-growing, open-source framework, enables reliable end-to-end testing and automation for modern web apps. Microsoft Playwright Testing is a fully managed service that uses the cloud to enable you to run Playwright tests with much higher parallelization across different operating system-browser combinations simultaneously. This means faster test runs with broader scenario coverage, which helps speed up delivery of features without sacrificing quality.

Get test suite results faster

Adding Playwright tests to your continuous integration (CI) workflow helps ensure that as the app evolves, your web app experiences continue to work the way you expect. But as the app becomes more complex, the test suite required for comprehensive testing across multiple browser and operating system combinations also increases in size. This leads to longer test suite completion times, potentially delaying your feature delivery. Development teams are already under pressure to quickly deploy app enhancements. To work around long wait times for test completion, it is common practice for development teams to selectively run only a small subset of tests. In a more detrimental scenario, a team may choose to execute tests less frequently, such as only a few times a week in an integration environment instead of with every pull request. This approach can potentially delay catching issues, complicate the process of pinpointing the cause of problems, and adversely affect the overall productivity of the development team.

With the @playwright/test runner, your tests run in independent, parallel worker processes with each process starting its own browser.  Increasing the number of parallel workers can reduce the time it takes to complete the full test suite. You can set the number of workers using the command line:

npx playwright test --workers=4

However, when you run tests locally or in your CI pipeline, you’re limited to the number of central processing unit (CPU) cores on your local machine or CI agent machine. At some point adding more workers will lead to resource contention, slowing down each worker and introducing test flakiness.

By using Microsoft Playwright Testing service you can increase the number of workers at cloud-scale to much bigger numbers. The worker processes orchestrated by @playwright/test continue to run locally but the browser instances, which are resource-intensive, now run in the cloud. You can see in the demo video below how thousands of tests run on 50 parallel browsers in the cloud managed by Microsoft Playwright Testing, significantly reducing the wait time for test results.

Consistent test results across multiple operating systems and browser combinations

App complexity isn’t the only factor in increasing test suite size. Modern web apps need to work flawlessly across numerous browsers, operating systems, and devices. Testing across all these variables increases the amount of time it takes to run your test suite. With Microsoft Playwright Testing you’ll use the scalable parallelism provided by the service to run these tests simultaneously across all modern rendering engine. This includes Chromium, WebKit and Firefox on Windows, and Linux and mobile emulation of Google Chrome for Android and Mobile Safari. Also, the service-managed browsers ensure consistent and reliable results for both functional and visual regression testing, whether tests are run from your CI pipeline or development machine. This extensive cross-compatibility testing helps ensure your web app delivers consistent performance and functionality across all platforms, optimizing the experience for any user, regardless of their browser or operating system.

Figure 1-Use Microsoft Playwright Testing service from your CI pipelines and code editors.

No test code changes required

If you’re using Playwright today, getting started with Microsoft Playwright Testing is easy! The service is designed to seamlessly integrate with your Playwright test suite, no changes to existing test code required. In just a few steps you can connect your test suite to the service and unlock the full potential of cloud-powered parallel testing. Plus, the service supports multiple versions of Playwright and updates with each new Playwright release, ensuring your tests run against the latest browser versions and technologies while helping to keep your app current, robust, and secure. Now you can focus on thorough application testing without the worry of managing a complex test infrastructure.

Source: microsoft.com