BrowserCat
Dashboard →
Playwright in the Cloud

Strengthen Selectors and Locators in Playwright

Browser automation is a powerful skill set, but it comes with unique challenges—including working with selectors and locators. The web is a dynamic environment, and writing scripts that are still likely to work in a few months (or years) takes some practice.

This article dives into numerous selector anti-patterns with advice for how to prepare your code for an everchanging world. Read to the end for advanced suggestions on improving the maintainability and performance of your scripts.

Intro to Selectors and Locators

To start off, let’s dig into the difference between selectors and locators, as well as how they work within Playwright…

What are Playwright Selectors?

Playwright selectors are search queries used to match elements within the page’s DOM. They leverage various strategies such as CSS, XPath, textual content, pseudo-selectors, and even custom properties you define. Selectors are strings, and they are the foundation of Playwright’s locators, though they don’t enable interaction by themselves.

Here’s some examples:

// create a locator using css
const $buttonByCSS = page.locator('css=button');

// create a locator using xpath
const $buttonByXPath = page.locator('xpath=//button');

// create a locator using psuedo-selectors
const $buttonByRole = page.getByRole('button');

Playwright also allows the registration of custom selectors, leveraging any logic you can dream up. At first blush, this might seem like overkill, but as code matures and you develop domain logic, custom selectors can dramatically simply your code.

Here’s a brief taste:

// target elements currently within a particular shadow DOM
await selectors.register('shadow', createShadowSelector);

// query all options within a custom select web component
const $options = page.locator('shadow=x-custom-select option');
await $options.count();

We’ll return to this technique later with a complete example.

What are Playwright Locators?

Locators are a powerful abstraction built on top of selectors. They provide a clean interface for selecting elements, interacting with them, and traversing the DOM subtree within them. You use them constantly in Playwright.

As an improvement over the now-deprecated $() API, they also come with some unique advantages. First, they are always up-to-date. A selector that targets a list elements will always select the full list, no matter how many items have been added, removed, or changed.

And second, locators will wait until the selector pattern finds a match on the page. This is incredibly useful when working with dynamic content, as you no longer have to build arbitrary or complex timeout logic into your scripts.

We’ll cover locators in great detail later, but here’s a quick sample:

// find the button with `name="submit"`
const $submitButton = page.getByRole('button', {name: 'submit'});
// or...
const $submitButton = page.locator('role=button', {name: 'submit'});

// click it!
await $submitButton.click();

Locators can also be chained to increase specificity without having to resort to long selector queries. Example:

const $taskList = page.locator('ul#tasks');
const $taskLinks = $taskList.locator('a');
const $completedTasks = $taskLinks.filter({
  has: page.locator('input[type="checkbox"]:checked'),
});

OK, basics covered. Now let’s go deep!

Selector Best Practices

Working with Playwright requires a good understanding of how to target elements effectively. Selectors are an essential part of this process, as they are used to identify a page’s elements. The precision of your selectors can heavily influence the performance and reliability of your tests, automation scripts, or web scrapers.

Let’s go through all of the best practices one at a time, so that you never need another article like this again.

Optimize Selector Specificity

Refining selector specificity involves finding an ideal equilibrium: you need selectors that are precise enough to unambiguously identify target elements across different page states, yet not so intricate that they shatter with the slightest DOM adjustments.

Steer clear of overly rigid selectors; such fragility means that a trivial HTML modification could render them useless. On the flip side, too loose selectors might lead you down a path of position-dependent logic (like indices) or unwarranted reliance on the webpage’s current implementation.

We’re looking for the Goldilocks zone of search queries.

Here’s a few anti-patterns and how to fix them:

// 🔴 BAD: Selectors based on fragile structural assumptions
const $link = page.locator('li:nth-child(3) > a');
// 🟢 GOOD: Selectors grounded in distinguishing features
const $link = page.locator('a[href*="privacy"]');

// 🔴 BAD: Selectors anchored to mutable text
const $button = page.locator('button', {hasText: 'Sign up'});
// 🟢 GOOD: Robust, pattern-based selectors
const $button = page.locator('button', {
  hasText: /(sign|start|subscribe|launch)/i,
});

// 🔴 BAD: Selectors tied to changeable style-specific classes
const $input = page.locator('input.email');
// 🟢 GOOD: Selectors focused on functional attributes
const $input = page.locator('input[type="email"]');

// 🔴 BAD: Excessively specific hierarchical selectors
const $list = page.locator('footer > ul.inline-links > li');
// 🟢 GOOD: Shallow, adaptive nesting
const $list = page.locator('footer:last-of-type li', {
  has: page.locator('a'),
});

To sum up:

  • Prioritize attributes that capture the essence of the target element, rather than arbitrary qualities.
  • If your target elements have no stable defining characteristcs, anchor your queries to parents or children that do have such qualities.
  • Keep chain selectors short, and try to only use stable elements within them.
  • Leverage regular expressions for text matching to accommodate variations.

Prefer Semantic Locator Methods

When it comes to selecting elements, it’s always better to prioritize semantic attributes over stylistic or functional attributes. While an element’s classes or its location in the DOM are typically the easiest to target, they’re also the most prone to change. On the other hand, semantic attributes (such as role, label, or title) change infrequently. And when they do, they tend to change in ways that can be accounted for ahead of time.

Given the benefit of targeting semantic attributes, Playwright provides utility methods for directly accessing them. Use them whenever possible. Here’s why:

  • Semantic methods encourage best practices. As you’ll see in the passages that follow, the most durable selector patterns have helper methods.
  • Semantic methods are typed, improving your IDE experience and alerting you to errors. Plain selector strings can only be debugged at runtime, resulting in more bugs.
  • Semantic methods are chainable. This dramatically simplifies the queries themselves. It also results in better error messages when queries fail.

Let’s run through each method that’s currently available:

// Target Test IDs when you have control of the HTML
// E.G. `<section data-testid="delete-modal" />`
const $modal = page.getByTestId('delete-modal');

// Target explicit or implied element roles
// "button" matches `<button>`, `<input type="button">`, or `<div aria-role="button">`
const $button = page.getByRole('button', {hasText: 'Buy'});

// Target text HTML attributes that are unlikely to change
const $input = page.getByLabel('Email');
const $search = page.getByPlaceholder(/^search/i);
const $image = page.getByAltText('Profile Picture');
const $icon = page.getByTitle('Info', {exact: false});

// Or target elements by `innerText`, when you can be sure it's stable
const $dialog = page.getByText(/^confirm/i);

I cannot stress enough how valuable it is to target based on role and data-testid in particular.

When you have control of the DOM, data-testid is a fantastic convention to implement across dev teams. Unlike every other HTML attribute, there is never any reason for data-testid to change, making it the least brittle of all possible selectors.

That said, given that you won’t always have control over the HTML of target pages, role is an excellent fallback. As shown in the example code above, role is an inherently forgiving selector. It can continue working even across substantial DOM changes.

Chain Locators, Not Selectors

As described above, it’s best to avoid long selector query strings. They’re inherently difficult to debug, and they result in less descriptive error messages.

Of course, you’re not always going to be able to avoid chaining. Sometimes, you need to target particularly evasive DOM nodes. More often, you simply want to break a page up into subtrees and drill down from an intermediate HTML element. (For example, a list of article in a blog feed.)

Here’s two strategies for designing locators that are both easy to read and easy to debug…

Drill Down Within Subtrees

Chaining locators is like adding layers to a sketch; each additional stroke refines the image. Begin with a broad locator and use methods like .filter(), .first(), .last(), and conditional parameters to progressively narrow down to your target element.

// Find the first article about BrowserCat
const $articles = page.locator('article');
const $aboutBrowserCat = $articles.filter({
  hasText: /BrowserCat/i
});
const $firstArticle = $aboutBrowserCat.first();

This approach separates concerns, ties the locators to logical entities, and keeps the code readable and adaptable. If your layout changes, you might only need to only adjust the parent locator instead of unraveling multiple complex strings.

Filter by Content with {has} and {hasNot}

Sometimes you need to select an element not only by its own properties but also by its relation to others. With {has} and {hasNot} parameters, you can define these relationships clearly, creating a robust context for your selectors.

// Find the first article with an image
const $articles = page.locator('article');
const $withImages = $articles.filter({ 
  has: page.locator('img'),
});
const $firstArticle = $withImages.first();

These parameters act as assertions about the presence of certain elements within a parent. This increases the number of stable, unique attributes you can leverage in creating good query patterns.

Use the Correct Wait Strategy

By default, Playwright’s locators will wait for at least one matching element to appear on the page. In many cases, this is all you need, but the deeper you get into automation, the more frequently this strategy will fall short.

Often, you’ll need to watch for other criteria before a particular locator is ready for action. Here’s a good example of why you can’t always trust the default wait strategy:

// 🟢 GOOD: The default wait works for a single matching element
const $modal = page.locator('#signup-modal');
// 🔴 BAD: But it fails when you need to wait for multiple elements
const $confetti = page.locator('.confetto');

Wait for global state

The “dynamic confetti” problem above can be solved by waiting for the global state to normalize.

// Wait for at least 100 confetti
await page.waitForFunction(async () => {
  return await page.locator('.confetto').count() >= 100;
});
const $confetti = page.locator('.confetto');

page.waitForFunction() will run the function until the return is truthy. This allows you to write very clean code that’s equally powerful.

Embrace the State of the Locator

Locators normally resolve when at least one matching DOM node is attached to the DOM. However, Playwright offers a few other convenience methods:

// Attached when added to the DOM (default)
const $attached = page.getByRole('button')
  .waitFor({state: 'attached'});

// Detached when removed from the DOM
const $detached = page.getByRole('button')
  .waitFor({state: 'detached'});

// Visible when attached and has some visible pixels
const $attached = page.getByRole('button')
  .waitFor({state: 'visible'});

// Hidden when detached, `visibility: hidden`, or `display: none`
const $attached = page.getByRole('button')
  .waitFor({state: 'hidden'});

The visible state will frequently come in handy, as a lot of dynamic HTML works by managing visibility rather than dynamically creating new subtrees.

Set a Custom Timeout

By default, Playwright will resolve a locator as soon as it possibly can. But if you’re waiting on a dynamically added list of elements, you may prefer to set an extended timeout.

// Wait for a waterfall of injected JS script tags
const $dynamicScripts = page.locator('script')
  .waitFor({timeout: 60_000});

That said, use caution with the timeout approach. If you find yourself leaning on such an imprecise tool, you may want to look twice for better solutions. After all, you can potentially save a lot of time by targeting the precise event that will allow your script to proceed.

Here’s one particular strategy…

Wait for DOM Events

Sometimes, it’s best to go back to basics and lean on the DOM to ensure the necessary content is loaded before proceeding with your selectors.

For example, lets say that you need a particular image to have loaded before taking a screenshot of the page. You might use the load event to ensure that this is the case:

const $logo = page.getByRole('img')
  .and(page.getByAltText('BrowserCat'));

$logo.evaluate(($el) => {
  return new Promise((resolve) => {
    $el.addEventListener('load', resolve, {once: true});
  });
});

// Logo is loaded!
await $logo.screenshot();

It’s worth highlighting the power of Playwright’s .evaluate() API. This simple construct gives you direct access to the browser context. Use it to get down to the metal, and control the flow of your scripts directly.

Handle Errors Gracefully

All robust automations share one thing in common: they can handle the tides of uncertainty the web throws at them day after day. Let’s explore some practical strategies for building resilience into Playwright scripts, ensuring they can handle errors, adapt to varied content, and provide clear control flow.

Try Multiple Locator Strategies

When scraping web pages, you may encounter variability in where and how data is presented. For instance, retrieving a star rating from a product page can be challenging because the value could be in a metadata field, within the text of an element, or even graphically represented with icons.

const maybeRatings = await Promise.all([
  page.locator('.rating-value').textContent(),
  page.locator('.star-rating img.star').count(),
  page.locator('meta[name="description"]')
    .getAttribute('content')
    .then((str) => (/\n\.\n+ stars/u).exec(str ?? '')?.[0]),
]);

const bestRating = maybeRatings.reduce((pick, rating) => {
  return pick ?? parseFloat(rating) || null;
}, null);

In this example, the script attempts multiple strategies to capture the rating, testing each result to ensure it looks like an actual rating number.

Quietly Skip Optional Content

Not all content is essential for the successful completion of a script. For example, let’s say you wanted to scrape a book product page, but you can’t always guarantee the page count is available. In this case, you’d only want the script to fail if critical values (like the book title) weren’t found, but to pass otherwise.

const title = await page.locator('.book-title')
  .textContent();
const author = await page.locator('.author-name')
  .textContent();
const pageCount = await page.locator('.page-count')
  .textContent();

const details = {
  title, 
  author,
  pageCount: parseInt(pageCount) || null,
};

assert(!!details.title && !!details.author);

The function demonstrates selective inclusion of optional data. If present, it enriches the results. But if absent, it doesn’t constitute failure.

Clarify Your Control Flow

Control flow can become unclear if scripts are peppered with conditional statements and error handling. Playwright’s locators can help us streamline the process.

In the following example, we only interact with the table of contents if it exists.

const $toc = page.locator('aside.contents');

if (await $toc.count() > 0) {
  const $lastLink = await $toc.locator('a').last();
  await $lastLink.click();

  const depth = await page.evaluate(() => window.scrollY);
  console.log(`Scrolled to depth: ${depth}`);
} else {
  console.log('No table of contents found.');
}

This example demonstrates a clear control flow where the presence of a table of contents triggers conditional behavior, but doesn’t result in an error otherwise.

Retry Failed Gestures

In some scenarios, you might want a script to retry an action until a certain condition is met, like rolling a digital die until a specific number appears.

const $d20 = page.getByRole('button', {name: 'Roll d20'});
const $result = page.locator('#d20-result');

page.waitForFunction(async () => {
  await $d20.click();
  const roll = await $result.textContent().then((str) => parseInt(str));

  return roll === 20;
});

This code snippet rolls a virtual die, retrying the gesture until the desired outcome is achieved, showcasing a controlled and persistent interaction pattern.

As you can see, these strategies equipt your web automation tasks with accuracy and fault tolerance, ensuring smooth interaction with a variety of web pages, content structures, and unexpected circumstances.

Use Asynchronous Interactions

The longer your scripts become, the more you’ll gain by ruynning page interactions simultanously. Playwright’s websocket communication model makes this very performant, and the Locator API is in fact built with asynchronous code in mind.

Let’s dive into some strategies worth remembering…

Reuse Locators with Dynamic Content

Playwright’s locators are dynamic by design, which means they resolve fresh each time they’re accessed. This is particularly handy when dealing with changeable content on a web page.

const $messages = page.locator('#chat li');
let prevMessage = null;

// listen for websocket messages
page.on('websocket', (socket) => {
  socket.on('framereceived', async (event) => {
    const lastMessage = await $messages.last().textContent();
    // verify the new message was added to the chat list
    assert(lastMessage !== prevMessage);
    prevMessage = lastMessage;
  });
});

In this example, we access the locator every time a websocket message is received, then we verify that the new message is added to the chat list.

Notice that the $messages locator is reused without being replaced. This unlocks the possibility of defining your primary locators in the initialization of your script, reusing them as necessary across otherwise complex automations.

Run Tasks in Parallel

Often, scripts wait for one task to complete before starting another. But this isn’t a necessary restriction. In Playwright, you can achieve easy parallelism just by employing Promise.all, saving a lot of time in the process.

const result = await Promise.all([
  // Starts a long-running calculation...
  page.locator('#long-running-calculation').click(),
  // Waits for the calculation result to come through...
  page.waitForResponse((res) => /calculation-results/.test(res.url())),
]).then(() => {
  // Retrieve the result
  return page.locator('.calculation-results').textContent();
});

console.log('Calculation completed. Results:', result);

With Promise.all, the script initiates the calculation and waits for the resulting response simultaneously, rather than sequentially. When leveraged strategically, this can shaving off precious seconds or even minutes from the total run time.

Leverage a Dependency Graph

Sometimes, you have tasks that depend on others but needn’t run in a singular chain. In these cases, you can create a dependency graph by initializing numerous promises, but only awaiting the results at the end of your script. This will ensure that every interaction happens both in the correct order and as swiftly as possible.

This is a somewhat contrived example, but it’s not difficult to scale the pattern up. You will assuredly find reasons to do so in your own code.

const doLogin = page.locator('#login').click();
const loadDashboard = doLogin.then(() => page.waitForNavigation({
  url: /dashboard/,
}));
const fetchData = page.waitForResponse((res) => {
  return /data-endpoint/.test(res.url());
});

// Do more stuff here...

const [dashboardLoaded, apiResponse] = await Promise.all([loadDashboard, fetchData]);
console.log('Dashboard has loaded!');
console.log('API data:', await apiResponse.json());

By setting up a structure of chained promises, this example initializes various tasks, but only forces those chains to resolve at the very end. Less time wasted, more money saved.

Extend Playwright with Custom Selectors

Playwright opens up a world of possibilities with its selector engine, not only by supporting a wide range of built-in selectors but also by empowering users to define their own.

Custom selectors can be tailored to unique application requirements, make scripts more readable, and support complex selection logic that goes beyond the basics.

Here’s some ideas:

  1. data-state: Select elements based on a data attribute representing state (e.g., data-state=\"active\") to target specific UI states in a single-page application.
  2. closest: Select the closest ancestor of an element that matches a certain selector—akin to the Element.closest() method in JavaScript.
  3. shadow: Target elements that are within a specific shadow DOM.
  4. rotation: Select an element rotated within a particular range.

To demonstrate the potential of custom selectors in Playwright, let’s create the data-state selector described above…

import * as pw from 'playwright';

// register custom selector
await pw.selectors.register('data-state', () => ({
  query(root: Node, selector: string) {
    return root.querySelector(`[data-state="${selector}"]`);
  },

  queryAll(root: Node, selector: string) {
    return Array.from(
      root.querySelectorAll(`[data-state="${selector}"]`)
    );
  },
}));

const browser = await pw.firefox.launch();
const page = await browser.newPage();
await page.goto('https://example.com');

// use the custom selector
const $active = page.locator('data-state=active');
await $active.click();

As you can see, it takes very little to create a custom selector. Since Playwright grants you access to the underlying DOM, the sky’s the limit.

Use the Page Object Model

The Page Object Model (POM) is a design pattern widely used in automation for improving maintainability and reducing duplication. A page object encapsulates the behaviors and elements of a specific page in your target site, making it easy to update the code as needed.

Here’s a example of what a page object might look like for a login page. Notice how it collects common locators as well as behaviors likely to be repeated in multiple different scripts…

import {Page} from 'playwright';

export class LoginPage {
  page: Page;

  constructor(page: Page) {
    this.page = page;
  }

  get $usernameInput() {
    return this.page.getByLabel('Username');
  }

  get $passwordInput() {
    return this.page.getByLabel('Password');
  }

  get $loginButton() {
    return this.page.getByText('Login');
  }

  public async login(username: string, password: string) {
    await this.$usernameInput.fill(username);
    await this.$passwordInput.fill(password);
    await this.$loginButton.click();
  }
}

Here’s how you might use the LoginPage POM:

import * as pw from 'playwright';
import {LoginPage} from './LoginPage';

const browser = await pw.webkit.launch();
const page = await browser.newPage();
await page.goto('https://app.browsercat.com/sign-in');

const loginPage = new LoginPage(page);
await loginPage.login('user@example.com', 'p1a2s3s4word!');
await loginPage.$loginButton.click();

I’ve found you can extend the wisdom of the Page Object Model further. Consider that most website are built from reusable components that are repeated on multiple pages. Rather than designing POMs from scratch, instead you can compose them from components found on page after page.

Here’s how we might leverage component objects for AppMenuBar, ChatWidget, and AppFooter:

// Define the separate components
class AppMenuBar {
  constructor(public page: Page) {}
  get $logo () {}
  get $menuButton () {}
}

class ChatWidget {
  constructor(public page: Page) {}
  get $chatToggle () {}
  get $messageInput () {}
  get $sendButton () {}

  async sendMessage(message: string) {}
}

class AppFooter {
  constructor(public page: Page) {}
  get $uptimeStatus () {}
}

// Compose these components into a complete page object
class ContactPage {
  menuBar: AppMenuBar;
  chatWidget: ChatWidget;
  footer: AppFooter;

  constructor(public page: Page) {
    this.menuBar = new AppMenuBar(page);
    this.chatWidget = new ChatWidget(page);
    this.footer = new AppFooter(page);
  }
}

// Use the composed POM
const browser = await pw.chromium.launch();
const page = await browser.newPage();
await page.goto('https://www.browsercat.com/contact');
const contactPage = new ContactPage(page);

// Use the component methods
await contactPage.menuBar.$menuButton.click();
await contactPage.chatWidget.sendMessage('Hello there!');

In the example above, the composed ContactPage object becomes a powerful and convenient abstraction, bundling the interactions for the menu bar, chat widget, and app footer, which are then used in repeated interactions. This approach keeps your scripts DRY (Don’t Repeat Yourself) and makes it easy to update component interactions when the UI changes, without having to change each individual usage.

Next Steps…

You’ve absorbed a lot of powerful techniques today. Now it’s time to put them into practice!

Begin by revisiting your current Playwright scripts. Integrate the best practices for selectors and locators discussed in this article. Update your error handling to be more graceful and your asynchronous interactions to be as efficient as possible. Is there room for custom selectors? How about refactoring to use the Page Object Model?

Remember, the proof is in the performance: track how much faster and more resilient your scripts become. It’s not crazy to expect dramatic reductions in execution time, bug occurences, or feature development time.

Once your scripts are in tip-top shape, consider outsourcing the hosting of your headless browsers to BrowserCat. We offer pay-as-you-go access to our fleet of browsers, at whatever scale you need. Don’t waste time on infrastructure when you can better spend that time writing code.

Happy automating!

Automate Everything.

Tired of managing a fleet of fickle browsers? Sick of skipping e2e tests and paying the piper later?

Sign up now for free access to our headless browser fleet…

Get started today!