Skip to main content

Test Automation Best Practices

Description​

End to end testing or Integration testing is an essential part of any software systems development life cycle. At Cancer Research UK, we have various products integrated as a part of our Supporter journey and hence it is crucial, that we conduct automated E2E(End to End) testing.

One of the vital aspects for writing/developing a test automation framework is to understand what to automate, why and when to apply a specific technique or how to use a particular framework.

This best practice guide details key factors to consider whilst designing automated tests.

Understanding your product and it's decisions​

It is highly encouraged that you have a team discussion to select the best options for your Product:

  • Scripting Language
  • Yarn or Npm (see comparison)
  • Test Framework: Assess how elaborate the current framework is, what is the cost of maintenance and any drawbacks being observed

Cypress was our chosen E2E testing tool however we started to run into some issues where Cypress support was limited specifically around cross domains/tabs, iframe and webkit support.

As a result, we re-explored the available E2E testing tools and found that Playwright was easily able to fill those gaps.

Cypress is however still being used within a few products as the primary E2E testing framework.

Always do a proof of concept (POC)​

Before starting to implement any new framework/tool, it is extremely important to identify what are you trying to do. Are you familiar with the language? Do you understand the framework? Is it possible to automate the critical user journeys for the product?

You only know if you try them yourself. Whilst deciding the test framework or designing tests one should make the decisions with little/no external dependency and document instructions where required (README's). Other people should be able to follow/execute the code without any external help that is not within the codebase itself.

It is also good practice to let the team know about the intentions of the POC so they can contribute or feedback any concerns or suggestions beforehand.

Think about isolated tests and code​

  • Always think about code isolation. All tests should ideally be isolated not dependent on other tests. Analyze your tests, and get rid of dependencies. For example , consider an e-commerce application and following tests:

🔲 Test 1: Create a shopping Product

🔲 Test 2: Clone the Product

🔲 Test 3: Edit the properties of Cloned Product

If test 1 or test 2 fail, the following test would fail too. So you have a waterfall of failing tests.

So make sure to avoid dependencies and start a brand new browser for each test.

Avoid conditions whenever possible​

❌ Problematic code:

async function login(user: Object): Promise<void> {
  if (user["Email"]) {
    await page.fill("email address", user["Email"]);
  } else if (user["Password"]) {
    await page.fill("password", user["Password"]);
  } else if (user["Cellphone"]) {
    await page.fill("email address", user["Cellphone"]);
  }
}

✅

async function login_with_email(
  email: String,
  password: String
): Promise<void> {
  await page.fill("email address", email);
  await page.fill("password", password);
}
async function login_with_cellphone(
  cellphone: number,
  password: String
): Promise<void> {
  await page.fill("email address", cellphone);
  await page.fill("password", password);
}

Avoid flaky tests​

There is nothing worse than discovering flaky tests during a release. It can lead to number of questions and unnecessary time spent whilst trying to investigate what has gone wrong. For example:

  • Will the Web App's users also face the same issue and be unhappy? 😕
  • Is it a bug in the App's code?
  • Is it an improperly written test case?
  • Did my commit/change/Pull Request just break something? Or was it an earlier change? The tests passed locally before I pushed!?!

It is useful to go through these links:

Selectors​

The robustness of an automation solution is dependent on how your tests find the web elements on a web page and how they interact with them. The more the tests resemble actual user behaviour, the more confidence they give us e,g., to click a log in button we could use the text displayed to the user on the UI (User Interface), instead of using an implemented class or html type or XPath.

It is best practice to prioritize the use of locators over selectors where possible mainly to mitigate flakiness in the tests. Locators are stricter then selectors and fail with better error messages e.g.: element resolved to multiple element in a page.

❌

cy.get(".btn.btn-large").click();
page.locator("button.buttonIcon.episode-actions-later");

✅

page.getByRole("button", { name: "submit" });
cy.contains("Submit").click();

Timeouts​

  • Avoid having arbitrary waitForTimeout() wherever possible as this can often lead to slow test execution.
  • Use action and navigation timeouts as required.

Adopt Page Object Model (POM)​

POM is industry recommended design pattern. POM allows us to represent all WebElements on given web page as variables and all user actions can be represented as methods.

Some of the advantages of using POM are:

  • Modularization of code.
  • Ease of maintenance since all the locators for a page are at one location.
  • In case one of the page changes we can easily replace the locators where as rest of the code is unaffected.

To implement POM consider each web page of an application as a different class file . Each class file will only contain web page elements that correspond to it. The automated tests can use these items to conduct operations on the website under test e.g., consider testing the log in functionality for wikipedia

//loginPage.ts
import { Page } from "@playwright/test";
export default class LoginPage {

    constructor(public page: Page) { }

    async function login(username: string, password: string) {
        await this.enterUsername(username);
        await this.enterLoginPassword(password);
        await this.clickLoginBtn();
    }

    async function enterUsername(username: string) {
        await this.page.getByPlaceholder("Enter your username")
            .type(username);
    }

    async function enterLoginPassword(password: string) {
        await this.page.getByPlaceholder("Enter your password")
            .type(password);
    }

    async function clickLoginBtn() {
        await Promise.all([
            this.page.waitForNavigation(),
            this.page.getByText("Log in")
        ])
    }
}

Then in the tests, we can use these classes as imports to create the user journeys like:

// login.spec.js
import { test } from "@playwright/test";
import LoginPage from "../pages/loginPage";

test("Test Login is successful", async ({ page, baseURL }) => {
  const login = new LoginPage(page);
  await page.goto(baseURL);
  await login("test123", "Pass$1234");
  expect(await page.getByText(`Hello, test123!`)).toBeVisible();
});

In Summary​

  • Use the technique to keep the operations and flows separate from assertions.

  • Stay DRY i.e. adhere to the principle of Do not repeat yourself.

  • Mostly will be useful to adopt for Larger test suites for ease of authoring and maintenance.

Code Style​

Last but not the least, make sure the code is well formatted.

References & Further Reading​