Scaling End-to-End User Interface Tests

At Slack, Quality is a shared responsibility. The Quality Engineering team is focused on creating a culture of testing, increasing test coverage, and helping the company ship high-quality features faster. We encourage all our developers to write and own end-to-end (E2E) tests. In turn, Quality Engineering (QE) is responsible for the frameworks used and provides best practices for writing reusable, scalable, and maintainable tests.

In this post, we are going to walk you through our journey on how we came up with a reusable automation framework and some of the positive impact we have seen.

Finding a framework for end-to-end tests

E2E UI automation at Slack started as a project built during HackDay, Slack’s internal hackathon. We wanted to implement and adopt a framework which supported JavaScript and used cypress.io for running tests. The framework built during the hackathon quickly gained adoption and several engineers contributed by adding new tests. Since there were no guardrails on how to add these tests, the framework ended up with a lot of duplicate code and flaky tests. This led to random test failures and longer triage shifts.

But where there is a problem, there is always an opportunity! The QE team decided to look for a better way to solve some of the pain points we were having within cypress. Our solution is a variation of the Page Object Model: We created a layer of abstraction between user interface and the actual test. We time-boxed the effort to one month and worked on using the proof of concept on a set of tests.

Abstract it all away

To make it easier to read, write, and maintain our end-to-end tests, we created a number of Slack-specific methods and bundled them up in a library. Thanks to this abstraction layer, we have a centralized place to define UI actions. Since Slack is largely an application as opposed to a traditional website, we thought about components rather than pages — allowing engineers to interact with, for instance, the channel sidebar. To make the code easier to write and read, our JavaScript objects are chainable.

For the UI Abstraction/Page Object Model approach, we broke things out in the components.

We came up with a few best practices to guide our work:

• Selecting Elements: Instead of relying on product-driven class names or element ids, we add a custom “data-qa” attribute to elements that we need to select for testing purposes. This allows us to provide context for our selectors so they aren’t impacted by JS/CSS changes.
• Only create new components when needed. We shouldn’t try to define every UI Action possible, but define those that are being used by our test.
• Methods within a component should only modify the piece of UI that they’re written for. The component for the channel sidebar shouldn’t interact with the message input, for instance.
• Try to only break items into components where it makes sense rather than creating a lot of smaller components.
• The UI Abstraction is stateless. The test should maintain the state and validate against it.

So, with those practices in mind, let’s take a look at how we organized our classes and components.

The above diagram represents a simplified version of our page object model (POM):

• Web Element Representation of a piece of UI. For example a button or a text box would be a WebItem. Here is where we define all the actions you can take on a element.
• BaseComponent Represents a component. Any component should extend this class. A component consists of Web Elements and other components.
• BaseModal Represents a modal. Any modal should extend this class.

If we take a look at Slack client we can break things up into components as such:

Let’s take a look at MessageInputComponent:

/**
 * MessageInputComponent
 * This is the component where people type to add a message and communicate.
 */
export class MessageInputComponent extends BaseComponent {
    constructor(parentSelector) {
        super('[data-qa="message_input"]', parentSelector);
        this.textBox = this.getChildElement('[data-qa="input_box"]');
    }
    
    /**
     * @description Type a message into the message input component
     * @param {string} text - text you want to type in
     * @param {string} submit - if true will type enter after. if false will just type in text
     * @returns {this} returns the MessageInputComponent component
     */
    typeMessage(text, submit = false) {
        this.textBox.type(text);
        if (submit) {
            this.textBox.type('{enter}');
        }
        return this;
    }
}

The MessageInputComponent extends BaseComponent where all the common methods and properties are defined for a component. From there, we pass in the selector for the MessageInputComponent and define what selectors live within the MessageInputComponent.

We created a top-level object called client. If, within a test, we want to type hello world, it would look like this:

client.messageInput.typeMessage(‘hello world’, true);

Takeaways

When we compare the test results from before versus the tests that have been migrated to use UI Abstraction framework, we’re able to see a 60% reduction in flakiness. Both automation engineers and front end engineers found it easier to add and maintain more tests.

Our team is currently working on migrating all the E2E tests to use UI Abstraction. There are several efforts underway to reduce test flakiness further.

As you can see, we are working on solving hard problems. Interested in joining the team? Check out our open roles!