Skip to main content

End-to-End Testing

End-to-end testing (E2E testing) is a software testing method that involves testing an application’s workflow from beginning to end. This method aims to replicate real user scenarios to validate the system for integration and data integrity.

How to implement End to End Testing​

The test files are located in apps/<appName>/tests/e2e directory.

We decide to use Playwright which is recommended by SvelteKit for End to End testing.

Finding components and actions​

  • In a normal situation, whatever components are, we will find them by using text. The function that is used depends on the type of component.

  • If components have the same type and same text, we will define their index of them for finding.

  • If it isn't above conditions such as a button without text, we use xpath to find those components. Please note that xpath will be changed when that page's UI is changed, try to avoid this method.

After finding a component, we can do an action for testing. The function that is used depends on the type of component.

example:

test.describe('example', async () => {
  await test('Enter text field', async ({ page }) => {
    // fill Textbox that has naming `Enter Text`
    await page.getByRole('textbox', { name: 'Enter Text' }).fill('test Text')
  })
  await test('Click create button', async ({ page }) => {
    // click first button that has `Create` as a title, The UI must have multi buttons with `Create` title
    await page.getByRole('button', { name: 'Create' }).first().click()
  })
  await test('Find button xpath', async ({ page }) => {
    await page
      .locator(
        `xpath=/html/body/div/div[1]/div/section/div[2]/div/div[2]/div[1]/div/button`
      )
      .click()
  })
})

Basic function:

// get a component by role
page.getByRole(`${role}`, { name: `${name}` })
// get a component by custom locator string
page.locator(`${customStr}`)

// click at a locator
locator.click()
// check at a locator: use for the checkbox
locator.check()

Expecting result​

We can verify a result by using expect function.

example:

test.describe('example', async () => {
        await test('test verify', async ({ page }) => {
        // go to product creation page
        await page.goto(`/admin/products/create`);
        // verify current url
        await expect(page).toHaveURL(`${process.env.TEST_BASE_URL}/admin/products/create`);
        // find product Title textbox
        const titleTextBox = page.getByRole('textbox', { name: 'Enter Title' };
        // check product Title textbox is existed
        if (await titleTextBox.isVisible()) {
            await titleTextBox.fill('test Product');
        }
        // verify product Title
        await expect(titleTextBox).toHaveValue('test Product');
    })
})

Basic function:

// validate a url
expect(page).toHaveURL(`${url}`)
// validate a value of the locator
expect(locator).toHaveValue(`${value}`)

Waiting for event​

  • After we do a component's action, in some situations we have to wait for the event to continue testing.
test.describe('example', async () => {
  await test('Create product', async ({ page }) => {
    // go to product creation page
    await page.goto(`/admin/products/create`)
    // wait for url to be /admin/products/create
    await page.waitForURL(`/admin/products/create`, { timeout: 5000 })
    // fill product Title
    await page
      .getByRole('textbox', { name: 'Enter Title' })
      .fill('test Product')
    // Click Save
    await page.getByRole('button', { name: 'Save' }).click()
    // wait for backend respone
    await page.waitForResponse(`${process.env.BACKEND_URL}`, { timeout: 5000 })
  })
})

Basic function:

    // wait for url
    await page.waitForURL(`${url}`, { timeout: 5000 });
    // wait for respone
    await page.waitForResponse(`${BACKEND_URL}`, { timeout: 5000 });
    // delay
    await page.waitForTimeout(${time_to_delay});
    // wait for target component by custom locator string
    await page.waitForSelector(`${customStr}`);

Please note that: If expected waiting is incorrect, the testing will crash with a timeout error(wait until timeout).

Handling corner cases​

To implement our applications, There are some corner cases that we have to handle because of our selected flamework.

Delay after navigating to a new page​

In some components, after we do a components' action, we need to wait for a UI's rendering or a response from the backend.

test.describe('example', async () => {
  await test('wait for text', async ({ page }) => {
    // click on Create button to open the popup
    await page.getByRole('button', { name: 'Create' }).first().click()
    // wait for response
    await page.waitForResponse(`${process.env.BACKEND_URL}`, {
      timeout: 5000,
    })
    // wait for the text
    await page.waitForSelector('text=Create Test', { timeout: 5000 })
  })
})

If we want to navigate to the target URL and there is some redirecting, we have to wait for the final URL.

test.describe('example', async () => {
  await test('wait for url', async ({ page }) => {
    // Go to root
    await page.goto(`/`)
    // by default, the website redirect to the login
    // wait for URL
    await page.waitForURL('/login', { timeout: 5000 })
  })
})

Because some pages use SSR for rendering. The page is loaded without any assets. we need to wait for a short time until those assets have loaded.

test.describe('example', async () => {
  await test('wait for assets loading', async ({ page }) => {
    // go to signup page
    await page.goto(`/sign_up`)
    // wait for assets
    await page.waitForTimeout(1000)
  })
})

Please note that: Avoid using delay as possible as we can due to it taking longer time than other solutions.

How to run tests in our projects

Before running e2e test, you need to run tilt with production environment.

tilt-env

On normal frontend development, the frontend service will be built with Vite for using its feature such as Hot Module Reload. And backend services will have Physical volume which stores data permanently on the host machine. This makes the backend service's containers can be restarted without losing data.

But using the environment(vite) for e2e testing is different. With Vite, the e2e test will crash because of reloading page stuff from Vite.

So, an environment for developing the e2e test must change from Vite to "Rollup" which is a production-like one.

On e2e testing, this e2e environment mustn't have physical volume component to persist the data from the test, and data from previous testing need to be clean before running new testing.

Setting up the environment for e2e testing is different from normal development by using make up-test (instead of make up).

# /tilt/Makefile

up-test: # Use this target to setup environment for e2e testing
  Build frontend with Rollout
  Use temporaly data store. The data vanish when restart container.

## Normal development
# up:
#   Build frontend with Vite
#   Use persistant data store

After you already set up the environment for e2e testing, you can run test with:

cd apps/<appName>/ && make test

If you want to run with the debugger, run make test-debug instead

cd apps/<appName>/ && make test-debug

What the target do? The test target will do 2 main operation. First, clean all data in backend's services and then run the test to it.

Congrats🎉 , now you done for e2e testing development setup and test the service. you can repeat all those step for setup the environment (just make up-test on /tilt then make test on your service)

Apart from setup e2e testing environment, you can switch between normal and e2e development by switch a command between make up and make up-test without losing data which you created when you develop feature on normal environment(with physical volume).