DEVHIDE
Login Or Sign up

How to get all errors and assert them (or add them into report) on the page instead of getting first error and stopping

83 Views Asked by At

I am testing html header orders in the following code, derived from this answer. But, test stops when it hits the first error. I want to get all errors on the page instead of getting first error and stops

I need to improve my code for it

async assertHtmlHeadingOrders() {
    await this.page.waitForLoadState('domcontentloaded')
    const failMsg = (prev, curr) =>
      `h${curr} was more than one level deeper than previous header h${prev}`
    const headers = await this.page
      .locator('h1, h2, h3, h4, h5, h6')
      .evaluateAll(els => els.map(el => +el.tagName.slice(1)))
    // eslint-disable-next-line no-plusplus, id-length
    for (let i = 1; i < headers.length; i++) {
      expect(headers[i] - headers[i - 1], failMsg(headers[i - 1], headers[i])).toBeLessThan(2)
    }
  }
typescript playwright playwright-test
Original Q&A
1

There are 1 best solutions below

3
On BEST ANSWER

You can put the failing pairs into an array, then assert one time on the whole array, enabling a clearer error message.

import {expect, test} from "@playwright/test"; // ^1.39.0

const failMsg = pairs =>
  `The following header pairs were invalid: ${
    pairs.map(e => `h${e.prev} -> h${e.curr}`).join(", ")}`;

const findInvalidHeaderPairs = async page => {
  const headers = await page
    .locator("h1, h2, h3, h4, h5, h6")
    .evaluateAll(els => els.map(el => +el.tagName.slice(1)));
  return headers
    .slice(1)
    .map((e, i) => ({prev: headers[i], curr: e}))
    .filter(e => e.curr - e.prev > 1);
};

test("fail so the assertion message can be seen", async ({page}) => {
  const html = `<!DOCTYPE html><html><body>
  <h1>1</h1>
  <h2>2</h2>
  <h4>4</h4>
  <h3>3</h3>
  <h5>5</h5>
  <h5>5</h5>
  <h6>6</h6>
  <h3>3</h3>
  <h6>6</h6>
  </body></html>`;
  await page.setContent(html); // replace with page.goto("<Your URL>"); when you're working with a real site
  const pairs = await findInvalidHeaderPairs(page);
  expect(pairs, failMsg(pairs)).toHaveLength(0);
});

Output:

$ npx playwright test

Running 1 test using 1 worker

  ✘  1 pw.test.js:17:5 › fail so the assertion message can be seen (98ms)


  1) pw.test.js:17:5 › fail so the assertion message can be seen ───────────────

    Error: The following header pairs were invalid: h2 -> h4, h3 -> h5, h3 -> h6

    expect(received).toHaveLength(expected)

    Expected length: 0
    Received length: 3
    Received array:  [{"curr": 4, "prev": 2}, {"curr": 5, "prev": 3}, {"curr": 6, "prev": 3}]

      30 |   const pairs = await findInvalidHeaderPairs(page);
    > 31 |   expect(pairs, failMsg(pairs)).toHaveLength(0);
         |                                 ^
      32 | });

        at pw.test.js:31:33

  1 failed
    pw.test.js:17:5 › fail so the assertion message can be seen ────────────────

The same suggestions as in this previous answer apply: adding a custom matcher or polling, as fits your needs.

Adding a custom matcher would abstract away the logic shown here into the matcher rather than into functions, which is a bit more elegant for the caller. If you're using this on many tests, it's probably worth the extra work. Here's an example that also polls to ensure the test eventually passes:

import {expect, test} from "@playwright/test";

expect.extend({
  async toHaveValidHeaders(page) {
    let pairs;
    let error;

    try {
      await expect.poll(async () => {
        const headers = await page
          .locator("h1, h2, h3, h4, h5, h6")
          .evaluateAll(els => els.map(el => +el.tagName.slice(1)));
        pairs = headers
          .slice(1)
          .map((e, i) => ({prev: headers[i], curr: e}))
          .filter(e => e.curr - e.prev > 1);
        return pairs;
      }).toHaveLength(0);
    }
    catch (e) {
      error = e;
    }

    if (!pairs) {
      return {
        message: () => error?.message ??
          "unknown page failure in toHaveValidHeaders",
        pass: false,
      };
    }
    else if (pairs.length) {
      return {
        message: () =>
          `The following header pairs were invalid: ${
            pairs.map(e => `h${e.prev} -> h${e.curr}`).join(", ")}`,
        pass: false,
      };
    }

    return {
      message: () => "passed",
      pass: true,
    };
  },
});

test("fail so the assertion message can be seen", async ({page}) => {
  const html = `<!DOCTYPE html><html><body>
  <h1>1</h1>
  <h2>2</h2>
  <h4>4</h4>
  <h3>3</h3>
  <h5>5</h5>
  <h5>5</h5>
  <h6>6</h6>
  <h3>3</h3>
  <h6>6</h6>
  </body></html>`;
  await page.setContent(html);
  await expect(page).toHaveValidHeaders();
});

test("page with valid headers", async ({page}) => {
  await page.goto("https://playwright.dev/docs/test-configuration");
  await expect(page).toHaveValidHeaders();
});

test("page with (eventual) valid headers", async ({page}) => {
  test.setTimeout(10_000);
  const html = `<!DOCTYPE html><html><body>
  <h1>1</h1>
  <h2>2</h2>
  <h4>4</h4>
  <script>
  setTimeout(() => {
    document.querySelector("h4").remove();
  }, 3000);
  </script>
  </body></html>`;
  await page.setContent(html);
  await expect(page).toHaveValidHeaders();
});

Now the assertion failure is clearer, and we can also work with pages that add more headers after navigation:

$ npx playwright test

Running 3 tests using 1 worker

  ✘  1 pw.test.js:47:5 › fail so the assertion message can be seen (5.0s)
  ✓  2 pw.test.js:63:5 › page with valid headers (533ms)
  ✓  3 pw.test.js:68:5 › page with (eventual) valid headers (4.0s)


  1) pw.test.js:47:5 › fail so the assertion message can be seen ───────────────

    Error: The following header pairs were invalid: h2 -> h4, h3 -> h5, h3 -> h6

      58 |   </body></html>`;
      59 |   await page.setContent(html);
    > 60 |   await expect(page).toHaveValidHeaders();
         |                      ^
      61 | });
      62 |
      63 | test("page with valid headers", async ({page}) => {

        at pw.test.js:60:22

  1 failed
    pw.test.js:47:5 › fail so the assertion message can be seen ────────────────
  2 passed (10.3s)

Here's a simpler custom matcher that doesn't include polling, which is better if you only want to check headers at page load, or want to explicitly use a poll in the test:

import {expect, test} from "@playwright/test";

expect.extend({
  async toHaveValidHeaders(page) {
    const headers = await page
      .locator("h1, h2, h3, h4, h5, h6")
      .evaluateAll(els => els.map(el => +el.tagName.slice(1)));
    const pairs = headers
      .slice(1)
      .map((e, i) => ({prev: headers[i], curr: e}))
      .filter(e => e.curr - e.prev > 1);

    if (pairs.length === 0) {
      return {
        message: () => "passed",
        pass: true,
      };
    }

    return {
      message: () =>
        `The following header pairs were invalid: ${
          pairs.map(e => `h${e.prev} -> h${e.curr}`).join(", ")}`,
      pass: false,
    };
  },
});

test("fail so the assertion message can be seen", async ({page}) => {
  const html = `<!DOCTYPE html><html><body>
  <h1>1</h1>
  <h2>2</h2>
  <h4>4</h4>
  <h3>3</h3>
  <h5>5</h5>
  <h5>5</h5>
  <h6>6</h6>
  <h3>3</h3>
  <h6>6</h6>
  </body></html>`;
  await page.setContent(html);
  await expect(page).toHaveValidHeaders();
});

test("page with valid headers", async ({page}) => {
  await page.goto("https://playwright.dev/docs/test-configuration");
  await expect(page).toHaveValidHeaders();
});

test("page with (eventual) valid headers", async ({page}) => {
  test.setTimeout(10_000);
  const html = `<!DOCTYPE html><html><body>
  <h1>1</h1>
  <h2>2</h2>
  <h4>4</h4>
  <script>
  setTimeout(() => {
    document.querySelector("h4").remove();
  }, 3000);
  </script>
  </body></html>`;
  await page.setContent(html);
  await expect(() => expect(page).toHaveValidHeaders()).toPass();
});

Related Questions in TYPESCRIPT

Related Questions in PLAYWRIGHT

Related Questions in PLAYWRIGHT-TEST

Trending Questions

Popular # Hahtags

javascript python java c# php android html jquery c++ css ios sql mysql r reactjs node.js arrays c asp.net json python-3.x ruby-on-rails .net sql-server swift django angular objective-c pandas excel

Popular Questions

Copyright © 2021 Jogjafile Inc.