Microsoft Power Query | Web Scraping | "422 Unprocessable Entity" error when adding a query parameter

Question

I'm using the Microsoft Power Query Tool to web scrape the Shopify app store. The URL I provide has a query parameter: (https://apps.shopify.com/search?q=scarcity), that being search?q=scarcity. When I attempt to scrape, I keep getting the same error: "422 Unprocessable Entity".

I've tried submitting the URL in its entirety and also breaking it into 2 parts using the Advanced option: https://apps.shopify.com/ and search?q=scarcity, each resulting in the same error: "422 Unprocessable Entity".

I scraped the root domain: https://apps.shopify.com/ and was successful - so I can't imagine there's an issue with headers, cookies, scraping restrictions, etc.

How can I successfully scarpe this URL, with the provided query parameter, using the Microsoft Power Query Tool?

Entry into the power query tool:

Resulting error

Answer 1

Try adding the Accept header with text/html, application/xhtml+xml. This will get you passed the 422 error.