Chrome DevTools Protocol Fetch Domain - getResponseBody - apparently fails with HTTP redirects

Question

I am wishing to collect the body of an HTTP request, including when the page redirects to elsewhere. Clearly, I can use non-Fetch domain mechanisms such as Network.getResponseBody. That works fine for the "final" page in a chain of redirections, but cannot be used for the intermediate pages because Chrome appears to dump the content when going to the next redirection target.

So, I implemented Fetch.enable( { patterns: [ { requestStage: Response } ] } ) (using PHP, but the details of that are irrelevant, as you will see). No error is returned from this method call. After then doing a Page.navigate, I wait for a Fetch.requestPaused event which contains members requestId, responseStatusCode and responseHeaders and then send a Fetch.getResponseBody (using the requestId from the Fetch.requestPaused) and the response I get depends on what the actual response to the page itself was. So, for a 200, I get a response body (hurray), but for a 30x (301, 302 etc), I always get error code -32000 with the message "Can only get response body on requests captured after headers received". Now, issuing that error message is inconsistent (in my view) with the Fetch.requestPaused event data, even if Chrome DevTools Protocol (CDP) was not intended to capture the bodies of HTTP redirected pages. By the way, pages with content triggered redirection (via a META element or JavaScript) are captured okay, I assume because they return a 200 status code.

So, is the issue in the sequence of calls I'm following or in the error message returned by Fetch.getResponseBody and am I correctly assuming CDP was not intended to capture the bodies of documents in a redirection chain (apart from the last one, obviously)?

Answer 1

You need to continue the request on a 301/302 and let the browser follow it (there is no body in a redirect):

    if (
      params.responseStatusCode === 301 || params.responseStatusCode === 302
    ) {
      await this.#client.send('Fetch.continueRequest', {
        requestId,
      });
    } else {
      // get body here
      const responseCdp = await this.#client.send('Fetch.getResponseBody', {
        requestId,
      });
    
      await this.#client.send('Fetch.fulfillRequest', {
        requestId,
        responseCode: params.responseStatusCode,
        responseHeaders: params.responseHeaders,
        body: responseCdp.body,
      });
    }