TypeError: terminated [cause]: SocketError: other side closed in fetch Nodejs

Question

I got this error when I keep fetching an external API, I am simply calling:

await fetch(url, method: "POST", headers:headers, body: JSON.stringify(payload))

TypeError: terminated
    at Fetch.onAborted (node:internal/deps/undici/undici:11442:53)
    at Fetch.emit (node:events:514:28)
    at Fetch.terminate (node:internal/deps/undici/undici:10695:14)
    at Object.onError (node:internal/deps/undici/undici:11537:36)
    at Request.onError (node:internal/deps/undici/undici:8310:31)
    at errorRequest (node:internal/deps/undici/undici:10378:17)
    at TLSSocket.onSocketClose (node:internal/deps/undici/undici:9811:9)
    at TLSSocket.emit (node:events:526:35)
    at node:net:337:12
    at TCP.done (node:_tls_wrap:631:7) {
  [cause]: SocketError: other side closed
      at TLSSocket.onSocketEnd (node:internal/deps/undici/undici:9790:26)
      at TLSSocket.emit (node:events:526:35)
      at endReadableNT (node:internal/streams/readable:1376:12)
      at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
    code: 'UND_ERR_SOCKET',
    socket: {
      localAddress: '172.17.0.2',
      localPort: 48798,
      remoteAddress: '13.232.157.196',
      remotePort: 443,
      remoteFamily: 'IPv4',
      timeout: undefined,
      bytesWritten: 1594938,
      bytesRead: 12318698
    }
  }
}

I have read SocketError: other side closed, however, I am still very confused how can I solve it or get more hint on this error.

How can I fix it or trace where's wrong with my request?

Answer 1

Note: everything below is unsure. I don’t have the Node knowledge for this.

We encountered this issue at work (Node 18.x, AWS Lambda environment), and what we found out is Node sometimes closes the TLS connection too soon.

According to an undici issue, the response must always be consumed (before ending your current JS execution block, if I properly get it).

So, this may sometimes fail:

const myFetcher = async (url) => await fetch(url)

This may be a workaround:

const myFetcher = async (url) => {
  const res = await fetch(url)

  /**
   * Response body must be consumed to avoid socket error.
   * https://github.com/nodejs/undici/issues/583#issuecomment-855384858
   */
  const clonedRes = res.clone() // alternative: `res.clone()`

  return clonedRes; // if using previous alternative: `return res`
};

This may be another one if you don’t need to use the response furthermore:

const myFetcher = async (url) => {
  const res = await fetch(url)

  /**
   * Response body must be consumed to avoid socket error.
   * https://github.com/nodejs/undici/issues/583#issuecomment-855384858
   */
  res.body.getReader() //

  // There’s no point returning `res`: it is consumed by the previous line.
};

I think the workarounds work because the response body is consumed inside the function definition (= in myFetcher) instead of outside like in the first example. I found a note about a TCP mechanism named backpressure on the MDN Object.clone, and it seems Node also has a full guide on it. My understanding of it in our case is that the fetch response is big enough to have Node slowing down the process of it: basically it says to the other side (either the system behind the URL called by fetch either an internal part of Node holding the response in memory/buffer) “can you wait one moment before we finish our transaction?”, and the remote part maybe can’t wait, so it gives up, and this error is thrown. That’s why consuming the body as soon as possible might solves this.

When looking at the stack, we have only one place referring “other side closed” in Node, and I guess that it’s not easy to isolate the exact fail leading to this behavior.

By looking for the stack trace line saying at TCP.done (node:_tls_wrap:631:7), I found a pull request in Node core to address our exact issue. According to the PR author and a commenter, Node doesn’t comply a lot with the part of the HTTP RFC about Message Body length, and apparently it’s currently not fixed neither in Node 20.

(I may be all wrong, of course.)