I have a try/catch around a fetch JavaScript call that reports caught errors. I periodically get the following report about "TypeError: Failed to fetch" exceptions from Googlebot requests:
[2023-09-21T00:00:00.031Z] POST https://findhumane.com/find_humane/api/get_location_details?v=12.1.20230917
[2023-09-21T00:00:00.034Z] handleErrors: Failed to fetch (class: TypeError)
...
HTTP_USER_AGENT: Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.5938.88 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
While there are many potential causes for TypeError during fetch, the most common is improper CORS response headers, but I don't see any problems from Chrome network inspector when I perform the same use case manually; for example, it shows good CORS headers:
POST https://findhumane.com/find_humane/api/get_location_details?v=12.1.20230917
200 OK
Response Headers:
Access-Control-Allow-Methods: GET, POST, PATCH, PUT
Access-Control-Allow-Origin: *
Access-Control-Max-Age: 7200
...
The odd thing is that since I'm getting the exception email from the exception catch that is being POSTed to the same domain, then I don't know what the difference is between that fetch and this application fetch.
Any ideas or how I can debug further? I have two ideas:
- Run a tcpdump on my backend all the time between my reverse proxy (HAProxy) and app server without TLS and then correlate to the Googlebot exception email to confirm 100% that the right CORS response headers are being sent.
- Maybe I can somehow route the console into an in-memory string and report other console warnings/errors in my exception email?
I do have a 60 second timeout abort on the fetch:
try {
const controller = new AbortController();
const timeoutId = setTimeout(() => controller.abort(), 60000);
const response = await fetch(
uri,
{
method: "POST",
headers: {
"Accept": "application/json",
"Content-Type": "application/json",
},
signal: controller.signal,
},
);
...
However, the timestamps above show the TypeError occurred about 3 milliseconds after the request started.