How can I access Chrome Devtools Network Logs data in NodeJS script?

Question

I'm currently trying to build a project that requires me to read data from a website (OpenSea marketplace). When I go to the page, it sends a POST request to an API (in pic 1), I want to access the data it returns (in pic 2) in my NodeJS/TypeScript project.

Is there maybe an API that can do this or is it not even possible?

Pic 1 (I'm trying to access the graphql/ request):

Pic 2 (This is the data I'm trying to get into my code):

Answer 1

you can hijack the XMLHttpRequest and fetch functions. unconventional, but it would work

Based on the comment I placed, a small function can be used on the client side to do such

function listen(fn){
  //this line simply adds functions to call in an array based on setup below
  if(listen.prototype.arguments){listen.prototype.arguments.push(fn)}
  
  //below is setup(functions hijacked so services that use them report to this too)
  listen.prototype.arguments=[] //array of functions to call
  let original1=XMLHttpRequest.prototype.send, original2=fetch
  function replace1(){
    let toReturn=original1.bind(this)(...arguments)
    this.addEventListener("load",res=>fn(this.responseText))
    return toReturn
  }
  function replace2(){
    let toReturn=original2.bind(this)(...arguments)
    toReturn.then(res=>res.text().then(fn))
    return toReturn
  }
  Object.defineProperty(XMLHttpRequest.prototype,"send",{value:replace1})
  Object.defineProperty(window,"fetch",{value:replace2})
}



//example usage
let socket=some_Web_Socket_Your_Backend_To_Handle_This_Data
//perhaps this is not the most elaborate example but I hope the point is understood
listen(socket.send.bind(socket))

Answer 2

• Do check out this service, to get started to build your own REST API & use it in your project. Leetcode Public REST API

• Below code might give you a deep understanding of how to extract the data. Assume this is the GraphQL Query.

export const recentSubmissionsQuery = `
    query recentAcSubmissions($username: String!, $limit: Int!) {
        recentAcSubmissionList(username: $username, limit: $limit) {
            id
            title
            titleSlug
            timestamp
        }
    }
`;

You can create an endpoint in NodeJS/ExpressJS like this

router.get("/:username/submissions", async (req, res) => {
  const query = recentSubmissionsQuery;
  const { username } = req.params;
  let { limit } = req.query;
  if (!limit) limit = 3;

  const data = await fetchGraphQLData(query, { username, limit });
  res.json(data);
});

And the fetchGraphQLData will look something similar to like this:

import fetch from "node-fetch";

const URL = BASE_URL;

const fetchGraphQLData = async (query, variables) => {
  const options = {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      query,
      variables,
    }),
  };

  try {
    const response = await fetch(URL, options);
    const data = await response.json();
    return data;
  } catch (error) {
    console.error(error);
  }
};

export default fetchGraphQLData;

---

• use node-fetch or axios libraries to do HTTP requests else, you will be facing issues in your deployment.