Dilemma and strategy for handling big JSON file in React app

27 Views Asked by At

So im building application mostly for learning purposes in React. The app is about car manufactures. So i started building my JSON file(i keep it localy) with all the brands, information and models. What i quickly realized is that the JSON file will get a lot big. For now its around 3MB and by the time i finished it may be 150MB or larger.

Now i learned that there is a tool like React Query where i can for specific page(for example if u go to brand Toyota on app), get only specific part of that large JSON which is only for Toyota(even though dont know how it workes behind scenes since u make a fetch request to the whole file, but main thing is that it can do the job, to "save" a user not getting whole JSON).

So ok this tool can help me, since for end user theres no need if he goes on toyota page, to get all the data from json, just to see toyota information.

My dilemma is should i keep all the information in one JSON file or should i split it up as much as i can, and perform different fetch requests to different end points for every different brand?(this seems like a lot more work)

What definitely ill do, is to make server with this database(friend will help me he was talking about sql lite or something like that), but again dont know as i explained above what is best for data in terms one file or more files.

What i most interested is for end user to have smooth experience.

I need advice from more experience people about this topic, and suggestion.

Thank you.

For now JSON file is small(30 000 lines) so even when i perform fetch request my computer and browser are dealing with it fine, but im thinking how it would act when it gets like 50 times more bigger than that.

1

There are 1 best solutions below

1
Robin van der Heijden On

Ideally you'll only ever want to fetch the data required to be able to properly display your page. Any data you don't use, shouldn't be fetched.

In a production app, you'd ideally have a back-end which you can call to query the database and generate a JSON response with only the data you need. This isn't possible without setting up an actual back-end to handle this logic.

If you truly want to use your own data, your best bet is probably to simply split the JSON into different files, based on your pages' needs. Alternatively you could set up a local database using something like MongoDB, however this is arguably overkill if you're simply learning React and don't need to have a highly dynamic database/interface.

My personal recommendation if you're simply trying to learn React though, is to use a free online API which provides you with data. You won't be able to choose exactly what data they provide, but you'll have a functional back-end so that you can focus on learning React rather than learning data modeling and back-end engineering.

A quick google search will probably find you a decent amount of results. This is a simple one I found, but it is quite small (30 cars total): https://freetestapi.com/apis/cars

EDIT: This is all with the assumption that you're using vanilla React. If you're using something like NextJS, you could potentially look into setting up an API route yourself, which uses Javascript to mutate your JSON data and return only the necessary data as a response