I am using a trading bot I created using Nodejs (Express), Postgres and React. React is only for the UI and all the work is done with Node. Postgres is used to store in DB trades information.
The application only runs on my localhost.
The workflow is like this :
- Fetch the list of every crypto there is on the trading platform (1 external API request)
- Loop on each crypto fetched and perform some "heavy" calculations on them (+150 external API requests and call lot of async helper functions)
- If one crypto meets required conditions, buy it (2 external API requests per bought crypto) then insert the trade data in DB (1 internal API request)
- When all cryptos have been looped through, repeat
I tried to simplify it the most I could because it is a little more complex than that but this is the basic workflow.
This works fine but is really slow. The loop on each crypto (2
) takes about a minute to complete (around 2.5 seconds per crypto).
Here is some code to give you an idea of the structure (again, simplified to the max because it uses lot of other helpers functions and is a few thousands lines of code total) :
// Controller :
exports.strategyBacktesting = async (req, res, next) => {
//... Some code ...
const markets = await getMarkets();
const calculationsResults = await calculations(markets);
//... More code ...
res.status(200).json({
// Return calculationResults and other stuff for the UI
)};
}
//----- Files in utils folders
// Called from controller
exports.getMarkets = async () => {
// Get all the listed cryptos via an external API request
// Loop through all the fetched data to filter the cryptos I want to work with
// Return a Promise (array containing all the cryptos I will do the calculations on)
}
// Called from controller
exports.calculations = async (markets) => {
// Loop through all the results from getMarkets() (markets param) (+150 results) = all the cryptos to work with
for (let cryptoPair in markets) ...
// In the loop:
// Call the getPairCandles() function (below) to get all the data needed per crypto :
// Note that the cryptoPair param is the object from the loop
const { /* data needed for the calculation */ } = await getPairCandles(cryptoPair)
// Perform all the needed calculations (one iteration = one crypto) => This is where it takes a long time (around 2.5 seconds per crypto)
// Calls lots of async helper functions and do the calculations
// If some required conditions are met, do :
//- 1 external async API call to buy the crypto
//- 1 external async API call to set a sell limit
//- One internal async API call to save the trade info in the DB
// When the loop is done on the +150 cryptos, return a Promise for the controller to return to UI
}
// Called from calculations() function above
const getPairCandles = async (cryptoPair) => {
// Make an external API request to get specific real-time data about one crypto (the cryptoPair param of the function)
// Loop through some data in the object received to filter the wanted properties
// Return a Promise containing 5 arrays (the data I need to calculate on this crypto)
}
I read about workers
and such in Nodejs that could help with heavy calculations. That being said, how could I implement it ? Is it even the good solution in my case ? The thing is, the loop that performs the calculations is the same that contains lots of calls to async
functions returning Promises
and async
API calls. This is the loop that takes around a minute to complete.
So I think I can greatly improve the performance, but don't know how. What can I do about it ? Could someone please guide me in the good direction ?
Thank you very much