We have developed a Node.js application with socket communication for GPS Tracker. All it have to do is listening to new TCP connection of GPS Tracker, read the packet, analyse the GPS data and store it in a database.
All running good, but we are facing an excessive use of RAM.
When we start the application, it takes approximatively 150MB of RAM to run. But few days later it takes more than 10 GB of RAM or even more if we don't restart it.
It doesn't seems to go up exponentialy, because when we restart it, even if we check every hours, the process dosent go more than 150/160 MB. It's more like something happend and then the RAM take 5MB every second ...
We have already tried things to inspect what could be the cause :
Trying to handle many events as possible.
We got 2 types of specific message :
snapshot --> Create a V8 heap snasphot.
GenereFichierConnexionActive --> Create a file with all the active socket connection.
So when the server taking RAM we ask to create thoses files.
Results are the connection files contains approximatively 10 active connections, so for us it's look good.
But for the heap snasphot, we have compared a snapshot 1 hour after the start and a snapshot when the excessiv RAM is there.
With the DevTools of Chrome, we can see "Mapping" constructor got a delta of +2869 (New=2869 / Deleted=0), but how can we see what cause this problem ? (and it is a problem after all) ?
We aren't expert of heap analysing, so we are seeking help for that. Or if you have any other idea of what problem could it be ?
Here is the main SocketServer class :
import dotenv from 'dotenv'
import ContextTrame from "../../context/ContextTrame";
import TeltonikaTrame from "../../trames/TeltonikaTrame";
import RedirectionTrame from "../../trames/RedirectionTrame";
import NS10Trame from "../../trames/NS10Trame";
import net, {Socket} from 'net';
import prisma from "../../lib/db";
import * as v8 from "v8";
import fs from 'fs';
dotenv.config();
const args = process.argv.slice(2);
const portIndex = args.indexOf("-p");
let port = 3000;
const activeConnection = new Map();
const TIMEOUT = 5 * 60 * 1000; // 5 minutes
if (portIndex !== -1 && args[portIndex + 1]) {
port = parseInt(args[portIndex + 1], 10);
}
class SocketServer
{
private static socketServer = net.createServer(SocketServer.onConnect);
private PORT: number = port;
public constructor()
{
SocketServer.socketServer.listen(this.PORT, () => {
console.log(`socket server bound on port ${this.PORT}`);
});
}
/**
* Méthode appelée lors de la connexion d'un client-backup.
* @param socket Socket du client-backup.
*/
public static onConnect(socket: Socket): void
{
console.log('a user connected : ' + socket.remoteAddress + ':' + socket.remotePort);
socket.setTimeout(TIMEOUT);
socket.on('data', (data) => SocketServer.onData(socket, data));
socket.on('error', (err) => SocketServer.onError(socket, err));
socket.on('close', (hadError) => SocketServer.onClose(socket,hadError));
socket.on('end', () => SocketServer.onEnd(socket));
socket.on('timeout', () => SocketServer.onTimeout(socket));
activeConnection.set(socket,{ lastActiveTime: Date.now() });
}
/**
* Méthode appelée lors de la réception de données.
* @param socket Socket du client-backup.
* @param data Données reçues.
*/
private static onData(socket: any, data: Buffer): void
{
// On maj la dateheure d'activité
activeConnection.get(socket).lastActiveTime = Date.now();
// Sanapshot de la mémoire ram avec V8
if(Buffer.from(data).toString() === 'snapshot') {
console.log('snapshot received'); // ! Comments
v8.writeHeapSnapshot(`./snapshots/heap_snapshot-${Date.now()}.heapsnapshot`);
}
// Si réception de la commande de snapshot.
if(Buffer.from(data).toString() === 'GenereFichierConnexionActive') {
this.generateActiveConnectionsFile();
}
const contextTrame = new ContextTrame();
// Si la trame est un IMEI.
if(Buffer.from(data).toString('hex').substring(0, 4) === '000f') {
console.log('IMEI received'); // ! Comments
socket.imei = Buffer.from(data).toString('ascii').substring(2);
socket.write(Buffer.from('01', 'hex'));
} else {
// Si la trame est une trame Teltonika.
if (socket.imei !== undefined) {
console.log('Teltonika trame received'); // ! Comments
contextTrame.setStrategy(new TeltonikaTrame(socket, data));
} else {
const hexString = Buffer.from(data).toString('hex').toUpperCase();
const last12Characters = hexString.substring(hexString.length - 12);
// Si la trame est une trame de redirection.
if (last12Characters === '0D0A0D0A0D0A') {
console.log('Redirection trame received'); // ! Comments
contextTrame.setStrategy(new RedirectionTrame(data));
} else {
// Si la trame est une trame NS10.
if (parseInt( Buffer.from(data).toString('ascii').substring(0, 1)) == 2) {
console.log('NS10 trame received'); // ! Comments
contextTrame.setStrategy(new NS10Trame(data));
}
}
}
}
contextTrame.isStrategySet() &&
contextTrame.cut().then((res: boolean) => {
if (res) {
console.log('trame cuted'); // ! Comments
contextTrame.insert().then((res: boolean) => {
if (res) {
console.log('trame inserted'); // ! Comments
contextTrame.redirect();
} else {
console.log('trame not inserted'); // ! Comments
}
});
} else {
console.log('trame not cuted'); // ! Comments
}
});
}
/**
* Méthode appelée lors d'une erreur.
* @param socket Socket du client-backup.
* @param err Erreur.
*/
private static onError(socket: any, err): void
{
console.log(err);
socket.destroy(); // On supprime le socket pour être sur qu'il soit bien fermé
}
private static onClose(socket: any, hadError: boolean): void
{
activeConnection.delete(socket);
}
private static onEnd(socket: any): void
{
console.log("Client à fermer la connection : "+socket.address());
socket.destroy(); // On supprime le socket pour être sur qu'il soit bien fermé
}
private static onTimeout(socket: any): void
{
console.log("Socket tiemout : "+socket.address());
socket.destroy(); // On supprime le socket pour être sur qu'il soit bien fermé
}
/**
* Méthode appelée lors de l'envoi de données.
* @param socket Socket du client-backup.
* @param message Données à envoyer.
*/
public write(socket: any, message: Buffer): void
{
socket.write(message);
}
// Function to generate text file containing active connections
private static generateActiveConnectionsFile(): void
{
const filename = './snapshots/connexionActive.txt';
let fileContent = '';
activeConnection.forEach((data, socket) => {
fileContent += `Socket: ${socket.remoteAddress}:${socket.remotePort}`;
fileContent += ` --> Dernière data reçu : ${new Date(data.lastActiveTime).toLocaleString()}\n`;
});
fs.writeFile(filename, fileContent, err => {
if (err) {
console.error('Erreur pendant la création du fichier des connexions actives:', err);
} else {
console.log('Fichier des connexions actives généré :', filename);
}
});
}
}
export { SocketServer };