I am trying to optimize memory usage of my python program owing to my limited memory in a server. Following coding optimization, I am doing memory optimization. Input dataset is rather big. For this, I am investigating whether I can optimize the integer columns in the input dataset I use during the program. These integer values are dynamically changing in it. It means that there are sometimes mostly in the range of INT64 value ranges or in the range of INT32 or INT16. Is there any way to store these integer values dynamically in a dataset as INT64, INT16 or etc while I run the program?
I investigated any possibility to store dataset's integer columns dynamically but I couldn't find any way so far.