sql.js FATAL ERROR: invalid array length Allocation failed - JavaScript heap out of memory

Question

We are trying to insert (1200 records) into sqlite database (existing database of size 106 MB) from node js, using sqlite-chiper node module and getting the below error.

We have even increased the --max-old-space-size=10192 till 10GB still we are getting the below error.

<--- Last few GCs --->

[16316:00000292A20A0C10]    15038 ms: Scavenge 1178.9 (1213.6) -> 1178.9 (1213.6) MB, 115.4 / 0.0 ms  (average mu = 0.999, current mu = 0.999) allocation failure
[16316:00000292A20A0C10]    16143 ms: Mark-sweep 1752.7 (1787.4) -> 1743.6 (1779.3) MB, 517.7 / 0.0 ms  (+ 5.4 ms in 12 steps since start of marking, biggest step 0.9 ms, walltime since start of marking 2513 ms) (average mu = 0.945, current mu = 0.933) al

<--- JS stacktrace --->

==== JS stack trace =========================================

    0: ExitFrame [pc: 00007FF7BDBA2C6D]
Security context: 0x02fbb1f408a1 <JSObject>
    1: write [0000029FB013E4D9] [E:\Product\Trunk\node_modules\sql.js\js\sql.js:~2114] [pc=00000379B65AE651](this=0x029fb01375f9 <Object map = 0000014672038741>,0x038bd5a3ff79 <JSObject>,0x032891c17b29 <Int8Array map = 0000014672002781>,6378736,4096,112812032,0x028694c804a9 <undefined>)
    2: write [0000032891C1CAB9] [E:\Product\Trunk\node_modules\s...

FATAL ERROR: invalid array length Allocation failed - JavaScript heap out of memory

Writing Node.js report to file: report.20200831.233420.16316.0.001.json
Node.js report completed
 1: 00007FF7BCFD094F napi_wrap+124431
 2: 00007FF7BCF72696 public: bool __cdecl v8::base::CPU::has_sse(void)const __ptr64+34502
 3: 00007FF7BCF73356 public: bool __cdecl v8::base::CPU::has_sse(void)const __ptr64+37766
 4: 00007FF7BD776F4E private: void __cdecl v8::Isolate::ReportExternalAllocationLimitReached(void) __ptr64+94
 5: 00007FF7BD75EF91 public: class v8::SharedArrayBuffer::Contents __cdecl v8::SharedArrayBuffer::Externalize(void) __ptr64+833
 6: 00007FF7BD62C85C public: static void __cdecl v8::internal::Heap::EphemeronKeyWriteBarrierFromCode(unsigned __int64,unsigned __int64,class v8::internal::Isolate * __ptr64)+1436
 7: 00007FF7BD64287D private: class v8::internal::HeapObject __cdecl v8::internal::Factory::AllocateRawFixedArray(int,enum v8::internal::AllocationType) __ptr64+61
 8: 00007FF7BD649C34 private: class v8::internal::Handle<class v8::internal::FixedArray> __cdecl v8::internal::Factory::NewFixedArrayWithFiller(enum v8::internal::RootIndex,int,class v8::internal::Object,enum v8::internal::AllocationType) __ptr64+52
 9: 00007FF7BD649BF1 public: class v8::internal::Handle<class v8::internal::FixedArray> __cdecl v8::internal::Factory::NewFixedArray(int,enum v8::internal::AllocationType) __ptr64+65
10: 00007FF7BD52AD9F public: class v8::Isolate * __ptr64 __cdecl v8::Message::GetIsolate(void)const __ptr64+7663
11: 00007FF7BD3DE55A public: class v8::internal::Object __cdecl v8::internal::OrderedHashMap::ValueAt(int) __ptr64+61274
12: 00007FF7BDBA2C6D public: virtual bool __cdecl v8::internal::SetupIsolateDelegate::SetupHeap(class v8::internal::Heap * __ptr64) __ptr64+567949
13: 00000379B65AE651

Any suggestion on how to resolve this problem ?

Answer 1

Since we are getting memory error in sql.js, we have moved to sqlite3 for node js to resolve this problem