Creating 2D std::vector as input vector for Tensor Flow Lite results in crashing ESP although there is enough heap memory

Question

I want to create a 2D input vector for my machine learning model. The model runs on a ESP32 but I am running into issues when it comes to setting up such an vector. I initialise a vector by std::vector<std::vector<float>> testing_vector; and reserve memory in my setup routine by testing_vector.reserve(1000);

This works fine when I reserve for 1000 elements. When I reserve for over 4700 elements though, my ESP crashes although ESP.getFreeHeap() and ESP.getMaxAllocHeap() shows me that there should be enough heap memory available.

Edit: Thanks to the answers of "Some programmer dude" and "molbdnilo" below I realised that my calculations for required space first was wrong. When I reserve 4700 elements of 12Byte vector objects this would now result in 56.400 Bytes which is still under Max Alloc Heap of 110580 though.

Input When I reserve memory for 4.700 elements by testing_vector.reserve(4700);the ESP crashes

Outcome My serial monitor with enabled "ESP32 Exception Decoder" gives me:

Rebooting...
ets Jul 29 2019 12:21:46

rst:0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0030,len:1184
load:0x40078000,len:13132
load:0x40080400,len:3036
entry 0x400805e4
Total heap: 298076
Free heap: 274232
Total PSRAM: 0
Free PSRAM: 0
Max Alloc Heap: 110580
Max Vector Size: 357913941

abort() was called at PC 0x4013457b on core 1


Backtrace:0x400834e1:0x3ffb26900x40088c5d:0x3ffb26b0 0x4008d6b5:0x3ffb26d0 0x4013457b:0x3ffb2750 0x401345c2:0x3ffb2770 0x401346bb:0x3ffb2790 0x4013461a:0x3ffb27b0 0x400d1b16:0x3ffb27d0 0x400d1d95:0x3ffb27f0 0x40129fe7:0x3ffb2820

  #0  0x400834e1:0x3ffb2690 in panic_abort at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/esp_system/panic.c:402 
  #1  0x40088c5d:0x3ffb26b0 in esp_system_abort at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/esp_system/esp_system.c:128
  #2  0x4008d6b5:0x3ffb26d0 in abort at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/newlib/abort.c:46
  #3  0x4013457b:0x3ffb2750 in __cxxabiv1::__terminate(void (*)()) at /builds/idf/crosstool-NG/.build/HOST-x86_64-w64-mingw32/xtensa-esp32-elf/src/gcc/libstdc++-v3/libsupc++/eh_terminate.cc:47
  #4  0x401345c2:0x3ffb2770 in std::terminate() at /builds/idf/crosstool-NG/.build/HOST-x86_64-w64-mingw32/xtensa-esp32-elf/src/gcc/libstdc++-v3/libsupc++/eh_terminate.cc:57
  #5  0x401346bb:0x3ffb2790 in __cxa_throw at /builds/idf/crosstool-NG/.build/HOST-x86_64-w64-mingw32/xtensa-esp32-elf/src/gcc/libstdc++-v3/libsupc++/eh_throw.cc:95
  #6  0x4013461a:0x3ffb27b0 in operator new(unsigned int) at /builds/idf/crosstool-NG/.build/HOST-x86_64-w64-mingw32/xtensa-esp32-elf/src/gcc/libstdc++-v3/libsupc++/new_op.cc:54
  #7  0x400d1b16:0x3ffb27d0 in __gnu_cxx::new_allocator<std::vector<float, std::allocator<float> > >::allocate(unsigned int, void const*) at c:\users\d.fluhr\.platformio\packages\toolchain-xtensa-esp32\xtensa-esp32-elf\include\c++\8.4.0\ext/new_allocator.h:111
      (inlined by) std::allocator_traits<std::allocator<std::vector<float, std::allocator<float> > > >::allocate(std::allocator<std::vector<float, std::allocator<float> > >&, unsigned int) at c:\users\d.fluhr\.platformio\packages\toolchain-xtensa-esp32\xtensa-esp32-elf\include\c++\8.4.0\bits/alloc_traits.h:436     
      (inlined by) std::_Vector_base<std::vector<float, std::allocator<float> >, std::allocator<std::vector<float, std::allocator<float> > > >::_M_allocate(unsigned int) at c:\users\d.fluhr\.platformio\packages\toolchain-xtensa-esp32\xtensa-esp32-elf\include\c++\8.4.0\bits/stl_vector.h:296
      (inlined by) std::vector<float, std::allocator<float> >* std::vector<std::vector<float, std::allocator<float> >, std::allocator<std::vector<float, std::allocator<float> > > >::_M_allocate_and_copy<std::move_iterator<std::vector<float, std::allocator<float> >*> >(unsigned int, std::move_iterator<std::vector<float, std::allocator<float> >*>, std::move_iterator<std::vector<float, std::allocator<float> >*>) at c:\users\d.fluhr\.platformio\packages\toolchain-xtensa-esp32\xtensa-esp32-elf\include\c++\8.4.0\bits/stl_vector.h:1398
      (inlined by) std::vector<std::vector<float, std::allocator<float> >, std::allocator<std::vector<float, std::allocator<float> > > >::reserve(unsigned int) at c:\users\d.fluhr\.platformio\packages\toolchain-xtensa-esp32\xtensa-esp32-elf\include\c++\8.4.0\bits/vector.tcc:74
  #8  0x400d1d95:0x3ffb27f0 in setup() at src/main.cc:104
  #9  0x40129fe7:0x3ffb2820 in loopTask(void*) at C:/Users/d.fluhr/.platformio/packages/framework-arduinoespressif32/cores/esp32/main.cpp:42

Conclusion

I don't understand this output very well but I understand the result is invoked by my "reserve" method. But what could be the reason for this crash?

---

Code unfortunately the tensorflow lib is very big. This is why I just present my main.cc here. I reduced the main loop part but kept the setup part as it is for a better understanding whats going on in the background.

/* Copyright 2020 The TensorFlow Authors. All Rights Reserved.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/


#include "tensorflow/lite/micro/all_ops_resolver.h"
#include "tensorflow/lite/micro/micro_interpreter.h"
#include "tensorflow/lite/micro/system_setup.h"
#include "tensorflow/lite/schema/schema_generated.h"

// #include "main_functions.h"
#include "model.h"
#include "constants.h"
#include "output_handler.h"

// additional libraries by Daniel
#include "Arduino.h"
#include <chrono>
#include <iostream>
#include <vector>

#include "input_data.h"

//setting timer
using namespace std::chrono;
unsigned long interval = 2000;
auto t_0 = high_resolution_clock::from_time_t(0);
auto now   = high_resolution_clock::now();
auto previousMillis = duration_cast<milliseconds>(now - t_0).count();
bool debug_flag = true;

// Globals, used for compatibility with Arduino-style sketches.
namespace {
const tflite::Model* model = nullptr;
tflite::MicroInterpreter* interpreter = nullptr;
TfLiteTensor* input = nullptr;
TfLiteTensor* output = nullptr;
int inference_count = 0;

// increase if esp spits out "Failed to resize buffer"
constexpr int kTensorArenaSize = 55000;
uint8_t tensor_arena[kTensorArenaSize];

}  // namespace

// initialise test vector
std::vector<std::vector<float>> testing_vector;

// The name of this function is important for Arduino compatibility.
void setup() {

  // Map the model into a usable data structure. This doesn't involve any
  // copying or parsing, it's a very lightweight operation.
  model = tflite::GetModel(g_model);
  if (model->version() != TFLITE_SCHEMA_VERSION) {
    MicroPrintf("Model provided is schema version %d not equal to supported "
                "version %d.", model->version(), TFLITE_SCHEMA_VERSION);
    return;
  }

  // This pulls in all the operation implementations we need.
  // NOLINTNEXTLINE(runtime-global-variables)
  static tflite::AllOpsResolver resolver;

  // Build an interpreter to run the model with.
  static tflite::MicroInterpreter static_interpreter(
      model, resolver, tensor_arena, kTensorArenaSize);
  interpreter = &static_interpreter;

  // Allocate memory from the tensor_arena for the model's tensors.
  TfLiteStatus allocate_status = interpreter->AllocateTensors();
  if (allocate_status != kTfLiteOk) {
    MicroPrintf("AllocateTensors() failed");
    return;
  }

  // Obtain pointers to the model's input and output tensors.
  input = interpreter->input(0);
  output = interpreter->output(0);

  // Keep track of how many inferences we have performed.
  inference_count = 0;

  //debugg information
  std::cout << "Total heap: " << ESP.getHeapSize() << "\n";
  std::cout << "Free heap: " << ESP.getFreeHeap() << "\n";
  std::cout << "Total PSRAM: " << ESP.getPsramSize() << "\n";
  std::cout << "Free PSRAM: " << ESP.getFreePsram() << "\n";
  std::cout << "Max Alloc Heap: " << ESP.getMaxAllocHeap() << "\n"; 
  std::cout << "Max Vector Size: " << testing_vector.max_size() << "\n";

  // reserve memory for vector
  testing_vector.reserve(4700);
}

// The name of this function is important for Arduino compatibility.
void loop() {
 
  // setting up timer for test ouput
  now   = high_resolution_clock::now();
  auto mseconds = duration_cast<milliseconds>(now - t_0).count();

  if (mseconds- interval > previousMillis){
    std::cout << "test ouput: I am in main loop";
    previousMillis = duration_cast<milliseconds>(now - t_0).count();
  }
}

---

Further Information:

I have a partition table and already played with configure different sizes

Partition Table

partition table image

Hardware

ESP32-S (NODEMCU-32) SPI Flash 32Mbit