I would like to use ApplicationVerifier provided by the Windows 10 SDK (version 10.0.18362.1) to accelerate debugging a (possible) heap corruption in a 32-bit application running under 64-bit Windows 10, build 1909. Unfortunately, when enabling the Heaps feature using its default settings under the Basic listing (and having only that feature enabled), the application runs out of memory as soon as its memory usage gets close to 2 GiB, even though the application itself is compiled as large address aware.
The issue occurs regardless of whether I use Application Verifier x86 or Application Verifier x64 to activate the heap checks. (And I am not even sure what the difference between the two versions is, anyway.)
I have searched the documentation available via the Help menu, which is actually quite extensive, but did not find any reason for that. I understand that inserting guard pages and aligning the beginning or end of allocations to page boundaries increases memory consumption but I do not expect it to cut the available memory in half or otherwise limit the virtual address space, at least not without enabling such settings explicitly in the options of the Heaps feature. Does anybody know the reason for this behavior and can tell me how to circumvent this limitation?
It is not enough to set the application Large Address flag. It makes sense, as if the system is not ready, the kernel memory can go over the 1GB limit and in this case you'll have overlapping address-ranges. You need to reboot the OS with the correct flag which is set using bcdedit.
The command should be (I haven't actually tested):