LARGEADDRESSAWARE turned off after installation?

Question

I have used the editbin /LARGEADDRESSAWARE command to change my .NET C# exe so that it can address memory > 2 GB. This works fine and I can run the dumpbin command to verify that it's been modified successfully.

I then proceed to create a .msi package that includes this exe. When I run this .msi on a windows 7 target machine it successfully installs the exe. But now if I run dumpbin on the same exe that got installed on the target machine is shows that it is no longer supporting memory > 2 GB.

What is causing this? Is is the process of building .msi or the running of .msi on the target machine?

Answer 1

I thinks in the end I found that the Installer was picking my exe from a different path than the one I was using to update using editbin post build command. I added this line in my post build command of my exe

copy "$(targetpath)"  "$(ProjectDir)\obj\x86\release"

So the entire post build command looks like this

set pathsave=%path%
set path=$(devenvdir);$(devenvdir)..\..\vc\bin
editbin.exe /nologo /largeaddressaware "$(targetpath)"
copy "$(targetpath)"  "$(ProjectDir)\obj\x86\release"
set path=%pathsave%

Thanks

Answer 2

Whatever is going wrong here, it surely has something to do with you doing this by hand. Let the build system do this for you. Project + Properties, Build Events tab. Paste this into the "Post-build event command line" box:

set pathsave=%path%
set path=$(devenvdir);$(devenvdir)..\..\vc\bin
editbin.exe /nologo /largeaddressaware "$(targetfilename)"
set path=%pathsave%

Answer 3

There are several issues with editbin to set the LARGEADDRESSAWARE flag in an msbuild post build step.

1. EditBin x32 does not run outside of VS command prompt because mspdb100.dll is not found. Why should you care? Well if you run a TFS build workflow msbuild is NOT called from a VS command prompt. This will cause issues ...
2. You can fix this one by using the one in bin\amd64\editbin.exe but then you can build your exe only on a x64 build machine.
3. If you patch the final file at $(TargetPath) then it will work but if you rebuild your project and you have set Inputs and Outputs for your task then it will not run again. This is an issue because during a rebuild the exe from the intermediate folder is copied again to the final location which was not patched.
4. This is still not it. Because if you did strong name your exe you need to re-sign it to make the strong name valid again. It will run on you dev machine because most of the time dev machines have disabled strong name verification but it will fail to run on customer machines.

Finally your task will look like this:

<Target Name="AfterBuild" BeforeTargets="CopyFilesToOutputDirectory" Inputs="$(IntermediateOutputPath)$(TargetFileName)" Outputs="$(IntermediateOutputPath)largaddessaware.tmp">
   <Exec Command="xxxxxbin\amd64\EditBin.exe /LARGEADDRESSAWARE &quot;$(IntermediateOutputPath)$(TargetFileName)&quot;"/>
   <Exec Command="sn -Ra &quot;$(IntermediateOutputPath)$(TargetFileName)&quot; &quot;$(AssemblyOriginatorKeyFile)&quot;"/>
   <Touch AlwaysCreate="true" Files="$(IntermediateOutputPath)largaddessaware.tmp"/>
</Target>

We need to patch the executable before CopyFiletoOutputDirectory has run otherwise we will patch the intermediate file after the unpatched file has already been copied to the ouptut folder. The final file cannot be patched because this target will not run when the exe has not changed to prevent breaking the incremental build.

This is a classic example of a simple task (set one bit in the PE header) which is quite difficult to get right. It is (nearly) never as easy as it might look like at the beginning.