Unable to build installer .vdproj with VS2022 on command line

Question

Previously, we used dotnet core 3.1 and Visual Studio 2019 to develop a WPF project. We also used the (somewhat old) Visual Studio Installer Projects. When we perform a build using our build server, we would use the following command line:

devenv.exe Solution.sln /build "Release" /project TheInstallerProjectName

This would work ... kind of. I actually had to have our build server execute that exact command three times in a row - the first two would always fail, but the final execution would actually produce an .msi file.

I'm attempting to transition this project to .NET 7. I've installed the VS2022 version of the installer projects extension, and have modified the project to the point where building in release mode, from within Visual Studio 2022, I get an .msi file as expected.

However, when I attempted to build at the command line using the same command above, I started to receive this error when it attempts to build that project. The version reported by devenv in the output when this happened was 17.6.2:

ERROR: An error occurred while validating. HRESULT = '8000000A'

However, our build server has 17.6.4 installed, and it receives a different error - devenv.exe complains that the command line is not correct! I updated my own installation to this same version, and I get the exact same behavior:

3:22:16 c:\redacted\master-824>devenv TheSolution.sln /build "Release" /project TheInstallerProject 
13:22:25 
13:22:25 Microsoft Visual Studio 2022 Version 17.6.4.
13:22:25 Copyright (C) Microsoft Corp. All rights reserved.
13:22:25 
13:22:25 The operation could not be completed. The parameter is incorrect. 
13:22:25 
13:22:25 Use:
13:22:25 devenv  [solutionfile | projectfile | folder | anyfile.ext]  [switches]
[snip]

There is a tool included in VS2019 that will add a registry entry to disable out-of-proc builds, which would supposedly fix the original problem, but now I can't even get there due to this command line error. However, I have run that program from within the VS2022 installation directory, and have confirmed that the registry entry was added. I added it as the same user I'm building with.

Why is this happening and how can I resolve it?

Answer 1

Faced a similar problem after upgrading our build server from VS2019 to vs2022. we too also have a solution file containing a Visual Studio Installer project (.vdproj).

Our exact version of Visual Studio is:

Microsoft Visual Studio Community 2022 (64-bit) - Current Version 17.7.2

Fortunately, DisableOutOfProcBuild.exe is our friend:

C:\>where /R . DisableOutOfProcBuild.exe
C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\IDE\CommonExtensions\Microsoft\VSI\DisableOutOfProcBuild\DisableOutOfProcBuild.exe

I amended our build script to execute DisableOutOfProcBuild.exe before any call to devenv.exe observing these words of wisdom from it3xl.

At the next invocation of the build, I noted the following addition to the registry

Following that, we have a working build once more!