Installing visual studio build tools in window container result in error "hcsshim::ImportLayer - failed failed in Win32..."

Question

I'm trying to build a docker file to install VS Build tools 2019. I create the following Dockerfile following this instructions: https://learn.microsoft.com/en-us/visualstudio/install/build-tools-container?view=vs-2019

I need to use windows server core container image version 2004, so I point to that in my docker file. This is how the Dockerfile looks like:

# escape=`

# Use the latest Windows Server Core image with .NET Framework 4.8.
FROM mcr.microsoft.com/dotnet/framework/sdk:4.8-windowsservercore-2004

# Restore the default Windows shell for correct batch processing.
SHELL ["cmd", "/S", "/C"]

# Download the Build Tools bootstrapper.
ADD https://aka.ms/vs/16/release/vs_buildtools.exe C:\tools\vs_buildtools.exe

# Install Build Tools, excluding workloads and components with known issues.
RUN C:\tools\vs_buildtools.exe --quiet --wait --norestart --nocache `
    --installPath 'C:\BuildTools' `
    --add Microsoft.VisualStudio.Workload.UniversalBuildTools;includeRecommended `
    --remove Microsoft.VisualStudio.Component.Windows10SDK.10240 `
    --remove Microsoft.VisualStudio.Component.Windows10SDK.10586 `
    --remove Microsoft.VisualStudio.Component.Windows10SDK.14393 `
    --remove Microsoft.VisualStudio.Component.Windows81SDK `
 || IF "%ERRORLEVEL%"=="3010" EXIT 0

# Define the entry point for the docker container.
# This entry point starts the developer command prompt and launches the PowerShell shell.
ENTRYPOINT ["C:\\BuildTools\\Common7\\Tools\\VsDevCmd.bat", "&&", "powershell.exe", "-NoLogo", "-ExecutionPolicy", "Bypass"]

I'm running the following command to build the image:

docker build -t buildtools2019:latest .

My Host OS is Microsoft Windows 10 Pro v2004 (OS Build 19041.630)

The output:

Sending build context to Docker daemon  42.66MB
Step 1/5 : FROM mcr.microsoft.com/dotnet/framework/sdk:4.8-windowsservercore-2004
4.8-windowsservercore-2004: Pulling from dotnet/framework/sdk
295f12394c4f: Already exists
63bace88024e: Already exists
19217c91f203: Pull complete
0accfd4ccbac: Pull complete
b34bf70de9e5: Pull complete
3f855de439e1: Pull complete
f11939b5c288: Pull complete
ba5106f84ea9: Pull complete
437a4aa4a672: Pull complete
75d38bc2eb59: Pull complete
e94356f41137: Pull complete
3f59c0b6fbd6: Pull complete
cdd6d627a7da: Pull complete
3ef86eb7425c: Pull complete
Digest: sha256:eda4d27414ac5ce28b4c031ac24968939759a0cb0637a517f0f9973df70a1cee
Status: Downloaded newer image for mcr.microsoft.com/dotnet/framework/sdk:4.8-windowsservercore-2004
 ---> a6ab7b3338e8
Step 2/5 : SHELL ["cmd", "/S", "/C"]
 ---> Running in 751c52c97349
Removing intermediate container 751c52c97349
 ---> 512e42837f7d
Step 3/5 : ADD https://aka.ms/vs/16/release/vs_buildtools.exe C:\tools\vs_buildtools.exe
Downloading [==================================================>]  1.409MB/1.409MB

 ---> 9c045d82f176
Step 4/5 : RUN C:\tools\vs_buildtools.exe --quiet --wait --norestart --nocache     --installPath 'C:\BuildTools'     --add Microsoft.VisualStudio.Workload.UniversalBuildTools;includeRecommended     --remove Microsoft.VisualStudio.Component.Windows10SDK.10240     --remove Microsoft.VisualStudio.Component.Windows10SDK.10586     --remove Microsoft.VisualStudio.Component.Windows10SDK.14393     --remove Microsoft.VisualStudio.Component.Windows81SDK  || IF "%ERRORLEVEL%"=="3010" EXIT 0
 ---> Running in f7a73366cc38
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\12e3333511c86bc9c91ba6\vs_bootstrapper_d15\HelpFile\1028\help.html...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\12e3333511c86bc9c91ba6\vs_bootstrapper_d15\HelpFile\2052\help.html...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\12e3333511c86bc9c91ba6\vs_bootstrapper_d15\HelpFile\1029\help.html...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\12e3333511c86bc9c91ba6\vs_bootstrapper_d15\HelpFile\1031\help.html...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\12e3333511c86bc9c91ba6\vs_bootstrapper_d15\HelpFile\1046\help.html...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\12e3333511c86bc9c91ba6\vs_bootstrapper_d15\HelpFile\1042\help.html...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\12e3333511c86bc9c91ba6\vs_bootstrapper_d15\HelpFile\1036\help.html...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\12e3333511c86bc9c91ba6\vs_bootstrapper_d15\HelpFile\3082\help.html...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\12e3333511c86bc9c91ba6\vs_bootstrapper_d15\HelpFile\1040\help.html...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\12e3333511c86bc9c91ba6\vs_bootstrapper_d15\HelpFile\1045\help.html...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\12e3333511c86bc9c91ba6\vs_bootstrapper_d15\HelpFile\1041\help.html...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\12e3333511c86bc9c91ba6\vs_bootstrapper_d15\HelpFile\1055\help.html...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\12e3333511c86bc9c91ba6\vs_bootstrapper_d15\HelpFile\1049\help.html...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\12e3333511c86bc9c91ba6\vs_bootstrapper_d15\HelpFile\1033\help.html...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\12e3333511c86bc9c91ba6\vs_bootstrapper_d15\vs_setup_bootstrapper.exe...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\12e3333511c86bc9c91ba6\vs_bootstrapper_d15\Microsoft.Diagnostics.Tracing.EventSource.dll...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\12e3333511c86bc9c91ba6\vs_bootstrapper_d15\Microsoft.VisualStudio.RemoteControl.dll...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\12e3333511c86bc9c91ba6\vs_bootstrapper_d15\Microsoft.VisualStudio.Setup.Common.dll...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\12e3333511c86bc9c91ba6\vs_bootstrapper_d15\Microsoft.VisualStudio.Setup.dll...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\12e3333511c86bc9c91ba6\vs_bootstrapper_d15\Microsoft.VisualStudio.Setup.Download.dll...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\12e3333511c86bc9c91ba6\vs_bootstrapper_d15\Microsoft.VisualStudio.Telemetry.dll...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\12e3333511c86bc9c91ba6\vs_bootstrapper_d15\Microsoft.VisualStudio.Utilities.Internal.dll...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\12e3333511c86bc9c91ba6\vs_bootstrapper_d15\Newtonsoft.Json.dll...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\12e3333511c86bc9c91ba6\vs_bootstrapper_d15\zh-Hant\vs_setup_bootstrapper.resources.dll...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\12e3333511c86bc9c91ba6\vs_bootstrapper_d15\fr\vs_setup_bootstrapper.resources.dll...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\12e3333511c86bc9c91ba6\vs_bootstrapper_d15\tr\vs_setup_bootstrapper.resources.dll...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\12e3333511c86bc9c91ba6\vs_bootstrapper_d15\it\vs_setup_bootstrapper.resources.dll...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\12e3333511c86bc9c91ba6\vs_bootstrapper_d15\pl\vs_setup_bootstrapper.resources.dll...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\12e3333511c86bc9c91ba6\vs_bootstrapper_d15\ko\vs_setup_bootstrapper.resources.dll...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\12e3333511c86bc9c91ba6\vs_bootstrapper_d15\ja\vs_setup_bootstrapper.resources.dll...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\12e3333511c86bc9c91ba6\vs_bootstrapper_d15\zh-Hans\vs_setup_bootstrapper.resources.dll...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\12e3333511c86bc9c91ba6\vs_bootstrapper_d15\cs\vs_setup_bootstrapper.resources.dll...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\12e3333511c86bc9c91ba6\vs_bootstrapper_d15\pt-BR\vs_setup_bootstrapper.resources.dll...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\12e3333511c86bc9c91ba6\vs_bootstrapper_d15\ru\vs_setup_bootstrapper.resources.dll...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\12e3333511c86bc9c91ba6\vs_bootstrapper_d15\es\vs_setup_bootstrapper.resources.dll...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\12e3333511c86bc9c91ba6\vs_bootstrapper_d15\de\vs_setup_bootstrapper.resources.dll...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\12e3333511c86bc9c91ba6\vs_bootstrapper_d15\vs_setup_bootstrapper.config...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\12e3333511c86bc9c91ba6\vs_bootstrapper_d15\vs_setup_bootstrapper.exe.config...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\12e3333511c86bc9c91ba6\vs_bootstrapper_d15\detection.json...
Preparing: C:\Users\ContainerAdministrator\AppData\Local\Temp\12e3333511c86bc9c91ba6\vs_bootstrapper_d15\vs_setup_bootstrapper.json...
re-exec error: exit status 1: output: hcsshim::ImportLayer - failed failed in Win32: The system cannot find the path specified. (0x3)

I've tried to increase the storage options for docker daemon to 50GB and 150GB as i read that that could be a problem but same issue. This is how my docker config file looks like:

{
  "registry-mirrors": [],
  "insecure-registries": [],
  "debug": false,
  "experimental": false,
  "storage-opts": [
    "size=150GB"
  ]
}

And here is my docker version:

Client: Docker Engine - Community
 Cloud integration  0.1.18
 Version:           19.03.13
 API version:       1.40
 Go version:        go1.13.15
 Git commit:        4484c46d9d
 Built:             Wed Sep 16 17:00:27 2020
 OS/Arch:           windows/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.13
  API version:      1.40 (minimum version 1.24)
  Go version:       go1.13.15
  Git commit:       4484c46d9d
  Built:            Wed Sep 16 17:14:20 2020
  OS/Arch:          windows/amd64
  Experimental:     false

Any ideas?