Terraform init: Error: failed to open temporary file with direnv

Question

I used direnv to install programs from a nix flake like this:

.envrc:

use flake ~/Documents/nixdesktop/shells/myshell#terraformAndOthers

myshell/flake.nix:

{
  description = "Shells with terraform, jdk, etc ";
  inputs = {
    nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable";
    flake-utils.url = "github:numtide/flake-utils/master";
  };

  outputs = { self, nixpkgs, flake-utils }:
    flake-utils.lib.eachDefaultSystem (system:
      let pkgs = nixpkgs.legacyPackages.${system};
      in {
        devShells.terraformAndOthers = pkgs.mkShell {
          buildInputs = [
            pkgs.openjdk17
            pkgs.bashInteractive
            pkgs.terraform
          ];
        }
      }
    );
}

Then I ran terraform init, which failed with the following error:

Initializing the backend...

Initializing provider plugins...
- terraform.io/builtin/terraform is built in to Terraform
- Reusing previous version of hashicorp/google from the dependency lock file
- Installing hashicorp/google v4.44.1...
╷
│ Error: Failed to install provider
│ 
│ Error while installing hashicorp/google v4.44.1: failed to open temporary file to download from
│ https://releases.hashicorp.com/terraform-provider-google/4.44.1/terraform-provider-google_4.44.1_darwin_arm64.zip
╵

Alternately, sometimes terraform is working for a while and then suddenly terraform apply stops working with error “failed to instantiate provider PROVIDER to obtain schema: Unrecognized remote plugin message:”:

│ Error: Failed to load plugin schemas
│ 
│ Error while loading schemas for plugin components: Failed to obtain provider schema: Could not load the schema for provider registry.terraform.io/hashicorp/google: failed to
│ instantiate provider "registry.terraform.io/hashicorp/google" to obtain schema: Unrecognized remote plugin message: 
│ 
│ This usually means that the plugin is either invalid or simply
│ needs to be recompiled to support the latest protocol...

How to fix this error?

Answer 1

Solution: call direnv allow again to have direnv create a new temporary directory.

Explanation:

Terraform (and any go program that calls os.TempDir("", …) or os.MkdirTemp("", …), or ioutil.TempFile("", …), or ioutil.TempDir("", …)) creates files in the directory returned by os.TempDir(), which is the value of the environment variable $TMPDIR. $TMPDIR must exist first, or else those functions give an error.

Meanwhile, direnv’s use flake calls nix print-dev-env, and nix print-dev-env --profile .direnv/flake-profile ~/Documents/nixdesktop/shells/myshell#terraformAndOthers prints a shell script which ends with

export NIX_BUILD_TOP="$(mktemp -d -t nix-shell.XXXXXX)"
export TMP="$NIX_BUILD_TOP"
export TMPDIR="$NIX_BUILD_TOP"
export TEMP="$NIX_BUILD_TOP"
export TEMPDIR="$NIX_BUILD_TOP"
…

Each time the flake is executed, mktemp should create a directory. But if $TMPDIR no longer exists, then you need to recreate it. I think $TMPDIR stopped existing because direnv cached the environmnet vars, but the temporary directory was deleted after a few hours or after a reboot. To fix this, re-run direnv allow.