everytime my Ubuntu is rebooted, Tensorflow with GPU support needs to be rebuild

Question

When I build tensorflow with gpu support(in configure I set cuda support to yes), bazel build -c opt --config=cuda //tensorflow/tools/pip_package:build_pip_package everything works well.

but after I reboot the system, and try to build tensorflow using above command, it gives me following error:

ERROR

/home/wolfson/.cache/bazel/_bazel_wolfson/41eaf6c788f09c81cffb135517d04fa2/external/local_config_cuda/crosstool/BUILD:4:1: Traceback (most recent call last):
    File "/home/wolfson/.cache/bazel/_bazel_wolfson/41eaf6c788f09c81cffb135517d04fa2/external/local_config_cuda/crosstool/BUILD", line 4
        error_gpu_disabled()
    File "/home/wolfson/.cache/bazel/_bazel_wolfson/41eaf6c788f09c81cffb135517d04fa2/external/local_config_cuda/crosstool/error_gpu_disabled.bzl", line 3, in error_gpu_disabled
        fail("ERROR: Building with --config=c...")
ERROR: Building with --config=cuda but TensorFlow is not configured to build with GPU support. Please re-run ./configure and enter 'Y' at the prompt to build with GPU support.
ERROR: no such target '@local_config_cuda//crosstool:toolchain': target 'toolchain' not declared in package 'crosstool' defined by /home/wolfson/.cache/bazel/_bazel_wolfson/41eaf6c788f09c81cffb135517d04fa2/external/local_config_cuda/crosstool/BUILD.

Answer 1

The bazel build checks if there is an environment variable TF_NEED_CUDA set to 1, otherwise it will complain. This variable is exported in the configure script but is then missing after rebooting.

Maybe executing export TF_NEED_CUDA=1 before building will solve your problem, but there might be other environment variables or files that are missing after a reboot.