Changing the Google Cloud Deep Learning VM image version after deployment?

Question

I deployed a deep learning VM on Google Cloud with a standard tensorflow-gpu==1.15 image. However, I later realized that I need tensorflow-gpu>=1.4. As I have already done a lot of processing on my VM instance, I was wondering if it's possible to upgrade the image version without having to deploy a new VM?

Thanks!

Answer 1

I am assuming when you say "if it's possible to upgrade the image version without having to deploy a new VM?". You mean downgrade.

Yes it is possible to downgrade the version of Tensorflow in a Deep Learning VM

I have run the following tests to see if it was possible.

1) Installed Deep Learning VM as per Creating a Deep Learning VM Instance From the Google Cloud Marketplace

After SSH'ing into the VM

2) Checked the installed version in interactive python shell

>>> import tensorflow as tf
 2020-03-30 11:57:03.076701: I 
 tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully 
 opened dynamic library libcudart.so.10.1
 2020-03-30 11:57:07.162626: I 
 tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully 
 opened dynamic library libnvinfer.so.6
 2020-03-30 11:57:07.222695: I 
 tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully 
 opened dynamic library libnvinfer_plugin.so.6
 >>> print(tf.__version__)
 2.1.0

Next step was to run;

pip install --upgrade tensorflow==1.14.0

The command succeeds but with following errors;

ERROR: scikit-image 0.15.0 requires pillow>=4.3.0, which is not installed.
ERROR: imageio 2.6.1 requires pillow, which is not installed.
ERROR: tensorflow-serving-api-gpu 1.14.0 has requirement tensorflow-gpu~=1.14.0, but you'll have tensorflow-gpu 2.1.0 which is incompatible.
ERROR: tensorflow-gpu 2.1.0 has requirement numpy<2.0,>=1.16.0, but you'll have numpy 1.15.4 which is incompatible.
ERROR: tensorflow-gpu 2.1.0 has requirement scipy==1.4.1; python_version >= "3", but you'll have scipy 1.1.0 which is incompatible.
ERROR: tensorflow-io 0.9.10 has requirement tensorflow==2.1.0rc0, but you'll have tensorflow 1.4.0 which is incompatible.

As you see this causes dependency issues.

Next steps were;

a) pip install --upgrade  pillow==4.3.0
b) pip install --upgrade tensorflow-gpu==1.14.0

Checking the version again in interactive python shell

Python 3.5.3 (default, Sep 27 2018, 17:25:39) 
[GCC 6.3.0 20170516] on linux
Type "help", "copyright", "credits" or "license" for more information.


>>> import tensorflow as tf

   /home/xxxxxxxxx/.local/lib/python3.5/site- 
   packages/tensorflow/python/framework/dtypes.py:516: FutureWarning: Passing 
   (type, 1) or '1type' as a synonym of type is deprecated; in a future version 
   of numpy, it will be understood as (type, (1,)) / '(1,)type'.
   _np_qint8 = np.dtype([("qint8", np.int8, 1)])
   /home/xxxxxxxx/.local/lib/python3.5/site- 
   packages/tensorflow/python/framework/dtypes.py:517: FutureWarning: Passing 
   (type, 1) or '1type' as a synonym of type is deprecated; in a future version 
   of numpy, it will be understood as (type, (1,)) / '(1,)type'.
   _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
   /home/xxxxxxxx/.local/lib/python3.5/site- 
   packages/tensorflow/python/framework/dtypes.py:518: FutureWarning: Passing 
   (type, 1) or '1type' as a synonym of type is deprecated; in a future version 
   of numpy, it will be understood as (type, (1,)) / '(1,)type'.
   _np_qint16 = np.dtype([("qint16", np.int16, 1)])
   /home/xxxxxxx/.local/lib/python3.5/site- 
   packages/tensorflow/python/framework/dtypes.py:519: FutureWarning: Passing 
   (type, 1) or '1type' as a synonym of type is deprecated; in a future version 
   of numpy, it will be understood as (type, (1,)) / '(1,)type'.
   _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
   /home/xxxxxxx/.local/lib/python3.5/site- 
   packages/tensorflow/python/framework/dtypes.py:520: FutureWarning: Passing 
   (type, 1) or '1type' as a synonym of type is deprecated; in a future version 
   of numpy, it will be understood as (type, (1,)) / '(1,)type'.
   _np_qint32 = np.dtype([("qint32", np.int32, 1)])
   /home/xxxxxxxx/.local/lib/python3.5/site- 
   packages/tensorflow/python/framework/dtypes.py:525: FutureWarning: Passing 
   (type, 1) or '1type' as a synonym of type is deprecated; in a future version 
   of numpy, it will be understood as (type, (1,)) / '(1,)type'.
   np_resource = np.dtype([("resource", np.ubyte, 1)])
   /home/xxxxxxxx/.local/lib/python3.5/site- 
   packages/tensorboard/compat/tensorflow_stub/dtypes.py:541: FutureWarning: 
   Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future 
   version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
   _np_qint8 = np.dtype([("qint8", np.int8, 1)])
   /home/xxxxxxx/.local/lib/python3.5/site- 
   packages/tensorboard/compat/tensorflow_stub/dtypes.py:542: FutureWarning: 
   Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future 
   version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
   _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
   /home/xxxxxxxx/.local/lib/python3.5/site- 
   packages/tensorboard/compat/tensorflow_stub/dtypes.py:543: FutureWarning: 
   Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future 
   version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
   _np_qint16 = np.dtype([("qint16", np.int16, 1)])
   /home/xxxxxxxx/.local/lib/python3.5/site- 
   packages/tensorboard/compat/tensorflow_stub/dtypes.py:544: FutureWarning: 
   Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future 
   version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
   _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
   /home/xxxxxxx/.local/lib/python3.5/site- 
   packages/tensorboard/compat/tensorflow_stub/dtypes.py:545: FutureWarning: 
   Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future 
   version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
   _np_qint32 = np.dtype([("qint32", np.int32, 1)])
   /home/xxxxxxxx/.local/lib/python3.5/site- 
   packages/tensorboard/compat/tensorflow_stub/dtypes.py:550: FutureWarning: 
   Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future 
   version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
   np_resource = np.dtype([("resource", np.ubyte, 1)])
   >>> print(tf.__version__)
   1.14.0

Running the "pip freeze" command to check the dependencies; Before changing the version the pip freeze output;

"....tensorboard==2.1.0
tensorflow-datasets==1.3.0
tensorflow-estimator==2.1.0
tensorflow-gpu==2.1.0
tensorflow-hub==0.7.0
tensorflow-io==0.9.10
tensorflow-metadata==0.21.1
tensorflow-probability==0.9.0
tensorflow-serving-api-gpu==1.14.0......."

After changing the version pip freeze output;

"....
tensorboard==1.14.0
tensorflow==1.14.0
tensorflow-datasets==1.3.0
tensorflow-estimator==1.14.0
tensorflow-gpu==1.14.0
tensorflow-hub==0.7.0
tensorflow-io==0.9.10
tensorflow-metadata==0.21.1
tensorflow-probability==0.9.0
tensorflow-serving-api-gpu==1.14.0
tensorflow-tensorboard==0.4.0...."