Monday, August 5, 2019

Install "tensorflow-gpu 2.0.0-beta1" on ubuntu 19.04 (T470p)

1. Install nvidia driver
    Open "Software & Updates" --> "Additional Drivers"
              --> select "nvidia-driver-418 (proprietary, tested)" --> "Apply Changes"

2. Installed CUDA 10.0 on Ubuntu 19.04
    (Note that the released binary 'tensorflow' doesn't support 'cuda 10.1/10.2' as of now, although you may try to compile 'tensorflow' from source against 'cuda 10.1/10.2'.  
    'cuda 10.0' is only compatable with Ubuntu 18.04, the method below is hacking.)

(1) Go to the CUDA download site. Click "Download Now"
      Select "Linux" --> "x86_64" --> "Ubuntu"
                  --> "18.04" (although we have 19.04) --> "runfile (local)"

(2) Open a new terminal environment and login as root.
      Stop the display manager: `service gdm3 stop`
      To find out your display manager  `pgrep -l dm`

(3) Run the CUDA installer with the override option:
      `./cuda_10.0.130_410.48_linux --override`
     Install the drivers, the toolkit and the samples.
     (Note that the 'override' option will ignore the compiler version check. CUDA requires gcc version 6, but the installation goes fine with gcc version 8 too. The 'override' option allow us to proceed using a later version of the compiler. )

(4) Restart the desktop manager
      `service gdm3 start`
      and switch again to the desktop environment.

(5) Add the 'CUDA include directory and the CUDA library directory' to the ".bashrc" file.
     If you installed CUDA under '/usr/local/cuda-10.0', edit ".bashrc" file, add:
     `export PATH="$PATH:/usr/local/cuda-10.0/bin"
       export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/local/cuda-10.0/lib64/:/usr/local/cuda-10.0/extras/CUPTI/lib64"
`

(6)  Go to the samples directory and compile them. (Optional)
       Install gcc version 6: `apt-get install g++-6`
       Compile the samples using g++ version 6 as the compiler:
       `make HOST_COMPILER=g++-6`

@reference_1_askubuntu.com

3.  Install 'TensorFlow-gpu 2.0 Beta' on 'miniconda'
(1) Create an environment which has Tensorflow-gpu:
     `conda create -n your_env_name python=3.6 pip tensorflow-gpu`
     (Now Anaconda is supporting install cudnn 7.6.0 with Tensorflow-gpu (1.1x) so we don’t have to manually copy cudnn to our environment anymore
     ??? see step (3)  -- How to avoid it ??)
     (Note that Python 3.7 is not compatible with 'tensorflow_text')

(2) Activate 'your_environment':
      `conda activate your_env_name`

(3) Uninstall 'Tensorflow-gpu 1.1x'
      `conda uninstall tensorflow-gpu`
     (In fact, 'cudnn' is uninstalled together with 'Tensorflow-gpu 1.1x' !!)

(4) Install 'Tensorflow-gpu 2.0 Beta' with pip in your environment:
      `pip install tensorflow-gpu==2.0.0-beta1`

(5) Install other packages
      `proxychains pip install IPython
      proxychains pip install pathlib
      proxychains pip install matplotlib
      proxychains pip install seaborn
      proxychains pip install tensorflow_datasets
      proxychains pip install tensorflow_hub
      proxychains pip install sklearn
      proxychains pip install pandas
      proxychains pip install tensorflow_text
      proxychains pip install numpy
      proxychains pip install pillow`

@reference_2_medium.com

4.  Manually copy "cudnn (libcudnn.so.7)" to the environment
(1)  Download "cuDNN v7.6.2 (July 22, 2019), for CUDA 10.0"
(2) Unzip package
     `tar xvzf cudnn-10.0-linux-x64-v7.6.2.24.tgz`
(3) Copy file to cuda directory
      `cp -P cuda/include/cudnn.h /usr/local/cuda-10.0/include`
      `cp -P cuda/lib64/libcudnn* /usr/local/cuda-10.0/lib64`
      `chmod a+r /usr/local/cuda-10.0/include/cudnn.h /usr/local/cuda-10.0/lib64/libcudnn*`

@reference_3_jermine.vdo.pub

5.  Set Pycharm environment to the 'conda TensorFlow environment'
     Note that Pycharm environment may be different from command line.
    ".bashrc" file is being read by bash (your command line interpreter) only.
     To preserve bash environment for PyCharm. Run PyCharm from the command line (from bash). Thus environment variables will be inherited from bash to pycharm.
     Just launch `./pycharm.sh` from command line.
    Or create launcher which invokes bash for PyCharm launching.

@reference_4_stackoverflow.com

6.  How to solve the above problem that Pycharm environment doesn't inherit from bash if not launching it from terminal ?

Another way to permanently add new path in 'LD_LIBRARY_PATH' is to edit ".conf" files in "/etc/ld.so.conf.d/".

Specifically,  add
`"/usr/local/cuda-10.0/bin"
  "/usr/local/cuda-10.0/lib64"
  "/usr/local/cuda-10.0/extras/CUPTI/lib64"` 3 lines
to the "x86_64-linux-gnu.conf" file.
Just in the next line. Save.
Then run `ldconfig`

@reference_5_unix.stackexchange.com

7.  If GPU not found, try reinstalling 'tensorflow-gpu==2.0.0-beta1'
    `pip uninstall tensorflow-gpu==2.0.0-beta1
    pip install tensorflow-gpu==2.0.0-beta1`

No comments:

Post a Comment