Tuesday, October 29, 2019

Bridge from Http proxy to Socks5 proxy using Privoxy

1.  Download Privoxy
2. Add the following 2 lines to the configuration file (config / config.txt)
`listen-address  127.0.0.1:8118
forward-socks5 / 127.0.0.1:1080 .
`

(The first line is the address where http proxy server listens,
the second line is the socks5 server address)

Monday, August 5, 2019

Reinstall ubuntu 19.04 dual boot on T470p - Install Pycharm, Clion, Intellij Idea

1. Download pycharm  (Clion, Intellij Idea)
2. Put it into the directory you want to install pycharm in, unzip it:
    `tar -xvzf pycharm-professional-2019.x.tar.gz`

3. Create a 'desktop shortcut launcher':
(1) `apt install --no-install-recommends gnome-panel`
      (The 'gnome-desktop-item-edit' command may be missing as it is not installed by default.)
(2) Create Desktop Launcher Dialog:     `gnome-desktop-item-edit ~/Desktop/ --create-new`
      Enter name, the path to the application binary executable and an optional comment. When ready, click on 'OK'.
(3) Double-click on the new shortcut to edit it:
      Change `Icon= ...` and `Icon[en_US]=...` path to the desirable icon path, e.g., '.../pycharm-2019.x/bin/pycharm.png'.

(4) Right-click on the new shortcut and select "Allow Launching".
      Double-click on the new shortcut to launch pycharm.

@reference_1_linuxconfig.org

4.  "File" --> "Setting" --> search "font" --> "Font" of "Editor" --> set font size '19'

Install "tensorflow-gpu 2.0.0-beta1" on ubuntu 19.04 (T470p)

1. Install nvidia driver
    Open "Software & Updates" --> "Additional Drivers"
              --> select "nvidia-driver-418 (proprietary, tested)" --> "Apply Changes"

2. Installed CUDA 10.0 on Ubuntu 19.04
    (Note that the released binary 'tensorflow' doesn't support 'cuda 10.1/10.2' as of now, although you may try to compile 'tensorflow' from source against 'cuda 10.1/10.2'.  
    'cuda 10.0' is only compatable with Ubuntu 18.04, the method below is hacking.)

(1) Go to the CUDA download site. Click "Download Now"
      Select "Linux" --> "x86_64" --> "Ubuntu"
                  --> "18.04" (although we have 19.04) --> "runfile (local)"

(2) Open a new terminal environment and login as root.
      Stop the display manager: `service gdm3 stop`
      To find out your display manager  `pgrep -l dm`

(3) Run the CUDA installer with the override option:
      `./cuda_10.0.130_410.48_linux --override`
     Install the drivers, the toolkit and the samples.
     (Note that the 'override' option will ignore the compiler version check. CUDA requires gcc version 6, but the installation goes fine with gcc version 8 too. The 'override' option allow us to proceed using a later version of the compiler. )

(4) Restart the desktop manager
      `service gdm3 start`
      and switch again to the desktop environment.

(5) Add the 'CUDA include directory and the CUDA library directory' to the ".bashrc" file.
     If you installed CUDA under '/usr/local/cuda-10.0', edit ".bashrc" file, add:
     `export PATH="$PATH:/usr/local/cuda-10.0/bin"
       export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/local/cuda-10.0/lib64/:/usr/local/cuda-10.0/extras/CUPTI/lib64"
`

(6)  Go to the samples directory and compile them. (Optional)
       Install gcc version 6: `apt-get install g++-6`
       Compile the samples using g++ version 6 as the compiler:
       `make HOST_COMPILER=g++-6`

@reference_1_askubuntu.com

3.  Install 'TensorFlow-gpu 2.0 Beta' on 'miniconda'
(1) Create an environment which has Tensorflow-gpu:
     `conda create -n your_env_name python=3.6 pip tensorflow-gpu`
     (Now Anaconda is supporting install cudnn 7.6.0 with Tensorflow-gpu (1.1x) so we don’t have to manually copy cudnn to our environment anymore
     ??? see step (3)  -- How to avoid it ??)
     (Note that Python 3.7 is not compatible with 'tensorflow_text')

(2) Activate 'your_environment':
      `conda activate your_env_name`

(3) Uninstall 'Tensorflow-gpu 1.1x'
      `conda uninstall tensorflow-gpu`
     (In fact, 'cudnn' is uninstalled together with 'Tensorflow-gpu 1.1x' !!)

(4) Install 'Tensorflow-gpu 2.0 Beta' with pip in your environment:
      `pip install tensorflow-gpu==2.0.0-beta1`

(5) Install other packages
      `proxychains pip install IPython
      proxychains pip install pathlib
      proxychains pip install matplotlib
      proxychains pip install seaborn
      proxychains pip install tensorflow_datasets
      proxychains pip install tensorflow_hub
      proxychains pip install sklearn
      proxychains pip install pandas
      proxychains pip install tensorflow_text
      proxychains pip install numpy
      proxychains pip install pillow`

@reference_2_medium.com

4.  Manually copy "cudnn (libcudnn.so.7)" to the environment
(1)  Download "cuDNN v7.6.2 (July 22, 2019), for CUDA 10.0"
(2) Unzip package
     `tar xvzf cudnn-10.0-linux-x64-v7.6.2.24.tgz`
(3) Copy file to cuda directory
      `cp -P cuda/include/cudnn.h /usr/local/cuda-10.0/include`
      `cp -P cuda/lib64/libcudnn* /usr/local/cuda-10.0/lib64`
      `chmod a+r /usr/local/cuda-10.0/include/cudnn.h /usr/local/cuda-10.0/lib64/libcudnn*`

@reference_3_jermine.vdo.pub

5.  Set Pycharm environment to the 'conda TensorFlow environment'
     Note that Pycharm environment may be different from command line.
    ".bashrc" file is being read by bash (your command line interpreter) only.
     To preserve bash environment for PyCharm. Run PyCharm from the command line (from bash). Thus environment variables will be inherited from bash to pycharm.
     Just launch `./pycharm.sh` from command line.
    Or create launcher which invokes bash for PyCharm launching.

@reference_4_stackoverflow.com

6.  How to solve the above problem that Pycharm environment doesn't inherit from bash if not launching it from terminal ?

Another way to permanently add new path in 'LD_LIBRARY_PATH' is to edit ".conf" files in "/etc/ld.so.conf.d/".

Specifically,  add
`"/usr/local/cuda-10.0/bin"
  "/usr/local/cuda-10.0/lib64"
  "/usr/local/cuda-10.0/extras/CUPTI/lib64"` 3 lines
to the "x86_64-linux-gnu.conf" file.
Just in the next line. Save.
Then run `ldconfig`

@reference_5_unix.stackexchange.com

7.  If GPU not found, try reinstalling 'tensorflow-gpu==2.0.0-beta1'
    `pip uninstall tensorflow-gpu==2.0.0-beta1
    pip install tensorflow-gpu==2.0.0-beta1`

Thursday, August 1, 2019

Reinstall ubuntu 19.04 dual boot on T470p - Setting (3)

1. Install 'Git'
`apt update`
`apt install git`
`git --version`

@reference_1_digitalocean.com

2. Check 'build-essential'
`apt install build-essential`
`gcc --version` , `g++ --version`, `make --version`

@reference_2_linuxize.com

3. Install 'openjdk'
`apt install default-jdk `
`java --version`, `javac --version`

@reference_3_ digitalocean.com

4. Install 'pip3' and 'virtualenv'
`apt install python3-pip`
`pip3 -V`
`pip3 install virtualenv`

5. Install  'miniconda'
(1) download miniconda
(2) `chmod +x Miniconda3-latest-Linux-x86_64.sh`
      `bash Miniconda3-latest-Linux-x86_64.sh`
      During the installation, choose the desired path to install.
(3) `source ~/.bashrc`
      `conda -V`, `conda list`
(4)  remove (base) from terminal prompt
       check "auto_activate_base":
      `conda config --show | grep auto_activate_base`
       Set it false:
      `conda config --set auto_activate_base False`

@reference_1_askubuntu.com
@reference_2_digitalocean.com

6. How to uninstall 'miniconda' ?
    (1) In order to uninstall miniconda, simply remove the miniconda folder:
          `rm -r ~/miniconda/`
    (2) Manually remove contents related to conda in '~/.bashrc'

@reference_3_stackoverflow.com

7. How to completely remove Thunderbird?
    `apt-get purge thunderbird* `

@reference_4_askubuntu.com

Reinstall ubuntu 19.04 dual boot on T470p - Setting (2)

1. Right click on the desktop, in the menu select "Display Settings"
   "Display Settings" --> "Resolution" --> "2048 x 1152 (16:9)"
                                  --> "Refresh Rate" --> "119.96Hz"
                                  --> "Scale" --> "100%"

2. Open "Terminal" --> "Preferences" --> "Unnamed" -->
               "Initial terminal size" --> "152 x 36"
               "Custom font" --> "12"

3.  Open "Settings"
(1) "Dock" --> scale dock to a proper size
(2) "Privacy" --> configure privacy setting
(3) "Power" -->
                   "Blank screen" --> select "Never"
                   "Automatic suspend" --> "off"
                   "When the Power Button is pressed" --> "Suspend"
(4) "Details" --> "About" --> modify "Device name" to make it short in terminal
                     --> "Date & Time" --> set proper date & time format
                     --> "User" --> select an image for login
                     --> "Default Application" --> select proper default application

4. Install "tweaks"
`sudo add-apt-repository universe`
`sudo apt install gnome-tweak-tool`
Open "tweaks":
(1) "General" --> turn off "Suspend when laptop lid is closed"
(2) "Top Bar" --> Select proper 'clock' and 'calendar' format
(3) "Windows" --> turn on "Center New Windows"

5. Install "VLC media player" from "Ubuntu Software"
(1) click 'loop one sign'
(2) "Tools" --> "Preferences" --> "Interface" -->
           "Show media change popup" --> select "Never"
           "Continue playback?" --> select "Never"
           "Save recently played items" --> "off"
           "Allow metedata network access"  --> "off"
     Open ubuntu system "Settings" -->
           "Notifications" --> "VLC media player" --> "off"
       
6.  Playing videos in firefox
If those streaming services use DRM, you must enable DRM in Firefox's settings: "Preferences -> General -> Play DRM-controlled content"
You might also have to install package libavcodec-extra to get the codecs:
`sudo apt install libavcodec-extra`

@reference_1_askubuntu.com

Reinstall ubuntu 19.04 dual boot on T470p - Setting (1)

1.  Fix Windows and Linux Showing Different Times When Dual Booting
 To make linux use Local Time (not UTC time):
 `timedatectl set-local-rtc 1 --adjust-system-clock`

 To check your current settings, run:
`timedatectl`



 To undo this change:
 `timedatectl set-local-rtc 0 --adjust-system-clock`
 
@reference_1_ howtogeek.com


2. Add a (command) directory to the '$PATH' in Ubuntu
Edit `.bashrc` in your home directory and add the following line: 
`export PATH="/path/to/dir:$PATH"`

You will need to source your '.bashrc' or logout/login (or restart the terminal) for the changes to take effect. To source your '.bashrc', simply type 
`source ~/.bashrc`

@reference_2_askubuntu.com

3.  Install "proxychains"
 `apt-get install proxychains`
Edit:   `/etc/proxychains.conf`               (`chmod...` etc.)
Add:  `socks5 127.0.0.1 1080`

Note that: there is a DNS issue in "tsocks"

Use the following script to switch ports:
`re='^[0-9]{4}$'
if [ "$#" -eq  "0" ]
  then
    echo "Error: No arguments supplied"
else
    if ! [[ $1 =~ $re ]] ; then
       echo "Error: argument '$1' is illegal!"
    else sudo sed -i -r 's/(127.0.0.1 )([0-9]{4})/\1'"$1"'/g' /etc/proxychains.conf
    fi
fi
`

@reference_3_yuzhangbit.github.io

4.  Add “New Document” back to the right-click menu
`touch ~/Templates/"Untitled Document"`

@reference_1_vitux.com

5. How to auto-mount windows 10 ntfs partition in dual-boot ubuntu system ?
    "Activities" --> search "Disks" --> select the partition to auto-mount -->
                click [gears icon] (Additional partition options) --> 
                click "Edit Mount Options..." --> 
                turn off "User Session Defaults" -->
                set "Mount Point" to `/media/xxxxx/D Volume` (or other pathes) -->
                click "OK" --> reboot system to check



Reinstall ubuntu 19.04 dual boot on T470p - system installation

1. Using "Rufus" to create a bootable USB flash drive (keep default configuration). Disable Secure Boot and Fast Boot options from UEFI settings (if supported).

2. Delete ubuntu partitions in winfdows 10 or delete it during the installation process. (or shrink windows 10 disk to create "free partition" for ubuntu installation if ubuntu dual boot hasn't been installed before)

3. Select "Something else"

4. create "root partition", "home partition" and "swap partition"
 root partition

 home partition

swap partition
 (Note that the size of swap partition being too small may slow down 'vmware' unless you select "Fit all virtual machine memory into reserved host RAM")

 5. From the Device for boot loader installation select the Windows Boot Manager.


@reference_1_linuxbsdos.com



Wednesday, July 31, 2019

Reinstall ubuntu 19.04 dual boot - install vmware workstation 15 pro

1. Uninstall Workstation Pro from a Linux Host
`su` root
`vmware-installer -u vmware-workstation`

(To inistall, 'sh' the bundle file)

@reference_1_docs.vmware.com

2. Prompt "No 3d support is available from the host" and "hardware graphics acceleration is not available" when booting up the virtual machine.

edit `.vmware/preferences`
`mks.gl.allowBlacklistedDrivers = "TRUE"`

@reference_2_askubuntu.com

3. “Install VMware Tools” greyed out

shutdown the machine and change both the "floppy drive" and "cd drive" to "Auto detect" and then power on the machine.

@reference_3_askubuntu.com

4.  Install VMware Tools 10.3.10 in windows10 vm on ubuntu host machine
(1) Download the 10.3.10 tools from VMware. https://my.vmware.com/web/vmware/details?downloadGroup=VMTOOLS10310&productId=491
(could be downloaded without proxy)
      Get the zip file with Windows and Linux tools.
(2) Unpack the zip file, and execute `chmod 777 -R VMware-Tools-core-10.3.10-12406962` .
(3) Set "floppy drive" to "pvscsi-Windows8.flp" and "cd drive" to "windows.iso" respectively.
(4) Restart the virtual machine, then click into the "cd drive" to install the "VMware Tools"

@reference_4_vm.knutsson.it
@reference_5_communities.vmware.com
@reference_6_my.vmware.com

5. Note that
There is NO "clean up disks" availability for Linux host VMware workstation.
In windows host there is an option to "reclaim disk space" but running with linux as host, that doesn't exist.

@reference_7_communities.vmware.com

6. System "Swap" memory problem
If the size of "Swap Partition" memory allocation is too low when you install the ubuntu dual boot system, your virtual window10 on the ubuntu host could be extremely slow.
To solve the problem:
(1) Run VMware under root (Terminal: `su` --> `vmware`)
(2) Click "Edit" --> "Preferences" --> "Memory" --> "Additional Memory"
      --> select "Fit all virtual machine memory into reserved host RAM"


Sunday, July 7, 2019

Build a 'trojan' server

Step 1: 
(a). install nginx
(b). install letsencrypt as previously posted

Step 2:
(a). obtaining a Let's Encrypt SSL certificate
(b). set  'live' and 'archive' folder in 'letsencrypt' to 755 recursively

@ref_github_1

Step 3:
change nginx configuration as follws:
`
server {
    listen 127.0.0.1:80 default_server;

    server_name example.com;

    location / {
        proxy_redirect off; 
        proxy_pass https://www.xxxx.net;
    }

}

server {
    listen 127.0.0.1:80;

    server_name 100.100.100.100; # server IP address

    return 301 https://example.com;
}

server {
    listen 0.0.0.0:80;
    listen [::]:80;

    server_name _;

    return 301 https://example.com;
}
`
@ref_blog.siaimes.me_2

Step 4:
Option 1: Install 'trojan' from github:
@ref_trojan-gfw.github.io_3
(a).
run
 `sudo apt -y install build-essential cmake libboost-system-dev libboost-program-options-dev libssl-dev default-libmysqlclient-dev`
to install all the necessary dependencies.

(b).
`git clone https://github.com/trojan-gfw/trojan.git`
` cd trojan/`

(c).
`mkdir build`
`cd build/`
`cmake ..`
`make`
`ctest`
`make install`

(d).
edit '/usr/local/etc/trojan/config.json'
set "password"
set "cert": "/etc/letsencrypt/live/example.com/fullchain.pem"
set "key": "/etc/letsencrypt/live/example.com/privkey.pem"
set "alpn": [ "http/2.0" ]

(e).
`systemctl restart trojan nginx`
`systemctl status trojan nginx`
`systemctl enable trojan nginx`

Option 2: Install 'trojan' from 'apt repository'
(a).
`
apt install software-properties-common
add-apt-repository ppa:greaterfire/trojan
apt update

apt install trojan
trojan version
vi /etc/trojan/config.json
`
@ref_blog.siaimes.me_4

Step 5:
Firewall:
'
apt install ufw
ufw allow ssh     # Or port number
ufw allow https
ufw allow http
ufw enable
ufw status
ufw status verbose
'
@ref_blog.siaimes.me_5