Sunday, August 6, 2023
Wednesday, May 5, 2021
Install caddy2 + naiveproxy on ubuntu 20.04
1. Install xcaddy
`cd /tmp`
`wget https://github.com/caddyserver/xcaddy/releases/download/v0.1.9/xcaddy_0.1.9_freebsd_amd64.tar.gz`
`tar xvf xcaddy_0.1.9_freebsd_amd64.tar.gz`
`mv xcaddy /usr/local/bin`
2. build and run Caddy v2 with naïve fork of forwardproxy:
`xcaddy build --with github.com/caddyserver/forwardproxy@caddy2=github.com/klzgrad/forwardproxy@naive`
Allow 80and 443 binding:
`setcap cap_net_bind_service=+ep ./caddy`
`mv caddy /usr/local/bincaddy version`
3. Create caddy user and group
`groupadd --system caddy-g`
`useradd --system --gid caddy-g --create-home --home-dir /var/lib/caddy --shell /usr/sbin/nologin --comment "Caddy web server" caddy-u`
`chown root:root /usr/local/bin/caddy`, `chmod 755 /usr/local/bin/caddy`
4. Create other directories for Caddy
`mkdir /etc/caddy`, `chown -R root:caddy-g /etc/caddy`
`mkdir /etc/ssl/caddy`, `chown -R root:caddy /etc/ssl/caddy`
`chmod 0770 /etc/ssl/caddy`
`mkdir /var/www`, `chown caddy-u:caddy-g /var/www`
5. Create configuration file, service and log for Caddy
`touch /etc/caddy/Caddyfile`, `chmod 644 /etc/caddy/Caddyfile`
Use `head /dev/urandom | tr -dc a-z0-9 | head -c 16 ; echo ''` to generate a password.
Copy the following text into the Caddyfile file:
`:443, example.com
tls example123@gmail.com
route {
forward_proxy {
basic_auth username password
hide_ip
hide_via
probe_resistance
}
file_server { root /var/www }
}
log {
output file /var/log/caddy/access.log {
roll_size 1gb
roll_keep 10
roll_keep_for 720h
}
}`
`touch /etc/systemd/system/caddy.service`
`chmod 644 /etc/systemd/system/caddy.service`
Copy the following text into the caddy.service file:
`[Unit]
Description=Caddy HTTP/2 web server
Documentation=https://caddyserver.com/docs/
After=network-online.target
Wants=network-online.target systemd-networkd-wait-online.service
StartLimitIntervalSec=14400
StartLimitBurst=10
[Service]
Restart=on-abnormal
User=caddy-u
Group=caddy-g
Environment=CADDYPATH=/etc/ssl/caddy
ExecStart= /usr/local/bin/caddy run --environ --config /etc/caddy/Caddyfile
ExecReload= /usr/local/bin/caddy reload --config /etc/caddy/Caddyfile
KillMode=mixed
KillSignal=SIGQUIT
TimeoutStopSec=5s
LimitNOFILE=1048576
LimitNPROC=512
PrivateTmp=true
PrivateDevices=false
ProtectHome=true
ProtectSystem=full
ReadWritePaths=/etc/ssl/caddy
ReadWriteDirectories=/etc/ssl/caddy
AmbientCapabilities=CAP_NET_BIND_SERVICE
[Install]
WantedBy=multi-user.target`
`touch /var/log/caddy/access.log`
`chown root:caddy-g /var/log/caddy/access.log`
`chmod 770 /var/log/caddy/access.log`
6. Install NaiveProxy server
`apt install libnss3 xz-utils`
`wget https://github.com/klzgrad/naiveproxy/releases/download/v90.0.4430.85-6/naiveproxy-v90.0.4430.85-6-linux-x64.tar.xz`
`tar -xf naiveproxy-v90.0.4430.85-6-linux-x64.tar.xz`
`cd naiveproxy-v90.0.4430.85-6-linux-x64`, `cp naive /usr/local/bin`
`touch /etc/systemd/system/naive.service`
Copy the following into the naive.service file:
`[Unit]
Description=NaiveProxy Server Service
After=network-online.target
[Service]
Type=simple
User=nobody
CapabilityBoundingSet=CAP_NET_BIND_SERVICE
ExecStart=/usr/local/bin/naive /etc/naive/config.json
[Install]
WantedBy=multi-user.target`
`mkdir /etc/naive`, `touch /etc/naive/config.json`
Copy the following into the config.json file:
`{
"listen": "socks://127.0.0.1:1080",
"proxy": "https://username:password@example.com",
"padding": true
}`
7. Start & Enable services
`systemctl daemon-reload`
`systemctl enable naive`, `systemctl enable caddy`
`systemctl start naive` , `systemctl start caddy`
`systemctl status naive`, `systemctl status caddy`
8. Install NaiveProxy client
Copy the following into the config.json file:
`{
"listen": "socks://127.0.0.1:1080",
"proxy": "https://username:password@example.com",
"log": "",
"padding": true
}`
Saturday, February 27, 2021
Install Cuda 11.0 & cudnn & TensorFlow 2.4 on ubuntu 20.04 (T470p)
1. Select `nvidia-driver-450(proprietary)` in `Software & Updates -> Additional Drivers`
2. Download latest CUDA Toolkit 11.0 and cuDNN v8.0.5 for CUDA 11.0 and Ubuntu 20.04
3. Only install the cuda toolkit (without driver) to `/usr/local`, run:
`sudo sh ./cuda_11.0.3_450.51.06_linux.run --toolkit --silent --override`
@reference_1_askubuntu
4. Install the runtime library:
`sudo dpkg -i libcudnn8_8.0.5.39-1+cuda11.0_amd64.deb`
Install the developer library:
`sudo dpkg -i libcudnn8-dev_8.0.5.39-1+cuda11.0_amd64.deb`
Install the code samples and the cuDNN library documentation:
`sudo dpkg -i libcudnn8-samples_8.0.5.39-1+cuda11.0_amd64.deb` @reference_2_docs.nvidia
5. Set Python & Pycharm environment
Sunday, April 19, 2020
Install Cuda 10.1 & TensorFlow 2.1 on ubuntu 18.04
@reference_1_tensorflow.org
(1) Install conda: `conda create -n your_env_name python=3.7 pip`
(2) `pip install tensorflow`
@reference_2_itfashion4u
2. Downgrade versions if necessary
`apt-get install -y --no-install-recommends libnvinfer6=6.0.1-1+cuda10.1 libnvinfer-dev=6.0.1-1+cuda10.1 libnvinfer-plugin6=6.0.1-1+cuda10.1 --allow-downgrades`
3. Uncheck Nvidia repositories to prevent ubuntu software updater from updating
"Software & Updates" => "Other Software" => uncheck Nvidia repositories
Tuesday, October 29, 2019
Bridge from Http proxy to Socks5 proxy using Privoxy
2. Add the following 2 lines to the configuration file (config / config.txt)
`listen-address 127.0.0.1:8118
forward-socks5 / 127.0.0.1:1080 .`
(The first line is the address where http proxy server listens,
the second line is the socks5 server address)
Monday, August 5, 2019
Reinstall ubuntu 19.04 dual boot on T470p - Install Pycharm, Clion, Intellij Idea
2. Put it into the directory you want to install pycharm in, unzip it:
`tar -xvzf pycharm-professional-2019.x.tar.gz`
3. Create a 'desktop shortcut launcher':
(1) `apt install --no-install-recommends gnome-panel`
(The 'gnome-desktop-item-edit' command may be missing as it is not installed by default.)
(2) Create Desktop Launcher Dialog: `gnome-desktop-item-edit ~/Desktop/ --create-new`
Enter name, the path to the application binary executable and an optional comment. When ready, click on 'OK'.
(3) Double-click on the new shortcut to edit it:
Change `Icon= ...` and `Icon[en_US]=...` path to the desirable icon path, e.g., '.../pycharm-2019.x/bin/pycharm.png'.
(4) Right-click on the new shortcut and select "Allow Launching".
Double-click on the new shortcut to launch pycharm.
@reference_1_linuxconfig.org
4. "File" --> "Setting" --> search "font" --> "Font" of "Editor" --> set font size '19'
Install "tensorflow-gpu 2.0.0-beta1" on ubuntu 19.04 (T470p)
Open "Software & Updates" --> "Additional Drivers"
--> select "nvidia-driver-418 (proprietary, tested)" --> "Apply Changes"
2. Installed CUDA 10.0 on Ubuntu 19.04
(Note that the released binary 'tensorflow' doesn't support 'cuda 10.1/10.2' as of now, although you may try to compile 'tensorflow' from source against 'cuda 10.1/10.2'.
'cuda 10.0' is only compatable with Ubuntu 18.04, the method below is hacking.)
(1) Go to the CUDA download site. Click "Download Now"
Select "Linux" --> "x86_64" --> "Ubuntu"
--> "18.04" (although we have 19.04) --> "runfile (local)"
(2) Open a new terminal environment and login as root.
Stop the display manager: `service gdm3 stop`
To find out your display manager `pgrep -l dm`
(3) Run the CUDA installer with the override option:
`./cuda_10.0.130_410.48_linux --override`
Install the drivers, the toolkit and the samples.
(Note that the 'override' option will ignore the compiler version check. CUDA requires gcc version 6, but the installation goes fine with gcc version 8 too. The 'override' option allow us to proceed using a later version of the compiler. )
(4) Restart the desktop manager
`service gdm3 start`
and switch again to the desktop environment.
(5) Add the 'CUDA include directory and the CUDA library directory' to the ".bashrc" file.
If you installed CUDA under '/usr/local/cuda-10.0', edit ".bashrc" file, add:
`export PATH="$PATH:/usr/local/cuda-10.0/bin"
export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/local/cuda-10.0/lib64/:/usr/local/cuda-10.0/extras/CUPTI/lib64"`
(6) Go to the samples directory and compile them. (Optional)
Install gcc version 6: `apt-get install g++-6`
Compile the samples using g++ version 6 as the compiler:
`make HOST_COMPILER=g++-6`
@reference_1_askubuntu.com
3. Install 'TensorFlow-gpu 2.0 Beta' on 'miniconda'
(1) Create an environment which has Tensorflow-gpu:
`conda create -n your_env_name python=3.6 pip tensorflow-gpu`
(Now Anaconda is supporting install cudnn 7.6.0 with Tensorflow-gpu (1.1x) so we don’t have to manually copy cudnn to our environment anymore
??? see step (3) -- How to avoid it ??)
(Note that Python 3.7 is not compatible with 'tensorflow_text')
(2) Activate 'your_environment':
`conda activate your_env_name`
(3) Uninstall 'Tensorflow-gpu 1.1x'
`conda uninstall tensorflow-gpu`
(In fact, 'cudnn' is uninstalled together with 'Tensorflow-gpu 1.1x' !!)
(4) Install 'Tensorflow-gpu 2.0 Beta' with pip in your environment:
`pip install tensorflow-gpu==2.0.0-beta1`
(5) Install other packages
`proxychains pip install IPython
proxychains pip install pathlib
proxychains pip install matplotlib
proxychains pip install seaborn
proxychains pip install tensorflow_datasets
proxychains pip install tensorflow_hub
proxychains pip install sklearn
proxychains pip install pandas
proxychains pip install tensorflow_text
proxychains pip install numpy
proxychains pip install pillow`
@reference_2_medium.com
4. Manually copy "cudnn (libcudnn.so.7)" to the environment
(1) Download "cuDNN v7.6.2 (July 22, 2019), for CUDA 10.0"
(2) Unzip package
`tar xvzf cudnn-10.0-linux-x64-v7.6.2.24.tgz`
(3) Copy file to cuda directory
`cp -P cuda/include/cudnn.h /usr/local/cuda-10.0/include`
`cp -P cuda/lib64/libcudnn* /usr/local/cuda-10.0/lib64`
`chmod a+r /usr/local/cuda-10.0/include/cudnn.h /usr/local/cuda-10.0/lib64/libcudnn*`
@reference_3_jermine.vdo.pub
5. Set Pycharm environment to the 'conda TensorFlow environment'
Note that Pycharm environment may be different from command line.
".bashrc" file is being read by bash (your command line interpreter) only.
To preserve bash environment for PyCharm. Run PyCharm from the command line (from bash). Thus environment variables will be inherited from bash to pycharm.
Just launch `./pycharm.sh` from command line.
Or create launcher which invokes bash for PyCharm launching.
@reference_4_stackoverflow.com
6. How to solve the above problem that Pycharm environment doesn't inherit from bash if not launching it from terminal ?
Another way to permanently add new path in 'LD_LIBRARY_PATH' is to edit ".conf" files in "/etc/ld.so.conf.d/".
Specifically, add
`"/usr/local/cuda-10.0/bin"
"/usr/local/cuda-10.0/lib64"
"/usr/local/cuda-10.0/extras/CUPTI/lib64"` 3 lines
to the "x86_64-linux-gnu.conf" file.
Just in the next line. Save.
Then run `ldconfig`
@reference_5_unix.stackexchange.com
7. If GPU not found, try reinstalling 'tensorflow-gpu==2.0.0-beta1'
`pip uninstall tensorflow-gpu==2.0.0-beta1
pip install tensorflow-gpu==2.0.0-beta1`