1. Install Cuda 10.1 & TensorFlow 2.1
@reference_1_tensorflow.org
(1) Install conda: `conda create -n your_env_name python=3.7 pip`
(2) `pip install tensorflow`
@reference_2_itfashion4u
2. Downgrade versions if necessary
`apt-get install -y --no-install-recommends libnvinfer6=6.0.1-1+cuda10.1 libnvinfer-dev=6.0.1-1+cuda10.1 libnvinfer-plugin6=6.0.1-1+cuda10.1 --allow-downgrades`
3. Uncheck Nvidia repositories to prevent ubuntu software updater from updating
"Software & Updates" => "Other Software" => uncheck Nvidia repositories
Showing posts with label Linux. Show all posts
Showing posts with label Linux. Show all posts
Sunday, April 19, 2020
Monday, August 5, 2019
Reinstall ubuntu 19.04 dual boot on T470p - Install Pycharm, Clion, Intellij Idea
1. Download pycharm (Clion, Intellij Idea)
2. Put it into the directory you want to install pycharm in, unzip it:
`tar -xvzf pycharm-professional-2019.x.tar.gz`
3. Create a 'desktop shortcut launcher':
(1) `apt install --no-install-recommends gnome-panel`
(The 'gnome-desktop-item-edit' command may be missing as it is not installed by default.)
(2) Create Desktop Launcher Dialog: `gnome-desktop-item-edit ~/Desktop/ --create-new`
Enter name, the path to the application binary executable and an optional comment. When ready, click on 'OK'.
(3) Double-click on the new shortcut to edit it:
Change `Icon= ...` and `Icon[en_US]=...` path to the desirable icon path, e.g., '.../pycharm-2019.x/bin/pycharm.png'.
(4) Right-click on the new shortcut and select "Allow Launching".
Double-click on the new shortcut to launch pycharm.
@reference_1_linuxconfig.org
4. "File" --> "Setting" --> search "font" --> "Font" of "Editor" --> set font size '19'
2. Put it into the directory you want to install pycharm in, unzip it:
`tar -xvzf pycharm-professional-2019.x.tar.gz`
3. Create a 'desktop shortcut launcher':
(1) `apt install --no-install-recommends gnome-panel`
(The 'gnome-desktop-item-edit' command may be missing as it is not installed by default.)
(2) Create Desktop Launcher Dialog: `gnome-desktop-item-edit ~/Desktop/ --create-new`
Enter name, the path to the application binary executable and an optional comment. When ready, click on 'OK'.
(3) Double-click on the new shortcut to edit it:
Change `Icon= ...` and `Icon[en_US]=...` path to the desirable icon path, e.g., '.../pycharm-2019.x/bin/pycharm.png'.
(4) Right-click on the new shortcut and select "Allow Launching".
Double-click on the new shortcut to launch pycharm.
@reference_1_linuxconfig.org
4. "File" --> "Setting" --> search "font" --> "Font" of "Editor" --> set font size '19'
Install "tensorflow-gpu 2.0.0-beta1" on ubuntu 19.04 (T470p)
1. Install nvidia driver
Open "Software & Updates" --> "Additional Drivers"
--> select "nvidia-driver-418 (proprietary, tested)" --> "Apply Changes"
2. Installed CUDA 10.0 on Ubuntu 19.04
(Note that the released binary 'tensorflow' doesn't support 'cuda 10.1/10.2' as of now, although you may try to compile 'tensorflow' from source against 'cuda 10.1/10.2'.
'cuda 10.0' is only compatable with Ubuntu 18.04, the method below is hacking.)
(1) Go to the CUDA download site. Click "Download Now"
Select "Linux" --> "x86_64" --> "Ubuntu"
--> "18.04" (although we have 19.04) --> "runfile (local)"
(2) Open a new terminal environment and login as root.
Stop the display manager: `service gdm3 stop`
To find out your display manager `pgrep -l dm`
(3) Run the CUDA installer with the override option:
`./cuda_10.0.130_410.48_linux --override`
Install the drivers, the toolkit and the samples.
(Note that the 'override' option will ignore the compiler version check. CUDA requires gcc version 6, but the installation goes fine with gcc version 8 too. The 'override' option allow us to proceed using a later version of the compiler. )
(4) Restart the desktop manager
`service gdm3 start`
and switch again to the desktop environment.
(5) Add the 'CUDA include directory and the CUDA library directory' to the ".bashrc" file.
If you installed CUDA under '/usr/local/cuda-10.0', edit ".bashrc" file, add:
`export PATH="$PATH:/usr/local/cuda-10.0/bin"
export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/local/cuda-10.0/lib64/:/usr/local/cuda-10.0/extras/CUPTI/lib64"`
(6) Go to the samples directory and compile them. (Optional)
Install gcc version 6: `apt-get install g++-6`
Compile the samples using g++ version 6 as the compiler:
`make HOST_COMPILER=g++-6`
@reference_1_askubuntu.com
3. Install 'TensorFlow-gpu 2.0 Beta' on 'miniconda'
(1) Create an environment which has Tensorflow-gpu:
`conda create -n your_env_name python=3.6 pip tensorflow-gpu`
(Now Anaconda is supporting install cudnn 7.6.0 with Tensorflow-gpu (1.1x) so we don’t have to manually copy cudnn to our environment anymore
??? see step (3) -- How to avoid it ??)
(Note that Python 3.7 is not compatible with 'tensorflow_text')
(2) Activate 'your_environment':
`conda activate your_env_name`
(3) Uninstall 'Tensorflow-gpu 1.1x'
`conda uninstall tensorflow-gpu`
(In fact, 'cudnn' is uninstalled together with 'Tensorflow-gpu 1.1x' !!)
(4) Install 'Tensorflow-gpu 2.0 Beta' with pip in your environment:
`pip install tensorflow-gpu==2.0.0-beta1`
(5) Install other packages
`proxychains pip install IPython
proxychains pip install pathlib
proxychains pip install matplotlib
proxychains pip install seaborn
proxychains pip install tensorflow_datasets
proxychains pip install tensorflow_hub
proxychains pip install sklearn
proxychains pip install pandas
proxychains pip install tensorflow_text
proxychains pip install numpy
proxychains pip install pillow`
@reference_2_medium.com
4. Manually copy "cudnn (libcudnn.so.7)" to the environment
(1) Download "cuDNN v7.6.2 (July 22, 2019), for CUDA 10.0"
(2) Unzip package
`tar xvzf cudnn-10.0-linux-x64-v7.6.2.24.tgz`
(3) Copy file to cuda directory
`cp -P cuda/include/cudnn.h /usr/local/cuda-10.0/include`
`cp -P cuda/lib64/libcudnn* /usr/local/cuda-10.0/lib64`
`chmod a+r /usr/local/cuda-10.0/include/cudnn.h /usr/local/cuda-10.0/lib64/libcudnn*`
@reference_3_jermine.vdo.pub
5. Set Pycharm environment to the 'conda TensorFlow environment'
Note that Pycharm environment may be different from command line.
".bashrc" file is being read by bash (your command line interpreter) only.
To preserve bash environment for PyCharm. Run PyCharm from the command line (from bash). Thus environment variables will be inherited from bash to pycharm.
Just launch `./pycharm.sh` from command line.
Or create launcher which invokes bash for PyCharm launching.
@reference_4_stackoverflow.com
6. How to solve the above problem that Pycharm environment doesn't inherit from bash if not launching it from terminal ?
Another way to permanently add new path in 'LD_LIBRARY_PATH' is to edit ".conf" files in "/etc/ld.so.conf.d/".
Specifically, add
`"/usr/local/cuda-10.0/bin"
"/usr/local/cuda-10.0/lib64"
"/usr/local/cuda-10.0/extras/CUPTI/lib64"` 3 lines
to the "x86_64-linux-gnu.conf" file.
Just in the next line. Save.
Then run `ldconfig`
@reference_5_unix.stackexchange.com
7. If GPU not found, try reinstalling 'tensorflow-gpu==2.0.0-beta1'
`pip uninstall tensorflow-gpu==2.0.0-beta1
pip install tensorflow-gpu==2.0.0-beta1`
Open "Software & Updates" --> "Additional Drivers"
--> select "nvidia-driver-418 (proprietary, tested)" --> "Apply Changes"
2. Installed CUDA 10.0 on Ubuntu 19.04
(Note that the released binary 'tensorflow' doesn't support 'cuda 10.1/10.2' as of now, although you may try to compile 'tensorflow' from source against 'cuda 10.1/10.2'.
'cuda 10.0' is only compatable with Ubuntu 18.04, the method below is hacking.)
(1) Go to the CUDA download site. Click "Download Now"
Select "Linux" --> "x86_64" --> "Ubuntu"
--> "18.04" (although we have 19.04) --> "runfile (local)"
(2) Open a new terminal environment and login as root.
Stop the display manager: `service gdm3 stop`
To find out your display manager `pgrep -l dm`
(3) Run the CUDA installer with the override option:
`./cuda_10.0.130_410.48_linux --override`
Install the drivers, the toolkit and the samples.
(Note that the 'override' option will ignore the compiler version check. CUDA requires gcc version 6, but the installation goes fine with gcc version 8 too. The 'override' option allow us to proceed using a later version of the compiler. )
(4) Restart the desktop manager
`service gdm3 start`
and switch again to the desktop environment.
(5) Add the 'CUDA include directory and the CUDA library directory' to the ".bashrc" file.
If you installed CUDA under '/usr/local/cuda-10.0', edit ".bashrc" file, add:
`export PATH="$PATH:/usr/local/cuda-10.0/bin"
export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/local/cuda-10.0/lib64/:/usr/local/cuda-10.0/extras/CUPTI/lib64"`
(6) Go to the samples directory and compile them. (Optional)
Install gcc version 6: `apt-get install g++-6`
Compile the samples using g++ version 6 as the compiler:
`make HOST_COMPILER=g++-6`
@reference_1_askubuntu.com
3. Install 'TensorFlow-gpu 2.0 Beta' on 'miniconda'
(1) Create an environment which has Tensorflow-gpu:
`conda create -n your_env_name python=3.6 pip tensorflow-gpu`
(Now Anaconda is supporting install cudnn 7.6.0 with Tensorflow-gpu (1.1x) so we don’t have to manually copy cudnn to our environment anymore
??? see step (3) -- How to avoid it ??)
(Note that Python 3.7 is not compatible with 'tensorflow_text')
(2) Activate 'your_environment':
`conda activate your_env_name`
(3) Uninstall 'Tensorflow-gpu 1.1x'
`conda uninstall tensorflow-gpu`
(In fact, 'cudnn' is uninstalled together with 'Tensorflow-gpu 1.1x' !!)
(4) Install 'Tensorflow-gpu 2.0 Beta' with pip in your environment:
`pip install tensorflow-gpu==2.0.0-beta1`
(5) Install other packages
`proxychains pip install IPython
proxychains pip install pathlib
proxychains pip install matplotlib
proxychains pip install seaborn
proxychains pip install tensorflow_datasets
proxychains pip install tensorflow_hub
proxychains pip install sklearn
proxychains pip install pandas
proxychains pip install tensorflow_text
proxychains pip install numpy
proxychains pip install pillow`
@reference_2_medium.com
4. Manually copy "cudnn (libcudnn.so.7)" to the environment
(1) Download "cuDNN v7.6.2 (July 22, 2019), for CUDA 10.0"
(2) Unzip package
`tar xvzf cudnn-10.0-linux-x64-v7.6.2.24.tgz`
(3) Copy file to cuda directory
`cp -P cuda/include/cudnn.h /usr/local/cuda-10.0/include`
`cp -P cuda/lib64/libcudnn* /usr/local/cuda-10.0/lib64`
`chmod a+r /usr/local/cuda-10.0/include/cudnn.h /usr/local/cuda-10.0/lib64/libcudnn*`
@reference_3_jermine.vdo.pub
5. Set Pycharm environment to the 'conda TensorFlow environment'
Note that Pycharm environment may be different from command line.
".bashrc" file is being read by bash (your command line interpreter) only.
To preserve bash environment for PyCharm. Run PyCharm from the command line (from bash). Thus environment variables will be inherited from bash to pycharm.
Just launch `./pycharm.sh` from command line.
Or create launcher which invokes bash for PyCharm launching.
@reference_4_stackoverflow.com
6. How to solve the above problem that Pycharm environment doesn't inherit from bash if not launching it from terminal ?
Another way to permanently add new path in 'LD_LIBRARY_PATH' is to edit ".conf" files in "/etc/ld.so.conf.d/".
Specifically, add
`"/usr/local/cuda-10.0/bin"
"/usr/local/cuda-10.0/lib64"
"/usr/local/cuda-10.0/extras/CUPTI/lib64"` 3 lines
to the "x86_64-linux-gnu.conf" file.
Just in the next line. Save.
Then run `ldconfig`
@reference_5_unix.stackexchange.com
7. If GPU not found, try reinstalling 'tensorflow-gpu==2.0.0-beta1'
`pip uninstall tensorflow-gpu==2.0.0-beta1
pip install tensorflow-gpu==2.0.0-beta1`
Thursday, August 1, 2019
Reinstall ubuntu 19.04 dual boot on T470p - Setting (3)
1. Install 'Git'
`apt update`
`apt install git`
`git --version`
@reference_1_digitalocean.com
2. Check 'build-essential'
`apt install build-essential`
`gcc --version` , `g++ --version`, `make --version`
@reference_2_linuxize.com
3. Install 'openjdk'
`apt install default-jdk `
`java --version`, `javac --version`
@reference_3_ digitalocean.com
4. Install 'pip3' and 'virtualenv'
`apt install python3-pip`
`pip3 -V`
`pip3 install virtualenv`
5. Install 'miniconda'
(1) download miniconda
(2) `chmod +x Miniconda3-latest-Linux-x86_64.sh`
`bash Miniconda3-latest-Linux-x86_64.sh`
During the installation, choose the desired path to install.
(3) `source ~/.bashrc`
`conda -V`, `conda list`
(4) remove (base) from terminal prompt
check "auto_activate_base":
`conda config --show | grep auto_activate_base`
Set it false:
`conda config --set auto_activate_base False`
@reference_1_askubuntu.com
@reference_2_digitalocean.com
6. How to uninstall 'miniconda' ?
(1) In order to uninstall miniconda, simply remove the miniconda folder:
`rm -r ~/miniconda/`
(2) Manually remove contents related to conda in '~/.bashrc'
@reference_3_stackoverflow.com
7. How to completely remove Thunderbird?
`apt-get purge thunderbird* `
@reference_4_askubuntu.com
`apt update`
`apt install git`
`git --version`
@reference_1_digitalocean.com
2. Check 'build-essential'
`apt install build-essential`
`gcc --version` , `g++ --version`, `make --version`
@reference_2_linuxize.com
3. Install 'openjdk'
`apt install default-jdk `
`java --version`, `javac --version`
@reference_3_ digitalocean.com
4. Install 'pip3' and 'virtualenv'
`apt install python3-pip`
`pip3 -V`
`pip3 install virtualenv`
5. Install 'miniconda'
(1) download miniconda
(2) `chmod +x Miniconda3-latest-Linux-x86_64.sh`
`bash Miniconda3-latest-Linux-x86_64.sh`
During the installation, choose the desired path to install.
(3) `source ~/.bashrc`
`conda -V`, `conda list`
(4) remove (base) from terminal prompt
check "auto_activate_base":
`conda config --show | grep auto_activate_base`
Set it false:
`conda config --set auto_activate_base False`
@reference_1_askubuntu.com
@reference_2_digitalocean.com
6. How to uninstall 'miniconda' ?
(1) In order to uninstall miniconda, simply remove the miniconda folder:
`rm -r ~/miniconda/`
(2) Manually remove contents related to conda in '~/.bashrc'
@reference_3_stackoverflow.com
7. How to completely remove Thunderbird?
`apt-get purge thunderbird* `
@reference_4_askubuntu.com
Reinstall ubuntu 19.04 dual boot on T470p - Setting (2)
1. Right click on the desktop, in the menu select "Display Settings"
"Display Settings" --> "Resolution" --> "2048 x 1152 (16:9)"
--> "Refresh Rate" --> "119.96Hz"
--> "Scale" --> "100%"
2. Open "Terminal" --> "Preferences" --> "Unnamed" -->
"Initial terminal size" --> "152 x 36"
"Custom font" --> "12"
3. Open "Settings"
(1) "Dock" --> scale dock to a proper size
(2) "Privacy" --> configure privacy setting
(3) "Power" -->
"Blank screen" --> select "Never"
"Automatic suspend" --> "off"
"When the Power Button is pressed" --> "Suspend"
(4) "Details" --> "About" --> modify "Device name" to make it short in terminal
--> "Date & Time" --> set proper date & time format
--> "User" --> select an image for login
--> "Default Application" --> select proper default application
4. Install "tweaks"
`sudo add-apt-repository universe`
`sudo apt install gnome-tweak-tool`
Open "tweaks":
(1) "General" --> turn off "Suspend when laptop lid is closed"
(2) "Top Bar" --> Select proper 'clock' and 'calendar' format
(3) "Windows" --> turn on "Center New Windows"
5. Install "VLC media player" from "Ubuntu Software"
(1) click 'loop one sign'
(2) "Tools" --> "Preferences" --> "Interface" -->
"Show media change popup" --> select "Never"
"Continue playback?" --> select "Never"
"Save recently played items" --> "off"
"Allow metedata network access" --> "off"
Open ubuntu system "Settings" -->
"Notifications" --> "VLC media player" --> "off"
6. Playing videos in firefox
If those streaming services use DRM, you must enable DRM in Firefox's settings: "Preferences -> General -> Play DRM-controlled content"
You might also have to install package libavcodec-extra to get the codecs:
`sudo apt install libavcodec-extra`
@reference_1_askubuntu.com
"Display Settings" --> "Resolution" --> "2048 x 1152 (16:9)"
--> "Refresh Rate" --> "119.96Hz"
--> "Scale" --> "100%"
2. Open "Terminal" --> "Preferences" --> "Unnamed" -->
"Initial terminal size" --> "152 x 36"
"Custom font" --> "12"
3. Open "Settings"
(1) "Dock" --> scale dock to a proper size
(2) "Privacy" --> configure privacy setting
(3) "Power" -->
"Blank screen" --> select "Never"
"Automatic suspend" --> "off"
"When the Power Button is pressed" --> "Suspend"
(4) "Details" --> "About" --> modify "Device name" to make it short in terminal
--> "Date & Time" --> set proper date & time format
--> "User" --> select an image for login
--> "Default Application" --> select proper default application
4. Install "tweaks"
`sudo add-apt-repository universe`
`sudo apt install gnome-tweak-tool`
Open "tweaks":
(1) "General" --> turn off "Suspend when laptop lid is closed"
(2) "Top Bar" --> Select proper 'clock' and 'calendar' format
(3) "Windows" --> turn on "Center New Windows"
5. Install "VLC media player" from "Ubuntu Software"
(1) click 'loop one sign'
(2) "Tools" --> "Preferences" --> "Interface" -->
"Show media change popup" --> select "Never"
"Continue playback?" --> select "Never"
"Save recently played items" --> "off"
"Allow metedata network access" --> "off"
Open ubuntu system "Settings" -->
"Notifications" --> "VLC media player" --> "off"
6. Playing videos in firefox
If those streaming services use DRM, you must enable DRM in Firefox's settings: "Preferences -> General -> Play DRM-controlled content"
You might also have to install package libavcodec-extra to get the codecs:
`sudo apt install libavcodec-extra`
@reference_1_askubuntu.com
Reinstall ubuntu 19.04 dual boot on T470p - Setting (1)
1. Fix Windows and Linux Showing Different Times When Dual Booting
To make linux use Local Time (not UTC time):
`timedatectl set-local-rtc 1 --adjust-system-clock`
To check your current settings, run:
`timedatectl`
To undo this change:
`timedatectl set-local-rtc 0 --adjust-system-clock`
@reference_1_ howtogeek.com
2. Add a (command) directory to the '$PATH' in Ubuntu
To make linux use Local Time (not UTC time):
`timedatectl set-local-rtc 1 --adjust-system-clock`
To check your current settings, run:
`timedatectl`
To undo this change:
`timedatectl set-local-rtc 0 --adjust-system-clock`
@reference_1_ howtogeek.com
2. Add a (command) directory to the '$PATH' in Ubuntu
Edit `.bashrc` in your home directory and add the following line:
`export PATH="/path/to/dir:$PATH"`
You will need to source your '.bashrc' or logout/login (or restart the terminal) for the changes to take effect. To source your '.bashrc', simply type
You will need to source your '.bashrc' or logout/login (or restart the terminal) for the changes to take effect. To source your '.bashrc', simply type
`source ~/.bashrc`
Edit: `/etc/proxychains.conf` (`chmod...` etc.)
Add:
Note that: there is a DNS issue in "tsocks"
Use the following script to switch ports:
`re='^[0-9]{4}$'
if [ "$#" -eq "0" ]
then
echo "Error: No arguments supplied"
else
if ! [[ $1 =~ $re ]] ; then
echo "Error: argument '$1' is illegal!"
else sudo sed -i -r 's/(127.0.0.1 )([0-9]{4})/\1'"$1"'/g' /etc/proxychains.conf
fi
fi`
@reference_3_yuzhangbit.github.io
4. Add “New Document” back to the right-click menu
`touch ~/Templates/"Untitled Document"`
@reference_1_vitux.com
5. How to auto-mount windows 10 ntfs partition in dual-boot ubuntu system ?
"Activities" --> search "Disks" --> select the partition to auto-mount -->
click [gears icon] (Additional partition options) -->
click "Edit Mount Options..." -->
turn off "User Session Defaults" -->
set "Mount Point" to `/media/xxxxx/D Volume` (or other pathes) -->
click "OK" --> reboot system to check
Add:
`socks5 127.0.0.1 1080`Note that: there is a DNS issue in "tsocks"
Use the following script to switch ports:
`re='^[0-9]{4}$'
if [ "$#" -eq "0" ]
then
echo "Error: No arguments supplied"
else
if ! [[ $1 =~ $re ]] ; then
echo "Error: argument '$1' is illegal!"
else sudo sed -i -r 's/(127.0.0.1 )([0-9]{4})/\1'"$1"'/g' /etc/proxychains.conf
fi
fi`
@reference_3_yuzhangbit.github.io
4. Add “New Document” back to the right-click menu
`touch ~/Templates/"Untitled Document"`
@reference_1_vitux.com
5. How to auto-mount windows 10 ntfs partition in dual-boot ubuntu system ?
"Activities" --> search "Disks" --> select the partition to auto-mount -->
click [gears icon] (Additional partition options) -->
click "Edit Mount Options..." -->
turn off "User Session Defaults" -->
set "Mount Point" to `/media/xxxxx/D Volume` (or other pathes) -->
click "OK" --> reboot system to check
Reinstall ubuntu 19.04 dual boot on T470p - system installation
1. Using "Rufus" to create a bootable USB flash drive (keep default configuration). Disable Secure Boot and Fast Boot options from UEFI settings (if supported).
2. Delete ubuntu partitions in winfdows 10 or delete it during the installation process. (or shrink windows 10 disk to create "free partition" for ubuntu installation if ubuntu dual boot hasn't been installed before)
3. Select "Something else"
4. create "root partition", "home partition" and "swap partition"
5. From the Device for boot loader installation, select the Windows Boot Manager.
@reference_1_linuxbsdos.com
2. Delete ubuntu partitions in winfdows 10 or delete it during the installation process. (or shrink windows 10 disk to create "free partition" for ubuntu installation if ubuntu dual boot hasn't been installed before)
3. Select "Something else"
4. create "root partition", "home partition" and "swap partition"
root partition
home partition
swap partition
(Note that the size of swap partition being too small may slow down 'vmware' unless you select "Fit all virtual machine memory into reserved host RAM")
5. From the Device for boot loader installation, select the Windows Boot Manager.
@reference_1_linuxbsdos.com
Wednesday, July 31, 2019
Reinstall ubuntu 19.04 dual boot - install vmware workstation 15 pro
1. Uninstall Workstation Pro from a Linux Host
`su` root
`vmware-installer -u vmware-workstation`
(To inistall, 'sh' the bundle file)
@reference_1_docs.vmware.com
2. Prompt "No 3d support is available from the host" and "hardware graphics acceleration is not available" when booting up the virtual machine.
edit `.vmware/preferences`
`mks.gl.allowBlacklistedDrivers = "TRUE"`
@reference_2_askubuntu.com
3. “Install VMware Tools” greyed out
shutdown the machine and change both the "floppy drive" and "cd drive" to "Auto detect" and then power on the machine.
@reference_3_askubuntu.com
4. Install VMware Tools 10.3.10 in windows10 vm on ubuntu host machine
(1) Download the 10.3.10 tools from VMware. https://my.vmware.com/web/vmware/details?downloadGroup=VMTOOLS10310&productId=491
(could be downloaded without proxy)
Get the zip file with Windows and Linux tools.
(2) Unpack the zip file, and execute `chmod 777 -R VMware-Tools-core-10.3.10-12406962` .
(3) Set "floppy drive" to "pvscsi-Windows8.flp" and "cd drive" to "windows.iso" respectively.
(4) Restart the virtual machine, then click into the "cd drive" to install the "VMware Tools"
@reference_4_vm.knutsson.it
@reference_5_communities.vmware.com
@reference_6_my.vmware.com
5. Note that
There is NO "clean up disks" availability for Linux host VMware workstation.
In windows host there is an option to "reclaim disk space" but running with linux as host, that doesn't exist.
@reference_7_communities.vmware.com
6. System "Swap" memory problem
If the size of "Swap Partition" memory allocation is too low when you install the ubuntu dual boot system, your virtual window10 on the ubuntu host could be extremely slow.
To solve the problem:
(1) Run VMware under root (Terminal: `su` --> `vmware`)
(2) Click "Edit" --> "Preferences" --> "Memory" --> "Additional Memory"
--> select "Fit all virtual machine memory into reserved host RAM"
`su` root
`vmware-installer -u vmware-workstation`
(To inistall, 'sh' the bundle file)
@reference_1_docs.vmware.com
2. Prompt "No 3d support is available from the host" and "hardware graphics acceleration is not available" when booting up the virtual machine.
edit `.vmware/preferences`
`mks.gl.allowBlacklistedDrivers = "TRUE"`
@reference_2_askubuntu.com
3. “Install VMware Tools” greyed out
shutdown the machine and change both the "floppy drive" and "cd drive" to "Auto detect" and then power on the machine.
@reference_3_askubuntu.com
4. Install VMware Tools 10.3.10 in windows10 vm on ubuntu host machine
(1) Download the 10.3.10 tools from VMware. https://my.vmware.com/web/vmware/details?downloadGroup=VMTOOLS10310&productId=491
(could be downloaded without proxy)
Get the zip file with Windows and Linux tools.
(2) Unpack the zip file, and execute `chmod 777 -R VMware-Tools-core-10.3.10-12406962` .
(3) Set "floppy drive" to "pvscsi-Windows8.flp" and "cd drive" to "windows.iso" respectively.
(4) Restart the virtual machine, then click into the "cd drive" to install the "VMware Tools"
@reference_4_vm.knutsson.it
@reference_5_communities.vmware.com
@reference_6_my.vmware.com
5. Note that
There is NO "clean up disks" availability for Linux host VMware workstation.
In windows host there is an option to "reclaim disk space" but running with linux as host, that doesn't exist.
@reference_7_communities.vmware.com
6. System "Swap" memory problem
If the size of "Swap Partition" memory allocation is too low when you install the ubuntu dual boot system, your virtual window10 on the ubuntu host could be extremely slow.
To solve the problem:
(1) Run VMware under root (Terminal: `su` --> `vmware`)
(2) Click "Edit" --> "Preferences" --> "Memory" --> "Additional Memory"
--> select "Fit all virtual machine memory into reserved host RAM"
Tuesday, October 10, 2017
Tuesday, September 19, 2017
Set no password for `su` in Ubuntu 17.04
1. Create new group: groupadd wheel
2. Add your user name: usermod -G wheel
3. gedit /etc/pam.d/su
# Uncomment this if you want wheel members to be able to
# su without a password.
auth sufficient pam_wheel.so trust
2. Add your user name: usermod -G wheel
your_user_name3. gedit /etc/pam.d/su
# Uncomment this if you want wheel members to be able to
# su without a password.
auth sufficient pam_wheel.so trust
Set Java environment variables for all users --- Difference between `su` & `su -`
1.
2. add:
3. restart or reopen shell or use
Difference between `su` & `su -`:
Using `su` won't load
birdonwire@win-u5n0t6:~$ su
Password:
root@win-u5n0t6:/home/birdonwire# $PATH
bash: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games: No such file or directory
root@win-u5n0t6:/home/birdonwire#
However `su -` will load
birdonwire@win-u5n0t6:~$ su -
Password:
root@win-u5n0t6:~# $PATH
-su: /home/birdonwire/Bash/com:/home/birdonwire/Develop/jdk1.8.0_144/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin: No such file or directory
root@win-u5n0t6:~#
Q: Then how to set persistent environment variables for root or all users?
A: To create persistent variables that are truly system-wide, you should set them both in
If you only set environment variables in
@reference_1_stackoverflow.com
How to set JAVA_HOME in Linux for all users
@reference_2_superuser.com
gedit /etc/profile2. add:
export JAVA_HOME=/home/birdonwire/Develop/jdk1.8.0_144/ export PATH=$JAVA_HOME/bin:$PATH
3. restart or reopen shell or use
source /etc/profile to apply changes immediatelyDifference between `su` & `su -`:
Using `su` won't load
/etc/profilebirdonwire@win-u5n0t6:~$ su
Password:
root@win-u5n0t6:/home/birdonwire# $PATH
bash: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games: No such file or directory
root@win-u5n0t6:/home/birdonwire#
However `su -` will load
/etc/profilebirdonwire@win-u5n0t6:~$ su -
Password:
root@win-u5n0t6:~# $PATH
-su: /home/birdonwire/Bash/com:/home/birdonwire/Develop/jdk1.8.0_144/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin: No such file or directory
root@win-u5n0t6:~#
Q: Then how to set persistent environment variables for root or all users?
A: To create persistent variables that are truly system-wide, you should set them both in
/etc/environment and /etc/profile .If you only set environment variables in
/etc/environment, `su - <root or other users>` won't load /etc/environment, but will load /etc/profile , although `su` will load /etc/environment, won't load /etc/profile . Therefore, you have to set environment variables both in /etc/environment and /etc/profile to make sure all users are covered. @reference_1_stackoverflow.com
How to set JAVA_HOME in Linux for all users
@reference_2_superuser.com
Adjusting $PATH in /etc/profile does not affect root
How do I set persistent environment variables for root?
Wednesday, September 13, 2017
About using iptables owner match to block a specific application
iptables -A OUTPUT -m owner --gid-owner black -j REJECT
User name (uid) or group name (gid) won't help because these are the IDs of the user executing the application. Thus it would apply to all applications by that particular user and not just a single one.
Blocking a specific application is usually done by blocking all ports the application uses. But this only works if the user can't change the ports the application will use.
@reference_1_unix.stackexchange.com
Block specific application with iptables
Then how to establish a white-list mode using iptables owner match?
#Allow Loopback Connections
iptables -A INPUT -i lo -j ACCEPT
iptables -A OUTPUT -o lo -j ACCEPT
iptables -A OUTPUT -m owner --uid-owner current_login_user -j REJECT
iptables -A OUTPUT -m owner --uid-owner root -j REJECT
iptables -A OUTPUT -m owner --uid-owner other_users -j REJECT
1. block all the users using above commands
2. create a white-list user allowed to use network
3. start your program using the white-list user under your current login user
User name (uid) or group name (gid) won't help because these are the IDs of the user executing the application. Thus it would apply to all applications by that particular user and not just a single one.
Blocking a specific application is usually done by blocking all ports the application uses. But this only works if the user can't change the ports the application will use.
@reference_1_unix.stackexchange.com
Block specific application with iptables
Then how to establish a white-list mode using iptables owner match?
#Allow Loopback Connections
iptables -A INPUT -i lo -j ACCEPT
iptables -A OUTPUT -o lo -j ACCEPT
iptables -A OUTPUT -m owner --uid-owner current_login_user -j REJECT
iptables -A OUTPUT -m owner --uid-owner root -j REJECT
iptables -A OUTPUT -m owner --uid-owner other_users -j REJECT
1. block all the users using above commands
2. create a white-list user allowed to use network
3. start your program using the white-list user under your current login user
root privilege & execute permission
sudo chmod 000 xxx
execution of the script(or program) xxx under the root account will be denied.
sudo chmod 100 xxx
sudo chmod 001 xxx
sudo chmod 010 xxx
either command above will make the execution of the script(or program) xxx under the root account accepted.
execution of the script(or program) xxx under the root account will be denied.
sudo chmod 100 xxx
sudo chmod 001 xxx
sudo chmod 010 xxx
either command above will make the execution of the script(or program) xxx under the root account accepted.
Difference between `sh xxx.sh` & `./xxx.sh` commands in ubuntu
`sh xxx.sh` requires +r Permission
`./xxx.sh` requires +rx Permission
@reference_1_forum.ubuntu.org.cn
./和sh的区别
`./xxx.sh` requires +rx Permission
@reference_1_forum.ubuntu.org.cn
./和sh的区别
Sunday, September 10, 2017
Install nmap on Ubuntu
1. Download
Back on your auditing machine, move into your home directory and use
@reference_1_digitalocean.com
How To Test your Firewall Configuration with Nmap and Tcpdump
@reference_2_nmap.org
Source Code Distribution (in case you wish to compile Nmap yourself)
Back on your auditing machine, move into your home directory and use
wget
to download the link you pasted. Make sure to update the link below to
reflect the most recent version you copied from the site:
- cd ~
- wget https://nmap.org/dist/nmap-7.60.tar.bz2
2. Decompress the file you downloaded and move into the resulting directory by typing:
- tar xjvf nmap*
- cd nmap*
3. Configure and compile the source code by typing:
- ./configure
- make
4. Once the compilation is complete, you can install the resulting executables and supporting files on your system by typing:
- sudo make install
5. Confirm your installation by typing:
- nmap -V
The output should match the version you downloaded from the nmap website.@reference_1_digitalocean.com
How To Test your Firewall Configuration with Nmap and Tcpdump
@reference_2_nmap.org
Source Code Distribution (in case you wish to compile Nmap yourself)
Install Wireshark on Ubuntu
1. Adding this PPA to your system
2. Install Wireshark
3. Allow non-root user to be able to sniff
select
4. Add user to
Add your username to the
@reference_1_launchpad.net
Wireshark stable releases
@reference_2_askubuntu.com
Ubuntu 16.04下安装网络流量分析工具 Wireshark
sudo add-apt-repository ppa:wireshark-dev/stable
sudo apt-get update
2. Install Wireshark
sudo apt-get install wireshark
3. Allow non-root user to be able to sniff
sudo dpkg-reconfigure wireshark-common
select
Yes and hit return. This adds a wireshark group. Anybody in that group will be able to sniff without being root.4. Add user to
wireshark groupsudo gedit /etc/group
Add your username to the
wireshark group.@reference_1_launchpad.net
Wireshark stable releases
@reference_2_askubuntu.com
How do I run wireshark, with root-privileges?
@reference_3_linuxidc.comUbuntu 16.04下安装网络流量分析工具 Wireshark
Tuesday, September 5, 2017
Install shadowsocks-libev from source on Ubuntu
1. Install the packages required:
3. build shadowsocks-libev and all its dependencies by script:
4.Edit
Ubuntu/Debian Installed from deb package(Installed from deb package will implicitly activate self start-up):
@reference_1_github.com
@reference_2_github.com
mkdir shadowsocks-libev && cd shadowsocks-libev
apt-get install build-essential autoconf libtool libssl-dev \
gawk debhelper dh-systemd init-system-helpers pkg-config git
2. Download the source code through git:git clone https://github.com/shadowsocks/shadowsocks-libev.git3. build shadowsocks-libev and all its dependencies by script:
mkdir -p ~/build-area/
cp ./scripts/build_deb.sh ~/build-area/
cd ~/build-area
./build_deb.sh
/etc/shadowsocks-libev/config.json, configure as follows:
5. Startup{ "server":"X.X.X.X", "server_port":443, "password":"password", "timeout":300, "method":"aes-256-cfb" }
Ubuntu/Debian Installed from deb package(Installed from deb package will implicitly activate self start-up):
service shadowsocks-libev restart@reference_1_github.com
@reference_2_github.com
Monday, September 4, 2017
Install Wine on Ubuntu 17.04
1. If you have disabled your system update options, enable them. Keep Ubuntu Up to date. Ubuntu automatically notifies when updates are available. (Important)
2. If your system is 64 bit, enable 32 bit architecture (if you haven't already):
If you have previously installed a Wine package from another repository, please remove it and any packages that depend on it (e.g., wine-mono, wine-gecko, winetricks) before attempting to install the WineHQ packages, as they may cause dependency conflicts.
3. Add the repository:
5.Then install one of the following packages:
If apt-get complains about missing dependencies, install them, then repeat the last two steps (update and install).
@reference_1_wiki.winehq.org
Ubuntu - Installing WineHQ packages
@reference_2_askubuntu.com
How do I resolve unmet dependencies after adding a PPA?
Other Advices:
1. Use aptitude instead of apt-get
recommend trying the following commands.
EDIT:
try the following command it should clean up your system.
3. Enable all the repositories
If the
Then make sure that the
Under
Do not forget to update with:
To get a list of repositories apt-get is checking, run the following command:
@reference_3_askubuntu.com
E: Unable to correct problems, you have held broken packages
@reference_4_askubuntu.com
sudo apt-get update
sudo apt-get upgrade
Update: Synchronizes your list of available packages with the servers in source repositories.
Upgrade: Downloads & installs any newer versions of your installed packages.2. If your system is 64 bit, enable 32 bit architecture (if you haven't already):
sudo dpkg --add-architecture i386
If you have previously installed a Wine package from another repository, please remove it and any packages that depend on it (e.g., wine-mono, wine-gecko, winetricks) before attempting to install the WineHQ packages, as they may cause dependency conflicts.
3. Add the repository:
wget https://dl.winehq.org/wine-builds/Release.key sudo apt-key add Release.key sudo apt-add-repository 'https://dl.winehq.org/wine-builds/ubuntu/'
4. Update packages:
sudo apt-get update
5.Then install one of the following packages:
| Stable branch | sudo apt-get install --install-recommends winehq-stable
|
|---|---|
| Development branch | sudo apt-get install --install-recommends winehq-devel
|
| Staging branch | sudo apt-get install --install-recommends winehq-staging
|
@reference_1_wiki.winehq.org
Ubuntu - Installing WineHQ packages
@reference_2_askubuntu.com
How do I resolve unmet dependencies after adding a PPA?
Other Advices:
1. Use aptitude instead of apt-get
Use aptitude instead of apt-get. It is more intelligent. It not
only will handle downgrading conflicting packages for you, but will make
a series of recommendations asking you which of many possible suggested
working scenarios you would like.
2. Clean up your system
sudo aptitude install myNewPackage
If you don't have aptitude on your machine yet, get it withsudo apt-get install aptitude
2. Clean up your system
sudo dpkg --configure -a
sudo apt-get install -f
Answer yes to any prompts and let apt-get try to resolve the issue for you.EDIT:
try the following command it should clean up your system.
sudo sh -c "apt-get update;apt-get dist-upgrade;apt-get autoremove;apt-get autoclean"
3. Enable all the repositories
If the
error shows something like this:<some-package>: Depends: <other-package> (= version) but this-version is to be installedThen make sure that the
restricted and universe repositories are enabled. Hit Alt+F2, type software-properties-gtk in search bar and hit Enter.Under
Ubuntu Software tab, enable all the repositories.
4. Remove or purge PPA
Use the
you can also disable or remove PPAs from the "Software Sources" section in Ubuntu Settings with a few clicks of your mouse (no terminal needed).
--remove flag, similar to how the PPA was added:sudo add-apt-repository --remove ppa:whatever/ppa
As a safer alternative, you can install ppa-purge:sudo apt-get install ppa-purge
And then remove the PPA, downgrading gracefully packages it provided to packages provided by official repositories:sudo ppa-purge ppa_name
Note that this will uninstall packages provided by the PPA, but not
those provided by the official repositories. If you want to remove them,
you should tell it to apt:sudo apt-get purge package_name
You can also remove PPAs by deleting the .list files from /etc/apt/sources.list.d directory.you can also disable or remove PPAs from the "Software Sources" section in Ubuntu Settings with a few clicks of your mouse (no terminal needed).
sudo apt-get update
apt-cache policy
5. Other solutions
- One possible cause of unmet dependencies could be corrupted
package database, and/or some packages weren’t installed properly. To
fix this problem, hit Alt+Ctrl+T to open terminal and try to run one of the following commands:
sudo apt-get clean
or,
sudo apt-get autoclean
apt-get cleanclears out the local repository of retrieved package files (the .deb files). It removes everything but the lock file from /var/cache/apt/archives/ and /var/cache/apt/archives/partial/.apt-get autocleanclears out the local repository of retrieved package files, but unlikeapt-get clean, it only removes package files that can no longer be downloaded, and are largely useless. - One of the most basic fixes to resolve dependencies problems is to run:
sudo apt-get -f install
The-fhere stands for “fix broken”.Aptwill attempt to correct broken dependencies. If you manually installed a package that had unmet dependencies,apt-getwill install those dependencies, if possible, otherwise it may simply remove the package that you installed in order to resolve the problem.
Then run:
sudo dpkg --configure -a
Then run this again:
sudo apt-get -f install
If the output is:
0 upgraded, 0 newly installed, 0 to remove and 1 not upgraded.
That means it failed. - Next solution is to run:
sudo apt-get -u dist-upgrade
If it shows any held packages, it is best to eliminate them. Packages are held because of dependency conflicts thataptcannot resolve. Try this command to find and repair the conflicts:
sudo apt-get -o Debug::pkgProblemResolver=yes dist-upgrade
If it cannot fix the conflicts, it will exit with:
0 upgraded, 0 newly installed, 0 to remove and 6 not upgraded.
Delete the held packages one by one, runningdist-upgradeeach time, until there are no more held packages. Then reinstall any needed packages. Be sure to use the--dry-runoption, so that you are fully informed of consequences:
sudo apt-get remove --dry-run package-name
Since removing the package you are trying to install may not be ideal, you might also try finding a repository that has the packages you need to satisfy the dependencies.
@reference_3_askubuntu.com
E: Unable to correct problems, you have held broken packages
@reference_4_askubuntu.com
How to fix dependencies / broken packages [duplicate]
@reference_6_askubuntu.com
How can PPAs be removed?
How to get a list of repositories apt-get is checking?
How iptables tables and chains are traversed
@reference_1_unix.stackexchange.com
How iptables tables and chains are traversed
Chapter 6. Traversing of tables and chains
Netfilter
Linux Firewall Tutorial: IPTables Tables, Chains, Rules Fundamentals
A Deep Dive into Iptables and Netfilter Architecture
Sunday, September 3, 2017
Iptables Chain Traversal Order
Netfilter Hooks
There are fivenetfilter hooks that programs can
register with. As packets progress through the stack, they will trigger
the kernel modules that have registered with these hooks. The hooks
that a packet will trigger depends on whether the packet is incoming or
outgoing, the packet's destination, and whether the packet was dropped
or rejected at a previous point.The following hooks represent various well-defined points in the networking stack:
NF_IP_PRE_ROUTING: This hook will be triggered by any incoming traffic very soon after entering the network stack. This hook is processed before any routing decisions have been made regarding where to send the packet.NF_IP_LOCAL_IN: This hook is triggered after an incoming packet has been routed if the packet is destined for the local system.NF_IP_FORWARD: This hook is triggered after an incoming packet has been routed if the packet is to be forwarded to another host.NF_IP_LOCAL_OUT: This hook is triggered by any locally created outbound traffic as soon it hits the network stack.NF_IP_POST_ROUTING: This hook is triggered by any outgoing or forwarded traffic after routing has taken place and just before being put out on the wire.
Assuming that the server knows how to route a packet and that the firewall rules permit its transmission, the following flows represent the paths that will be traversed in different situations:
- Incoming packets destined for the local system:
PREROUTING->INPUT - Incoming packets destined to another host:
PREROUTING->FORWARD->POSTROUTING - Locally generated packets:
OUTPUT->POSTROUTING
PREROUTING chains of the raw, mangle, and nat tables. It will then traverse the INPUT chains of the mangle, filter, security, and nat tables before finally being delivered to the local socket.@reference_1_digitalocean.com
A Deep Dive into Iptables and Netfilter Architecture
Subscribe to:
Posts (Atom)









