read() method return -1 when reach the End of Stream

read()

The read() method of an InputStream returns an int which contains the byte value of the byte read. Here is an InputStream read() example:

int data = inputstream.read();

You can case the returned int to a char like this:

char aChar = (char) data;

Subclasses of InputStream may have alternative read() methods. For instance, the DataInputStream allows you to read Java primitives like int, long, float, double, boolean etc. with its corresponding methods readBoolean(), readDouble() etc.

End of Stream

If the read() method returns -1, the end of stream has been reached, meaning there is no more data to read in the InputStream. That is, -1 as int value, not -1 as byte or short value. There is a difference here!
When the end of stream has been reached, you can close the InputStream.

@reference_1_tutorials.jenkov.com
Java IO: InputStream

Reading from a Socket

To read from a Java Socket you will need to obtains its InputStream. Here is how that is done:

Socket socket = new Socket("jenkov.com", 80);
InputStream in = socket.getInputStream();

int data = in.read();
//... read more data...

in.close();
socket.close();

Pretty simple, right?
Keep in mind that you cannot always just read from the Socket's InputStream until it returns -1, as you can when reading a file. The reason is that -1 is only returned when the server closes the connection. But a server may not always close the connection. Perhaps you want to send multiple requests over the same connection. In that case it would be pretty stupid to close the connection.
Instead you must know how many bytes to read from the Socket's InputStream. This can be done by either the server telling how many bytes it is sending, or by looking for a special end-of-data character.

@reference_2_tutorials.jenkov.com
Java Networking: Socket

Set no password for `su` in Ubuntu 17.04

1. Create new group: groupadd wheel
2. Add your user name: usermod -G wheel your_user_name
3. gedit /etc/pam.d/su

# Uncomment this if you want wheel members to be able to
# su without a password.
auth       sufficient pam_wheel.so trust

Set Java environment variables for all users --- Difference between `su` & `su -`

1.  gedit /etc/profile
2.  add:  export JAVA_HOME=/home/birdonwire/Develop/jdk1.8.0_144/
           export PATH=$JAVA_HOME/bin:$PATH
3.  restart or reopen shell or use source /etc/profile to apply changes immediately

Difference between `su` & `su -`:

Using `su` won't load /etc/profile

birdonwire@win-u5n0t6:~$ su
Password:
root@win-u5n0t6:/home/birdonwire# $PATH
bash: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games: No such file or directory
root@win-u5n0t6:/home/birdonwire#

However `su -` will load /etc/profile

birdonwire@win-u5n0t6:~$ su -
Password:
root@win-u5n0t6:~# $PATH
-su: /home/birdonwire/Bash/com:/home/birdonwire/Develop/jdk1.8.0_144/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin: No such file or directory
root@win-u5n0t6:~#


Q:   Then how to set persistent environment variables for root or all users?
A:   To create persistent variables that are truly system-wide, you should set them both in /etc/environment and /etc/profile .

If you only set environment variables in /etc/environment, `su - <root or other users>` won't  load /etc/environment, but will load /etc/profile , although `su` will load /etc/environment, won't load /etc/profile . Therefore, you have to set environment variables both in /etc/environment and /etc/profile to make sure all users are covered.


@reference_1_stackoverflow.com
How to set JAVA_HOME in Linux for all users
@reference_2_superuser.com

Adjusting $PATH in /etc/profile does not affect root

@reference_3_askubuntu.com 

How do I set persistent environment variables for root?

About using iptables owner match to block a specific application

iptables -A OUTPUT -m owner --gid-owner black -j REJECT

User name (uid) or group name (gid) won't help because these are the IDs of the user executing the application. Thus it would apply to all applications by that particular user and not just a single one.
Blocking a specific application is usually done by blocking all ports the application uses. But this only works if the user can't change the ports the application will use.

@reference_1_unix.stackexchange.com
Block specific application with iptables


Then how to establish a white-list mode using iptables owner match?

#Allow Loopback Connections
iptables -A INPUT -i lo -j ACCEPT
iptables -A OUTPUT -o lo -j ACCEPT

iptables -A OUTPUT -m owner --uid-owner current_login_user -j REJECT
iptables -A OUTPUT -m owner --uid-owner root -j REJECT
iptables -A OUTPUT -m owner --uid-owner other_users -j REJECT

1. block all the users using above commands
2. create a white-list user allowed to use network
3. start your program using the white-list user under your current login user

root privilege & execute permission

sudo chmod 000 xxx
execution of the script(or program) xxx under the root account will be denied.

sudo chmod 100 xxx
sudo chmod 001 xxx
sudo chmod 010 xxx
either command above will make the execution of the script(or program) xxx under the root account accepted.

Difference between `sh xxx.sh` & `./xxx.sh` commands in ubuntu

`sh xxx.sh` requires +r Permission
`./xxx.sh` requires +rx Permission


@reference_1_forum.ubuntu.org.cn
./和sh的区别

Install nmap on Ubuntu

1. Download
Back on your auditing machine, move into your home directory and use wget to download the link you pasted. Make sure to update the link below to reflect the most recent version you copied from the site:

cd ~

wget https://nmap.org/dist/nmap-7.60.tar.bz2

2. Decompress the file you downloaded and move into the resulting directory by typing:

tar xjvf nmap*

cd nmap*

3. Configure and compile the source code by typing:

./configure

make

4. Once the compilation is complete, you can install the resulting executables and supporting files on your system by typing:

sudo make install

5. Confirm your installation by typing:

nmap -V

The output should match the version you downloaded from the nmap website.

@reference_1_digitalocean.com
How To Test your Firewall Configuration with Nmap and Tcpdump
@reference_2_nmap.org
Source Code Distribution (in case you wish to compile Nmap yourself)

Install Wireshark on Ubuntu

1. Adding this PPA to your system

sudo add-apt-repository ppa:wireshark-dev/stable
sudo apt-get update

2. Install Wireshark

sudo apt-get install wireshark

3. Allow non-root user to be able to sniff

sudo dpkg-reconfigure wireshark-common

select Yes and hit return. This adds a wireshark group. Anybody in that group will be able to sniff without being root.

4. Add user to wireshark group

sudo gedit /etc/group

Add your username to the wireshark group.

@reference_1_launchpad.net
Wireshark stable releases
@reference_2_askubuntu.com

How do I run wireshark, with root-privileges?

@reference_3_linuxidc.com
Ubuntu 16.04下安装网络流量分析工具 Wireshark

Install shadowsocks-libev from source on Ubuntu

1. Install the packages required：

mkdir shadowsocks-libev && cd shadowsocks-libev
apt-get install build-essential autoconf libtool libssl-dev \
  gawk debhelper dh-systemd init-system-helpers pkg-config git

2. Download the source code through git：
git clone https://github.com/shadowsocks/shadowsocks-libev.git
3. build shadowsocks-libev and all its dependencies by script:

mkdir -p ~/build-area/
cp ./scripts/build_deb.sh ~/build-area/
cd ~/build-area
./build_deb.sh

4.Edit /etc/shadowsocks-libev/config.json， configure as follows：

{
 "server":"X.X.X.X",
 "server_port":443,
 "password":"password",
 "timeout":300,
 "method":"aes-256-cfb"
}

5. Startup

Ubuntu/Debian Installed from deb package（Installed from deb package will implicitly activate self start-up）：
service shadowsocks-libev restart

@reference_1_github.com
@reference_2_github.com

Install Wine on Ubuntu 17.04

1. If you have disabled your system update options, enable them. Keep Ubuntu Up to date. Ubuntu automatically notifies when updates are available. (Important)

sudo apt-get update
sudo apt-get upgrade

Update: Synchronizes your list of available packages with the servers in source repositories. Upgrade: Downloads & installs any newer versions of your installed packages.

2. If your system is 64 bit, enable 32 bit architecture (if you haven't already):

sudo dpkg --add-architecture i386 

If you have previously installed a Wine package from another repository, please remove it and any packages that depend on it (e.g., wine-mono, wine-gecko, winetricks) before attempting to install the WineHQ packages, as they may cause dependency conflicts.

3.  Add the repository:

wget https://dl.winehq.org/wine-builds/Release.key
sudo apt-key add Release.key
sudo apt-add-repository 'https://dl.winehq.org/wine-builds/ubuntu/'

4. Update packages:

sudo apt-get update 

5.Then install one of the following packages:

Stable branch	sudo apt-get install --install-recommends winehq-stable
Development branch	sudo apt-get install --install-recommends winehq-devel
Staging branch	sudo apt-get install --install-recommends winehq-staging

If apt-get complains about missing dependencies, install them, then repeat the last two steps (update and install).

@reference_1_wiki.winehq.org
Ubuntu - Installing WineHQ packages
@reference_2_askubuntu.com
How do I resolve unmet dependencies after adding a PPA?


Other Advices:

1. Use aptitude instead of apt-get

Use aptitude instead of apt-get. It is more intelligent. It not only will handle downgrading conflicting packages for you, but will make a series of recommendations asking you which of many possible suggested working scenarios you would like.

sudo aptitude install myNewPackage

If you don't have aptitude on your machine yet, get it with

sudo apt-get install aptitude

2. Clean up your system

recommend trying the following commands.

sudo dpkg --configure -a
sudo apt-get install -f

Answer yes to any prompts and let apt-get try to resolve the issue for you.
EDIT:
try the following command it should clean up your system.

sudo sh -c "apt-get update;apt-get dist-upgrade;apt-get autoremove;apt-get autoclean"

3. Enable all the repositories

If the error shows something like this:
<some-package>: Depends: <other-package> (= version) but this-version is to be installed
Then make sure that the restricted and universe repositories are enabled. Hit Alt+F2, type software-properties-gtk in search bar and hit Enter.
Under Ubuntu Software tab, enable all the repositories.

 

 4. Remove or purge PPA

Use the --remove flag, similar to how the PPA was added:

sudo add-apt-repository --remove ppa:whatever/ppa

As a safer alternative, you can install ppa-purge:

sudo apt-get install ppa-purge

And then remove the PPA, downgrading gracefully packages it provided to packages provided by official repositories:

sudo ppa-purge ppa_name

Note that this will uninstall packages provided by the PPA, but not those provided by the official repositories. If you want to remove them, you should tell it to apt:

sudo apt-get purge package_name

You can also remove PPAs by deleting the .list files from /etc/apt/sources.list.d directory.

you can also disable or remove PPAs from the "Software Sources" section in Ubuntu Settings with a few clicks of your mouse (no terminal needed).

Do not forget to update with:

sudo apt-get update

To get a list of repositories apt-get is checking, run the following command:

apt-cache policy

5. Other solutions

• One possible cause of unmet dependencies could be corrupted package database, and/or some packages weren’t installed properly. To fix this problem, hit Alt+Ctrl+T to open terminal and try to run one of the following commands:
  

  sudo apt-get clean

  or,
  

  sudo apt-get autoclean

  apt-get clean clears out the local repository of retrieved package files (the .deb files). It removes everything but the lock file from /var/cache/apt/archives/ and /var/cache/apt/archives/partial/. apt-get autoclean clears out the local repository of retrieved package files, but unlike apt-get clean, it only removes package files that can no longer be downloaded, and are largely useless.
• One of the most basic fixes to resolve dependencies problems is to run:
  

  sudo apt-get -f install

  The -f here stands for “fix broken”. Apt will attempt to correct broken dependencies. If you manually installed a package that had unmet dependencies, apt-get will install those dependencies, if possible, otherwise it may simply remove the package that you installed in order to resolve the problem.
  Then run:
  

  sudo dpkg --configure -a

  Then run this again:
  

  sudo apt-get -f install

  If the output is:
  0 upgraded, 0 newly installed, 0 to remove and 1 not upgraded.
  That means it failed.
• Next solution is to run:
  

  sudo apt-get -u dist-upgrade

  If it shows any held packages, it is best to eliminate them. Packages are held because of dependency conflicts that apt cannot resolve. Try this command to find and repair the conflicts:
  

  sudo apt-get -o Debug::pkgProblemResolver=yes dist-upgrade

  If it cannot fix the conflicts, it will exit with:
  0 upgraded, 0 newly installed, 0 to remove and 6 not upgraded.
  Delete the held packages one by one, running dist-upgrade each time, until there are no more held packages. Then reinstall any needed packages. Be sure to use the --dry-run option, so that you are fully informed of consequences:
  

  sudo apt-get remove --dry-run package-name

  Since removing the package you are trying to install may not be ideal, you might also try finding a repository that has the packages you need to satisfy the dependencies.

Finally, if all else fails, you can attempt to satisfy the dependencies yourself, either by finding and installing the necessary packages, or by installing them from source and then creating “deb” packages for them.

@reference_3_askubuntu.com
E: Unable to correct problems, you have held broken packages
@reference_4_askubuntu.com

How to fix dependencies / broken packages [duplicate]

@reference_5_askubuntu.com

How do I resolve unmet dependencies after adding a PPA?

@reference_6_askubuntu.com

How can PPAs be removed?

@reference_7_stackoverflow.com 

How to get a list of repositories apt-get is checking?

How iptables tables and chains are traversed

@reference_1_unix.stackexchange.com

How iptables tables and chains are traversed

@reference_2_iptables.info 

Chapter 6. Traversing of tables and chains

@reference_3_wikipedia.org 

Netfilter

@reference_4_thegeekstuff.com 

Linux Firewall Tutorial: IPTables Tables, Chains, Rules Fundamentals

@reference_5_digitalocean.com 

A Deep Dive into Iptables and Netfilter Architecture

Iptables Chain Traversal Order

Netfilter Hooks

There are five netfilter hooks that programs can register with. As packets progress through the stack, they will trigger the kernel modules that have registered with these hooks. The hooks that a packet will trigger depends on whether the packet is incoming or outgoing, the packet's destination, and whether the packet was dropped or rejected at a previous point.
The following hooks represent various well-defined points in the networking stack:

• NF_IP_PRE_ROUTING: This hook will be triggered by any incoming traffic very soon after entering the network stack. This hook is processed before any routing decisions have been made regarding where to send the packet.
• NF_IP_LOCAL_IN: This hook is triggered after an incoming packet has been routed if the packet is destined for the local system.
• NF_IP_FORWARD: This hook is triggered after an incoming packet has been routed if the packet is to be forwarded to another host.
• NF_IP_LOCAL_OUT: This hook is triggered by any locally created outbound traffic as soon it hits the network stack.
• NF_IP_POST_ROUTING: This hook is triggered by any outgoing or forwarded traffic after routing has taken place and just before being put out on the wire.

Assuming that the server knows how to route a packet and that the firewall rules permit its transmission, the following flows represent the paths that will be traversed in different situations:

• Incoming packets destined for the local system: PREROUTING -> INPUT
• Incoming packets destined to another host: PREROUTING -> FORWARD -> POSTROUTING
• Locally generated packets: OUTPUT -> POSTROUTING

If we combine the above information with the ordering laid out in the previous table, we can see that an incoming packet destined for the local system will first be evaluated against the PREROUTING chains of the raw, mangle, and nat tables. It will then traverse the INPUT chains of the mangle, filter, security, and nat tables before finally being delivered to the local socket.

@reference_1_digitalocean.com
A Deep Dive into Iptables and Netfilter Architecture

Iptables Commands Examples

#Allow Loopback Connections
iptables -A INPUT -i lo -j ACCEPT
iptables -A OUTPUT -o lo -j ACCEPT

#Allow Established and Related Incoming Connections
iptables -A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
#Allow Established Outgoing Connections
iptables -A OUTPUT -m conntrack --ctstate ESTABLISHED -j ACCEPT

#Internal to External
#iptables -A FORWARD -i eth1 -o eth0 -j ACCEPT

#Drop Invalid Packets
iptables -A INPUT -m conntrack --ctstate INVALID -j DROP

#Block an IP Address
#iptables -A INPUT -s 15.15.15.51 -j DROP
#iptables -A INPUT -s 15.15.15.51 -j REJECT
#Block Connections to a Network Interface
#iptables -A INPUT -i eth0 -s 15.15.15.51 -j DROP

#Allow All Incoming SSH
iptables -A INPUT -p tcp --dport 22 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
#iptables -A OUTPUT -p tcp --dport 22 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
iptables -A OUTPUT -p tcp --sport 22 -m conntrack --ctstate ESTABLISHED -j ACCEPT

#Allow Incoming SSH from Specific IP address or subnet
iptables -A INPUT -p tcp -s 112.11.0.0/16 --dport 22 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
iptables -A INPUT -p tcp -s 172.104.167.8 --dport 22 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
iptables -A INPUT -p tcp -s 139.162.68.174 --dport 22 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
iptables -A INPUT -p tcp -s 23.92.25.19 --dport 22 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
iptables -A OUTPUT -p tcp --sport 22 -m conntrack --ctstate ESTABLISHED -j ACCEPT

#Allow Outgoing SSH
iptables -A OUTPUT -p tcp --dport 22 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
iptables -A INPUT -p tcp --sport 22 -m conntrack --ctstate ESTABLISHED -j ACCEPT

#Allow Incoming Rsync from Specific IP Address or Subnet
#iptables -A INPUT -p tcp -s 15.15.15.0/24 --dport 873 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
#iptables -A OUTPUT -p tcp --sport 873 -m conntrack --ctstate ESTABLISHED -j ACCEPT

#Allow All Incoming HTTP
iptables -A INPUT -p tcp --dport 80 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
iptables -A OUTPUT -p tcp --sport 80 -m conntrack --ctstate ESTABLISHED -j ACCEPT

#Allow All Incoming HTTPS
iptables -A INPUT -p tcp --dport 443 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
iptables -A OUTPUT -p tcp --sport 443 -m conntrack --ctstate ESTABLISHED -j ACCEPT

#Allow All Incoming HTTP and HTTPS
iptables -A INPUT -p tcp -m multiport --dports 80,443 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
iptables -A OUTPUT -p tcp -m multiport --dports 80,443 -m conntrack --ctstate ESTABLISHED -j ACCEPT

#Allow MySQL from Specific IP Address or Subnet
#iptables -A INPUT -p tcp -s 15.15.15.0/24 --dport 3306 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
#iptables -A OUTPUT -p tcp --sport 3306 -m conntrack --ctstate ESTABLISHED -j ACCEPT

#Allow MySQL to Specific Network Interface
#iptables -A INPUT -i eth1 -p tcp --dport 3306 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
#iptables -A OUTPUT -o eth1 -p tcp --sport 3306 -m conntrack --ctstate ESTABLISHED -j ACCEPT

#PostgreSQL from Specific IP Address or Subnet
#iptables -A INPUT -p tcp -s 15.15.15.0/24 --dport 5432 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
#iptables -A OUTPUT -p tcp --sport 5432 -m conntrack --ctstate ESTABLISHED -j ACCEPT

#Allow PostgreSQL to Specific Network Interface
#iptables -A INPUT -i eth1 -p tcp --dport 5432 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
#iptables -A OUTPUT -o eth1 -p tcp --sport 5432 -m conntrack --ctstate ESTABLISHED -j ACCEPT

#Block Outgoing SMTP Mail
iptables -A OUTPUT -p tcp --dport 25 -j REJECT

#Allow All Incoming SMTP
#iptables -A INPUT -p tcp --dport 25 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
#iptables -A OUTPUT -p tcp --sport 25 -m conntrack --ctstate ESTABLISHED -j ACCEPT

#Allow All Incoming IMAP
#iptables -A INPUT -p tcp --dport 143 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
#iptables -A OUTPUT -p tcp --sport 143 -m conntrack --ctstate ESTABLISHED -j ACCEPT

#Allow All Incoming IMAPS
#iptables -A INPUT -p tcp --dport 993 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
#iptables -A OUTPUT -p tcp --sport 993 -m conntrack --ctstate ESTABLISHED -j ACCEPT

#Allow All Incoming POP3
#iptables -A INPUT -p tcp --dport 110 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
#iptables -A OUTPUT -p tcp --sport 110 -m conntrack --ctstate ESTABLISHED -j ACCEPT

#Allow All Incoming POP3S
#iptables -A INPUT -p tcp --dport 995 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
#iptables -A OUTPUT -p tcp --sport 995 -m conntrack --ctstate ESTABLISHED -j ACCEPT

#iptables -A INPUT -p tcp --dport 22 -j ACCEPT
#iptables -A INPUT -p tcp --dport 8443 -j ACCEPT
#iptables -I INPUT 1 -i lo -j ACCEPT

#Allow DNS
iptables -A OUTPUT -p udp --dport 53 -j ACCEPT
iptables -A INPUT -p udp --sport 53 -j ACCEPT

#Default rules
iptables -A INPUT -j DROP
iptables -A OUTPUT -j DROP

##Source:
@reference_1_digitalocean.com
Iptables Essentials: Common Firewall Rules and Commands

@reference_2_digitalocean.com
How To Set Up a Firewall Using Iptables on Ubuntu 14.04

@reference_3_digitalocean.com
How the Iptables Firewall Works

Activate TCP BBR on Linode VPS

Linux kernel 4.9+ required to support "tcp_bbr"

If your kernel doesn't support  "tcp_bbr", update it.

1. Download the latest Linux kernel, for example:

`wget http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.10.17/

linux-image-4.10.17-041017-generic_4.10.17-041017.201705201051_amd64.deb`

2. Install the kernel:

`dpkg -i linux-image-4.*.deb`

3. Delete old kernel:

`dpkg -l | grep linux-image `
`apt-get purge (old kernel)`

4. Update grub & reboot

`update-grub`
`reboot`

5. Edit Configuration Profile --> Kernel --> GRUB 2 in Linode Manager Console, then reboot

Turn on BBR

1. execute command `uname -r` to see if the kernel version >= 4.9

2. execute command `lsmod | grep bbr`，if there is no `tcp_bbr` in the result, execute the following commands first:

`modprobe tcp_bbr`
`echo "tcp_bbr" >> /etc/modules-load.d/modules.conf` 

3. Execute:

`echo "net.core.default_qdisc=fq" >> /etc/sysctl.conf`
`echo "net.ipv4.tcp_congestion_control=bbr" >> /etc/sysctl.conf`

4. Save:

`sysctl -p`

5. Execute:

`sysctl net.ipv4.tcp_available_congestion_control`
`sysctl net.ipv4.tcp_congestion_control`

If there is `tcp_bbr` in the result, it means that your kernel has turned BBR on. The BBR has been activated when you see the `tcp_bbr`.

@reference_1_github.com
开启TCP BBR拥塞控制算法
@reference_2_orchidflower.oschina.io
在Linode节点上开启BBR算法

update:
http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.13.5/linux-image-
4.13.5-041305-generic_4.13.5-041305.201710050600_amd64.deb