Monday, November 5, 2012

Simulating network packet loss

Recently i had to simulate packet loss to test an application
At first i tried netem; however it errored out

[root@prithvivm3 ~]# /sbin/tc qdisc change dev eth0 root netem loss 0.1%
RTNETLINK answers: No such file or directory
i tried very hard, but couldnt fix this so tried packet loss using iptable
first i had to upgrade my current iptable

--> to install the iptables use the below command
# rpm -Uvh http://ftp.srce.hr/redhat/extras/el5/x86_64/iptables-1.3.5-5.3.el5_4.3.x86_64.rpm http://ftp.srce.hr/redhat/extras/el5/x86_64/iptables-ipv6-1.3.5-5.3.el5_4.3.x86_64.rpm

--> clear all rules of iptables
# iptables --flush
--> run the below command to view all rules in iptables. the below should not return any rules
#/sbin/iptables -vnL INPUT --line-numbers

--> now use the below to add a new rule. here we are telling IPTables to drop 1 packet for every 10 packets statistically random
The above returns all rules configured. as we said flush before this it will return a single line. now run the below command to add a rule
# iptables -A INPUT -m statistic --mode random --probability 0.1 -j DROP

--> save the above rule to ip tables file
# /etc/init.d/iptables save
--> view all rules of iptables
#/sbin/iptables -vnL INPUT --line-numbers

--> it should be visible as below
Chain INPUT (policy ACCEPT 5328 packets, 673K bytes)
num   pkts bytes target     prot opt in     out     source               destination
1     3510  480K DROP       all  --  *      *       0.0.0.0/0            0.0.0.0/0           statistic mode random probability 0.100000

--> so we added our rule successfully

--> now restart the iproute table for the above rules to take affect
# /etc/init.d/iptables restart



to monitor the packets run the below command
watch 'iptables -vL'

the output will be as follows
Every 2.0s: iptables -vL  Mon Nov  5 14:17:13 2012

Chain INPUT (policy ACCEPT 5680 packets, 702K bytes)
 pkts bytes target     prot opt in     out     source               destination
 3726  496K DROP       all  --  any    any     anywhere             anywhere            statistic mode random probability 0.400000

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain OUTPUT (policy ACCEPT 7336 packets, 593K bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain LOGGING (0 references)
 pkts bytes target     prot opt in     out     source               destination

Chain RH-Firewall-1-INPUT (0 references)
 pkts bytes target     prot opt in     out     source               destination



After having done the above .. i ran my client that tries to communicate to server on plain sockets.

Learning:
Initially i was expecting some issues... 'cos we are dropping packets that are going to the socket. So on server side i was expecting server to read some wrong data/ throw some array out of bound exception. however no such thing had happend. I am trying to understand it.

So... In transport layer two things happen
  1. The packets are checked for corruption; by CRC- cyclic redundancy check of the packet data
  2. packet sequencing; 
if any of the above is not valid/ out of sequence tcp/ip layer takes corrective action i.e.. ignore the packets and then ask for a resend. My initial belief was the packets get dropped (because of iptables rules is) after the above check is done. But when running the experiment using VI editor; or when my local windows java program was sending bytes tothe linux server.. i never noticed server misbehaving. Atmost i could see was only delay in communication; as such server was never getting incorrect data/ garbled data from the socket.

so by this I think...
  1. the CRC check is first done (at hardware lever in ether net card)
  2. then the packets are sent to the OS layer
  3. before which (i.e.. before the point 2 above) the packets are intercepted and acted upon
  4. Now the packet drop(by iptables rules) happens (ie.. in point 3 above) and then it reaches the OS layer
  5. OS layer now checks for sequencing and because packet drop (of point 4) has happened asks the tcpip to re-fetch the full packet.
Given this understanding.. i m wondering what will happen if in iptables we asked to drop all packets fro a given ip; will...
  1. OS keep indefinitely asking for resend?
  2. or as OS is totally unaware of the packets it would just ignore.
i was trying to get an evidence to the above points; however couldnt... so incase u find any please let me know the same...






1 comment:

  1. try /sbin/tc qdisc add dev eth0 root netem loss 0.1%

    ReplyDelete