by Kulbir Saini on Administration, Hacks, Linux, Programming

Hack: A Fast Network Scanning Program

I was searching for a simple tool which can do a port scanning in a huge network quickly without making me wait for ages. I first thought of using nmap, but it was a bit too complex and it takes a lot of time to discover the machines even after optimizing the parameters. After searching a lot, I wrote to one of my seniors, Sandeep Kumar, asking the details of his program which maintains a list of active FTP servers in the network. He replied with a reference to his own findings about the network scanning tools. He is using an enhanced version of a program originally written by Troy Robinson. I tried the program out of curiosity and found out that its damn fast as compared to nmap (no literal comparison) 🙂 The program can be downloaded from here.

How to use

Compile the program using gcc as

[root@localhost ~]# gcc NetworkScanner.c [ENTER]

Now create a file IPRange.txt containing the IP address ranges for your network. The contents of the file may be

172.16.*.* Meaning all the IP address with first two parts as 172.16 and rest of the address will be generated by permutations.

172.16.1-16.* Meaning the first two parts are fixed. Third part will vary from 1 to 16. And the fourth part will be permuted from 0 to 255.

So an IPRange.txt may look like

1
2

172.16.1-16.*
192.168.36.*

Now run the program as

[root@localhost ~]# ./a.out port_to_be_scanned Parallel_attempts IP_list_file output.txt [ENTER]

Parallel_attempts is the number of processes that’ll be forked for scanning the network port. It is safe to have its value as 255. A very high value may hog the network or may even slow down your machine. So an example run would be

[root@localhost ~]# ./a.out 21 255 IPRange.txt Output.txt [ENTER]

Benchmarks

I carried out a lot of test on my network using the following setup and parameters

Machine : AMD X2 5600+ (2.6GHz Dual Core), 4GB 800MHz DDR2 RAM, Gigabit Ethernet Card (on 100mbps network).

Port : 21 (FTP)

IPRange.txt : Total 16896 IP Addresses

1
2
3
4
5

Machines on wired (100mbps) network
172.16.1-48.* 
192.168.36.*
Machines on wireless (54mbps) network
172.17.0-16.*

Network Scanner Benchmarks

Parallel Attempts

Scanning Time (seconds)

Upload Bandwidth (kbps)
255 180 13
512 90 25
1024 47 55
2048 25 100
4096 14 205
6144 11 307
8192 9 374

The interval between two scans was almost 30-40 seconds. I think parallelism beyond 8192 will crash my machine, so I didn’t try. You can try it at your own risk 🙂 I hope this program help you scan your network.

 
by Kulbir Saini on Configuration, GSOC, Installation, IntelligentMirror, Open Source, Proxy Server, Server, Squid

IntelligentMirror Gets Even More Intelligent (1.0.1)

Warning : This version of IntelligentMirror is compatible with only squid-2.7 as of now. It is NOT compatible even with squid-3.0.

IntelligentMirror Version 1.0.1

I have been following squid development regularly (at least the part in which I am interested) and they have introduced a new directive in squid-2.7 known as StoreUrlRewrite (storeurl_rewrite_program). Using this directive you can instruct squid to cache url A (http://abc.com/foo/bar/version/crap.rpm) as url B (http://proxy.fedora.co.in/intelligentmirror/crap.rpm). In simple words you can direct squid to cache any url as any other url without any extra efforts.

So keeping the above directive in mind, I have worked out a different version of intelligentmirror especially for squid-2.7.

IntelligentMirror : Old method of operation

  1. IntelligentMirror gets a client request for a URL.
  2. Check: if URL is not in (RPM, metadata file)
    • Then its none of our business.
    • Let proxy handle it the normal way.
    • Done and exit.
  3. Check: if RPM/metadata is available in cache
    • Stream the RPM/metadata from cache.
    • Done and exit.
  4. Check: if RPM/metadata is not available in cache
    • Download in parallel for caching in some dir and stream.
    • Done and exit.

IntelligentMirror : New method of operation

  1. IntelligentMirror gets a client request for a URL.
  2. Check: if request for rpm
    1. Direct squid to cache the request as http://<same_host_all_the_time>/intelligentmirror/<rpmname>.rpm
  3. Check: if request for deb
    1. Direct squid to cache the request as http://<same_host_all_the_time>/intelligentmirror/<debname>.deb
  4. Done and exit.

So your squid will see every request for an rpm package as a request http://<same_host_all_the_time>/intelligentmirror/<rpmname>.rpm. So, if you happen to request the same rpm from a different mirror, it’ll still be served from cache 🙂

Improvements

  1. No need to check if the url supplied by squid is for rpm or not because storeurl_rewrite_program has an acl controller attached which will invoke intelligentmirror for urls ending in .rpm .
  2. No need to check if the url is already cached or not. No need to worry about the directory where you are going to store the packages. No human intervention is needed in maintaining the cache. Almighty squid is doing everything for us.
  3. No need to worry if the target package has changed because of the resigning or whatever because squid will do that for you.
  4. No need to actually download the package in parallel for caching because squid is already doing that.
  5. No need to worry about the hashing algorithms and storage optimizations for the cached content.

Availability

  1. RPM for Fedora/Red Hat
  2. Source RPM for Fedora/Red Hat
  3. Source Tarball

Install and Configure

The install and configure files should be enough to guide you through the installation if you choose the tar ball way. Otherwise you can always install from rpm from the above link.

Note1: You have to configure your squid to use intelligentmirror as a plugin even if you install via rpm. Check the configure file at the above link.

Note2: StoreUrlRewrite will probably be available in squid-3.1.

 
by Kulbir Saini on Configuration, GSOC, Installation, IntelligentMirror, Open Source, Proxy Server, Server, Squid

IntelligentMirror: RPM and DEB Caching Improved (0.4)

IntelligentMirror version 0.4 is available now. There have been significant improvements in intelligent mirror since last release.

Improvements

  1. Fixed defunct process problem. You will not see defunct python processes hanging around anymore. Previously every forked daemon used to got defucnt because parent never waited for the forked child to finish.
  2. IntelligentMirror now supports caching of Debian packages just like rpms. So now IntelligentMirror is best suited shared environments where people have different tastes.
  3. Intelligent Mirror now uses url_rewrite_program instead of redirect_program. This boosts the efficiency of IntelligentMirror by a significant factor as url_rewrite_program has an acl controller url_rewrite_access. And using url_rewrite_access only requests for rpm/deb packages will be passed to Intelligent Mirror. So, IM now need not process each and every incoming request. Also, it has redirector_bypass directive which will bypass IM in case all the instances of IM are busy serving requests. So, squid will not die with a fatal error in case of huge requests.
  4. Options to enable/disable caching for rpm and Debian packages have been added.
  5. Options to control the total size of caching directories and the size of individual package to be cached have also been introduced.
  6. Proxy authentication is also supported now just the way it is supported in yum.
  7. Packages are not checked for last-modified time anymore. Because in principle two rpms A and B can only have same name iff they have the same contents. So, the delay in response time in case of hits has reduced.

Availability

  1. RPMs for Fedora/Red Hat
  2. Source RPMs for Fedora/Red Hat
  3. Source Tar balls

Installation and configuration is easy and the INSTALL and README files should serve the purpose.

In case you have any suggestions or problems, leave a comment here or file a ticket on project page.

 
by Kulbir Saini on Adobe, Hardware, Laptop, Linux, News

Humour: Ubuntu is Costlier than Microsoft Windows XP

Dell started selling their new Dell Inspiron Mini 9 series yesterday as informed by slashdot article. Lately I have been looking for a mobile solution which should be easy to carry and have enough computing power to meet my expectation. I liked the SSD concept replacing the mechanical hard disk. I just moved over to Dell website and tried configuring one for myself (go here to configure them yourself). I am an open source addict but I am not an Ubuntu fan but still going with ubuntu sounds better than going with M$ stuff. So, started configuring the Ubuntu machine and pushed everything (main memory, hard disk and other components) to maximum. A few minutes later I realized that the Ubuntu model’s price is more than the equivalent config with M$ Windows.

Below is a table comparing Windows XP and Ubuntu model of Dell Inspiron Mini 9 series. The differences are highlighted in red color.

Ubuntu 8.04
Microsoft Windows XP Home Premium
$494
$479
My Components
Obsidian Black
Obsidian Black
Mini OS powered by Ubuntu 8.04
Genuine Windows® XP Home Edition
1GB DDR2 at 533MHz
1GB DDR2 at 533MHz
16GB Solid State Drive
16GB Solid State Drive
Built-in Bluetooth 2.1 capability
Built-in Bluetooth 2.1 capability
Integrated 1.3M Pixel Webcam
Integrated 1.3M Pixel Webcam
My Service
1Yr Ltd Warranty and Mail-In Service
1Yr Ltd Warranty and Mail-In Service
Also Includes
Intel® Atom Processor® N270 (1.6GHz/533Mhz FSB/512K cache) Intel® Atom Processor® N270 (1.6GHz/533Mhz FSB/512K cache)
Glossy 8.9 inch LED display (1024X600)
Glossy 8.9 inch LED display (1024X600)
Intel Graphics Media Accelerator (GMA) 950
Intel Graphics Media Accelerator (GMA) 950
Wireless 802.11g Mini Card
Wireless 802.11g Mini Card
32WHr Battery (4 cell)
32WHr Battery (4 cell)
Base LCD Assembly
Base LCD Assembly
Base LCD Assembly in Black
Base LCD Assembly in Black
No Installation
No Installation
Adobe® Acrobat® Reader 8.1, English
McAfee SecurityCenter 30-day Trial
Microsoft Works 9 DOES NOT INCLUDE MS WORD
No ISP requested

According to the above table Ubuntu alone is costlier than Genuine Microsoft Windows XP Home Premium + Adobe Acrobat Reader 8.1 English + McAfee SecurityCenter 30-day Trial + Microsoft Works 9 (MS Word Excluded). I am not sure if Dell is trying to promote Microsoft by selling Windows XP model at a lower price or they are generating more revenues for Ubuntu by selling it at a higher price or they are just fooling the customers. Below are the picture proofs if Dell changes the prices.

Inspiron Ubuntu

Inspiron Windows

 
by Kulbir Saini on Fedora, Grub, Hacks, HowTo, Installation, Kernel, Tips - Tricks

How To: Install Fedora without CD or DVD

Note: If you are new to Fedora/Linux, I highly recommend the book “Fedora Linux Toolbox”

[amazon-product alink=”0000FF” bordercolor=”000000″ height=”240″]0470082917[/amazon-product]

Use Case

  1. When you don’t have CD / DVD drive on your system.
  2. You have Fedora DVD but your system has only a CD Drive.
  3. You don’t want to waste time and resources in burning iso on optical media.

Pre-requisites

  1. You have a Fedora DVD iso or rescue cd iso.
  2. You have a Linux installation on your system.
  3. You have a partition (FAT32, ext2, ext3) which you will not format while installing the new OS.

How to proceed

Let us assume you want to install Fedora 9 on your system and you have a Linux distro already installed on your system. You have downloaded the Fedora DVD iso (Fedora-9-DVD-i686.iso). And you have a FAT32/ext2/ext3 partition /stuff/ which you will not format during installation.

Step 1 : Move the Fedora DVD iso to /stuff/ directory.

[root@saini saini]# mv Fedora-9-DVD-i686.iso /stuff/ [Enter]

Step 2 : Mount Fedora DVD iso on /mnt/

[root@saini saini]# mount /stuff/Fedora-9-DVD-i686.iso /mnt/ -ro loop [Enter] (do as root)

Step 3 : Copy the initrd.img and vmlinuz to /boot/ partition

[root@saini saini]# cd /mnt/isolinux/ [Enter]
[root@saini isolinux]# cp initrd.img vmlinuz /boot/ [Enter] (do as root)

Step 4 : Create grub entry for booting into Fedora 9

Add these lines at the end of your /boot/grub/grub.conf file.

title Fedora 9 (New installation)
    kernel /vmlinuz
    initrd /initrd.img

Step 5 : Note the device having Fedora DVD iso

[root@saini saini]# df -h [Enter]
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda3              15G  9.5G  4.1G  70% /
/dev/sda8             135G  116G   13G  91% /stuff
/dev/sda5             4.8G  1.2G  3.4G  26% /home
/dev/sda1              99M   12M   82M  13% /boot

In this case /dev/sda8 contains Fedora DVD iso. Note this down as you need it later.

Step 6 : Reboot

Reboot your system and boot into the Fedora 9 (New installation) grub entry.

Step 7 : Install from hard disk

While in installation wizard, select “Hard drive” as installation method and choose /dev/sda8 as it contains the Fedora DVD iso. And rest is damn easy.

Want MB5-292 ? Check out our latest MB7-226 study guides and 70-271 give you 100% exam pass guarantee of 70-158. You can also get MB6-285 from this website.