Internet

by Kulbir Saini on Administration, Configuration, HowTo, Internet, Proxy Server, Server, Squid

How To: Configure Hierarchicy of Proxy Servers (Squid)

Yesterday I came across this idea of caching all the data that I browse on my hard disk so that the average load time of a website decreases. Actually the idea is I’ll cache all the static data that I browse like images, static html pages, CSS files and similar things which does not change frequently and can be served from the cache. But while setting up the proxy server on my machine, I faced the problem that my machine which is going to act as a proxy server is behind my institute’s proxy. So, a simple caching proxy server can’t serve my needs and I have to really figure out how to setup a hierarchical proxy server. Below we’ll see how to setup a hierarchical proxy server.

Approach

When I thought of setting up a caching proxy server, squid immediately struck my mind. Actually I don’t know about any other proxy servers. I never setup proxy server before this ( I tried a lot of time, but in vain). So, I started googling about squid setup. There were a lot of tutorials, but either they were too small to get things going or they were too verbose that I couldn’t manage to read them. So, I directly jump into squid configuration file squid.conf . And with references from here and there, I managed to setup the proxy server successfully.

Note: The configurations below worked on Fedora 7 with squid 2.6STABLE16. The same configurations may work with other squid versions and on other operating systems as well, but try them at your own risk.

Part 1 : Setting up simple proxy server with squid

Setting up a very simple and usable proxy server is really easy. You need to add/edit only 2-3 lines /etc/squid/squid.conf to get started.

Add your ip to the access list.

1
2
3

acl myip src 172.17.8.175 #<your_ip_which_will_use_the_proxy_server> (e.g. )
http_access allow myip
http_port 8080 #<http_proxy_port> (this is 3128 by default. you can set it to anything you like. e.g. 8080)

Save the squid.conf file. Then issue these commands.

1
2

[root@localhost squid]# squid -z [Enter] (as root) (This needs to be executed only once.)
[root@localhost squid]# service squid start [Enter] (as root)

If you want to start the squid server on boot, issue this command.

[root@localhost squid]# chkconfig --level 345 squid on [Enter] (as root)

Now, your machine is a proxy server. You can setup your browser to use the machine as a proxy server.

Conditions

The proxy server will work only if your machine has a public IP and is directly connected to internet.

Part 2: Setting up a hierarchical caching proxy server with squid

The above setup works fine if a machine is directly connected to internet. But my machine itself is behind a proxy, so setting up a proxy on my machine is of no use unless the proxy on my machine uses the institute proxy for connecting to internet. So, here we jump into squid.conf again and this time we have to really do some brain storming. If you are a newbie to Linux and don’t know how to make a system work when nothing seems to help, you will probably be better off by using institute’s proxy.

Here is the scenario.

1
2
3
4
5
6
7
8
9
10
11
12
13

1. Your browser sends a content request to proxy on your machine.
2. Check: if a cache HIT from institute proxy cache (HIT means content was found in cache)
	2a. Check: if content is older than the original upstream content
		2aa. Fetch content from upstream and serve the client
	2b. else
		2ba. Serve the content from the cache
3. Check: if cache HIT from proxy on your machine
	3a. Check: if content is older than the original upstream content
		3aa. Fetch content from upstream and serve the client
	3b. else
		3ba. Serve the content from the cache
4. Cache MISS from both the proxies
	4a. Fetch the content from upstream and serve the client

The above method of operation is very basic and is my understanding of squid. It may not be the exact squid behavior.

Now, lets see the configurations needed for setting up the hierarchical caching proxy server with squid.

Assumptions

I assume that we already have squid setup at institute’s proxy whether in caching mode or not. The best way to add/edit the following lines in your squid.conf is to search for particular parameter and then edit the value to set as given.

I also assume that you have simple proxy server setup on your machine and now we want to make it act as child proxy of the institute’s proxy.

Configuration

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71

# Your local machine will act as a sibling proxy
cache_peer 172.17.8.175 sibling 3128 3130 no-query weight=10
# The institute's proxy server will act as a parent proxy
# 'default' mean the last-resort
cache_peer 192.168.36.204 parent 8080 3130 no-query proxy-only no-digest default
# allow accessing peer cache for access list 'myip'
cache_peer_access 172.17.8.175 allow myip
# Don't cache dynamic content
hierarchy_stoplist cgi-bin ?
acl QUERY urlpath_regex cgi-bin \?
cache deny QUERY
# Size of main memory to be used for caching
cache_mem 200 MB
# max size of content to be stored in main memory
maximum_object_size_in_memory 7000 KB
# policy for cache replacement if memory is full
cache_replacement_policy heap LFUDA
# the directory to be used for storing cache on your hdd
cache_dir aufs /var/spool/squid 200 16 256
# max file descriptor open at a time .. 0(unlimited)
max_open_disk_fds 0
# min object size to cache on hdd
minimum_object_size 0 KB
# max object size to cache on hdd
maximum_object_size 16384 KB
# access log
access_log /var/log/squid/access.log squid
refresh_pattern ^ftp:           1440    20%     10080
refresh_pattern ^gopher:        1440    0%      1440
refresh_pattern .               0       20%     4320
store_avg_object_size 20 KB
acl apache rep_header Server ^Apache
broken_vary_encoding allow apache
refresh_stale_hit 5 seconds
acl SSL_ports port 443 563 1863 5190 5222 5050 6667
# Allow AIM protocols
acl AIM_ports port 5190 9898 6667
acl AIM_domains dstdomain .oscar.aol.com .blue.aol.com .freenode.net
acl AIM_domains dstdomain .messaging.aol.com .aim.com
acl AIM_hosts dstdomain login.oscar.aol.com login.glogin.messaging.aol.com toc.oscar.aol.com irc.freenode.net
acl AIM_nets dst 64.12.0.0/255.255.0.0
acl AIM_methods method CONNECT
http_access allow AIM_methods AIM_ports AIM_nets
http_access allow AIM_methods AIM_ports AIM_hosts
http_access allow AIM_methods AIM_ports AIM_domains
# Allow Yahoo Messenger
acl YIM_ports port 5050
acl YIM_domains dstdomain .yahoo.com .yahoo.co.jp
acl YIM_hosts dstdomain scs.msg.yahoo.com cs.yahoo.co.jp
acl YIM_methods method CONNECT
http_access allow YIM_methods YIM_ports YIM_hosts
http_access allow YIM_methods YIM_ports YIM_domains
# Allow GTalk
acl GTALK_ports port 5222 5050
acl GTALK_domains dstdomain .google.com
acl GTALK_hosts dstdomain talk.google.com
acl GTALK_methods method CONNECT
http_access allow GTALK_methods GTALK_ports GTALK_hosts
http_access allow GTALK_methods GTALK_ports GTALK_domains
# Allow MSN
acl MSN_ports port 1863 443 1503
acl MSN_domains dstdomain .microsoft.com .hotmail.com .live.com .msft.net .msn.com .passport.com
acl MSN_hosts dstdomain messenger.hotmail.com
acl MSN_nets dst 207.46.111.0/255.255.255.0
acl MSN_methods method CONNECT
http_access allow MSN_methods MSN_ports MSN_hosts
# Turn this off if hierarchical behavior is needed
nonhierarchical_direct off
never_direct deny myip
hosts_file /etc/hosts
coredump_dir /var/spool/squid

That’s the minimal configuration you need for running squid in hierarchical way. Save the squid.conf file and start/restart/reload the squid service. Setup your browser to use your machine as proxy and while using it’ll cache all the static content. You should experience some reduction in average page load time.

Advantages

I am currently using squid in above configuration. And its turning out to be nice for me. I am browsing websites faster and saving a chunk of bandwidth for my institute.

Disadvantages

Introduction of another proxy server increases the latency for dynamic content.

Notice

The above configurations and views are a result of my understanding of squid. If you feel this may break your system or it may have adverse effects, don’t use them. At least don’t use these on a production system.

 
by Kulbir Saini on Extensions, Firefox, Fonts, FTP, Internet, Linux, Review, Screenshot

Review: Firefox 3 Beta 1

Firefox 3 Beta 1 is available now. I downloaded the package yesterday morning and started using it right away. While using, I figured out some of the good things and at the same time there are some bad things about this test release. Here is what I feel about Firefox 3 Beta 1.

BTW, you can get beta 1 for Firefox 3 here. I am not going to tell, how to install firefox 🙂

Positives:

1.Nice Default Font:

Well someone may argue that you can configure any font in any version of Firefox, so whats the good thing about this default font in Firefox 3 Beta1. Well, I’ll say that configuration is always available but nobody gives a damn to configure the font in browser. I am liking this default font in FF3 Beta1 and everything now seems interesting to read. I have Read more than 40 wiki pages since last two days.

2.New FTP Listing Style:

The FTP listings in Firefox 3 Beta 1 has got new stylesheet. Its not dirty any more. It feels good while browsing ftp now. See the screen-shot below.

FTP Listing In Firefox

3.Website Identity:

This is new feature, but I couldn’t find it useful. When you click on favicon in the location bar, a pop-up kinda thing comes up, which tells about the identity of the site you are visiting. Check out the snapshot below.

Website Identity in Firefox

4.Drag n Drop:

You can now hold any thing in Firefox 3 Beta 1 and drag it anywhere you want. Though I could not drop the images to gimp or anywhere else, it looks good. I think its in the development stage right now.

Drag and Drop in Firefox Drag and Drop in Firefox

5.Bookmarks:

Bookmarking a website or editing your bookmarks is far easier now. You can edit a bookmarked site while its loading or can bookmark the site by just clicking a button. A new star-shaped button has been added in the location bar before the go button. If a site is not bookmarked, clicking the button will just bookmark it without asking for anything. If you again click on the button, now you can edit or delete the site you just bookmarked.

Bookmark Editing in Firefox

6.Adding Search Engine:

In firefox 3 beta 1, you can add the site you are browsing in you search engine list just like google, imdb, wikipedia, yahoo or whatever. But this is not valid for every website. Every website in this world can’t be a search engine 😛 Checkout the snapshot or KDE.

Adding Search Engine in Firefox

7.Link sorting/searching in location bar:

Firefox 3 beta 1 implements a new searching/sorting mechanism for links in location bar. As you type some letters, those are now searched in the link as well as the title of the page and then the links are sorted and displayed as list. In drop down list, you can also see the sites on your bookmarks.

8.Places Folder:

A new bookmark folder named ‘Places’ has been added. It has 6 sub-folders which contains links to different site based on a particular criterion. Some of them are ‘Recently Starred Pages’, ‘Recently Visited Starred Pages’, ‘Most Visited Starred Pages’ etc. I was looking for something like this since a long time. Because every time I login into my PC and fire up my firefox, first of all, I visit 8-10 sites (students mail, slashdot, iiit blogroll, yahoo mail, gmail, forum, orkut etc.). Now, I can open all of them in just one click. I click on ‘Most Visited Starred Pages’ and done.

9.Low Memory:

Well, a review of Firefox 3 Beta 1 on zdnet, say that Firefox 3 beta 1 is consuming low memory than Firefox 2 and IE 7. That review is for Firefox in windows. But in Fedora 7, I didn’t experience any reduction in memory usage.

10.Faster:

I don’t know how to justify this, but Firefox 3 Beta 1 seems to load pages faster. What I could think was that, when you click go, it loads the page in the background for some-time and then suddenly flushes the page to the screen and it appears that the entire page has been loaded in one go. I may be wrong though.

11.Improved GUI:

The rendering of buttons, input fields, images in the pages have improved a lot and buttons and drop down menus look better now.

Negatives:

1.Extensions:

Life is a bit or I should say a byte difficult with Firefox extensions. Now, I am too much used to use these extensions that I can’t live without them. I’ll crash if I don’t have ‘Undo Closed Tab’ extension. I keep making stupid mistakes. All the extensions are not yet available for Firefox 3, so its a problem.

2.Multiple Tabs:

Firefox 3 Beta 1 immediately dies if I open more than 20 tabs. I opened my wiki saved session (39 tabs) and after sometime I was searching for Firefox. Where the hell firefox window has gone?

3.Crashes:

I don’t know what exactly is the problem, but I think Firefox 3 revolts against Yahoo Mail and GMail. Whenever I open Yahoo Mail or GMail, Firefox 3 Beta 1 crashes. Either there is some problem with the heavily loaded mail pages or it doesn’t want me to use GMail or Yahoo.

Well, there may be lot more positives and negatives. These are the things I noticed about Firefox 3 Beta 1 in last two days. As the positives list is ruling the negatives, I am using Firefox 3 Beta 1 full time these days.

PS : @Paresh I think FF3 is better than FF2 in many ways. But you may want to wait for all the extensions to be compatible with FF3.

 
by Kulbir Saini on Drivers, Fedora, Hacks, Hardware, HowTo, Internet, Wireless

How To: Configure Wireless with Ralink (rt2500) Level One WNC 0301 in Fedora 7

If you are searching for wireless lan configuration in Fedora Core 6, a detailed description is available here. Though Fedora 7 – Moonshine detect the Ralink rt2500 Level One WNC-0301 wireless lan card, the network doesn’t work properly with default drivers. You may be able to connect sometime, but some other time it may not function properly. Because ralink drivers are not yet stable. See the discussions here. So, here is a step by step complete reference to how to make it work properly.

Step 1: Download the latest CVS release of Ralink rt2500 drivers from here.

Step 2: Unload the kernel module for rt2500 drivers

[root@bordeaux kulbirsaini] rmmod rt2500pci [Enter]

Step 3: Go to the directory /lib/modules/2.6.21-1.3194.fc7/kernel/drivers/net/wireless/mac80211/rt2x00 and take backup of the current rt2500pci driver module.

[root@bordeaux rt2x00] mv rt2500pci.ko back.rt2500pci.ko [Enter]

Step 4: Untar the download driver tar ball and change to the directory ./rt2500-cvs-XXXXXXXXXX/Module/

1
2

[root@bordeaux Module] make [Enter]
[root@bordeaux Module] make install-fedora [Enter]

Step 5: Open /etc/modprobe.conf file and add a line

alias wlan0 rt2500

Save the file and load the rt2500 driver.

[root@bordeaux kulbirsaini] modprobe rt2500 [Enter]

Step 6: Go to the directory /etc/sysconfig/network-scripts/ and create a file named ifcfg-wmaster0. Copy the contents of ifcfg-wlan0 to the ifcfg-wmaster0. If you don’t have ifcfg-wlan0 file, then issue command ‘neat’ as root and add new wireless device with appropriate configurations and ifcfg-wlan0 will be created in the process.

Step 7: Activate the device, what else ???

PS1 : You can refer to my ifcfg-wlan0 and ifcfg-wmaster0 files.

PS2 : You can ask for more details on any issues in the any of the above steps.

 
by Kulbir Saini on Drivers, Fedora, Hacks, Hardware, HowTo, Internet, Kernel, Ndiswrapper, Wireless

How To: Configure Wireless with Ralink (rt2500) Level One WNC-0301 in Fedora Core 6

Well … Yesterday I switched from Fedora Core 4 to Fedora Core 6. The main problem was the wireless lan. As I was using ndiswrapper for wlan in FC4, I tried compiling it for FC6. But FC6 doesn’t have build packages so it didn’t work out. Then I installed the kernel-devel packages from here. Now the build packages were not a problem. I installed ndiswrapper from here. Now while installing ndiswrapper it gave an error that your kernel is using 4k stack, while for windows driver you need to have a kernel with 16k stack. I googled and searched the 16k stack version for my kernel but didnt get one. I used to download 16k stack kernel from here. But 16k stack kernel is not yet out for FC6. So its almost impossible for me to get ndiswrapper to work if I dont want to mess up with the patched and all.

Then I thought of using the native drivers for rt2500 (Level One wnc-0301). I downloaded the drivers from here.
The step by step installation is here ….

Step 0

Install the kernel-devel package from the above specified site.

[root@localhost ~]# rpm -ivh kernel-devel-yourkernel.rpm

Step 1

Untar the drivers.

[root@localhost ~]# tar -xvzf rt2500-1.1.0-b4.tar.gz

Step 2

Change the directory to Module

[root@localhost ~]# cd rt2500-1.1.0-b4/Module/

Step 3

compile the modules

[root@localhost ~]# make

If this gives error like this

1
2
3
4
5
6
7
8
9

make[1]: Entering directory '/usr/src/kernels/2.6.18-1.2798.fc6-xen-i686'
CC [M]  /home2/Softwares/Drivers/rt2500-1.1.0-b4/Module/rtmp_main.o
In file included from /home2/Softwares/Drivers/rt2500-1.1.0-b4/Module/rtmp_main.c:50:
/home2/Softwares/Drivers/rt2500-1.1.0-b4/Module/rt_config.h:58:40: error: linux/config.h: No such file or directory
make[2]: *** [/home2/Softwares/Drivers/rt2500-1.1.0-b4/Module/rtmp_main.o] Error 1
make[1]: *** [_module_/home2/Softwares/Drivers/rt2500-1.1.0-b4/Module] Error 2
make[1]: Leaving directory '/usr/src/kernels/2.6.18-1.2798.fc6-xen-i686'
rt2500.ko failed to build!
make: *** [module] Error 1

Then open the file rt_config.h and comment the line #include<linux/config.h> and again compile by issuing make.
If you get a different error try to debug if you know c-programming a bit.
Step 4. [Do as root]
Install the module

[root@localhost ~]# make install-fedora

Step 5 [Do as root]

Configure the wlan card.

[root@localhost ~]# neat

Now select a new connection and wireless and the the Ralink driver. Thats it. Activate the wlan0.

Hope this help a bit.
More suggestions are welcome.

Edit : If you are looking for wireless configuration in Fedora 7, here is a reference.

 
by Kulbir Saini on Configuration, Drivers, Fedora, Hacks, Hardware, HowTo, Installation, Internet, Ndiswrapper, Wireless

How To: Configure Wireless with Ralink (RT2500) Level One WNC 0301 in Fedora Core 4

Yesterday after a lot of trials with Fedora Core 5 to activate my wireless LAN card, I switched to Fedora Core 4, which looked much more stable than FC5. Actually with FC5, I mainly faced problems with gui , my mouse which is Microsoft USB mouse and my wlan card which is level one Ralink RT2500. In FC5, I tried both with ndiswrapper and madwifi but nothing worked out for me. Also my mouse was not working well. I can click things but the pointer was not visible like microsoft gave me a invisible mouse or this microsoft mouse don’t want to work with Linux. After installing FC4, I felt very comfortable with guis and my mouse. They worked fine. But now there were two problems. One is that FC4 was not able to detect my soundcard and the wlan was not working again. Anyway soundcard is not a big problem, as it worked in FC5, I’ll make it work with some up-gradation or things like that. But the major problem is wlan. And here goes the methods which I tried to activate my wlan card.

Method 1. With rpms

I downloaded these rpms

1
2
3

madwifi-0.9.4.12-16.rhfc4.at.i386.rpm
madwifi-kmdl-2.6.11-1.1369_FC4-0.9.4.12-16.rhfc4.at.i686.rpm
madwifi-hal-kmdl-2.6.16-1.2111_FC4smp-0.9.6.0-20.rhfc4.at.i686.rpm

from the site atrpms and tried installing them but they failed the dependency /boot/vmlinuz-2.6.16-1.2111_FC4 , then i tried to find this rpm everywhere on google , rpmfind etc …. but I could not find it. Then I installed it with –nodeps option and configured according to the data given here .
But in vain.

Method 2. With ndiswrapper using Microsoft Windows XP drivers

I downloaded the ndiswrapper-1.16 from sourceforge.
I untarred it and used these commands.

1
2
3

make uninstall
make
make install

Please don’t forget to the read the INSTALL and README files in the ndiswrapper-1.* directory. Then I inserted my LAN card driver cd for XP and copied the drivers directory which contains .inf and .sys files to my root directory. Then I used the commands .

1
2
3
4
5

ndiswrapper -i Rt2500.INF
ndiswrapper -l
modprobe ndiswrapper
iwconfig "wlan0" key open "wep 128 bit hex key" ESSID "IIIT WLAN"
dhclient wlan0

I ran all this command and I am very happy to say that wireless LAN did not work. 🙂 Then I ran this command

touch /etc/sysconfig/network-scripts/ifcfg-wlan0

and then edited this file. I entered all the data suggested here except that second line I wrote

DEVICE=wlan0

OK. Then i ran

neat

and edited the properties of wlan device by double clicking it and gave the 26 letters wlan key there and activated the device and it worked 😀