Tips – Tricks

by Kulbir Saini on Fedora, HowTo, Tips - Tricks, Virtualization

How To: Test Fedora Pre-release

If you want your Fedora to be best ever, just grab a copy of Fedora DVD or live iso from here and test the features you like. Checkout the Fedora 10 feature list and test as many as possible.

If you don’t have access to optical media (cd/dvd rom) and want to try the live iso, VirtualBox is your friend. Grab the latest version of VirtualBox for your distro and platfrom from Sun download center and install using rpm or yum. Using VirtualBox is quite easy. Here is a video tutorial for noobs on how to test pre-release live ISOs using VirualBox. Have fun 🙂

 
by Kulbir Saini on Hacks, HowTo, Linux, Programming, Proxy Server, Python, Server, Squid, Tips - Tricks

How To: Write Custom Redirector or Rewritor Plugin For Squid in Python

Mission

To write a custom Python program which can act as a plugin for Squid to redirect a given URL to another URL. This is useful when already existing redirector plugins for Squid doesn’t suit your needs or you want everything of your own.

Use Cases

  1. When you want to redirect URLs using a database like mysql or postgresql.
  2. When you want to redirect based on mappings stored in simple text files.
  3. When you want to build a redirector which can learn by itself using AI techniques 😛

How to proceed

From Squid FAQ,

The redirector program must read URLs (one per line) on standard input, and write rewritten URLs or blank lines on standard output. Note that the redirector program can not use buffered I/O. Squid writes additional information after the URL which a redirector can use to make a decision.

The format of the line read from the standard input by the program is as follows.

1
2
3

URL ip-address/fqdn ident method
# for example
http://saini.co.in 172.17.8.175/saini.co.in - GET -

The implementation sounds very simple and it is indeed very simple to implement. The only thing that should be taken care of is the unbuffered I/O. You should immediately flush the output to standard output once decision is taken.

For this howto, we assume we have a method called ‘modify_url()‘ which returns either a blank line or a modified URL to which the client should be redirected.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

#!/usr/bin/env python
 
import sys
def modify_url(line):
    list = line.split(' ')
    # first element of the list is the URL
    old_url = list[0]
    new_url = '\n'
    # take the decision and modify the url if needed
    # do remember that the new_url should contain a '\n' at the end.
    if old_url.endswith('.avi'):
        new_url = 'http://fedora.co.in/errors/accessDenied.html' + new_url
    return new_url
 
while True:
    # the format of the line read from stdin is
    # URL ip-address/fqdn ident method
    # for example
    # http://saini.co.in 172.17.8.175/saini.co.in - GET -
    line = sys.stdin.readline().strip()
    # new_url is a simple URL only
    # for example
    # http://fedora.co.in
    new_url = modify_url(line)
    sys.stdout.write(new_url)
    sys.stdout.flush()

Save the above file somewhere. We save this example file in /etc/squid/custom_redirect.py. Now, we have the function for redirecting clients. We need to configure squid to use custom_redirect.py . Below is the squid configuration for telling squid to use the above program as redirector.

1
2
3
4
5
6

# Add these lines to /etc/squid/squid.conf file.
# /usr/bin/python should be replaced by the path to python executable if you installed it somewhere else.
redirect_program /usr/bin/python /etc/squid/custom_redirect.py
# Number of instances of the above program that should run concurrently.
# 5 is good enough but you should go for 10 at least. Anything below 5 would not work properly.
redirect_children 5

Now, start/reload/restart squid. That’s all we need to write and use a custom redirector plugin for squid.

 
by Kulbir Saini on Hacks, HowTo, Linux, Programming, Proxy Server, Python, Security, Server, Squid, Tips - Tricks

How To: Write Custom Basic Authentication Plugin for Squid in Python

Mission

To write a Python program which can be used to authenticate for Squid proxy server. This is useful when you don’t want to configure complex systems like LDAP, ntlm etc.

Use Cases

  1. When you want to authenticate clients using mysql database.
  2. When you want to authenticate clients using flat files or /etc/passwd file or some custom service on your network.

How to proceed

From auth_param section in squid.conf file:

Specify the command for the external authenticator. Such a program reads a line containing "username password" and replies "OK" or "ERR" in an endless loop. "ERR" responses may optionally be followed by a error description available as %m in the returned error page.

By default, the basic authentication scheme is not used unless a program is specified.

That clearly states that our python program should read a line from standard input (stdin) and write the appropriate response to the standard output (stdout). But there are some issues with I/O. The output should be unbuffered and should be flushed to standard output immediately after the response is known.

So, lets see a small program where we authenticate using a function ‘matchpassword()‘. This function returns True when username, password pair matches and returns False when they mismatch.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

#!/usr/bin/python
 
import sys
import socket
"""USAGE:The function returns True if the user and passwd match False otherwise"""
def matchpasswd(login,passwd):
    # Write your own function definition. 
    # Use mysql, files, /etc/passwd or some service or whatever you want
    pass
 
while True:
    # read a line from stdin
    line = sys.stdin.readline()
    # remove '\n' from line
    line = line.strip()
    # extract username and password from line
    username = line[:line.find(' ')]
    password = line[line.find(' ')+1:]
 
    if matchpasswd(username, password):
        sys.stdout.write('OK\n')
    else:
        sys.stdout.write('ERR\n')
    # Flush the output to stdout.
    sys.stdout.flush()

Save the above file somewhere. We save this example file in /etc/squid/custom_auth.py .Now, we have the function for authenticating clients. We need to configure squid to use custom_auth.py . Below is the squid configuration for telling squid to use the above program as basic authenticator.

1
2
3
4
5
6
7
8
9
10
11

# you need to specify /usr/bin/python if your file is not executable and needs an interpreter to be invoked.
# Replace /usr/bin/python with /usr/bin/php , if you write auth program in php.
auth_param basic program /usr/bin/python /etc/squid/custom_auth.py
# how many instances of the above program should run concurrently
auth_param basic children 5
# display some message to clients when they are asked for username, password
auth_param basic realm Please enter your proxy server username and password
# for how much time the authentication should be valid
auth_param basic credentialsttl 2 hours
# whether username, password should be case sensitive or not
auth_param basic casesensitive on

Now, to force clients to authenticate, configure the acls as follow. Below we assume, you want to force all clients on your lan to authenticate for using proxy server.

1
2
3
4
5
6

# acl to force proxy authentication
acl authenticated proxy_auth REQUIRED
# acl to define IPs from your lan
acl lan src 192.168.0.0/16
# acl to force clients on your lan to authenticate
http_access allow lan authenticated

Now, reload/restart squid. That’s all we need to write and use a custom authentication plugin for squid.

Limitation

Username can’t contain spaces. Otherwise program will not be able to parse/extract username, password from standard input.

 
by Kulbir Saini on Configuration, Hacks, HowTo, Installation, Linux, Server, Tips - Tricks

How To: Install and Configure Shoutcast Radio

Shoutcast is a mp3 broadcasting/streaming media server software provided by NullSoft. One can setup a server on any system, GNU/Linux / Windows/ MacOS and can stream mp3 over the network, internet/intranet. I setup shoutcast on my system a long back and found it very useful. Here is a step by step how to on how you can setup shoutcast on a GNU/Linux system.

Shoutcast can be installed even if you don’t have root privileges. But in that case you can’t use port less than 1024 for broadcasting. Below, I’ll explain how to install it for a non-root user.

Shoutcast server depends on a tool shoutcast DNAS for audio input in Linux. So, here we go

Step 1

Download the latest version of shoutcast from here. Download the one for Linux (glibc).

Step 2

Let us assume we want to install shoutcast in a directory named ‘shoutcast’ in user’s home directory and we want to broadcast punjabi songs.

1
2
3

[saini@bordeaux shoutcast]# tar -xvzf sc_serv_1.9.8_Linux.tar.gz [Enter]
[saini@bordeaux shoutcast]# mkdir punjabi [Enter]
[saini@bordeaux shoutcast]# mv sc_serv.conf punjabi/sc_serv_punjabi.conf [Enter]

Step 3

Open sc_serv_punjabi.conf in your favorite editor and modify certain parameters as per you requirements. The essentials are below.

1
2
3
4
5
6
7
8
9
10
11
12
13

MaxUser=20
Password=yourPassword
PortBase=8300 #(Confirm that this port and the port PortBase+1 is not being used)
LogFile=none
RealTime=0
ScreenLog=0
ShowLastSongs=10
SrcIP=ANY
DestIP=ANY
Yport=80
NameLookups=0
AdminPassword=yourAdminPassword
TitleFormat=%s [IIIT Radio]

etc. My sc_serv.conf can be accessed here. That was all for installing the shoutcast server.

Now, the installation of DNAS tool is still pending. Here is a step by step procedure to install DNAS tool.

Step 1

Download the latest version of DNAS tool from here.

Step 2

1
2
3
4

[saini@bordeaux shoutcast]# tar -xvzf sc_trans_posix_040.tgz [Enter]
[saini@bordeaux shoutcast]# cd sc_trans_040/ [Enter]
[saini@bordeaux sc_trans_040]# mv sc_trans_linux ../ [Enter]
[saini@bordeaux sc_trans_040]# mv sc_trans.conf ../punjabi/sc_trans_punjabi.conf [Enter]

Step 3

Go to punjabi directory and open sc_trans_punjabi.conf in your favorite editor and make changes according to your needs. Here are some

1
2
3
4
5
6
7
8
9

PlaylistFile=/exactPathTo/punjabi.lst
ServerIP=
ServerPort=
 # 8300 in this case
Password=
 # yourPassword in this case
StreamTitle= %s
StreamURL=
Shuffle=1 # (1 for random songs)

etc. My sc_trans.conf can be accessed here.

Step 4

Generate a list of all the songs (mp3) you have and put it in punjabi.lst in punjabi directory.

[saini@bordeaux punjabi]# find /pathToPunjabiDir/ -type f -name "*.mp3" > punjabi.lst [Enter]

My dummy punjabi.lst can be accessed here.

The configuration part of shoutcast server with audio input is complete. Now we have to run the server so that we can listen to music.

Go to the shoutcast directory and run the sc_serv first and then run the sc_trans_linux. Here is way to do that.

1
2

[saini@bordeaux shoutcast]# ./sc_serv punjabi/sc_serv_punjabi.conf > /dev/null 2> /dev/null &
[saini@bordeaux shoutcast]# ./sc_trans_linux punjabi/sc_trans_punjabi.conf > /dev/null 2> /dev/null &

Now your system is a shoutcast server. Any client can use mplayer, vlc, amarok or any other multimedia player that support streaming media to listen to the music being played on your server.

1
2

[saini@bordeaux saini]# mplayer http://yourIp:port [Enter]
[saini@bordeaux saini]# mplayer http://localhost:8300 [Enter] # (in the above case).

If you want shoutcast to start every time your system boots. Put these lines in /etc/rc.local

1
2

/home/saini/shoutcast/sc_serv /home/saini/shoutcast/punjabi/sc_serv_punjabi.conf > /dev/null 2> /dev/null &
/home/saini/shoutcast/sc_trans_linux /home/saini/shoutcast/punjabi/sc_trans_punjabi.conf > /dev/null 2> /dev/null &

Shoutcast is fun and its more fun when everyone listens to what you are listening to 🙂

 
by Kulbir Saini on Compiz, Drivers, Eclipse, Fedora, GNOME, Graphics, Hardware, NVIDIA, Remote Desktop, Review, Screenshot, Sound, Tips - Tricks, VNC Server, VNC Viewer, Wireless, Xorg

Review: Fedora 8 – Warewolf

I installed Fedora 8 32 bit from a leaky mirror on Nov 7th and I just had a very bad experience with it. Nothing seemed to be working. But I can’t accept that. As I am a hardcore fan of Fedora, I just can’t sit back and say “ah, Fedora 8 sucks, i am not gonna use that”. I fetched Fedora 8 x86_64 (64bit) from a mirror yesterday, after the release. I installed it and everything worked out of the box. I can’t believe that I wrote something wrong about Fedora. How could I do that ?

First of all, I would like to say that The artwork team at Fedora has done a very fantastic job. The graphics right from installation up to the desktop are just awesome. Especially the default background is very nice. Here is shot of the default Gnome Fedora 8 Desktop.

GNOME Fedora 8 Desktop

Right after the installation, I fetched the nVidia proprietary drivers from here and installed them. And those were installed successfully without giving any errors or problems. [ If you want a complete howto on installing nvidia drivers. Its here.] A reboot after the installation and compiz worked out of the box. Here is shot.

Compiz Fusion

Ok, graphics done. What now ? I just realized that there is no mp3 support. No worries. Codeina aka Codec Buddy is there. Just issue ‘codeina’ command from command line and a window like this will appear.

Codeina Audio Codec Fetcher

Check Fluendo MP3 Audio Decoder and click get selected, accept the license conditions and you’ll see that codeina is fetching the codecs. [If codeina does not fetch codecs or give error like timeout or some other network error. Try checking your proxy setting in System -> Preferences -> Internet And Network -> Network Proxy . It may help. ]

Codeina Installing MP3 Support

Ok. Now, codeina has done the job. Lets play some mp3. Note that amarok still can’t play mp3 files because it uses xine engine. So, you can choose either Totem or Rhythmbox to play your mp3 files. Here is a shot of Rhythmbox. So, Codeina also works out of the box.

Rhythmbox Playing MP3

Another major improvement in Fedora 8 in audio section is introduction or pulseaudio. Issue command ‘pulseaudio’ from command line and you will see a window like this.

Pulseaudio Device and Application Control

You can control the sound stream from different players or whatever. You can mute individual streams and can even set the default devices for certain streams through this fantastic gui.

Another good thing in Fedora 8 is Eclipse. Eclipse 3.3 is back in Fedora 8. They excluded it from Fedora 7. I am happy to see it back here in Fedora 8.

Eclipse In Fedora 8

Another utility that I found helpful is Remote Desktop utility. Launch System -> Preferences -> Internet And Network -> Remote Desktop and you’ll see a window like this.

Remote Desktop Utility

Set your preferences and now you can browse your desktop from anywhere using ‘vncviewer <yourIP>:0’. Though one call always configure vncserver to get that done. But for newbies it’ll be a great help.

Also, My wireless lan card, Ralink rt2500 WNC-0301 is detected successfully in Fedora 8. But I am not sure whether it works or not, because there is not wifi environment in my lab and I can’t check it without that. [ Anyway if your wifi card doesn’t work, here is a howto on installing Ralink rt2500 WNC-0301 using drivers from serailmonkey. ]

Another improvement is that cursor was never invisible. Up to Fedora 7, I suffered cursor invisible problem on first login. [ If you are facing the same problem, add line

Options "HWCursor" off

to “screens” section in your /etc/X11/xorg.conf file and issue ‘gdm-restart’ command. It’ll be fine afterwards. ]

The boot time has also improved significantly. My Fedora 8 boots in just 45 seconds.

These Fedora 8 Screenshots and other related to Fedora 8 can be reached here.