Installation

by Kulbir Saini on Configuration, GSOC, Installation, IntelligentMirror, Open Source, Proxy Server, Server, Squid

IntelligentMirror: RPM and DEB Caching Improved (0.4)

IntelligentMirror version 0.4 is available now. There have been significant improvements in intelligent mirror since last release.

Improvements

  1. Fixed defunct process problem. You will not see defunct python processes hanging around anymore. Previously every forked daemon used to got defucnt because parent never waited for the forked child to finish.
  2. IntelligentMirror now supports caching of Debian packages just like rpms. So now IntelligentMirror is best suited shared environments where people have different tastes.
  3. Intelligent Mirror now uses url_rewrite_program instead of redirect_program. This boosts the efficiency of IntelligentMirror by a significant factor as url_rewrite_program has an acl controller url_rewrite_access. And using url_rewrite_access only requests for rpm/deb packages will be passed to Intelligent Mirror. So, IM now need not process each and every incoming request. Also, it has redirector_bypass directive which will bypass IM in case all the instances of IM are busy serving requests. So, squid will not die with a fatal error in case of huge requests.
  4. Options to enable/disable caching for rpm and Debian packages have been added.
  5. Options to control the total size of caching directories and the size of individual package to be cached have also been introduced.
  6. Proxy authentication is also supported now just the way it is supported in yum.
  7. Packages are not checked for last-modified time anymore. Because in principle two rpms A and B can only have same name iff they have the same contents. So, the delay in response time in case of hits has reduced.

Availability

  1. RPMs for Fedora/Red Hat
  2. Source RPMs for Fedora/Red Hat
  3. Source Tar balls

Installation and configuration is easy and the INSTALL and README files should serve the purpose.

In case you have any suggestions or problems, leave a comment here or file a ticket on project page.

 
by Kulbir Saini on Fedora, Grub, Hacks, HowTo, Installation, Kernel, Tips - Tricks

How To: Install Fedora without CD or DVD

Note: If you are new to Fedora/Linux, I highly recommend the book “Fedora Linux Toolbox”

[amazon-product alink=”0000FF” bordercolor=”000000″ height=”240″]0470082917[/amazon-product]

Use Case

  1. When you don’t have CD / DVD drive on your system.
  2. You have Fedora DVD but your system has only a CD Drive.
  3. You don’t want to waste time and resources in burning iso on optical media.

Pre-requisites

  1. You have a Fedora DVD iso or rescue cd iso.
  2. You have a Linux installation on your system.
  3. You have a partition (FAT32, ext2, ext3) which you will not format while installing the new OS.

How to proceed

Let us assume you want to install Fedora 9 on your system and you have a Linux distro already installed on your system. You have downloaded the Fedora DVD iso (Fedora-9-DVD-i686.iso). And you have a FAT32/ext2/ext3 partition /stuff/ which you will not format during installation.

Step 1 : Move the Fedora DVD iso to /stuff/ directory.

[root@saini saini]# mv Fedora-9-DVD-i686.iso /stuff/ [Enter]

Step 2 : Mount Fedora DVD iso on /mnt/

[root@saini saini]# mount /stuff/Fedora-9-DVD-i686.iso /mnt/ -ro loop [Enter] (do as root)

Step 3 : Copy the initrd.img and vmlinuz to /boot/ partition

[root@saini saini]# cd /mnt/isolinux/ [Enter]
[root@saini isolinux]# cp initrd.img vmlinuz /boot/ [Enter] (do as root)

Step 4 : Create grub entry for booting into Fedora 9

Add these lines at the end of your /boot/grub/grub.conf file.

title Fedora 9 (New installation)
    kernel /vmlinuz
    initrd /initrd.img

Step 5 : Note the device having Fedora DVD iso

[root@saini saini]# df -h [Enter]
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda3              15G  9.5G  4.1G  70% /
/dev/sda8             135G  116G   13G  91% /stuff
/dev/sda5             4.8G  1.2G  3.4G  26% /home
/dev/sda1              99M   12M   82M  13% /boot

In this case /dev/sda8 contains Fedora DVD iso. Note this down as you need it later.

Step 6 : Reboot

Reboot your system and boot into the Fedora 9 (New installation) grub entry.

Step 7 : Install from hard disk

While in installation wizard, select “Hard drive” as installation method and choose /dev/sda8 as it contains the Fedora DVD iso. And rest is damn easy.

Want MB5-292 ? Check out our latest MB7-226 study guides and 70-271 give you 100% exam pass guarantee of 70-158. You can also get MB6-285 from this website.

 
by Kulbir Saini on Configuration, GSOC, Installation, IntelligentMirror, Open Source, Proxy Server, Server, Squid

IntelligentMirror: Available for Testing

Note : A newer version of intelligentmirror is available now. Please check this.

Intelligent Mirror is basically a tool or squid plugin (redirector) to cache rpm packages so that the subsequent requests for the same package can be served from the local cache which will eventually save a lot of bandwidth and downloading time.

Who needs Intelligent Mirror?

  1. If you are on a shared network where a lot of people use linux distros with RPM as their package manager, then you need this. Universities should come under this category.
  2. If you have a set of systems having red hat derivatives and almost identical OS versions, you need this. LAN setups at home should come under this category.
  3. If you can’t afford to or don’t want to mirror entire fedora repo for local access due to bandwidth limitations, you need this.

What it does?

As described above, Intelligent Mirror, just caches rpms which are requested by the clients in a shared network. And subsequent requests for those rpms are served from the cache. For a detailed description, check the project page.

Why not use Squid in caching mode?

Squid caching is based on url hashing. Let me explain with an example how Intelligent Mirror is actually intelligent as compared to squid while caching rpms.

Let us say there is an rpm yum-3.2.0-1.fc7.i386.rpm . You executed “yum update yum“. And let us say the newer version of yum is yum-3.2.18-1.fc9.i386.rpm which was fetched from one of the fedora mirrors http://abc.com/ (say). Now someone on the same network launched “yum update yum” and he got the same rpm yum-3.2.18-1.fc9.i386.rpm. But this time rpm was fetched from another mirror http://xyz.com/ (say).

Case I : Squid caching

Squid will cache http://abc.com/linux/fc9/updates/i386/yum-3.2.18-1.fc9.i386.rpm . And when http://xyz.com/linux/fc9/updates/i386/yum-3.2.18-1.fc9.i386.rpm will be requested, it’ll result in a cache miss and squid will again download the same package and will cache this one as well. Now there are two problems

  1. Squid is not able to serve from the cache, though the package was the same.
  2. Additional storage space is being wasted in caching the same package. And this can really harm if unluckily a different mirror is picked in all the subsequent queries.

Case II : IntelligentMirror caching

Intelligent Mirror will cache the package yum-3.2.18-1.fc9.i386.rpm without bothering about its origin. And even if yum picks up a different mirror for the subsequent request, the package will be served from the cache and will not be fetched from upstream. So, the obvious advantage of saving the bandwidth and downloading time.

Download

Intelligent Mirror source tarball, rpm, source rpm are available for download from here.

Installing and Configuring Intelligent Mirror

Install Guide

Configuration Guide

Issues and Suggestions

If you see any issue or you have any suggestions for improving the functionality, either mail me at kulbirsaini25 AT GMAIL DoT COM or file a ticket on the project page.

 
by Kulbir Saini on Administration, Configuration, HowTo, Installation, Proxy Server, Server, Squid

How To: Configure Squid Proxy Server

Mission

To configure squid for simple proxying without caching anything.

Use Cases

  1. When you want to have control on what people browse on your lan.
  2. When number of machine is more than the number of IP addresses you can afford to buy.
  3. When you want to help this holy world in saving some IPV4 addresses 😛

Assumptions

  1. You have a machine connected directly to internet that you are going to use as a proxy server for other machines on your network.
  2. The machines on your network are using 192.168.0.0/16 as private address space. You can use anyone/multiple address spaces of the available but for this howto we assume 192.168.0.0/16 as the local network.
  3. The local IP address of the machine which will run squid proxy server is 192.168.36.204. You can have any IP, but for this howto we assume this.

How to proceed

First of all ensure that you have squid installed. After installing squid, you need to set access control in squid configuration file which resides in /etc/squid by default. Open /etc/squid/squid.conf and add/edit following lines according to your preferences. Few lines already exist in the configuration file, you can add the rest.

# The port on which squid will listen for requests
http_port 8080
# If 'cgi-bin' or '?' is in query, squid should not check with neighbours'/parents' cache
# and should go to target web-server.
hierarchy_stoplist cgi-bin ?
# If url contains 'cgi-bin' or '?', then it must not be cached
acl QUERY urlpath_regex cgi-bin \?
cache deny QUERY
acl apache rep_header Server ^Apache
broken_vary_encoding allow apache
# Absolute path to squid access log.
access_log /var/log/squid/access.log squid
refresh_pattern ^ftp:           1440    20%     10080
refresh_pattern ^gopher:        1440    0%      1440
refresh_pattern .               0       20%     4320
# Access control list to control every IP address
acl all src 0.0.0.0/0.0.0.0
# Access control list for source machine in LAN
acl lan_src src 192.168.0.0/16
# Access control list for destination machine in LAN
acl lan_dst dst 192.168.0.0/16
# Access control list to manage squid cache
acl manager proto cache_object
# Access control list to define IP address allowed for source localhost
acl localhost src 127.0.0.1/255.255.255.255
# Access control list to define IP addresses allowed for localhost as destination
acl to_localhost dst 127.0.0.0/8
# Access control list to define Safe ports that should be allowed by default
acl SSL_ports port 443 563 1863 5190 5222 5050 6667
acl Safe_ports port 80          # http
acl Safe_ports port 21          # ftp
acl Safe_ports port 443         # https
acl Safe_ports port 70          # gopher
acl Safe_ports port 210         # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280         # http-mgmt
acl Safe_ports port 488         # gss-http
acl Safe_ports port 591         # filemaker
acl Safe_ports port 777         # multiling http
acl CONNECT method CONNECT
# Allow cache management only from localhost
http_access allow manager localhost
# Deny cache management from remote hosts
http_access deny manager
# Deny http access via all the ports which are not listed as safe
http_access deny !Safe_ports
# Deny all connections via all ports which are not listed as safe
http_access deny CONNECT !SSL_ports
# Allow http access from localhost
http_access allow localhost
# Allow http access from machines on LAN
http_access allow lan_src
http_access deny all
http_reply_access allow all
icp_access allow all
# Deny caching for everyone so that there is not caching at all
cache deny all
coredump_dir /var/spool/squid
# Never allow direct connection to machines on the internet
prefer_direct off
never_direct allow all
# Allow direct connetion if the destination machine is on LAN
always_direct allow lan_dst
# Delete this line if you don't have /etc/hosts file
hosts_file /etc/hosts
# Allow AIM connections
# Delete the following 9 lines if you don't want people to connect to AIM
acl AIM_ports port 5190 9898 6667
acl AIM_domains dstdomain .oscar.aol.com .blue.aol.com .freenode.net
acl AIM_domains dstdomain .messaging.aol.com .aim.com
acl AIM_hosts dstdomain login.oscar.aol.com login.glogin.messaging.aol.com toc.oscar.aol.com irc.freenode.net
acl AIM_nets dst 64.12.0.0/255.255.0.0
acl AIM_methods method CONNECT
http_access allow AIM_methods AIM_ports AIM_nets
http_access allow AIM_methods AIM_ports AIM_hosts
http_access allow AIM_methods AIM_ports AIM_domains
# Allow connections to Yahoo Messenger
# Delete the following 6 lines if you don't want people to connect to Yahoo Messenger
acl YIM_ports port 5050
acl YIM_domains dstdomain .yahoo.com .yahoo.co.jp
acl YIM_hosts dstdomain scs.msg.yahoo.com cs.yahoo.co.jp
acl YIM_methods method CONNECT
http_access allow YIM_methods YIM_ports YIM_hosts
http_access allow YIM_methods YIM_ports YIM_domains
# Allow connections to Google Talk
# Delete the following 6 lines if you don't want people to connect to Google Talk
acl GTALK_ports port 5222 5050
acl GTALK_domains dstdomain .google.com
acl GTALK_hosts dstdomain talk.google.com
acl GTALK_methods method CONNECT
http_access allow GTALK_methods GTALK_ports GTALK_hosts
http_access allow GTALK_methods GTALK_ports GTALK_domains
# Allow connections to MSN
# Delete the following 6 lines if you don't want people to connect to Google Talk
acl MSN_ports port 1863 443 1503
acl MSN_domains dstdomain .microsoft.com .hotmail.com .live.com .msft.net .msn.com .passport.com
acl MSN_hosts dstdomain messenger.hotmail.com
acl MSN_nets dst 207.46.111.0/255.255.255.0
acl MSN_methods method CONNECT
http_access allow MSN_methods MSN_ports MSN_hosts

Now, start the squid proxy server as

service squid start

Also, if you want squid to be started every time you boot the machine, execute the following command

chkconfig --level 345 squid on

You have a squid proxy server running now. You can ask clients to configure there browsers to use 192.168.36.204 as a proxy server with 8080 as proxy port. Command line utilities like elinks, lynx, yum, wget etc. can be asked to use proxy by exporting http_proxy variable as below. Users can also add these lines to ~/.bashrc file to avoid exporting every-time.

export http_proxy='http://192.168.36.204:8080'
export ftp_proxy='http://192.168.36.204:8080'

I highly recommend the book “Squid Proxy Server 3.1: Beginner’s Guide (Paperback)” for further reading.

You can get our complete MB5-294 exam pass resources including our latest MB6-288 and MB7-227 training courses. Our 70-272 and 70-162 are also playing vital role in IT world.

 
by Kulbir Saini on Apache, Configuration, FFMPEG, Hacks, HowTo, Installation, LAMP, Linux, PHP, Server

How To: Install FFMPEG and FFMPEG-PHP

I was randomly browsing the internet and reading about making a website look better and I encountered ffmpeg-php. ffmpeg is a very powerful tool to record, convert and stream audio and video. Its a very rich tool almost supporting every format out there in the world. It can convert any format to any other format provided the codec. ffmpeg-php is an extension for PHP that provides a rich library to access info about audio and video files. The good thing about ffmpeg-php is that it can retrieve all info about any audio/video file subjected to the condition that the particular audio/video format is supported by your ffmpeg installation. So, now you have a clear idea that you can do wonders with audio/videos while showing them on your site 🙂

I tried some of the functionalities and they worked out of the box. Here’s is complete how to on installing ffmpeg and ffmpeg-php.

FFMPEG:

I tried installing ffmpeg from rpms provided by several Fedora repositories but after installation ffmpeg doesn’t seem to work. After several tries, I installed ffmpeg from source rpms and it worked. Below, I will describe how to install ffmpeg from source rpm.

Step 1:

Make sure that you have ‘rpmbuild’ installed by issuing

[root@bordeaux saini]# rpm -q rpmbuild [Enter]

command. If the above says that rpmbuild is not installed, then install it using yum as given below

[root@bordeaux saini]# yum install rpmbuild [Enter] (do as root)

Step 2:

Download the latest src rpm of ffmpeg from rpmfind.net. Issue the command given below

[root@bordeaux saini]# rpm -hiv ffmpeg-x.x.x.xx-xxx.src.rpm [Enter] (do as root)

Step 3:

Go to ‘/usr/src/redhat/SPECS/’ directory and issue the command given below

1
2

[root@bordeaux saini]# cd /usr/src/redhat/SPECS/ [Enter]
[root@bordeaux SPECS]# rpmbuild -ba ffmpeg.spec [Enter] (do as root)

If it gives an error like package ‘xyz’ is need by ffmpeg. Then install the package ‘xyz’ using yum as

[root@bordeaux SPECS]# yum install xyz [Enter] (do as root)

After installing the dependencies, issue the rpmbuild command ‘rpmbuild -ba ffmpeg.spec’. Now ffmpeg rpms will be build and they will be stored in ‘/usr/src/redhat/RPMS/i386/’.

Step 4:

Go the ‘/usr/src/redhat/RPMS/i386/’ (x86_64 instead of i386 if your OS is 64 bit). Install all the rpms that were built by rpmbuild.

[root@bordeaux saini]# rpm -hiv *.rpm [Enter] (do as root)

Thats it. ffmpeg is now successfully installed on your computer. Half the job is done. Now lets proceed with ffmpeg-php installation.

FFMPEG-PHP:

We will install ffmpeg-php from source bundle.

Step 1:

Make sure that ‘php-devel’ installed on your machine by issuing

[root@bordeaux saini]# rpm -q php-devel [Enter]

command. If the above command says the ‘php-devel’ is not installed, then install it using the following command.

[root@bordeaux saini]# yum install php-devel [Enter] (do as root)

Step 2:

Download the latest version of ffmpeg-php from here. Unpack the file you have downloaded.

1
2

[root@bordeaux saini]# bunzip2 -d ffmpeg-php-0.5.1.tbz2 [Enter]
[root@bordeaux saini]# tar -xvf ffmpeg-php-0.5.1.tar [Enter]

Step 3:

Issue the following command in sequence if everything goes fine.

1
2
3
4
5

[root@bordeaux saini]# cd ffmpeg-php-0.5.1 [Enter]
[root@bordeaux ffmpeg-php-0.5.1]# phpize [Enter]
[root@bordeaux ffmpeg-php-0.5.1]# ./configure [Enter]
[root@bordeaux ffmpeg-php-0.5.1]# make [Enter]
[root@bordeaux ffmpeg-php-0.5.1]# make install [Enter] (do as root)

Step 4:

Open ‘/etc/php.ini’ and add a line ‘extension=ffmpeg.so’ in the category ‘Dynamic Extensions’. For help see the image below.
FFMPEG PHP PHPini Module

Step 5:

Restart apache web server aka ‘httpd’ service by issuing the command.

[root@bordeaux saini]# service httpd restart [Enter] (do as root)

Step 6:

Write a test php file and test your ffmpeg-php installation.

phpinfo();

Save the above code in ‘info.php’ and save the file in ‘/var/www/html/’ and browse http://localhost/info.php . If you see something like this.
FFMPEG PHP Linux
Then the ffmpeg-php is successfully installed on your machine. Now you can jump into the world of video manipulation via your website.