How To: Configure Procmailrc to Reduce Spam

Last Monday in IIIT Linux Users Group (LUG) meeting, I gave a small presentation regarding how to configure .procmailrc to make very effective filters.

Procmail is a mail delivery agent or mail filter which is widely used on Unix systems to process incoming mails. It is automatically invoked by the mail transport agents like Sendmail whenever there is an incoming mail. Procmail has the power to process all the incoming mails based on the recipes provided by the user and deliver them to the provided destination(either a mail folder or email id or something else like a file or stdout and many more).

Procmail by default searches for a configuration file named .procmailrc in user’s home directory. All the recipes, global variables and other things are provided here by the user to let Procmail know what to do.

Here is an example .procmailrc

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
# .procmailrc 
 
PATH = $PATH
MAILDIR = $HOME/mail
DEFAULT = $HOME/mbox
SHELL = /bin/bash
 
# Backup for testing mode.
#:0 c # Uncomment for testing mode
#Backup
 
:0: # Spam mails should go to Spam folder
* ^(From|Cc|To).*(hi5.com|auctionit|newegg|voilin|mingle)
Spam
 
:0: # Spam mails should go to Spam folder
* ^Subject:.*(Goonj|Spam|Disarmed|Pictures|Re\. Pictures|Sperm|Penis|Viagra|Filename|voilin)
Spam
 
:0: # Mail from Fedora mailing list should go to Fedora
* ^(From|Cc|To).*(fedora-devel|fedorawiki-noreply|bugzilla)
Fedora
 
:0: # Mail from yum mailing list should go to Yum
* ^(From|Cc|To).*yum-devel*
Yum
 
:0: # Mail from/to lug should go to LUG folder
* ^(From|Cc|To).*lug@students.iiit.ac.in
LUG
 
:0: # Lost found mails should go to LostFound folder
* ^Subject:.*(lost|found)
LostFound
 
:0: # Mail from/to life should go to Life folder
* ^(From|Cc|To).*life@students*
Life
 
:0: # Birc mails should go to BIRC folder
* ^(From|Cc|To).*birc@students*
${HOME}/mbox
 
:0: # Mail from/to course should go to Courses folder
* ^(From|Cc|To).*(ec5303|cs3600|cs3150|cs3350|cs3155|cs4460|cs4110)
{
	:0 c
	! kulbirsaini25@gmail.com
 
	:0:
	Courses
}
 
:0: # Mail from/to clubs should go to Clubs folder
* ^(From|Cc|To).*(agents|campusgreen|campusgreenclub|cybergames|dpscm|guitar|music|nss|photography|quizzers|signet|sigops|videography|movie)
Clubs
 
:0: # Mail from/to clubs should go to Clubs folder
* ^Subject:.*(agents|campusgreen|dance|cybergames|dpscm|guitar|music|nss|photography|quizzers|signet|sigops|videography|movie)
Clubs
 
:0: # House mails should go to House folder
* ^Subject:.*(IBCT|House|Tournament|Champion|championship|Inter\ House|chess|cultural|basket|cricket|foot|ball|Physical|PEC|carrom|dumb|TT|IHFT)
House
 
:0: # Mails from Physical Education Center
* ^(From|Cc|To).*pec@iiit.ac.in
House
 
:0: # Returned mail transcriptions to Bounced folder
* ^Subject:.*(Returned mail: see transcript for details|could not deliver mail|bounced|could not send message for past)
Trash
 
:0: # Mails from TopCoder should go to TopCoder folder
* ^(From|Cc|To).*topcoder*
Trash
 
:0: # Default
* ^(From|Cc|To).*
{
	:0 c
	! kulbirsaini25@gmail.com
 
	:0:
	${HOME}/mbox
}

The top few lines are global variables which you need to declare so that Procmail can detect your default mailbox and mail folders, path etc.

These are configured in accordance with the Students mail server at IIIT-H. These global variable declarations are followed by the recipes which guide Procmail to process the incoming mails.

The usual syntax of a recipe is

1
2
3
:0 [flags] [: [lock-file]]
zero or more conditions
one action line or nested actions

Lets start with the conditions line with second recipe from above image. ‘*’ specifies the start of the action line. This ‘*’ is followed by a regular expression which Procmail egreps in the header by default. ‘^’ in regular expression species the start of the line. Then all the mail which are from/to/cced to mail ids which contain auctionit or newegg or violin . e.g. newegg@newegg.com. The condition can span only one line. You can’t write comment in the action line otherwise Procmail will treat it as a part of regular expression. Anywhere else all the characters that follows a ‘#’ in a line are treated as comments or are ignored by the Procmail while processing mails.

Then comes the action line. There can only be action line per recipe unless and until its not nested. Action line may be just a mail folder name or path (relative or absolute). Spam means that all the mails satisfying the regular expression in condition line will be delivered to the Spam mail folder.

The action lines can be nested as in the last recipe in the above image. Procmail can support any level of nesting but the nesting should be proper.

The action line may also be used to forward mails to some other email id. ‘!’ is used in starting of the line followed by the email id to forward the mail.

Now some tips about the first line in the recipe. ‘:0’ is must. But optional flags may be specified. The second ‘:’ asks Procmail to use a lock-file. The need of lock-file is because if your mail account is being swamped with a lot of mails. The Sendmail invokes one copy of Procmail per incoming mail. In that case if two or more Procmail processes try to write the same mail folder, there will be conflicts. So, using the second ‘:’ protect the same mail folder from being written by the two different Procmail processes.

Procmail processes .procmailrc in top-down fashion and stops whenever it finds a matching regular expression in any of the recipes. But you may make it work further buy using the flag ‘c’ as in last recipe. If flag ‘c’ is specified, it will create a carbon copy and give it to the both recipes.

I think thats enough. If you want to explore Procmail more, read man page ‘procmailrc’ and for example procmailrcs read ‘procmailex’ man page.

You can use my procmailrc if you are interested. Find it here.

 

Hack: Mail Filter – Shell Script

This is an attempt by me to filter mails on a Linux machine. You can run this file on your home directory at mail server and it will help you to sort mails on the basis of

  • From which user the mail has come or from which domain it came,
  • to which user or domain mail was sent and
  • the keywords in the subject part of the mail.

Actually its very slow as shell is very slow , I cant help it . It scan your entire mbox file in the home directory by default. Copy the code and paste in a file say filter.sh and change its permissions to executable. Place it in you home directory at the mail server and run.

Example
You want to see all mails from username@gmail.com just run and it will ask for input,
press 1
From: username [Enter]
It will display all the mails from that are lying there in your mbox one by one.

Get the script here or copy the code below.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
#!/bin/bash
#################################
#   Author - Kulbir Saini       #
#   Home - http://sain.ico.in/  #
#################################
 
#shell script to filter mails.
echo -e -n "\033[32mWelcome to the mail filter system \nWhat do you want to do? \nCheck mails on basis of \n1) Sender... \n2) Receiver... \n3) Subject... \n4) Quit... \nPress(1,2,3,4): \033[0m"
read int <&2
while [[ 1 ]];do
while [[ 1 ]];do
count=0;	line=""
if [[ $int == 1 ]];then
	echo -n -e "\033[32mFrom:\033[0m "
	read from <&2
	i=0
	clear
	while read line ;do
	loopbreak=0;	nextmessagestatus=1;	mailtostatus=1;		mailsubstatus=1;	stat=1;
	toecho=`echo $line | grep -E "^From:+" | grep -E "${from}"`
	if [ "$toecho" != "" ];then
		i=`expr $i + 1`
		echo -e "${toecho}"
		while read line ; do
		mailto=`echo $line | grep -E "^To:+"`
		mailsub=`echo $line | grep -E "^Subject:+"`
		if [[ "$mailto" != "" ]] && [[ $mailtostatus == 1 ]];then
			echo $mailto
			mailtostatus=0
		fi
		if [[ "$mailsub" != "" ]] && [[ $mailsubstatus == 1 ]];then
			echo $mailsub
			mailsubstatus=0
		fi
		if [[ $mailtostatus == 0 ]] && [[ $mailsubstatus == 0 ]] && [[ $stat == 1 ]];then
			echo -e "\033[33mStarting of the Message${i}:\033[0m"
			stat=0
		fi
		fromstatus=`echo $line | grep -E "^X-UID:+"`
			if [[ $fromstatus != "" ]];then
			while read line; do
				loopstatus=`echo $line | grep -E "^From +"`
				if [[ $loopstatus != "" ]];then
				loopbreak=1
				break
				fi
				echo $line
			done
			if [[ $loopbreak == 1 ]];then
			echo -e -n "\033[33mEnd of the Message${i}.\nDisplay next message(y/n):\033[0m"
			read messagestatus <&2
			if [[ "$messagestatus" == "y" ]];then
				clear
			else
				nextmessagestatus=0
			fi
			break
			fi
			fi
		done
		count=`expr $count + 1`
	fi
	if [[ $nextmessagestatus == 0 ]];then
		break
	fi
	done
	if [[ $count == 0 ]];then
		echo -e "\033[32mNo mail(s) From: ${from}.\n\033[0m"
	else
		echo -e "\033[32mTotal $count mail(s) From: ${from}.\n\033[0m"
	fi
elif [[ $int == 2 ]];then
	echo -n -e "\033[32mTo: \033[0m"
	read to <&2
	i=0
	clear
	while read line ;do
	loopbreak=0;	nextmessagestatus=1
	toecho=`echo $line | grep -E "^To:+" | grep -E "${to}"`
	if [[ "$toecho" != "" ]];then
		echo -e "$toecho"
		i=`expr $i + 1`
		while read line; do
			tostatus=`echo $line | grep -E "^X-UID:+"`
			if [[ $tostatus != "" ]];then
			echo -e "\033[33mStarting of the Message${i}:\033[0m"
			while read line; do
				loopstatus=`echo $line | grep -E "^From +"`
				if [[ $loopstatus != "" ]];then
				loopbreak=1
				break
				fi
				echo $line
			done
			if [[ $loopbreak == 1 ]];then
			echo -e -n "\033[33mEnd of the Message${i}. \nDisplay next message(y/n): \033[0m"
			read messagestatus <&2
			if [[ $messagestatus == "y" ]];then
				clear
			else
				nextmessagestatus=0
			fi
			break
			fi
			fi
		done
		count=`expr $count + 1`
	fi
	if [[ $nextmessagestatus == 0 ]];then
		break
	fi
	done
	if [[ $count == 0 ]];then
		echo -e "\033[32mNo mail(s) to: ${to}.\n\033[0m"
	else 
		echo -e "\033[32mTotal $count mail(s) To: ${to}.\n\033[0m"
	fi
elif [[ $int == 3 ]];then
	echo -n -e "\033[32mSubject:\033[0m "
 
	read subject <&2
	clear
	i=0
	while read line ;do
	loopbreak=0;	nextmessagestatus=1
	toecho=`echo $line | grep -E "^Subject:+" | grep -E "${subject}"`
	if [[ "$toecho" != "" ]];then
		echo "$toecho"
		i=`expr $i + 1`
		while read line; do
		substatus=`echo $line | grep -E "^X-UID:+"`
		if [[ $substatus != "" ]];then
		echo -e "\033[33mStarting of the Message${i}:\033[0m"
		while read line ; do
			loopstatus=`echo $line | grep -E "^From +"`
			if [[ $loopstatus != "" ]];then
			loopbreak=1
			break
			fi
			echo $line
		done
		if [[ $loopbreak == 1 ]];then
		echo -e -n "\033[33mEnd of the message${i}. \nDisplay next message(y/n): \033[0m"
		read messagestatus <&2
		if [[ $messagestatus == "y" ]];then
			clear
		else
			nextmessagestatus=0
		fi
		break
		fi
		fi
		done
		count=`expr $count + 1`
	fi
	if [[ $nextmessagestatus == 0 ]];then
		break
	fi
	done
	if [[ $count == 0 ]];then
		echo -e "\033[32mNo mail(s) with Subject: ${subject}.\n\033[0m"
	else
		echo -e "\033[32mTotal $count mail(s) with Subject: ${subject}.\033[0m\n"
	fi
elif [[ $int == 4 ]];then
	echo -e "\033[32mThank you for using this utility. \nPlease visit again.\033[0m"
	exit
else
	echo -e "\033[32mI could not understand that. \nPlease try again.\033[0m\n"
fi
echo -e -n "\033[32m1) Sender... \n2) Receiver... \n3) Subject... \n4) Quit... \nPress(1,2,3,4):\033[0m "
read int <&2
break
done < mbox 
done