Configure sender rate limits to prevent spam, using cluebringer (policyd) with Postfix
Par Mathieu le vendredi 13 mars 2015, 09:18 - Hacks - Lien permanent
This small how-to will show you how to configure cluebringer (aka policyd) to set a per-hour/per-user limit for sent mails. Note that sending to multiple recipient will count like multiple mails were sent.
This how-to is Debian-oriented but should apply to any unix operating system.
Requirements
A mail server with Postfix installed.
Installation
Install a DBMS (MySQL for instance), cluebringer, and cluebringer-webui :
apt-get install mysql-server cluebringer cluebringer-mysql cluebringer-webui
Note that cluebringer-webui will install apache as a dependency if you don’t already have a webserver.
Set-up the Cluebringer database
Get the initial database schema that correspond to your DBMS, for instance mysql :
cp /usr/share/doc/postfix-cluebringer/database/policyd-db.mysql.gz ~/ && gunzip ~/policyd-db.mysql.gz
Create the database, and populate it with the initial dump :
# cd ~/ && mysql -u root -p mysql> CREATE DATABASE cluebringer; mysql> CREATE USER 'cluebringer'@'localhost' IDENTIFIED BY 'mypassword'; mysql> GRANT ALL PRIVILEGES ON cluebringer.* TO 'cluebringer'@'localhost'; mysql> \. policyd-db.mysql mysql> quit mysql> Bye
Note that on Debian I had to modify the dump to make it work, TYPE=InnoDB was rejected by MySQL as an invalid syntax.
Configure Cluebringer
Add your DBMS credentials to the file /etc/cluebringer/cluebringer.conf :
DSN=DBI:mysql:dbname=cluebringer;host=localhost DB_Type=mysql DB_Host=localhost DB_Port=3306 DB_Name=cluebringer Username=cluebringer Password=mypassword
And start it :
service postfix-cluebringer start
Configure Cluebringer webui
Configure the file /etc/cluebringer/cluebringer-webui.conf with your DBMS credentials :
<?php $DB_DSN="mysql:host=localhost;dbname=cluebringer"; $DB_USER="cluebringer"; $DB_PASS="mypassword";
Cluebringer Webui needs a web server to run. Copy the sample configuration from the package documentation :
cp /usr/share/doc/postfix-cluebringer-webui/examples/httpd/cluebringer-httpd.conf /etc/apache2/conf.d/
Restart Apache :
service apache2 restart
You may need to adjust a few things to access it from the outside. If you a really lazy, just make a ssh tunnel to access the webserver from localhost :
ssh -L 8008:localhost:80 mylogin@mymailserver
Don’t forget : you have to make this tunnel from the outside, do not run this command on server, it won’t work.
You should now be able to open http://localhost:8080/ and see your fresh new Cluebinger Webui !
Configure Cluebringer using its webui
Add a policy
Under Policies -> Main, disable Test policy (select policy and choose Action -> Change and switch Disabled to yes, validate)
Add a new policy : Action -> Add, give it a name and a description
Activate your new policy : select policy and choose Action -> Change (switch Disabled to no)
Add a new member to your policy : select it and choose Action -> Members, and then Action -> Add. Specify any as source and any as destination.
Go back to your policy, choose Action -> Members, and the select your member, do Action -> Change, and activate your new member (switch Disabled to no).
Add a quota
Under Quotas -> Configure, disable Test quotas.
Add a new quota : Choose Action -> Add
- Name : whatever you want
- Track : user@domain
- Period (seconds) : 3600
- Link to policy : specify the policy you created here
- Verdict : Defer
- Data : set a custom error message here
- Comment : whatever you want
Activate your quota : switch Disabled to no.
Add a limit to your quota : select your quota, and choose Action -> Limits, then Action -> Add.
- Type : MessageCount
- Counter Limit : 200
Activate your limit : switch Disabled to no.
Configure Postfix to call Cluebringer for each mail sent
Open /etc/postfix/main.cf and locate the line smtpd_sender_restrictions.
Add check_policy_service inet:127.0.0.1:10031 at the end of the line, for instance :
smtpd_sender_restrictions = check_sender_access mysql:/etc/postfix/mysql-virtual_sender.cf, check_policy_service inet:127.0.0.1:10031
If the line does not exists, simply add it.
Don’t forget to restart Postfix :
service postfix restart
Check your config
You can now send some mails to see what happens. To check if these mails are passed to Cluebringer, connect to MySQL as the cluebringer user :
# mysql -u cluebringer -p cluebringer
And execute the query :
mysql> SELECT * FROM quotas_tracking;
You should see the value LastUpdate and Counter updating when sending a mail. Note that sending to multiple recipient will count like multiple mails were sent.
Pitfalls, bleeding edges, etc
Cluebringer versions prior to 2.1.x does not support IPv6, your customers won’t be able to send any mail if they have an IPv6 connection.
Unfortunately, the Debian stable version (wheezy) provides Cluebringer 2.0.10 within its repositories, as well as the experimental release of Debian (sid). As an alternative, you should consider installing the 2.1.x experimental Cluebringer from official website instead of Debian packages from repositories.
Commentaires
I have a question: In all other sites explaining the use of this policy with policyd, they ask to put the directive check_policy service within smptd_recipient_restrictions and you are the only one that instructs to put it into smptpd_sender_restrictions. Why? I have tried in both and It doesn’t work, what am I missing?
Hi,
Using smtpd_recipient_restriction will filter the incoming mails (but not the outgoing), by doing that you may filter the incoming spam (if the spammer is using the same “From” address every time, which is pretty unlikely), but not the outgoing mail. It’s not what I wanted, and my configuration may not be what you are looking for. It should be possible to use Policyd to filter out incoming spam, but not with such simple filters and policies. To filter the incoming spam, it may be useful to make a filter based on the IP address.
While searching how to restrict outgoing mails, I actually read a lot of articles talking about smtpd_client_restrictions. I am using smtpd_sender_restrictions instead of smtpd_client_restrictions because I wanted to be very specific to SASL authenticated outgoing users (that’s why i chain my policy after check_sender_access). Because smtpd_client_restrictions applies to all clients, i didn’t want the filter to apply to incoming mails (I guess incoming mail servers are also “clients” as they are sending data on port 25, but I may be wrong). One of the other benefit is that local users (like mail sent by cronjobs) will not be concerned by this restriction, as they are “clients” because they talk to the MTA, they can send mail because they are in $mynetwork (actually localhost), but they are not “sasl authenticated” clients so the “smtpd_sender_restrictions” would not apply to them. It may be seen as a downside, thought.
As a conclusion, I finally abandonned Cluebringer to filter the volume of outgoing mails, because it was unavailable for Debian and complicated to maintain, now I use a custom policy script that does exactly what I want : say “stop” for an authenticated client that is trying to send an unusual volume of mail : https://github.com/mpellegrin/ratelimit-policyd . My use case is : block an hosted account whose password has been stolen (or guessed because too simple) by a spammer.
I hope my answer helps you.
Hello,
Very good article, exactly what I’m looking for :)
But you lost me with /etc/postfix/mysql-virtual_sender.cf ! What must this file contain?
I’ll try with :
user = cluebringer
password = mypassword
hosts = 127.0.0.1
dbname = cluebringer
no luck
Hello and thank you for your comment.
As I said in the previous comment, I do not recommend cluebringer anymore, and at that time I am using a custom script to throttle the mail rates. However, here are some explanations.
The file /etc/postfix/mysql-virtual_sender.cf is part of the check that verifies that the user is allowed to send mail. In my configuration it indicates that the file contains the MySQL credential to access the users list stored in a mysql database (because of the prefix “mysql:”). It could have been a plain text file (“hash:”), the return value of a command (“unix:”), etc. This file does not contains the cluebringer credentials, it contains references to your users database (either plain file or mysql table, or ldap…).
As you can see, it depends on your setup, the configuration explained here is must not be seen as a configuration “from scratch”.
For instance, here is the table schema I am using for storing my users (from ISPConfig) :
and the corresponding content in mysql-virtual_sender.cf :
To send mail, you have the following workflow :
You can see that the checks are chained, if it passes, it does the next check, if it fails, it aborts. That’s why the checks are written sequentially :
smtpd_sender_restrictions = check_sender_access … , check_policy_service … , check_something …
Hope it helps.
I followed this guide but when I tried to sent email I can't the error is Recipient address rejected: Access denied .
what is the problem?
Hello,
Either you messed up your cluebringer Rules (outgoing/incoming), or your Postfix configuration was not working before adding cluebringer.
You have this error while sending the mail from your server, or when sending to your server ?
What is in your directives mydestination, virtual_mailbox_domains, smtpd_recipient_restrictions, smtpd_client_restrictions, relay_domains, etc ?
Hello,
is it possible in ratelimit-policyd to set different limits per SASL login? Let's say that 95% of users doesn't send more than 50 e-mails per hour but there are some that do send mass mailing. Is it possible to set higher limit for them? I tried to manually change quota in database to higher value but it's reset to default one every hour :(
Regards,
Marcin
Oh, there's "persist" collumn which does exactly what i need :)