# uname -a - Mot-clé - jessie

Debian 8 : Configure Nginx and Passenger to supercharge your PuppetMaster

Mathieu — Wed, 22 Jun 2016 21:16:00 +0200

The Puppet master comes by default with a basic WEBrick server. It allow a quick start for those that are not familiar with Puppet, but when the number of Puppet nodes grows, the performances of the default WEBrick server are going down quickly.

The Puppet documentation show how to configure Apache and Passenger to replace the default WEBrick server, but what if you have a lot of nodes ? What if you want to apply your configuration within minutes, instead of the default half-hour threshold before the agent asks the master if something changed ?

Or you may just want a fancy Nginx instead of your plain-old-reliable Apache.

Here is how.

Check your hostname

Your hostname is the base configuration for your node, you should check that it's correct, otherwise you will run into problems after Puppet installation.

# hostname -f

If everything is okay, check your hosts file

# cat /etc/hosts

If your hostname is inside your host file, carry on. Otherwise, set it.

Install Puppet and Puppetmaster

I suppose that you also need the puppet agent installed on the Puppetmaster server.

Install Puppet and Puppetmaster :

# apt-get install puppet puppetmaster

Stop the Puppetmaster :

# service puppetmaster stop

Prevent the puppetmaster from starting. Nginx will spawn on the right port instead of the WEBrick server, previously spawn by Puppetmaster service. Edit the file /etc/defaults/puppetmaster :

# Start puppetmaster on boot?
START=no

Configure the Puppet agent : edit the file /etc/puppet/puppet.conf to point your agent on the master (for instance puppetmaster.example.com).

Also, comment the two lines that are "needed for passenger", our configuration don't need them. Actually, if you keep it, it will not work.

[main]
logdir=/var/log/puppet
vardir=/var/lib/puppet
ssldir=/var/lib/puppet/ssl
rundir=/var/run/puppet
factpath=$vardir/lib/facter
prerun_command=/etc/puppet/etckeeper-commit-pre
postrun_command=/etc/puppet/etckeeper-commit-post
server=puppetmaster.example.com

[master]
# These are needed when the puppetmaster is run by passenger
# and can safely be removed if webrick is used.
#ssl_client_header = SSL_CLIENT_S_DN
#ssl_client_verify_header = SSL_CLIENT_VERIFY

[agent]
report = true

Enable your puppet agent :

# puppet agent --enable

Install Nginx and Passenger

We will install the bundle Nginx+Passenger shipped by Phusion repositories.

Add the key to your keyring :

# apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 561F9B9CAC40B2F7

The Phusion repository uses HTTPS, add HTTPS transport to APT :

# apt-get install apt-transport-https ca-certificates

Finally, install Nginx and Passenger :

# apt-get update
# apt-get install nginx-extras passenger

Configure Nginx and Puppetmaster application

Edit /etc/nginx/nginx.conf and uncomment the reference to passenger config :

    ##
    # Phusion Passenger config
    ##
    # Uncomment it if you installed passenger or passenger-enterprise
    ##

    include /etc/nginx/passenger.conf;

Create the file /etc/nginx/nginx/sites-available/puppet.conf with the following content :

server {
    listen                     8140 ssl;
    server_name                puppet puppetmaster puppetmaster.example.com;

    passenger_enabled          on;
    passenger_app_env          production;

    passenger_set_header       X-Client-Verify  $ssl_client_verify;
    passenger_set_header       X-Client-DN $ssl_client_s_dn;
    passenger_set_header       X-SSL-Subject    $ssl_client_s_dn;
    passenger_set_header       X-SSL-Issuer     $ssl_client_i_dn;

    access_log                 /var/log/nginx/puppet_access.log;
    error_log                  /var/log/nginx/puppet_error.log;

    root                       /etc/puppet/rack/public;

    ssl_certificate            /var/lib/puppet/ssl/certs/puppetmaster.example.com.pem;
    ssl_certificate_key        /var/lib/puppet/ssl/private_keys/puppetmaster.example.com.pem;
    ssl_crl                    /var/lib/puppet/ssl/ca/ca_crl.pem;
    ssl_client_certificate     /var/lib/puppet/ssl/certs/ca.pem;
    ssl_ciphers                'EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH';
    ssl_prefer_server_ciphers  on;
    ssl_verify_client          optional;
    ssl_verify_depth           1;
    ssl_session_cache          shared:SSL:128m;
    ssl_session_timeout        5m;
}

Remove the default virtual host from Nginx as we don't need it :

# rm /etc/nginx/sites-enabled/default

And enable your newly created server :

# ln -s /etc/nginx/sites-available/puppet.conf /etc/nginx/sites-enabled/puppet.conf

Before restarting Nginx, we will configure the Ruby application for Puppetmaster.

Create the directory /etc/puppet/rack and its subdirectories /etc/puppet/rack/public and /etc/puppet/rack/tmp

# mkdir -p /etc/puppet/rack/public /etc/puppet/rack/tmp

Create the file /etc/puppet/rack/config.ru with the following content :

# a config.ru, for use with every rack-compatible webserver.
# SSL needs to be handled outside this, though.

# if puppet is not in your RUBYLIB:
# $LOAD_PATH.unshift('/opt/puppet/lib')

$0 = "master"

# Set the PATH in environment variable
ENV['PATH'] = "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"

# if you want debugging:
# ARGV << "--debug"

ARGV << "--rack"

# Rack applications typically don't start as root.  Set --confdir, --vardir,
# --logdir, --rundir to prevent reading configuration from
# ~/ based pathing.
ARGV << "--confdir" << "/etc/puppet"
ARGV << "--vardir"  << "/var/lib/puppet"
ARGV << "--logdir"  << "/var/log/puppet"
ARGV << "--rundir"  << "/var/run/puppet"
#ARGV << "--codedir"  << "/etc/puppet/code"

# always_cache_features is a performance improvement and safe for a master to
# apply. This is intended to allow agents to recognize new features that may be
# delivered during catalog compilation.
ARGV << "--always_cache_features"

# NOTE: it's unfortunate that we have to use the "CommandLine" class
#  here to launch the app, but it contains some initialization logic
#  (such as triggering the parsing of the config file) that is very
#  important.  We should do something less nasty here when we've
#  gotten our API and settings initialization logic cleaned up.
#
# Also note that the "$0 = master" line up near the top here is
#  the magic that allows the CommandLine class to know that it's
#  supposed to be running master.
#
# --cprice 2012-05-22

require 'puppet/util/command_line'
# we're usually running inside a Rack::Builder.new {} block,
# therefore we need to call run *here*.
run Puppet::Util::CommandLine.new.execute

Chown the file for Puppet user :

# chown puppet:puppet /etc/puppet/rack/config.ru

And finally, restart Nginx :

# service nginx restart

Then, test you configuration by running the agent :

# puppet agent --test

Troubleshooting and errors

Error 500

Warning: Error 500 on SERVER: Internal Server Error

Read the logs at /var/log/nginx/error.log and /etc/nginx/puppet_error.log

Error 403

Warning: Error 403 on SERVER: Forbidden request: localhost(127.0.0.1) access to /node/puppetmaster.example.com [find] at :119

Check that your hostname resolves, and that your host file is clean. In particular, you should have the host name of your server on the same line than localhost :

127.0.0.1    localhost puppetmaster puppetmaster.example.com

Also check that your Puppet configuration is correct, in particular check that the two lines "required for Passenger" are commented.

Sources

Debian 8 : Limit SSH users to SFTP

Mathieu — Thu, 18 Feb 2016 10:38:00 +0100

Let’s say you want to configure a secure remote file access for you users, but you can’t use FTPS for some reasons (problems with passive mode and commercial firewalls ? Yes !). Your only secure solution is either a VPN, or a SFTP access.

SFTP is great, but it may implies giving full command line access to your end users. In order to prevent that, you could set-up a jailed SSH access with Jailkit and some bind mount, but it’s not that trivial to configure and to maintain ; and it may not work with software virtualization (Docker, LXCs…). There is a simpler solution.

The solution is : use the native chroot and limitations abilities of OpenSSH. Here is how.

Warning!

You should not configure this on your primary SSH access. By doing so, you will simply lock you out of your server.

In this article, we will set up a completely new instance of OpennSSH server, running next to the original, and handling SFTP only.

1. Setup the secondary SSH access (SFTP-only)

Create a new configuration file by copying the primary configuration :

cp /etc/ssh/sshd_config /etc/ssh/sftp_config

Now edit the file /etc/ssh/sftp_config and change the listening port (for instance 10022) :

Port 10022

Change the PID file for this new instance, set something meaningful :

PidFile /var/run/sftp.pid

Then add these lines to /etc/ssh/sftp_config :

ChrootDirectory %h
ForceCommand internal-sftp
AllowTcpForwarding no

Here is a sample of a full configuration :

# Package generated configuration file
# See the sshd_config(5) manpage for details

# What ports, IPs and protocols we listen for
Port 10022
# Use these options to restrict which interfaces/protocols sshd will bind to
#ListenAddress ::
#ListenAddress 0.0.0.0
Protocol 2
# HostKeys for protocol version 2
HostKey /etc/ssh/ssh_host_rsa_key
HostKey /etc/ssh/ssh_host_dsa_key
HostKey /etc/ssh/ssh_host_ecdsa_key
HostKey /etc/ssh/ssh_host_ed25519_key
#Privilege Separation is turned on for security
UsePrivilegeSeparation yes

PidFile /var/run/sftp.pid

# Lifetime and size of ephemeral version 1 server key
KeyRegenerationInterval 3600
ServerKeyBits 1024

# Logging
SyslogFacility AUTH
LogLevel INFO

# Authentication:
LoginGraceTime 120
PermitRootLogin no
StrictModes yes

RSAAuthentication yes
PubkeyAuthentication yes
AuthorizedKeysFile    %h/.ssh/authorized_keys

# Don't read the user's ~/.rhosts and ~/.shosts files
IgnoreRhosts yes
# For this to work you will also need host keys in /etc/ssh_known_hosts
RhostsRSAAuthentication no
# similar for protocol version 2
HostbasedAuthentication no
# Uncomment if you don't trust ~/.ssh/known_hosts for RhostsRSAAuthentication
#IgnoreUserKnownHosts yes

# To enable empty passwords, change to yes (NOT RECOMMENDED)
PermitEmptyPasswords no

# Change to yes to enable challenge-response passwords (beware issues with
# some PAM modules and threads)
ChallengeResponseAuthentication no

# Change to no to disable tunnelled clear text passwords
#PasswordAuthentication yes

# Kerberos options
#KerberosAuthentication no
#KerberosGetAFSToken no
#KerberosOrLocalPasswd yes
#KerberosTicketCleanup yes

# GSSAPI options
#GSSAPIAuthentication no
#GSSAPICleanupCredentials yes

X11Forwarding no
X11DisplayOffset 10
PrintMotd no
PrintLastLog yes
TCPKeepAlive yes
#UseLogin no

#MaxStartups 10:30:60
#Banner /etc/issue.net

# Allow client to pass locale environment variables
AcceptEnv LANG LC_*

Subsystem sftp /usr/lib/openssh/sftp-server
ChrootDirectory %h
ForceCommand internal-sftp
AllowTcpForwarding no

# Set this to 'yes' to enable PAM authentication, account processing,
# and session processing. If this is enabled, PAM authentication will
# be allowed through the ChallengeResponseAuthentication and
# PasswordAuthentication.  Depending on your PAM configuration,
# PAM authentication via ChallengeResponseAuthentication may bypass
# the setting of "PermitRootLogin without-password".
# If you just want the PAM account and session checks to run without
# PAM authentication, then enable this but set PasswordAuthentication
# and ChallengeResponseAuthentication to 'no'.
UsePAM yes

Now, let’s configure autostart. Copy /lib/systemd/system/ssh.service to /lib/systemd/system/sftp.service and adjust settings :

[Unit]
Description=OpenBSD Secure Shell server (SFTP only)
After=network.target auditd.service
ConditionPathExists=!/etc/ssh/sshd_not_to_be_run

[Service]
EnvironmentFile=-/etc/default/ssh
ExecStart=/usr/sbin/sshd -D $SSHD_OPTS -f /etc/ssh/sftp.conf
ExecReload=/bin/kill -HUP $MAINPID
KillMode=process
Restart=on-failure

[Install]
WantedBy=multi-user.target
Alias=sftp.service

And enable your service :

systemctl enable sftp.service

Make sure the symlink /etc/systemd/system/sftp.service is created.

And try to start it :

service sftp start

2. Reconfigure the primary SSH access

In order to prevent normal users to log into a full shell, we have to change the primary configuration.

The configuration file should be located in /etc/ssh/sshd_config . Add an AllowUsers or AllowGroups directive to this file :

# One or the other but not both!
AllowUsers root admin
#AllowGroups sudo

Here is a sample of a full configuration :

# Package generated configuration file
# See the sshd_config(5) manpage for details

# What ports, IPs and protocols we listen for
Port 22
# Use these options to restrict which interfaces/protocols sshd will bind to
#ListenAddress ::
#ListenAddress 0.0.0.0
Protocol 2
# HostKeys for protocol version 2
HostKey /etc/ssh/ssh_host_rsa_key
HostKey /etc/ssh/ssh_host_dsa_key
HostKey /etc/ssh/ssh_host_ecdsa_key
HostKey /etc/ssh/ssh_host_ed25519_key
#Privilege Separation is turned on for security
UsePrivilegeSeparation yes

# Lifetime and size of ephemeral version 1 server key
KeyRegenerationInterval 3600
ServerKeyBits 1024

# Logging
SyslogFacility AUTH
LogLevel INFO

# Authentication:
LoginGraceTime 120
PermitRootLogin without-password
StrictModes yes
AllowUsers root admin

RSAAuthentication yes
PubkeyAuthentication yes
AuthorizedKeysFile    %h/.ssh/authorized_keys

# Don't read the user's ~/.rhosts and ~/.shosts files
IgnoreRhosts yes
# For this to work you will also need host keys in /etc/ssh_known_hosts
RhostsRSAAuthentication no
# similar for protocol version 2
HostbasedAuthentication no
# Uncomment if you don't trust ~/.ssh/known_hosts for RhostsRSAAuthentication
#IgnoreUserKnownHosts yes

# To enable empty passwords, change to yes (NOT RECOMMENDED)
PermitEmptyPasswords no

# Change to yes to enable challenge-response passwords (beware issues with
# some PAM modules and threads)
ChallengeResponseAuthentication no

# Change to no to disable tunnelled clear text passwords
#PasswordAuthentication yes

# Kerberos options
#KerberosAuthentication no
#KerberosGetAFSToken no
#KerberosOrLocalPasswd yes
#KerberosTicketCleanup yes

# GSSAPI options
#GSSAPIAuthentication no
#GSSAPICleanupCredentials yes

X11Forwarding no
X11DisplayOffset 10
PrintMotd no
PrintLastLog yes
TCPKeepAlive yes
#UseLogin no

#MaxStartups 10:30:60
Banner /etc/issue.net

# Allow client to pass locale environment variables
AcceptEnv LANG LC_*

Subsystem sftp /usr/lib/openssh/sftp-server

UsePAM yes

Restart your primary SSH access, but don’t close your terminal afterwards :

service ssh restart

Now open a new terminal and check that your primary SSH is still working. If not, rollback your configuration.

3. Conclusion

Now you should have two SSH sockets listening : one for everyone using exclusively SFTP, and the other with full SSH access for authorized accounts.

Don’t hesitate to reply in comments if you encounter problems.

Sources

https://wiki.archlinux.org/index.php/SFTP_chroot

Set-up SQL quarantine with Amavisd-new and ISPConfig

Mathieu — Sun, 06 Dec 2015 16:56:00 +0100

It's documented, but it took me two days to do it correctly, so here is how to reconfigure an ISPConfig installation of Amavis to store quarantined mail in SQL database, in order to install a quarantine viewer like Mailzu.

1. Prerequisites

A working Postfix+Amavis stack with ISPConfig
A working SQL (PostgreSQL, MySQL...) database
Optional : a working mail server with PHP (for Mailzu)

Jump to Installation if you know what you are doing.

2. Off-subject generic explanations

2.1 Amavis and ISPConfig policies

ISPConfig policies, Tag-Levels

In a default ISPConfig installation per-user ISPConfig policies are loaded. The configuration file for Amavis, written by ISPConfig contains :

@lookup_sql_dsn =
   ( ['DBI:mysql:database=dbispconfig;host=127.0.0.1;port=3306', 'ispconfig', 'xxxx'] );
$sql_select_policy =
   'SELECT *,spamfilter_users.id'.
   ' FROM spamfilter_users LEFT JOIN spamfilter_policy ON spamfilter_users.policy_id=spamfilter_policy.id'.
   ' WHERE spamfilter_users.email IN (%k) ORDER BY spamfilter_users.priority DESC';
$sql_select_white_black_list = 'SELECT wb FROM spamfilter_wblist'.
    ' WHERE (spamfilter_wblist.rid=?) AND (spamfilter_wblist.email IN (%k))' .
    ' ORDER BY spamfilter_wblist.priority DESC';

It means that whatever you would set as $sa_spam_subject_tag, $sa_tag_level_deflt, $sa_tag2_level_deflt, $sa_kill_level_deflt, $sa_dsn_cutoff_level, it will be overridden by per-user policies.

The ISPConfig policies can be changed in tab Email => Spamfilter => Policy in ISPConfig panel. If you struggle wondering why your message keeps getting smashed at level 4.5, look at the sa_tag_level in policies. We will have to change values in that policies, to make the SQL quarantine working.

2.2 Lookup DSN and Storage DSN

DSN (Data Source Name) are the connection strings with host, username, and password, used to connect to databases.

Amavis can set two DSN : one for Policies lookup (used to retrieve ISPConfig policies from Panel), and one for storage of mail meta informations and quarantine. We will use the Storage DSN to set up a secondary database for quarantine storage, to not mess with existing ISPConfig database.

2.3 Levels and cutoffs

Amavis uses Spamassassin to score the mail, in order to decide what to do with it. The category of test (spam test, antivirus, etc) and the score along with levels determines the actions Amavis will trigger, and the final destiny where the mail belongs.

Spamassassin levels are :

tag_level : a message above that score will be tagged with X-Spam-Status, X-Spam-Score and X-Spam-Level headers.
tag2_level : a message above that score will be marked as X-Spam-Status: Yes and the subject is changed if sa_spam_modifies_subj is set to true.
kill_level : a message above that score is taken to the final_spam_destiny, and quarantined, it will not be delivered unless D_PASS is set to final_spam_destiny.
dsn_cutoff_level : a message above that level will never trigger a bounce or a reject, whatever spam_destiny is.
quarantine_cutoff_level : a message above that level will not be quarantined.

2.4 Final destinations

Once the message is categorized by Amavis tests (through SpamAssassin, ClamAV, etc), Amavis decides if it should be delivered to user mailbox or not, and if a bounce will be issued.

This is the purpose of $final_virus_destiny, $final_spam_destiny, $final_banned_destiny, $final_bad_header_destiny.

They can take the following values :

D_PASS : mail will be delivered to inbox.
D_BOUNCE : mail will not be delivered, and a delivery status notification will be returned by Postifx to sender (except if the score exceeds the dsn_cutoff level)
D_REJECT : Postfix will answer REJECT to the distant mail server, and the distant mail server may produce a delivery status notification to the user
D_DISCARD : forgive and forget : the mail will not be delivered and the sender is not informed. The mail may be quarantined if the quarantine_cutoff level is not exceeded.

3. Installation

For Amavis : Nothing ! Amavis comes out-of-the-box with SQL storage.
For Mailzu : see Mailzu section.

4. Configuration

4.1 Database

Create an user and a database for quarantine storage :

# mysql -u root -p
mysql> CREATE DATABASE amavis_storage;
mysql> CREATE USER 'amavis_storage'@'localhost' IDENTIFIED BY 'xxxx';
mysql> GRANT ALL PRIVILEGES ON amavis_storage.* TO 'amavis_storage'@'localhost';
mysql> FLUSH PRIVILEGES;

Load the initial schema from Amavis docs (usually located in /usr/share/doc/amavisd-new/ ).

Delete unnecessary tables, as we will be using this database only for mail storage and not for lookups :

# mysql -u amavis_storage -p amavis_storage
mysql> DROP TABLE users;
mysql> DROP TABLE mailaddr;
mysql> DROP TABLE policy;
mysql> DROP TABLE wblist;

Nota Bene : while executing DROP TABLE users, don't be silly, and do not remove mysql users database.

4.2 Amavis

Update your Amavis configuration /etc/amavis/conf.d/50_user :

@storage_sql_dsn = ( ['DBI:mysql:database=amavis_storage;host=127.0.0.1;port=3306', 'amavis_storage', 'xxxx'] );  # none, same, or separate database

# Quarantine SPAM into SQL server.
$spam_quarantine_to = 'spam-quarantine';
$spam_quarantine_method = 'sql:';

# Quarantine VIRUS into SQL server.
$virus_quarantine_to = 'virus-quarantine';
$virus_quarantine_method = 'sql:';

# Quarantine BANNED message into SQL server.
$banned_quarantine_to = 'banned-quarantine';
$banned_files_quarantine_method = 'sql:';

# Quarantine Bad Header message into SQL server.
$bad_header_quarantine_method = 'sql:';
$bad_header_quarantine_to = 'badheader-quarantine';

# Do not store non-quarantined messages info
# You can set it to 1 (the default) to test if Amavis is filling correctly the tables maddr, msgs, and msgcrpt
$sql_store_info_for_all_msgs = 0;

#
# SQL Select statements
#

$sql_select_policy =
   'SELECT *,spamfilter_users.id'.
   ' FROM spamfilter_users LEFT JOIN spamfilter_policy ON spamfilter_users.policy_id=spamfilter_policy.id'.
   ' WHERE spamfilter_users.email IN (%k) ORDER BY spamfilter_users.priority DESC';

$sql_select_white_black_list = 'SELECT wb FROM spamfilter_wblist'.
    ' WHERE (spamfilter_wblist.rid=?) AND (spamfilter_wblist.email IN (%k))' .
    ' ORDER BY spamfilter_wblist.priority DESC';

#
# Quarantine settings
#

$final_virus_destiny = D_BOUNCE;
$final_spam_destiny = D_DISCARD;
$final_banned_destiny = D_BOUNCE;
$final_bad_header_destiny = D_PASS;

# Default settings, we st this very high to not filter aut emails accidently
$sa_spam_subject_tag = '[SPAM] ';
$sa_tag_level_deflt  = 20.0;  # add spam info headers if at, or above that level
$sa_tag2_level_deflt = 60.0; # add 'spam detected' headers at that level
$sa_kill_level_deflt = 60.0; # triggers spam evasive actions
$sa_dsn_cutoff_level = 100;   # spam level beyond which a DSN is not sent
#$sa_debug = 1;

#
# Disable spam and virus notifications for the admin user.
# Can be overridden by the policies in mysql
#

$virus_admin = undef;
$spam_admin = undef;

#
# Enable Logging
#

$DO_SYSLOG = 1;
$LOGFILE = "/var/log/amavis.log";  # (defaults to empty, no log)

# Set the log_level to 5 for debugging
$log_level = 0;                # (defaults to 0)

4.3 ISPConfig policies

Remember that ISPconfig policies are overriding a lot of our configuration in 50_user. In order to majke the quarantine work, you have to reconfigure all the available policies in ISPConfig Panel.

Look at your policies list, you have to change the quarantine settings for every policies :

ISPConfig Mail Spamfilter Policy

When editing a policy, on the Quarantine tab, set the destinations :

ISPConfig Mail Spamfilter Policy Quarantine destinations

If you do not fill something in these fields, Amavis will not store quarantined mails in SQL database, and will just discard it !

These fields correspond to the virus_quarantine_to, spam_quarantine_to, banned_quarantine_to, bad_header_quarantine_to variables in Amavis configuration, and an empty value is overriding those we set in Amavis configuration.

4.4 Test

Send some spam to your server, check if the tables are populated :

mysql> SELECT * FROM maddr;

Check if meta informations are populated :

mysql> SELECT * FROM msgs;
mysql> SELECT * FROM msgrcpt;

And if quarantine is filling :

mysql> SELECT * FROM quarantine;

5. Cleanup !

You should not "setup and forget" your quarantine SQL storage. Messages has to be deleted periodically, otherwise your database will grow forever. Look at the documentaion in /usr/share/docs/amavisd-new to make a cronjob like this :

#!/bin/bash

SQL_HOST="localhost";
SQL_LOGIN="amavis_storage"
SQL_PASSWORD="xxxx"
SQL_DB="amavis_storage"

mysql --user="$SQL_LOGIN" --password="$SQL_PASSWORD" --host="$SQL_HOST" $SQL_DB -e " \
  DELETE FROM msgs WHERE time_num < UNIX_TIMESTAMP() - 30*24*3600; \
  DELETE FROM msgrcpt WHERE NOT EXISTS (SELECT 1 FROM msgs WHERE mail_id=msgrcpt.mail_id); \
  DELETE FROM quarantine WHERE NOT EXISTS (SELECT 1 FROM msgs WHERE mail_id=quarantine.mail_id); \
  DELETE FROM maddr WHERE NOT EXISTS (SELECT 1 FROM msgs WHERE sid=id) AND NOT EXISTS (SELECT 1 FROM msgrcpt WHERE rid=id); \
"

6. Mailzu

I have to admit, Mailzu seems a bit obsolete as I had to patch to make it working with Amavis 3.3 tables. But it still works pretty well for a simple task like reading and releasing quarantine mails.

6.1 Installation

Download the source files at http://sourceforge.net/projects/mailzu/.

62. Patch

The existing Mailzu source code is quite old, and the schema of Amavis SQL tables changed. Download and apply this patch in to make Mailzu work.

6.3 Configuration

I suppose that you know how to spawn PHP with CGI to serve the Mailzu files.

Configure your database login and password in config/config.php :

$conf['db']['dbType'] = 'mysql';
$conf['db']['dbUser'] = 'amavis_storage';
$conf['db']['dbPass'] = 'xxxx';
$conf['db']['dbName'] = 'amavis_storage';
$conf['db']['hostSpec'] = 'localhost:3306';

I am using IMAP login to authenticate in Mailzu. Unfortunately, I had to turn off SSL authentication, as it wasn't working. Here is my configuration :

$conf['auth']['serverType'] = 'imap';
$conf['auth']['imap_hosts'] = array( 'localhost:143' );
$conf['auth']['imap_type'] = 'imaptls';
$conf['auth']['imap_domain_name'] = 'example.com';

Don't forget to set yourself "super" :

$conf['auth']['s_admins'] = array ('me@example.com');

And to set your web uri :

$conf['app']['weburi'] = 'https://example.com/mailzu';

6.4 Configure in-app release

Mailzu can also release quarantined mail. I did not implement this function, but you have to set up the amavisd-release internface on an inet socket on port 9998, instead of the existing unix socket located at /var/lib/amavis/amavisd.sock . Read more.

References

Configurer dibbler-client pour IPv6 sur une Dedibox (Online.net) avec Debian 8 (Jessie)

Mathieu — Wed, 21 Oct 2015 14:24:00 +0200

La documentation d’Online pour IPv6 ne traite pas le cas de Debian, Ubuntu, ou toute distribution utilisant systemd (CentOS6, etc). La documention doit être étendue pour ajouter le service sysetmd au démarrage, en lieu et place de l’init script.

Vérifiez que l’IPv6 est activé

Cela devrait normalement être le cas, puisque le noyau par défaut de Debian 8 inclut nativement IPv6 et ne peut pas être désactivé. Mais au cas où vous ne verriez pas le link-local sur vos interfaces :

Changer dans /etc/modprobe.d/local.conf :

options ipv6 disable=0

Ajouter dans /etc/modules :

ipv6

Il faudra sans doute redémarrer pour appliquer les changments.

Récupérez votre préfixe et votre DUID depuis la console d’Online

Vous devez demander l’activation de IPv6 au support et créer votre /64 dans la console d’Online pour obtenir votre DUID.

Créez un /64 et n’utilisez pas le /48 ou le 56 directement, vous pourriez le regretter si vous souhaitez redécouper le réseau. Vous n’avez qu’un seul /48 par compte, un /56 pour chaque serveur, et un /64 pour chaque failover souscrit.

Configurez l’interface réseau

Encore une fois, c’est sans doute facultatif puisque Dibbler reconfigure l’interface lorsqu’il se lance, donc explicitons ça dans le fichier /etc/network/interfaces, juste au cas où :

iface eth0 inet6 static
    address your_ipv6_address
    netmask 64
    accept_ra 2

Notes sur Proxmox et le forwarding

Sous Proxmox on travaille sur l’interface bridge, c’est vmbr0 et non pas eth0.

Si le forwarding est activé, vous devez forcer le accept_ra à 2, une valeur de 1 fera ignorer les router advertisements lorsque le forwarding est activé. Explications. Ajoutez dans sysctl.conf :

net.ipv6.conf.vmbr0.accept_ra = 2

Notez bien que dans le cas où vous adressez vos machines virtuelles en IPv6 vous ne devez pas brancher l’interface IPv6 de vos VM directement sur l’interface vmbr0. Voilà pourquoi.

Compilez et installez Dibbler

Téléchargez Dibbler 1.0.1 à partir du site officiel
Installez build-essential : apt-get install build-essential
Décompressez Dibbler : tar -xzf dibbler-1.0.1.tar.gz && cd dibbler-1.0.1
Compilez et installez Dibbler : ./configure && ./make

Configurez Dibbler

Créez les dossiers et fichiers de configuration comme le précise la documentation :

Configurez le DUID dans /var/lib/dibbler/client-duid :

mkdir /var/lib/dibbler/
touch /var/lib/dibbler/client-duid
chmod 640 /var/lib/client-duid
# Set up you duid in client-duid
vim /var/lib/client-duid

Configurez /etc/dibbler/client.conf :

mkdir /etc/dibbler
vim /etc/dibbler/client.conf

Voici le contenu de mon client.conf :

auth-protocol reconfigure-key
auth-replay monotonic
auth-methods digest-hmac-md5
duid-type duid-ll
inactive-mode
log-level 8
iface eth0 {
    pd
    ia
}

Encore une fois, c’est à adapter en fonction du nom de votre interface.

Démarrez le client pour tester la connectivité :

dibbler-client run

Pour vérifier que Dibbler a configuré l’interface, pressez CTRL+Z pour suspendre le processus et vérifiez que l’IP et les routes sont bien configurées. Tapez la commande “fg” pour retourner au processus en cours d’exécution, et tapez CTRL+C pour stopper Dibbler.

Si pendant l’opération les routes et les IPs sont incorrectes, vérifiez que votre pare feu accepte les connexions entrantes par le port 546 UDP.

Configurez Dibbler au démarrage

Cette section est différente de la documentation d’Online.

Avant systemd, le système d’init par dépendances se contentait d’un script dans /etc/init.d/. Avec systemd, il faut créer un fichier service dans /etc/systemd/system/ .

Créez le fichier suivant : /etc/systemd/system/dibbler.service :

[Unit]
Description=Dibbler
After=network.target

[Service]
Type=simple
ExecStart=/usr/local/sbin/dibbler-client start
ExecStop=/usr/local/sbin/dibbler-client stop
PrivateTmp=true
NonBlocking=yes

[Install]
WantedBy=multi-user.target

Lancez la commande suivante pour que systemd lise le fichier :

systemctl daemon-reload

Et activez le service :

systemctl enable dibbler.service

Et essayez de le lancer pour la première fois :

systemctl start dibbler.service

Vérifions que tout est correct :

service dibbler status
ifconfig
route -6
ping6 whatever-you-want.com

Si tout va bien, redémarrez et l’IPv6 fonctionnera dès le démarrage !

Addendum : juste au cas où, n’utilisez pas resolvconf pour pousser les DNS IPv6 automatiquement, conservez autant que possible vos DNS IPv4. Ça serait tellement dommage que votre connectivité IPv6 saute et que vous vous retrouviez sans DNS (oui ça m’est arrivé…).

Sources

Configure dibbler-client for IPV6 networking on Dedibox (or any Online.net) servers with Debian 8 (Jessie)

Mathieu — Wed, 21 Oct 2015 12:50:00 +0200

The Online documentation for IPv6 is not dealing with the case of Debian 8, Ubuntu, or any distribution using systemd. Systemd replaces upstart, so the procedure has to be extended to add systemd service for startup, replacing the previoux behavior that was using init scripts.

Ensure that you are IPv6-proof

It should be the case as Debian 8 is shipping a kernel with native IPv6, but just to be sure :

In /etc/modprobe.d/local.conf :

options ipv6 disable=0

In /etc/modules :

ipv6

You may have to reboot in order to apply the changes.

Set up your IPv6 prefix and get your DUID on Online console

You have to request IPv6 activation to support and create your /64 on Online console before getting your DUID working.

Make a /64 and do not use your /48 or your /56 directly, as you may regret it. You can have only one /48 by account, one /56 by server, and one /64 by IP failover (the /48 is divided to make the /56 and so on).

Configure your network interface

It may not be mandatory as Dibbler will reconfigure your interface, but you have to ensure that you accept router advertisements. Add to /etc/network/interfaces :

iface eth0 inet6 static
    address your_ipv6_address
    netmask 64
    accept_ra 2

Notes about Proxmox and forwarding

On Proxmox, you are working on the bridge interface, it should be vmbr0 instead of eth0.

If you enabled forwarding on this interface (to give your VM an access to IPv6 network), you have to force the accept_ra to 2, while the default value of 1 wil make your Debian to ignore router advertisements when forwarding is enabled ! Read more. Add to sysctl.conf :

net.ipv6.conf.vmbr0.accept_ra = 2

Also do not set the IPv6 interface of your VM to vmbr0, as you can break your network access. Read more.

Compile and install Dibbler

Download Dibbler 1.0.1 from the offical website.
Install build-essential : apt-get install build-essential
Extract Dibbler : tar -xzf dibbler-1.0.1.tar.gz && cd dibbler-1.0.1
Compile and install Dibbler : ./configure && ./make

Configure Dibbler

Make the directories and set up according to the documentation :

Set up duid :

mkdir /var/lib/dibbler/
touch /var/lib/dibbler/client-duid
chmod 640 /var/lib/client-duid
# Set up you duid in client-duid
vim /var/lib/client-duid

Set up client.conf :

mkdir /etc/dibbler
vim /etc/dibbler/client.conf

The content of my client.conf :

auth-protocol reconfigure-key
auth-replay monotonic
auth-methods digest-hmac-md5
duid-type duid-ll
inactive-mode
log-level 8
iface eth0 {
    pd
    ia
}

Note that if your interface name is different you have to change it accordingly. For instance, on a Proxmox host server, it should be vmbr0 instead of eth0.

Start dibbler to try the connectivity :

dibbler-client run

Hit CTRL+Z to suspend the process and check that the IP and routes are configured. Type the command “fg” to get back to the running process and hit CTRL+C to stop it.

If it doesn’t work, check your firewall, dibbler needs to listen port 546 UDP.

Set Dibbler at startup

This section differs from Online documentation.

Before systemd, the dependency-based boot sequence only needed an init script, for instance /etc/init.d/dibbler . With systemd, you have to make a service file in /etc/systemd/system/ .

Make the following file : /etc/systemd/system/dibbler.service :

[Unit]
Description=Dibbler
After=network.target

[Service]
Type=simple
ExecStart=/usr/local/sbin/dibbler-client start
ExecStop=/usr/local/sbin/dibbler-client stop
PrivateTmp=true
NonBlocking=yes

[Install]
WantedBy=multi-user.target

Run the following command to make systemd read file :

systemctl daemon-reload

Enable the service :

systemctl enable dibbler.service

And then, try to start it :

systemctl start dibbler.service

Check everything is fine :

service dibbler status
ifconfig
route -6
ping6 whatever-you-want.com

Reboot and the IPv6 networking should work at startup !

Addendum : just in case, don’t use resolvconf with IPv6 DNS pushing and if possible keep IPv4 DNS, it would be such a pity if the IPv6 networking crashes while your DNS servers are set to IPv6 addresses (yeah, it happened to to me)…

Sources

Migrate an OpenVPN configuration to Debian 8 (Jessie) with systemd

Mathieu — Sat, 23 May 2015 10:32:00 +0200

This article could have been avoided if the Debian documentation was up-to-date. Actually it is not, and the solution came from Fedora documentation for OpenVPN.

Debian 8 uses systemd by default, and it implies several changes, in particular the way you start/stop your services.

The main topic : systemd

What changes

A new fancy command now manage the startup : systemctl (don't mess with the sysctl command used for network configuration !)
The startup dependencies are no longer in the LSB headers in startup scripts (way too simple, boy), the dependencies are stored as symlinks in subdirectories located in /etc/systemd/* . Note that /etc/systemd/ contains some static configuration files, and that the real services configuration files are stored in /lib/systemd/* (this is where the symlinks from /etc/systemd/* points to).
Some services have been split from monolithic startup to dynamic. It means that you potentially have to enable and run multiple "services" in order to actually start the full "service". For instance, OpenVPN no longer runs every available configuration in /etc/openvpn/*.conf , you have to explicitely activate each *.conf file as a service in systemd !

What doesn't change

You can still start your services with service servicename start.
The init scripts in /etc/init.d/* still exists, and some are still usable (monolithics services).

Run your OpenVPN configuration

While you can still use the command openvpn --config /etc/openvpn/yourconfigfile.conf, you should do it with systemd. If your configuration file is /etc/openvpn/sample.conf, you should start your VPN connexion with systemctl start openvpn@sample.service .

Note that service openvpn@sample start also works.

Start your VPN at boot

Again, the auto startup was too simple. You now have to enable every *.conf file at boot. Enable you newly sample.conf at startup with the command systemctl enable openvpn@sample.service

This actually creates a symlink in /etc/systemd/system/multi-user.target.wants/openvpn@sample.service pointing to /lib/systemd/system/openvpn@.service

Ok, it's simpler for dynamic loads, but who needs to dynamically enable and disable configuration at boot ? If I want a different configuration, I simply write different files in the right folder...

Meanwhile, in Debian Apache package

You can enable and disable VirtualHosts by using the /etc/apache2/sites-available/ and /etc/apache2/sites-enabled/ folders. No hidden features, no boot startup configuration, all configuration files in /etc/programname/ . Way too simple ?

Hey, what if we had to configure a startup script for every VirtualHost ? Should be fun, don't you think ?