This document describes incompatible changes introduced in recent Rspamd versions and details how to update your rules and configuration accordingly.
Due to implementation of the new milter interface, there is an important incompatible change that you might need to handle if you use rmilter_headers module. This module has been renamed to milter_headers and the according protocol section is now named milter instead of rmilter. If you configured this module inside local.d/rmilter_headers.conf or in override.d/rmilter_headers.conf then you don’t need to undertake any actions: these files are still loaded by the renamed module. Otherwise, you need to change section name from rmilter_headers to milter_headers.
Milter_headers module now skips adding headers for local networks & authenticated users by default; this can be re-enabled by setting skip_local = false and/or skip_authenticated = false in the module configuration; or alternatively you could set authenticated_headers and/or local_headers to a list of headers that should not be skipped.
Proxy worker has been added to the default configuration and listens on all interfaces on TCP port 11332. If you don’t need it you can set enabled = false in local.d/worker-proxy.inc.
This release removes the config split for systemd/sysv platforms. If you have custom init scripts you should ensure that these use rspamd.conf rather than rspamd.sysvinit.conf. If you use systemd and prefer to log to the systemd journal, you should add the following to local.d/logging.inc:
systemd = true;
type = "console";
A major rework of lua libraries has taken place in Rspamd 1.6. Some of the custom scripts might be broken if they are loaded before rspamd.lua or if you have edited rspamd.lua manually. To ensure that everything is fine you need to load vendor rspamd.lua before all of your custom scripts. It is a default behaviour, however, in some highly custiomised setups it might cause issues. In general, you need to ensure that the following line is somewhere in your code (it is at the very beginning of rspamd.lua):
require "global_functions" ()
Rmilter tool is now deprecated in honor of milter protocol support in rspamd proxy. There are examples of some particular features that were previously implemented in Rmilter in milter headers module. You should consider migrating from Rmilter as soon as possible since Rspamd 1.6 will be the last version that supports Rmilter tool. In future major releases (starting from 1.7), there are absolutely no guarantees of compatibility with Rmilter.
For example, if you need the old behaviour for extended_spam_headers in Rmilter, then you can use the following snippet added to the local.d/milter_headers.conf:
# local.d/milter_headers.conf
extended_spam_headers = true;
The rspamd_update module has been disabled by default; if you need it please set enabled = true in local.d/rspamd_update.conf.
New configuration files have been added for the following modules which previously missed them; if you have previously configured one of these modules in rspamd.conf.local please move your configuration to rspamd.conf.override to ensure that it is preserved verbatim or rework your configuration to use local.d/[module_name].conf instead.
enabled = true if you use local.d)New configuration files have been added for the following modules which previously missed them: greylist, metadata_exporter and metric_exporter. If you have previously configured one of these modules in rspamd.conf.local please move your configuration to rspamd.conf.override to ensure that it is preserved verbatim or rework your configuration to use local.d/[module_name].conf instead.
If you have composites defined in local.d/composites.conf or override.d/composites.conf these will need to be moved to rspamd.conf.local or reworked to the new format, see /etc/rspamd/composites.conf for an example.
You are also suggested to disable outdated and no longer supported features of Rmilter and switch them to Rspamd:
The list of features includes the following ones:
All duplicating features are still kept in Rmilter for compatibility reasons. However, no further development or bug fixes will likely be done for them.
From version 1.9.1 it is possible to specify enable option in greylisting and ratelimit sections. It is also possible for dkim section since 1.9.2. These options are true by default. Here is an example of configuration where greylisting and ratelimit are disabled:
# /etc/rmilter.conf.local
limits {
enable = false;
}
greylisting {
enable = false;
}
dkim {
enable = false;
}
These options are in their default enabled states merely for compatibility purposes. In future Rmilter releases, they will be DISABLED by default.
The default passwords, namely q1 and q2 are no longer allowed to be used for remote authentication. This is done due to many misusages of these example passwords and dangerous security flaws introduced by some Rspamd users.
Systemd socket activation has been removed in these releases. Rmilter may not restart correctly on upgrade on Debian platforms. Please run systemctl restart rmilter after installing the package if necessary. Rspamd is expected to restart correctly on upgrade. Both Rspamd & Rmilter should be automatically configured to run on reboot post-upgrade.
There are couple of things that are no longer supported:
beanstalk section from the configuration fileawl options from the greylisting sectionIf you have used beanstalk for some purposes then you could move to Redis pub/sub. There are settings for sending spam (spam_servers and spam_channel) and for sending messages copies (copy_servers, copy_prob and copy_channel) in the redis section that allow you to reproduce beanstalk functions using Redis.
Rmilter now supports configuration override from rmilter.conf.local and from rmilter.conf.d/*.conf files. You should consider using these methods for your local configuration options.
Rmilter no longer adds several SpamAssassin-compatible headers: namely X-Spam-Status, X-Spam-Level and X-Spamd-Bar. Support has been added for adding/removing custom headers under instruction of Rspamd (Requires Rspamd 1.3.0+). Example script which restores the removed headers is shown below (to be added to /etc/rspamd/rspamd.local.lua):
rspamd_config:register_symbol({
name = 'RMILTER_HEADERS',
type = 'postfilter',
priority = 10,
callback = function(task)
local metric_score = task:get_metric_score('default')
local score = metric_score[1]
local required_score = metric_score[2]
-- X-Spamd-Bar & X-Spam-Level
local spambar
local spamlevel = ''
if score <= -1 then
spambar = string.rep('-', score*-1)
elseif score >= 1 then
spambar = string.rep('+', score)
spamlevel = string.rep('*', score)
else
spambar = '/'
end
-- X-Spam-Status
local is_spam
local spamstatus
local action = task:get_metric_action('default')
if action ~= 'no action' and action ~= 'greylist' then
is_spam = 'Yes'
else
is_spam = 'No'
end
spamstatus = is_spam .. ', score=' .. string.format('%.2f', score)
-- Add headers
task:set_rmilter_reply({
add_headers = {
['X-Spamd-Bar'] = spambar,
['X-Spam-Level'] = spamlevel,
['X-Spam-Status'] = spamstatus
},
remove_headers = {
['X-Spamd-Bar'] = 1,
['X-Spam-Level'] = 1,
['X-Spam-Status'] = 1
}
})
end
})
There are no incompatible changes introduced in Rspamd 1.3 version.
There are no incompatible changes introduced in Rspamd 1.2 version.
The only change here affects users with per-user statistics enabled. There is an incompatible change in sqlite3 and per-user behaviour:
Now both redis and sqlite3 follow common principles for per-user statistics:
If you need the old behaviour, then you need to use a separate classifier for per-user statistics, for example:
classifier {
tokenizer {
name = "osb";
}
name = "bayes_user";
min_tokens = 11;
backend = "sqlite3";
per_language = true;
per_user = true;
statfile {
path = "/tmp/bayes.spam.sqlite";
symbol = "BAYES_SPAM_USER";
}
statfile {
path = "/tmp/bayes.ham.sqlite";
symbol = "BAYES_HAM_USER";
}
}
classifier {
tokenizer {
name = "osb";
}
name = "bayes";
min_tokens = 11;
backend = "sqlite3";
per_language = true;
statfile {
path = "/tmp/bayes.spam.sqlite";
symbol = "BAYES_SPAM";
}
statfile {
path = "/tmp/bayes.ham.sqlite";
symbol = "BAYES_HAM";
}
}
In Rspamd 1.0 the default settings for statistics tokenization have been changed to modern, meaning that tokens are now generated from normalized words and there are various improvements which are incompatible with the statistics model used in pre-1.0 versions. To use these new features you should either relearn your statistics or continue using your old statistics without new features by adding a compat parameter:
classifier {
...
tokenizer {
compat = true;
}
...
}
The recommended way to store statistics now is the sqlite3 backend (which is incompatible with the old mmap backend):
classifier {
type = "bayes";
tokenizer {
name = "osb";
}
cache {
path = "${DBDIR}/learn_cache.sqlite";
}
min_tokens = 11;
backend = "sqlite3";
languages_enabled = true;
statfile {
symbol = "BAYES_HAM";
path = "${DBDIR}/bayes.ham.sqlite";
spam = false;
}
statfile {
symbol = "BAYES_SPAM";
path = "${DBDIR}/bayes.spam.sqlite";
spam = true;
}
}
The Rspamd web interface is now a part of the Rspamd distribution. Moreover, all static files are now served by Rspamd itself so you won’t need to set up a separate web server to distribute static files. At the same time, the WebUI worker has been removed and the controller acts as WebUI+old_controller which allows it to work with both a web browser and the rspamc client. However, you might still want to set up a full-featured HTTP server in front of Rspamd to enable, for example, TLS and access controls.
Now there are two password levels for Rspamd: password for read-only commands and enable_password for data changing commands. If enable_password is not specified then password is used for both commands.
Here is an example of the full configuration of the Rspamd controller worker to serve the WebUI:
worker {
type = "controller";
bind_socket = "localhost:11334";
count = 1;
password = "q1";
enable_password = "q2";
secure_ip = "127.0.0.1"; # Allows to use *all* commands from this IP
static_dir = "${WWWDIR}";
}
The settings system has been completely reworked. It is now a lua plugin that registers pre-filters and assigns settings according to dynamic maps or a static configuration. Should you want to use the new settings system then please check the recent documentation. The old settings have been completely removed from Rspamd.
There are many changes in the lua API and some of them are, unfortunately, breaking ones.
rspamd_config. This affects the following modules:
rspamd_loggerrspamd_iprspamd_httprspamd_cdbrspamd_regexprspamd_trielocal rspamd_logger = require "rspamd_logger"
local rspamd_trie = require "rspamd_trie"
local rspamd_cdb = require "rspamd_cdb"
local rspamd_ip = require "rspamd_ip"
local rspamd_regexp = require "rspamd_regexp"
rspamd_config object. Old version:local reconf = config['regexp']
reconf['SYMBOL'] = function(task)
...
end
new one:
rspamd_config.SYMBOL = function(task)
...
end
rspamd_message is removed completely; you should use task methods to access message data. This includes such methods as:
get_date - this method can now return a date for task and message based on the arguments:local dm = task:get_date{format = 'message'} -- MIME message date
local dt = task:get_date{format = 'connect'} -- check date
get_header - this function is totally reworked. Now get_header version returns just a decoded string, get_header_raw returns an undecoded string and get_header_full returns the full list of tables. Please consult the corresponding documentation for details. You also might want to update the old invocation of task:get_header to the new one.
Old version:function kmail_msgid (task)
local msg = task:get_message()
local header_msgid = msg:get_header('Message-Id')
if header_msgid then
-- header_from and header_msgid are tables
for _,header_from in ipairs(msg:get_header('From')) do
...
end
end
return false
end
new one:
function kmail_msgid (task)
local header_msgid = task:get_header('Message-Id')
if header_msgid then
local header_from = task:get_header('From')
-- header_from and header_msgid are strings
end
return false
end
or with the full version:
rspamd_config.FORGED_GENERIC_RECEIVED5 = function (task)
local headers_recv = task:get_header_full('Received')
if headers_recv then
-- headers_recv is now the list of tables
for _,header_r in ipairs(headers_recv) do
if re:match(header_r['value']) then
return true
end
end
end
return false
end
get_from and get_recipients now accept optional numeric arguments that specifies where to get sender and recipients for a message. By default, this argument is 0 which means that data is initially checked in the SMTP envelope (meaning MAIL FROM and RCPT TO SMTP commands) and if the envelope data is inaccessible then it is grabbed from MIME headers. Value 1 means that data is checked on envelope only, while 2 switches mode to MIME headers. Here is an example from the forged_recipients module:-- Check sender
local smtp_from = task:get_from(1)
if smtp_from then
local mime_from = task:get_from(2)
if not mime_from or
not (string.lower(mime_from[1]['addr']) ==
string.lower(smtp_from[1]['addr'])) then
task:insert_result(symbol_sender, 1)
end
end
Rspamd now uses HTTP protocols for all operations, therefore an additional client library is unlikely to be needed. The fallback to the old spamc protocol has also been implemented to be automatically compatible with rmilter and other software that uses the rspamc protocol.