The RBL module offers support for checking various message elements, such as the sender’s IP addresses, URLs, emails, Received headers chains, SMTP data (e.g. HELO domain), and more, against a set of Runtime Black Lists (RBL) typically provided through dedicated DNS zones.
By default, Rspamd comes with a set of RBL rules pre-configured for popular resources that are often free for non-profit usage, subject to fair usage policies. If you require a different level of support or access, please contact the relevant vendors.
For example, you can use Abusix Mail Intelligence or Spamhaus DQS or any other RBL provider that suits your needs.
Configuration for this module is structured as following:
# local.d/rbl.conf
# 'rbls' subsection under which the RBL definitions are nested
rbls {
# rbl-specific subsection
an_rbl {
## required settings
# checks to enable for this RBL
checks = ["from"];
# Address used for RBL-testing
rbl = "rbl.example.net";
## some optional settings
# Explicitly defined symbol
symbol = "SOME_SYMBOL";
# redefined defaults for IPv6 only RBL
ipv4 = false;
ipv6 = true; # Define IPv6 only RBL
# Possible responses from RBL and symbols to yield
returncodes = {
# Name_of_symbol = "address";
EXAMPLE_ONE = "127.0.0.1";
EXAMPLE_TWO = "127.0.0.2";
}
}
}
local_exclude_ip_map: map containing additional IPv4/IPv6 addresses/subnets that should be considered private and excluded from checks where exclude_local is true (the default).url_whitelist: map containing host names which should be skipped by URL checks.The required parameters rbl and checks set the address used for testing and the checks to be performed respectively. Valid values for checks can include any of the following:
content_urls - URLs extracted by lua_content (eg. from PDFs)dkim - domain that provided DKIM signature for a messageemails - email addresses found in a message-bodyfrom - the sending IP that sent the messagehelo - HELO provided by the senderimages - URLs of images linked in message bodynumeric_urls - IP addresses featuring in the hostname part of URLs (since 3.7)rdns - sender’s hostname as provided to Rspamd (expected to be forward-confirmed)received - IP addresses found in Received headersreplyto - address from the Reply-To header of a messageurls - URLs extracted from message bodyYou can use selectors to look up arbitrary data. Please refer to the section on selectors for more information.
# /etc/rspamd/local.d/rbl.conf
rules {
# minimal configuration example
SIMPLE_RBL {
rbl = "rbl.example.net";
checks = ["from"];
}
}
Optional parameters (and their defaults if applicable) are as follows:
dkim_domainonly (true) - lookup eSLD associated with DKIM signature rather than full labeldkim_match_from (false) - only check DKIM signatures matching the From headeremails_domainonly (false) - lookup domain of address instead of full addressenabled (true) - allow for disabling of RBLsexclude_local (true) - do not check messages from private IPs against this RBL (for received check: do not check private IPs at all)exclude_users (false) - do not check this RBL if sender is an authenticated userhash - valid for helo and emails RBL types - lookup hashes instead of literal strings. Possible values for this parameter are sha1, sha256, sha384, sha512 and md5 or any other value for the default hashing algorithm.hash_format - encoding to use for hash: hex, base32 or base64ignore_whitelist (false) - allow whitelists to neutralise this RBLimages (false) - whether image URLs should be checked by urls checkipv4 (true) - if IPv4 addresses should be checkedipv6 (true) - if IPv6 addresses should be checkedis_whitelist (false) - denotes that this RBL is an whitelistlocal_exclude_ip_map - map containing IPv4/IPv6 addresses/subnets which should be considered private (and treated as local by exclude_local)monitored_address (1.0.0.127) - fixed address to check for absence; see section on monitoring for more informationno_ip (false) - do not look up IP addresses in this RBLrequests_limit (9999) - maximum number of entities extracted by URL checksresolve_ip - resolve the domain to IP addressreturnbits - dictionary of symbols mapped to bit positions; if the bit in the specified position is set the symbol will be returnedreturncodes - dictionary of symbols mapped to lua patterns; if result returned by the RBL matches the pattern the symbol will be returnedreturncodes_matcher - a specific mechanism for testing returncodes, see detailsselector_flatten (true) - when disabled will lookup result of chained selector as a single label without any separatorselector - one or more selectors producing data to look up in this RBL; see section on selectors for more informationunknown (false) - yield default symbol if returncodes or returnbits is specified and RBL returns unrecognised resultwhitelist_exception - for whitelists; list of symbols which will not act as whitelistsSome examples of using RBL:
rbls {
blocklist {
symbol = "BLOCKLIST";
rbl = "blocklist.bl";
checks = ['from', 'received'];
}
WHITELIST_BASE {
checks = ['from', 'received'];
is_whitelist = true;
rbl = "whitelist.wl";
symbol = "WL_RBL_UNKNOWN";
unknown = true;
returncodes = {
"WL_RBL_CODE_2" = "127.0.0.2";
"WL_RBL_CODE_3" = "127.0.0.3";
}
}
DNS_WL {
symbol = "DNS_WL";
rbl = "dnswl.test";
checks = ['dkim'];
dkim_domainonly = false;
dkim_match_from = true;
ignore_whitelist = true;
unknown = false;
returncodes_matcher = "luapattern";
returncodes {
DNS_WL_NONE = "127.0.%d+.0";
DNS_WL_LOW = "127.0.%d+.1";
DNS_WL_MED = "127.0.%d+.2";
DNS_WL_HI = "127.0.%d+.3";
DNS_WL_BLOCKED = "127.0.0.255";
}
}
}
From version 3.7.2 Rspamd supports different strategies for handling returncodes via the returncodes_matcher setting.
By default return codes are tested for equality with the result of the DNS query. For backwards compatibility values containing the percent character implicitly enable the luapattern matcher if one is not set.
Matcher types:
equality: the default, not useful for actual configurationluapattern: match query results using Lua patterns, the old defaultradix: check for query results inside collection of subnets and IP addressesglob: match query results against “globbed” stringsregexp: match query results using regular expressionsExamples:
returncodes_matcher = "radix";
returncodes {
SYMBOL_ONE = "127.0.0.0/24";
SYMBOL_TWO = ["192.168.0.0/16", "1.2.3.4"];
}
returncodes_matcher = "glob";
returncodes {
SYMBOL_ONE = "127.0.0.*";
SYMBOL_TWO = ["192.168.*.*", "1.2.3.4"];
}
returncodes_matcher = "regexp";
returncodes {
# regexp is not automatically anchored
SYMBOL_ONE = '^127\.0\.0\.\d+$';
SYMBOL_TWO = ['^192\.168\.\d+\.\d+$", '^1\.2\.3\.4$'];
}
returncodes_matcher = "luapattern";
returncodes {
# lua patterns are automatically anchored by ^ and $
SYMBOL_ONE = '127%.0%.0%.%d+';
SYMBOL_TWO = ['192%.168%.%d+%.%d+", '1%.2%.3%.4'];
}
Starting from version 2.0, both the Emails and SURBL modules are deprecated in favour of the rules for the RBL module. Rspamd automatically converts the old rules on start. If you have custom rules in either the SURBL or Emails module, they are converted to have priority over RBL modules for a smooth transition. However, new rules should only be written for the RBL module, as the transition phase for the SURBL and Emails modules will not last forever.
Previously, the SURBL module was responsible for scanning URLs found in messages against a list of known RBLs. However, these functions are now transferred to the RBL module.
URLs extracted from the message body & URLs extracted from content such as PDFs can be checked by adding urls and/or content_urls respectively to the checks setting.
Image URLs are not extracted by default, to include image URLs, add images to the checks setting. If images is used in checks without urls only image URLs are extracted.
Numeric URLs (IP addresses) are extracted by default and are looked up in reverse notation, to exclude them set no_ip = true; or to extract IP addresses only set checks = ["numeric_urls"] (since 3.7). Combining this check with the urls check doesn’t make logical sense; if combined with content_urls or images then numeric URLs from content and images are respectively included (by default they are excluded as with urls check).
Rspamd defines a set of URL lists in the configuration by default. However, their terms of usage typically prohibit commercial or extensive usage without purchasing a specific type of license.
Nevertheless, these lists can be used free of charge by personal services or low-volume requests.
Here are the default lists specified:
# local.d/rbl.conf
# List of domains that are not checked by surbl
url_whitelist = "file://$CONFDIR/local.d/maps.d/surbl-whitelist.inc.local";
# 'rbls' subsection under which the SURBL definitions are nested
rbls {
"SURBL_MULTI" {
ignore_defaults = true; # for compatibility with old defaults
rbl = "multi.surbl.org";
checks = ['emails', 'dkim', 'urls'];
emails_domainonly = true;
urls = true;
returnbits = {
CRACKED_SURBL = 128; # From February 2016
ABUSE_SURBL = 64;
CT_SURBL = 32;
MW_SURBL_MULTI = 16;
PH_SURBL_MULTI = 8;
DM_SURBL = 4;
SURBL_BLOCKED = 1;
}
}
"URIBL_MULTI" {
ignore_defaults = true; # for compatibility with old defaults
rbl = "multi.uribl.com";
checks = ['emails', 'dkim', 'urls'];
emails_domainonly = true;
returnbits = {
URIBL_BLOCKED = 1;
URIBL_BLACK = 2;
URIBL_GREY = 4;
URIBL_RED = 8;
}
}
"RSPAMD_URIBL" {
ignore_defaults = true; # for compatibility with old defaults
rbl = "uribl.rspamd.com";
checks = ['emails', 'dkim', 'urls'];
# Also check images
images = true;
# Check emails for URLs
emails_domainonly = true;
# Hashed BL
hash = 'blake2';
hash_len = 32;
hash_format = 'base32';
returncodes = {
RSPAMD_URIBL = [
"127.0.0.2",
];
}
}
"DBL" {
ignore_defaults = true; # for compatibility with old defaults
rbl = "dbl.spamhaus.org";
no_ip = true;
checks = ['emails', 'dkim', 'urls'];
emails_domainonly = true;
returncodes = {
# spam domain
DBL_SPAM = "127.0.1.2";
# phish domain
DBL_PHISH = "127.0.1.4";
# malware domain
DBL_MALWARE = "127.0.1.5";
# botnet C&C domain
DBL_BOTNET = "127.0.1.6";
# abused legit spam
DBL_ABUSE = "127.0.1.102";
# abused spammed redirector domain
DBL_ABUSE_REDIR = "127.0.1.103";
# abused legit phish
DBL_ABUSE_PHISH = "127.0.1.104";
# abused legit malware
DBL_ABUSE_MALWARE = "127.0.1.105";
# abused legit botnet C&C
DBL_ABUSE_BOTNET = "127.0.1.106";
# error - IP queries prohibited!
DBL_PROHIBIT = "127.0.1.255";
# issue #3074
DBL_BLOCKED_OPENRESOLVER = "127.255.255.254";
DBL_BLOCKED = "127.255.255.255";
}
}
"SEM_URIBL_UNKNOWN" {
ignore_defaults = true; # for compatibility with old defaults
rbl = "uribl.spameatingmonkey.net";
no_ip = true;
checks = ['emails', 'dkim', 'urls'];
emails_domainonly = true;
returnbits {
SEM_URIBL = 2;
}
}
}
Each list should have a suffix parameter that defines the list itself, and optionally, some replies processing logic either by returnbits or returncodes sections.
As some URL lists do not accept IP addresses, it is possible to disable the sending of URLs with IP addresses in the host to such lists. This can be done by specifying the no_ip = true option.
"DBL" {
rbl = "dbl.spamhaus.org";
# Do not check numeric URLs
no_ip = true;
}
URL blacklists can also be used to check DKIM signature domains, HTML image URLs, and email addresses (domain part) in the mail’s body part for URLs.
"RSPAMD_URIBL" {
ignore_defaults = true; # for compatibility with old defaults
rbl = "uribl.rspamd.com";
checks = ['emails', 'dkim', 'urls'];
# Also check images
images = true;
# Check emails for URLs
emails_domainonly = true;
# Hashed BL
hash = 'blake2';
hash_len = 32;
hash_format = 'base32';
returncodes = {
RSPAMD_URIBL = [
"127.0.0.2",
];
}
}
In this example, we also enable privacy for requests by hashing all elements before sending. This feature is supported by a limited number of RBLs, such as Rspamd URL blacklist or MSBL EBL.
Rspamd checks each RBL rule to ensure it’s a valid DNS list as defined in RFC 5782 by default. This is done to avoid situations where a single RBL blacklists the entire world or becomes unresponsive. For the IP-based rules, meaning that an IP address is queried, Rspamd will query for the 127.0.0.1 address, as per the RFC, this must return an NXDOMAIN response. However, some DNS lists are non RFC compatible, so you can disable monitoring for them as follows:
"HOSTKARMA_URIBL" {
rbl = "hostkarma.junkemailfilter.com";
no_ip = true;
enabled = false;
returncodes = {
URIBL_HOSTKARMA_WHITE = "127.0.0.1";
URIBL_HOSTKARMA_BLACK = "127.0.0.2";
URIBL_HOSTKARMA_YELLOW = "127.0.0.3";
URIBL_HOSTKARMA_BROWN = "127.0.0.4";
URIBL_HOSTKARMA_NOBLACK = "127.0.0.5";
URIBL_HOSTKARMA_24_48H = "127.0.2.1";
URIBL_HOSTKARMA_LAST_10D = "127.0.2.2";
URIBL_HOSTKARMA_OLDER_10D = "127.0.2.3";
}
disable_monitoring = true;
}
For non IP lists (DKIM, URL, Email and so on), Rspamd will just produce some long random string to query expecting that this random string will very likely return NXDOMAIN by its nature.
In this section, we define how RBL module performs its checks.
By default, we want to check some top level domain, however, many domains contain two components while others can have 3 or even more components to check against the list. By default, rspamd takes top level domain as defined in the public suffixes. Then one more component is prepended, for example:
sub.example.com -> [.com] -> example.com
sub.co.uk -> [.co.uk] -> sub.co.uk
However, sometimes even more levels of domain components are required. In this case,
the exceptions map can be used. For example, if we want to check all subdomains of
example.com and example.co.uk, then we can define the following list:
example.com
example.co.uk
Here are new composition rules:
sub.example.com -> [.example.com] -> sub.example.com
sub1.sub2.example.co.uk -> [.example.co.uk] -> sub2.example.co.uk
Starting from Rspamd 2.5, it is now possible to define custom composition rules per RBL rule, using the lua_urls_compose library. Below is a basic explanation of how these composition rules work:
-- First one is the input hostname, the second is the expected results
cases = {
{'example.com', 'example.com'},
{'baz.example.com', 'baz.example.com'},
{'3.baz.example.com', 'baz.example.com'},
{'bar.example.com', 'example.com'},
{'foo.example.com', 'foo.example.com'},
{'3.foo.example.com', '3.foo.example.com'},
}
-- Just a domain means domain + 1 level
-- *.domain means the full hostname if the last part matches
-- !domain means exclusion
-- !*.domain means the same in fact :)
-- More rules can be added easily...
local excl_rules1 = {
'example.com',
'*.foo.example.com',
'!bar.example.com'
}
To define a specific map for these rules, the following syntax can be used:
# local.d/rbl.conf
rules {
EXAMPLE_RBL = {
suffix = "example.url.bl.com";
url_compose_map = "${CONFDIR}/maps.d/url_compose_map.list";
checks = ['emails', 'dkim', 'urls'];
emails_domainonly = true;
ignore_defaults = true;
}
}
Where in maps you can use something like this:
*.dirty.sanchez.com
!not.dirty.sanchez.com
41.black.sanchez.com
So it will check 5 hostname components for all urls in dirty.sanchez.com (e.g. sub.some.dirty.sanchez.com will be transformed to just some.dirty.sanchez.com) but not for not.dirty.sanchez.com where the normal tld rules will apply (e.g. some.not.dirty.sanchez.com -> sanchez.com), and for 41.black.sanchez.com all 5 components will be checked, e.g. something.41.black.sanchez.com.
SURBL module composes the DNS request of two parts:
For example, to form a request to multi.surbl.org, the following applied:
example.com -> example.com.multi.surbl.com
Normally, DNS blacklists encode reply in A record from some private network
(namely, 127.0.0.0/8). Encoding varies from one service to another. Some lists
use bits encoding, where a single DNS list or error message is encoded as a bit
in the least significant octet of the IP address. For example, if bit 1 encodes LISTA
and bit 2 encodes LISTB, then we need to perform bitwise OR for each specific bit
to decode reply:
127.0.0.3 -> LISTA | LISTB -> both bit symbols are added
127.0.0.2 -> LISTB only
127.0.0.1 -> LISTA only
The use of this encoding can reduce DNS requests needed to query multiple lists individually.
However, some lists use a direct encoding method where specific addresses are assigned to each list. In such cases, the decoding principle for the results should be defined in the ips section instead of the bits section since bitwise rules do not apply to these lists. In the ips section, the IP address returned by a list is explicitly matched with its corresponding meaning.
Starting from rspamd 1.1, it is also possible to perform two-step checks:
In general, this procedure can be represented as follows:
A or AAAA records for example.comexample.com is 1.2.3.4, then checks would be for 4.3.2.1.uribl.tldTo disable a rule in SURBL, you can set the enabled setting to false. This makes it easy to disable specific SURBLs without overriding the entire default configuration. For instance, if you want to disable the RAMBLER_URIBL URIBL, you can add the following example to /etc/rspamd/local.d/surbl.conf:
rules {
"RAMBLER_URIBL" {
enabled = false;
}
}
The SURBL module is designed to work in conjunction with the url_redirector module which is capable of resolving known redirectors and extracting the actual URL for the SURBL module to check. You can refer to the url_redirector module’s documentation for more information on how to use it. Once the url_redirector module has resolved the actual URL, the SURBL module will automatically use the results to perform its checks.
Selectors can be used to look up arbitrary values in RBLs.
The selector field could be configured as a string in the rule settings if only one selector is needed:
checks = ["replyto"];
selector = "from('mime'):addr";
hash = "sha1";
symbols_prefixes {
replyto = "REPLYTO";
selector = "FROM";
}
Or they could be specified as a map of user-specified names to selectors if more than one is needed:
selector = {
mime_domain = "from('mime'):domain";
subject_digest = "header('subject').lower.digest('hex')";
}
symbols_prefixes {
mime_domain = "MIME_DOMAIN";
subject_digest = "SUBJECT_DIGEST"
}
To combine multiple selectors in one query you can set selector_flatten to false. For example to create query to RBL like: from-sha256.rcpt-sha256:
selector_flatten = false;
selector = "from('mime').lower.digest('hex','sha256').append('.');rcpts('smtp').first.lower.digest('hex','sha256')";