This document describes upstreams: list of servers that are selected by Rspamd using specific algorithm to establish a connection.
List of upstreams is a common structure used in various Rspamd configuration options when you need to setup some remote servers. For example, upstreams are used to connect to a Redis server, to select a DNS server and to establish a connection by Rspamd proxy. Servers in upstream list can be defined by IP addresses (IPv6 addresses should be enclosed in brackets) or Unix domain sockets:
You can also specify custom ports if they differ from the default ones (e.g.
53 for DNS):
It is also possible to define upstreams priorities (described later), but in this case you must also specify a port number:
Unix sockets (starting with
.) can also be specified, but priorities are not supported:
Upstreams line can be separated by commas or by semicolons in any combination. You can prepend rotation algorithm to the upstreams line to override the default rotation method (specific for each upstream list definition):
There are several algorithms available in Rspamd so far:
This algorithm always select the upstream with highest weight unless it is not alive. For example,
master-slave:127.0.0.1:53:10,220.127.116.11:53:1, line specifies that
127.0.0.1 will be always used if possible. You can skip priorities, then the first element is treated as master and the subsequent ones are used as slaves:
master-slave:127.0.0.1,18.104.22.168 is the equivalent of the previous definition.
In this algorithm, upstreams are selected based on its weight, but, after selection, the upstream’s weight is decreased by one. For example,
round-robin:127.0.0.1:53:10,22.214.171.124:53:1 will select
127.0.0.1 10 times and
126.96.36.199 merely one time. After all upstreams are rotated, Rspamd reset current weights to the initial ones. Hence, this could be treated as
10:1 distribution for these two upstreams. Upstreams with errors pending have their priorities penalised according to number of errors pending, so Rspamd prefers to select upstreams with no errors.
Selects upstreams sequentially ignoring priorities.
Selects upstreams randomly ignoring priorities.
Selects upstream based on hash value of the input key. Rspamd uses a consistent hash algorithm that allows you to split data between shards based on some key value. This rotation is available for specific upstreams, for example, some Redis upstreams. Otherwise,
round-robin algorithm is used.
Each upstream is monitored by Rspamd for errors. If an error occur Rspamd places an upstream in monitoring mode during which it analyses errors rate (this is usually set by options
error_time where rate is calculated by
time elapsed since monitoring start). If this error rate is higher than desired (
error_time) then Rspamd marks upstream as inactive unless there are no active upstreams. Any successful connection during
monitoring state returns an upstream to the
active state. Upon reaching error rate limit, an upstream is marked as inactive and Rspamd waits for some time configured by option
dead_time to restore upstream in the active list. The overall process is depicted in the following scheme:
Rspamd has a special treatment for upstreams defined with their names. During
dead_time, Rspamd tries to re-resolve names and insert new IP addresses into upstream. If a name has multiple addresses, then Rspamd inserts all. Addresses are selected using round-robin rotation with error checking. Unlike upstreams configurations, errors are persistent and not cleared after successful attempts, so Rspamd always select an address with fewer errors count. This is done to turn off an IPv6 address, for example, if IPv6 is improperly configured in the system.
From version 2.0 Rspamd also resolves all upstreams in the background each
jitter(0.1 * lazy_resolve_time). By default, this value is 1 hour but you can set this setting to a different value in the configuration (options -> upstreams section).