Rspamd proxy worker

This worker provides different functionality to build multiple layers systems and to handle Milter protocol. Here is a short list of functions provided by proxy worker:

  • Forwarding of messages to scanning layer
  • Directly interacting with MTA using Milter protocol
  • Perform load balancing, retransmitting and health checks for scanning layer
  • Add encryption and/or compression to the scan requests
  • Mirror some portion of traffic to some test server
  • Compare results of mirrored request
  • Perform messages scan by own (self-scan mode)

Milter support

From Rspamd 1.6, rspamd proxy worker supports milter protocol which is supported by some of the popular MTA, such as Postfix or Sendmail. The introducing of this feature also finally obsoletes the Rmilter project in honor of the new integration method. Milter support is presented in rspamd_proxy only, however, there are two possibilities to use milter protocol:

  • Proxy mode (for large instances) with a dedicated scan layer
  • Self-scan mode (for small instances)

If your setup does not allow your MTA to reject mails set discard_on_reject (>= 1.6.2) to true for discarding spam-mails.

Self-scan mode

In this mode, rspamd_proxy scans messages itself and talk to MTA directly using Milter protocol. The advantage of this mode is its simplicity. Here is a sample configuration for this mode:

# local.d/worker-proxy.inc
upstream "local" {
  self_scan = yes; # Enable self-scan
}

# Proxy worker is listening on *:11332 by default
#bind_socket = localhost:11332;

Also you can disable1 normal worker to free up system resources as it is not necessary in self-scan mode:

# local.d/worker-normal.inc
enabled = false;

But there is a drawback: since rspamc uses normal worker by default you need to explicitly point it to controller worker port (11334)2:

rspamc -h rspamd.example.org:11334 input-file

 

1. The enabled option is available for workers since Rspamd 1.6.2, in previous versions you can use count = 0; instead.

2. When connecting to local IP rspamc uses controller port by default (1.7+).

Proxy mode

In this mode, there is a dedicated layer of Rspamd scanners with load-balancing and optional encryption and/or compression. For this setup, the configuration might be different. Here is a small sample of proxy mode with 4 scanners where 2 scanners are more powerful and receive more requests:

# local.d/worker-proxy.inc
upstream "scan" {
  default = yes;
  hosts = "round-robin:host1:11333:10,host2:11333:10,host3:11333:5,host4:11333:5";
  key = "..."; # Public key for encryption, generated by rspamadm keypair (optional)
  compression = yes; # Use zstd compression (optional)
}

Mirroring

Proxy can be used to test:

  • new versions of Rspamd;
  • new plugins;
  • new rules;
  • configuration changes;
  • ML models.

In this mode, Rspamd mirrors some portion of traffic to a test cluster. Results of their scans are ignored when returning reply to a client, however, optional compare scripts could be started to evaluate mirror results. Here is a sample configuration of this setup (we are not using milter mode in this sample):

# local.d/worker-proxy.inc
# Main scan layer
upstream "scan" {
  default = "yes";
  hosts = "round-robin:host1:11333:10,host2:11333:10,host3:11333:5,host4:11333:5";
  key = "..."; # Public key for encryption, generated by rspamadm keypair
  compression = yes; # Use zstd compression
}

mirror "test" {
  hosts = "test:11333";
  probability = 0.1; # Mirror 10% of traffic
  key = "..."; # Public key for encryption, generated by rspamadm keypair
  compression = yes; # Use zstd compression
}

Compare scripts

Compare scripts are intended to perform some simple actions with the results returned by a mirror and main cluster machine. They cannot do async request, so all you can do is to write to logs or to files. Here is a simple example of such a script in the configuration:

# local.d/worker-proxy.inc
  script =<<EOD
return function(results)
  local log = require "rspamd_logger"

  for k,v in pairs(results) do
    if type(v) == 'table' then
      log.infox("%s: %s", k, v['score'])
    else
      log.infox("err: %s: %s", k, v)
    end
  end
end
EOD;