# Writing tests for Rspamd

Testing is an important part of maintaining stable product. In case of Rspamd, there was a historical lack of proper testing, however, we are working on improvements here. Any help from the community with regard to tests is much appreciated.

## Introduction

Rspamd has two types of tests:

• Unit tests - those type of tests are intended to test some particular function in Rspamd and are written in Lua + FFI (if testing plain C function) using Telescope framework;
• Functional tests - are used to test the whole daemon behaviour with complex setup involving custom configuration, external services, such as Redis, and so on. Functional tests are written using Robot Framework.

## Unit tests

Unit tests are placed in test/lua/unit. Each tests defines testing context, where there are main definitions used by all test cases, for example, FFI definitions:

context("Inet addr check functions", function()
local ffi = require("ffi")

ffi.cdef[[
const char *src);
]]

...
end)


Then, there could be some test cases:

  local cases = {
{'192.168.1.1', true},
{'2a01:4f8:190:43b5::99', true},
{'256.1.1.1', false},
{'/tmp/socket', true},
{'./socket', true},
{'[fe80::f919:8b26:ff93:3092%5]', true},
{'[fe80::f919:8b26:ff93:3092]', true},
}

for i,c in ipairs(cases) do
test("Create inet addr from string " .. i, function()
assert_equal(res, c[2], "Expect " .. tostring(c[2]) .. " while parsing " .. c[1])
if res then
end
end)


Bear in mind, that a single test invocation should define one specific case.

Running unit tests requires to build a special rspamd-test target. If you use make to build Rspamd from the sources, you should type make rspamd-test to do it. After that, you would have test/rspamd-test binary available in your build directory.

To run unit tests, just type test/rspamd-test -p /rspamd/lua

Unfortunately, it is currently impossible to execute specific unit tests only.

## Functional tests

Functional tests are intended to test the whole setup of Rspamd and you should first learn some basics about the Robot Framework that is used to write tests.

Functional tests live in test/functional directory. To run functional tests, you first need to install Rspamd in your system (or a container). Then you can run them manually using something like RSPAMD_INSTALLROOT=/usr/local robot -s '280*' ~/rspamd/test/functional/cases, where:

• RSPAMD_INSTALLROOT - a prefix where Rspamd is installed (e.g. /usr for the vast majority of Linux installations)
• -s - pattern to match tests (may be skipped if all tests are needed)
• ~/rspamd/test/functional/cases - directory where test cases are placed

Functional tests are also executed by Rspamd CI. It also covers pull requests you send on the GitHub site.

### Functional tests structure

Each test usually has 3 components:

• Test case (written in Robot) that lives in test/functional/cases
• Some configuration that lives in test/functional/configs
• Messages to scan in test/functional/messages

In many cases you’d also need to have some specific Lua code that should be placed in test/functional/lua. For complicated setups, e.g. if you need some fake or real external service, you could need to write some Python code that should be placed to test/functional/lib and, for fake services, in test/functional/util.

You could find plenty of examples about how to run those fake servers in the existing tests.

### Test case structure

Each test is enclosed within some specific test case. Test case consist of setup, set of tests and teardown.

General testing advice on the Robot Framework is listed here.

Test case has also a preamble that defines some common variables and setup + teardown procedures:

*** Settings ***
Suite Setup     Rbl Setup
Suite Teardown  Rbl Teardown
Library         ${TESTDIR}/lib/rspamd.py Resource${TESTDIR}/lib/rspamd.robot
Variables       ${TESTDIR}/lib/vars.py *** Variables ***${CONFIG}       ${TESTDIR}/configs/plugins.conf${MESSAGE}      ${TESTDIR}/messages/spam_message.eml${RSPAMD_SCOPE}  Suite
${URL_TLD}${TESTDIR}/../lua/unit/test_tld.dat


Setup procedure starts Rspamd using specific config file, teardown procedure, in turn, switches everything off. Both are listed in the Keywords section:

*** Keywords ***
Rbl Setup
${PLUGIN_CONFIG} = Get File${TESTDIR}/configs/rbl.conf
Set Suite Variable  ${PLUGIN_CONFIG} Generic Setup PLUGIN_CONFIG Rbl Teardown Normal Teardown Terminate All Processes kill=True  Test cases are listed within *** Test Cases *** section: *** Test Cases *** RBL FROM MISS${result} =  Scan Message With Rspamc  ${MESSAGE} -i 1.2.3.4 Check Rspamc${result}  FAKE_RBL_CODE_2  inverse=True

RBL FROM HIT
${result} = Scan Message With Rspamc${MESSAGE}  -i  4.3.2.1