List of changes

Release 3.8.2: 20 Feb 2024

Feature

Add extraction type for `from` maps

Feature

Allow to add templates to redis history prefix

Feature

Implement dynamic keys map in fuzzy storage

Feature

Lua_url: Add `to_http` method

Feature

Support JSON logging when in syslog mode

Fix

Deal with `Connection` and `Host` headers on proxying

Fix

Encode headers in metadata exporter

Fix

Fix initial maps load

Fix

Make stat tokens allocation consistent

Fix

Resolve issue with bayes stat in `rspamadm` mode

Fix

Try to fix url path issue

Rework

Breaking: Do not report module as action

Rework

Use khash instead of glib hashes for many reasons

Release 3.8.1: 25 Jan 2024

Fix

Fix headers insertion in the ordered list

Fix

Fix learn error propagation

Fix

Fix learning with long prefixes

Fix

Fix potential double free by expclicitly zeroing pointers

Fix

Fix storing of the bayes tokens

Fix

Use short comments rather than long one

Release 3.8.0: 19 Jan 2024

CritFix

- Fix reported length of logging structure

Feature

Allow to set `max_users` for Redis bayes backend

Feature

Escape JSON when needed

Feature

Proxy: Allow `encrypted_only` option

Feature

Reiterate on gtube patterns

Feature

Support ping command in fuzzy storage

Feature

Support suppressing DMARC reporting for particular recipients

Feature

rbl: support disabling or replacing url_whitelist per RBL

Fix

Another try to fix setproctitle

Fix

Cleanup session on exit

Fix

Do not cleanup hyperscan files unless new ones are loaded

Fix

Fix bad access when printing pending events on failure path

Fix

Fix issues with the raw header on header insertion/modification

Fix

Fix overflow in fuzzy_stats command

Fix

Fix some corner cases when parsing maps with no newlines

Fix

Fix various issues with canonicalisation of the paths

Fix

Fix wrong raw_len usage

Fix

Make words selection random deterministic upon content

Fix

Properly set config field when creating tasks from Lua

Fix

Really fix the language detector statistical heuristic

Fix

Set loaded variable explicitly

Fix

dkim_signing: siging_table: lowercase before lookup

Fix

known_senders: fix config handling

Fix

rbl: fix `exclude_local`

Release 3.7.5: 15 Dec 2023

Fix

Fix rspamadm lua logger

Fix

Fix systemd logging

Fix

dkim_signing: siging_table: lowercase before lookup

Release 3.7.4: 14 Nov 2023

Fix

Another try to fix setproctitle

Fix

Do not cleanup hyperscan files unless new ones are loaded

Fix

Fix various issues with canonicalisation of the paths

Fix

Properly set config field when creating tasks from Lua

Fix

Set loaded variable explicitly

Fix

known_senders: fix config handling

Fix

rbl: fix `exclude_local`

Release 3.7.3: 27 Oct 2023

Fix

Emergency fix for the hyperscan path error

Release 3.7.2: 26 Oct 2023

Feature

rbl: support checking returncodes by CIDR

Feature

rbl: support checking returncodes by regex

Feature

rbl: support globbed return codes

Fix

DMARC reporting: fix reporting for subdomains

Fix

Deal with fmtlib exceptions properly

Fix

backport fix for dlfcn.h from backward-cpp

Rules

Blank spam detection

Release 3.7.1: 07 Oct 2023

CritFix

Fix leak in `gzip` function

Feature

Add ICAP Content-Type and Filename

Feature

Add `logging`->`task_max_elts` option

Feature

Add utility to split string like stuff for C++ code

Feature

Allow to set HTTP auth parameters for the maps

Feature

Check for plugin configuration errors on `configtest`

Feature

`known_senders` plugin

Feature

Use backward-cpp instead of manual libunwind stuff

Feature

rbl: support checking numeric URLs in isolation

Fix

CMakeLists.txt remove whitespace added by linter as it makes tests fail

Fix

Change Date: header location to conform with RFC

Fix

Correct format pattern for RE tree tempfile name

Fix

Correct format string for unw_word_t

Fix

Do not accept invalid ucl object types

Fix

Do not pollute public headers with libev internals

Fix

Do not set output type if list application failed

Fix

Fix `url:set_redirected` method

Fix

Fix format string and some length issues

Fix

Fix grammar definition for content-disposition attributes

Fix

Fix lua schema enrichment logic for Redis params

Fix

Fix lua stack corruption when logging large tables

Fix

Fix merge table utility

Fix

Fix output of non-RSA DKIM keys

Fix

Fix some corner cases of single-host urls parsing

Fix

Fix various issues in the `url_redirector` plugin

Fix

MISSING_MIMEOLE: avoid matching messages from Android GMail app (#4561)

Fix

Prevent DNSWL sabotage

Fix

Try to fix unzip function

Fix

rbl: really fix dependency registration when symbols_prefixes is used

Fix

rspamadm mime: arguments beginning with letter `t`

Rework

Breaking: return back to semver

Rework

Move rcl logic to C++

Conf

Add `one_shot` to some specific multimap rules

Conf

Add language detection configuration

Conf

Add missing attributes for the language detection configuration

Conf

Remove outdated composite rules

Feature

Add `sentinel_password` option

Feature

Add ability to deny specific fuzzy flags by default

Feature

Add controller endpoint to get fuzzy hashes from messages

Feature

Add extra symbol when URL redirector reaches nested limit

Feature

Add function to transliterate utf8 to ascii with some normalisation

Feature

Add html parsing limit

Feature

Add order to urls structure

Feature

Add some missing functions to `lua_rsa` library

Feature

Allow fuzzy workers to exchange blocked information

Feature

Allow to have weak flags in fuzzy storage

Feature

Allow to read options from maps in the multimap plugin

Feature

Allow to use other methods when fasttext detection is enabled

Feature

Count stats per key per flag

Feature

Finish all features of dkim_keygen in Lua

Feature

Khash: Allow static initialisation

Feature

Maps: Add on_load support

Feature

Preliminary implementation of dynamic composites

Feature

Process HTML parts before text ones

Feature

Reorganise struct rspamd_url to be 64 bytes size

Feature

Save fuzzy ratelimit buckets

Feature

Use in-place deflation for strings

Feature

external_relay: add ip_map strategy

Fix

Avoid race between config new/free by using a counter

Fix

Do not use `rspamadm.dkim_keygen`

Fix

Feed fasttext language model with the pre-tokenized words

Fix

Fix `rspamd_has_only_html_part`

Fix

Fix an old issue with order of destruction race between redis pool and lua

Fix

Fix format string usage

Fix

Fix parsing due to old bug revealed

Fix

Fix parsing of the mask values that are invalid

Fix

Ignore non-unique stop words

Fix

Include the last character when parsing the last header with no value

Fix

More fixes to fuzzystat

Fix

Set proper counter

Fix

Try harder to clean pending bucket

Fix

Try harder to remove bad hyperscan files

Fix

Update stats before encryption...

Fix

dmarc gramar - allow spaces before ";"

Fix

rbl: fix dependency registration when symbols_prefixes is used

Fix

remove obsolete rspamd-redirector files

Fix

test external_relay: count should always be the last rule, as it have no matching condition

Rework

Default max shots must not influence options

Rework

Write dkim keygen tool in lua

Rules

Add thread hijacking composite rule

Conf

Clarify `timeout` in rspamd_proxy

Conf

Fix hashbl

Conf

Add SURBL hashbl support (WIP)

Conf

RBL: Fix selector

Conf

Reduce the default timeout

CritFix

Deserialise hyperscan to the page-aligned space to prevent alignment issues

CritFix

Fill path field in hyperscan notice command

Feature

Add `thresholds` field to the scan result

Feature

Add possibility to execute lua scripts for blocked fuzzy clients

Feature

Add preliminary support of the external maps in the multimap plugin

Feature

Allow to build a map by combining tuples of selectors

Feature

Allow to query external maps for settings

Feature

Allow to specify `selector_alias` in the maps definition

Feature

Enable Mime part filters on antivirus module

Feature

Improve ratelimit redis scripts

Feature

Selectors: Add specific_urls_filter_map extractor

Feature

Some rework of the selectors framework

Fix

Add O_CREAT flag when creating a file

Fix

Add a database check function unless we have anything from Hyperscan

Fix

Add hyperscan databases sanity check

Fix

Add workaround for ENOBUFS error on sending

Fix

Do not do `lstat` when we are creating file

Fix

Finally get rid of RSPAMD_USE_47BIT_LIGHTUSERDATA_HACK

Fix

Fix boundaries that contain only dashes

Fix

Fix off-by-one error in css tokenizer

Fix

Fix url reputation plugin

Fix

Fix usage of the Redis config schema as `extra_fields`

Fix

Further checks for the hs_scratch_alloc

Fix

Honor group flag for one shot

Fix

Normalize glob paths to avoid hash table misses

Fix

Ratelimit: Use unpack for `HMGET` return value

Fix

Rbl: Fix helo check pipeline

Fix

Replace broken strict_domains with phishing_exceptions

Fix

Restore `strict_domains` support

Fix

Return true from has_urls(true) if only emails are present

Fix

Rework lists applications

Fix

Set symcache item in coroutine calls

Fix

Treat `hs_allocate_scratch` errors as non-fatal

Fix

Treat hostnames with no dots as eSLD of their own

Fix

received: filtering of artificial header

Rework

Breaking: Do not report soft reject in history

Rework

Convert chartable plugin to c++ for convenience

Rework

Use a different approach for customization of the settings

Rules

Mid: Add MID_END_EQ_FROM_USER_PART rule

CritFix

Restore compatibility with the integrations and headers alterations

Feature

Milter_headers: Add `x-rspamd-action` routine

Feature

Share hyperscan database among processes

Fix

Another corner case in url parsing

Fix

Another fix for the enable password

Fix

Another try to fix close method in lua_tcp

Fix

Fix additional fields in the Redis schema

Fix

Fix emoji joiner FP

Fix

Fix favicon.ico Content-Type header

Fix

Fix hang when close is used

Fix

Lua_tcp: Sigh, another try to fix `close` invocation

Fix

Mx_check: Cache the fact of a missing MX record

Fix

Try to fix parsing of the unencoded `>` characters in html attributes

Fix

Try to fix the case where password == enable_password

Rework

Convert multipattern to use hyperscan tools

Rework

Make http normalize path function a generic function

Rework

Split locked and unlocked files, as mmap does not need flock normally

Rework

Start movement of the hyperscan related routines into a single unit

Rework

Store the current worker, so other libraries could use this information

Rework

Use blocking socket for IPC between main and workers

Rework

Use more predictable size for commands buffers

Rules

Do not insert ONCE_RECEIVED_STRICT on RDNS missing

Rules

Reduce score of HTTP_TO_HTTPS - subject to remove completely

Conf

Add missing groups for whitelist module symbols

CritFix

Neural: Fix keys regression after #3968

Feature

Accept upstream in lua_tcp

Feature

Add ability to statically maintain disabled/enabled patterns

Feature

Add function to store upstreams for HTTP urls

Feature

Allow augmentations set in Lua API

Feature

Allow lua_http module to accept upstreams

Feature

Allow to limit write access to fuzzy storage by key

Feature

Allow to sort symbols output

Feature

Check content for binary stuff before dumping it to Lua

Feature

Implement symbols augmentations

Fix

Add missing flags

Fix

Add more sanity checks for rua in dmarc_report

Fix

Adjust length of the fuzzy checks for short text parts

Fix

Another try to fix add headers compatibility logic

Fix

Another try to fix race condition in the runtime destruction

Fix

Avoid cyclic references in symcache and fix memory leaks

Fix

Avoid overriding IP with Sender IP

Fix

BAD_REP_POLICIES did not trigger when message was classified as spam by Bayes

Fix

Bind AF_UNIX DGRAM client connection to annonymous address

Fix

Disable IPv6 lookups for Blocklist.de RBL

Fix

Distinguish dynamic and static items

Fix

Dkim: Ignore unknown DKIM kv pairs as stated in RFC

Fix

Dmarc report: Use local timezone instead of GMT

Fix

Do not exclude authenticated users from URIBL lookups

Fix

Empty envelopes should not be emitted as arrays (json+messagepack) when populated envelopes are objects. This greatly complicates decoding in strictly typed languages.

Fix

External_relay: Restore the originating hostname check

Fix

Fix DKIM keys with spaces still allowing errors on invalid base64

Fix

Fix copying of sockaddr_un addresses

Fix

Fix crash with cname replies

Fix

Fix dependencies propagation

Fix

Fix iteration over milter headers

Fix

Fix ordering when sorting symcache

Fix

Fix reading of the cached maps

Fix

Fix several issues with the HTTP keepalive parsing

Fix

Fix stack smashing

Fix

Fix synchronous auth/select in lua_redis

Fix

Fix various symcache issues

Fix

Ignore all (I hope) unknown DKIM signature KV pairs

Fix

Ignore directories in RarV5 archives

Fix

Libucl: avoid memory leak on objects merging

Fix

Lua_tcp: Another try to fix closing logic

Fix

Mempool: Fix alloc_array function to actually multiply nmembers by size

Fix

Only check allowed fuzzy worker update ips for non-unix sockets

Fix

Plug memory leak in regexp destruction with pcre2

Fix

Properly check the original email flag

Fix

Properly deal with `get_symbol/get_metric_symbol` ambiguity

Fix

Properly parse expressions atoms

Fix

Properly set `Host` in rspamd_proxy

Fix

Rbl: Fix received positioned checks

Fix

Remove check for a score with no symbol being registered

Fix

Same fix for lua_tcp

Fix

Skip cname records when processing SPF records

Fix

Skip sending dmarc reports in no-opt mode fixes https://github.com/rspamd/rspamd/issues/4241

Fix

Stop slow timer on task destruction

Fix

Symcache: Do not use C style comparators in C++ sorts

Fix

Try to avoid a corner case for `@` pattern

Fix

Try to fix dkim reputation adjustements

Fix

Try to fix passthrough results processing logic

Fix

Try to fix the mess with read only flag

Fix

Upstreams: Don't ignore revive_time config option

Fix

Use proper format string, sigh...

Fix

Use space category in ragel automata to resolve space characters

Fix

Zstd: Fix compression with the new Zstd API

Fix

milter_headers: Header fields may be inserted at wrong position.

Rework

Another movement

Rework

Augmentations can now imply flags

Rework

Further steps

Rework

Further work on deps processing

Rework

Implement cache resorting

Rework

Isolate disable/enable logic for the configuration ucl objects

Rework

Move item implementation to a separate header

Rework

Multimap: Avoid prefilters usage where augmentations can be used

Rework

Pass upstream when sending TCP requests

Rework

Re-implement cache sorting

Rework

Reimplement saving/loading the cache items

Rework

Reiterate on priorities

Rework

Rework files structure

Rework

Rewrite rspamc in C++

Rework

Simplify scores check and extend it to pre/post filters

Rework

Switch minimum C++ standard version to C++20

Rework

Try to fix the mess with types & flags

Rework

Use another version of hash table from the same author

Rework

Use dynamic items for calling callbacks

Rework

Use dynamic items in the callbacks

Rework

Use hash map for id->symbol mappings

Rework

Use khash instead of uthash in rdns compression logic

Conf

Score MIME_OBFUSCATED_ARCHIVE to 8 points

Conf

Set one_shot for URIBL rules by default

CritFix

Fix upstreams name resolution when there is also a port

Feature

Add ROC feature to neural network plugin

Feature

Add public suffic compilation utility

Feature

Add support of Cloudmark

Feature

Allow hyperscan for ppc64, as vectorscan now suports it.

Feature

Allow to skip DNS resolution for keep-alive connections

Feature

Aws_s3: Allow to store large parts separately

Feature

BIMI: Add preliminary version of the BIMI plugin

Feature

JSON endpoint for querying maps

Feature

Lua_magic: Add a sane CSV heuristic

Feature

Lua_mime: Add schema for message transfer

Feature

Output average scan time in /stat endpoint

Feature

Show average scan time in `rspamc stat` output

Fix

Add guards to avoid race condition on TCP connection

Fix

Allow spaces in DKIM key records

Fix

Apply the similar fix to the url_reputation

Fix

Avoid overwriting whitelisted_signers_map

Fix

Backport PR from libucl

Fix

Clear SSL errors

Fix

ClickHouse cleanup of old partitions

Fix

Do not double call error handler on ssl errors in the timeout path

Fix

Do not forget to clear pointers on IOC reset

Fix

External_relay: Remove useless check of the map value

Fix

Find suspicious url encodings that could break url extraction

Fix

Fix HTTP(s) client timeout

Fix

Fix exclude flags setting

Fix

Fix expanding of the variables

Fix

Fix host header usage in lua_http

Fix

Fix http maps shared memory cache cleanup

Fix

Fix logic in HTML processing FSM

Fix

Fix parsing of the compound mailto urls

Fix

Fix processing captures from pcre2

Fix

Fix removing from khash

Fix

Fix stuctured headers pushing

Fix

Further fix for i386 compilation

Fix

Improve duplicate settings error reporting

Fix

Lua: task:remove_result didn't work in some cases

Fix

Output service parts as well

Fix

Phishing: Deal with phishing + redirected URL

Fix

Phishing: Fix finding domains in the phishing map

Fix

Plug memory leak by using mempool for a copied address

Fix

Properly find the request and the number of requested entries

Fix

Rbl: Fix inversed logic of the url_full_hostname

Fix

Read file maps if they were not pre-read during preload

Fix

Restrict x86_64 assembly to x86_64

Fix

Return a real number of recipients when dealing with aliases

Fix

Rework unshedule DNS request function

Fix

Support definition of ungrouped symbol in conf file, use group info from lua or other conf file

Fix

Unschedule DNS request when clearing IO channel

Fix

When checking for phishing, we need to convert punicode -> UTF8, not vice versa

Fix

lua_cfg_transform - actions without score (discard)

Fix

lua_cfg_transform - silly break break actions

Fix

ratelimit - symbol per bucket

Rework

Allow to restore SSL handlers after keepalive pooling

Rework

Allow to set a different behaviour for actions from settings

Rework

Include SSL flag into keepalive hash

Rework

Make `rspamadm dmarc_report` default behaviour more sane

Rework

Mempool: Use explicit alignment

Rework

Rdns: Use faster and more compact hash table for DNS requests

Rework

Rework SSL flag operations

Rework

Take disabled flag into account

Rework

Timeouts are now global per event and not reseted by IO activity

Rework

Use xxh3 as a default hash and fix memory/alignment issues

Rules

Fix old rules to stop global functions usage

Rules

Fix symbol for DKIM temporary failure

Rules

Remove ancient and inefficient rules

Rules

Slightly reduce MULTIPLE_FROM score

Feature

Add junk_threshold for autolearn

Feature

Add neural test command

Feature

Antivirus: Allow to set fake eicar patterns for testing AV engines

Feature

Lua_cdb: Add cdb building interface

Feature

Ratelimit: Add per bucket configurations

Feature

S3: Allow to store structured data in messagepack

Fix

Add concept of uncancellable events to prevent use-after-free

Fix

Add temporary guard to prevent linked list exploitation

Fix

Another rework of the ucl hashing

Fix

Another try to fix references safety

Fix

Another try to fix rspamd_text passing in the selectors

Fix

Avoid copy for received structure as it has raw C pointers

Fix

Avoid dangling reference

Fix

Correctly check numeric URLs in URL DNS lists

Fix

Delete the correct pointer type

Fix

Dmarc: Always lowercase domain

Fix

Fix compilation of the hyperscan databases with errors

Fix

Fix hash table lookup

Fix

Fix http message flag shift

Fix

Fix parsing of the from_hostname when it is an IP address

Fix

Fix parsing of the unquoted attributes in HTML

Fix

Fix passing of rspamd_text in selectors pipelines

Fix

Fix rubbish QP sequences decoding

Fix

Fix some complicated case with the closing tags parsing

Fix

Fix the case when l tag is too small

Fix

Html: Fix the case where only bgcolor is explicitly set

Fix

Libucl: Fix deletion from ucl objects

Fix

Namespace and add metadata for OpenMetrics, fix interleaving

Fix

Plug memory leak in http settings reload

Fix

Preserve SPF top record in the mempool variable

Fix

Remove aarch64 GC64 workaround

Fix

Remove bogus G_LIKELY

Fix

Spf: Do not parse non TXT DNS replies as TXT replies

Fix

Try to use on_connect/on_disconnect callbacks to handle internal Redis failures

Fix

buffer overflow in rspamc counters

Fix

fix static building

Fix

lua_scanners - message_min_words logic

Fix

src/lua/lua_mimepart.c: fix null dereference

Rework

Further rework of the redis pool

Rework

Redis_pool: fix issues found

Rework

Rework learn and add classify condition

Rework

Save invisible content to a separate buffer

Rework

Start rewriting of the redis pool logic

Rules

Improve zero font rule

Conf

Align ARC scores with DKIM scores

CritFix

Neural: Fix sorting application

Feature

Add a simple dumper for bayes tokens

Feature

Add lua_maps.fill_config_maps function

Feature

Add preliminary exporter to AWS S3

Feature

Add preliminary restore bayes support

Feature

Add race condition protection against hs_helper restarts

Feature

Add rspamd_utf8_strcmp utility

Feature

Add zstd streaming API

Feature

Allow to log severity level explicitly

Feature

Allow to save and show attachment name when inserting AV scan results

Feature

Allow to sort urls for Lua

Feature

Allow to specify different timeouts/retransmits for fuzzy rules

Feature

Aws_s3: Allow to compress data stored

Feature

CMakeLists.txt: Change check and run-test to use rspamd-test-cxx * fixes #3807

Feature

Dmarc_report: allow sending reports in batches

Feature

Fuzzy_check: Allow to disable subject when making short text hash

Feature

Lua_cryptobox: Add keyed ssl hash functions via HMAC

Feature

Lua_task: Add get_urls_filtered method

Feature

Make monitored checks less frequent

Feature

Milter_headers: Add x-rspamd-pre-result header

Feature

Neural: Allow to balance FP/FN for the network

Feature

Ppopagate monitored errors from rbl module

Feature

Pyzor calculate score dynamically Count - WL-Count of default_score in percent

Feature

Rbl: Distinguish flattened and non-flattened selectors in RBL requests

Feature

Re-add pyzor support

Feature

Settings: add ip_map check and rework structure slightly

Feature

Spamassassin: Allow to set the default priority for SA scores

Feature

Strip smtp comments from message id

Feature

add SYSTEM_ZSTD cmake option To use the system zstd instead on the bundled version

Feature

external_relay plugin

Feature

rspamadm clickhouse neural_train subcommand

Fix

#3400 milter_headers: fix inverted logic for extended_headers_rcpt

Fix

ASN: fix _FAIL symbol for when main symbol is disabled

Fix

Add a special logic for text part with no text extraction

Fix

Add diacritics flag for several eu languages

Fix

Another FSM fix to accomodate possibility of multiple consequent ?

Fix

Avoid curse of dynamic array referencing

Fix

Avoid reinitialising neural settings

Fix

Check remain before processing TXT records

Fix

Enable error multiplier on http errors

Fix

Finally rework parsing entities logic

Fix

Fix '==' parsing in the content type attributes parser

Fix

Fix IPv6 expansion for SPF macros

Fix

Fix Mozilla Message-ID detection

Fix

Fix an edge case in BITCOIN_ADDR rule

Fix

Fix brain-damaged behaviour when http request has a custom Host header

Fix

Fix check of limits in email address parsing

Fix

Fix copy&paste error and rework

Fix

Fix expressions logic for and/or and float values

Fix

Fix fuzzy retransmits

Fix

Fix http maps with no or invalid expires data

Fix

Fix last quote character parsing in the content-type state machine

Fix

Fix normalisation flags propagation

Fix

Fix overflow when appending many broken tags

Fix

Fix parsing of rfc2047 tokens with '?' inside

Fix

Fix phishing flag set

Fix

Fix rfc2047 embedded into rfc2231 pieces in special headers

Fix

Fix round-robin rotation

Fix

Fix searching for symbols

Fix

Fix storing of the regexps inside variant

Fix

Fix tokenization near exceptions

Fix

Fix visibility calculations

Fix

Html: Attach inline tags to the structure

Fix

Html: Do not treat empty tags as block tags

Fix

Ical: Do not extract urls from all flags using merely specific ones

Fix

Initialise symcache even if it cannot be loaded properly

Fix

Lua_fuzzy: Remove text parts check when checking image dimensions

Fix

Lua_maps: Fix adjustments for the map type in the complex map definitions

Fix

Lua_task: Fix deleted symbols in has_symbol/get_symbol

Fix

Move metric and symcache link from validation to the init stage

Fix

Oletools: Another try to fix table sorting

Fix

One more default behaviour fix

Fix

Phishing: Rework urls processing

Fix

RBL: was missing some config schema

Fix

Replies: Fix 'Reply-To' handling in task:get_reply_sender

Fix

Rework metrics handling

Fix

Save symcache on exit

Fix

Selectors: Filter nil elements in lists

Fix

Selectors: Properly fix implicit tostring for nils

Fix

Try to fix some broken code in DMARC reporting plugin

Fix

Urls: Fix processing of html urls when it comes to the flags

Fix

Use proper buffer length

Fix

Various visibility fixes

Fix

: ASN: dns cb func should also return in case of an error

Rework

Add composites manager concept

Rework

Add tags definitions

Rework

Allow C code to be compiled with C++ compiler

Rework

Clickhouse: Store url flags

Rework

Composites: Rewrite the composites logic

Rework

Composites: Start rework of the composites framework

Rework

Dmarc: Move check policy function to the common utils

Rework

Dmarc: Rework reports keys structure

Rework

Further work to make html content private

Rework

Html/CSS: Remove css C bindings as they are useless now

Rework

Html/CSS: Rework Lua bindings

Rework

Html/Css: Start rework of the html blocks

Rework

Html: Add images processing logic

Rework

Html: Add traverse function

Rework

Html: Another steps to get rid of gnode

Rework

Html: Convert to variant

Rework

Html: Deal with the utf_content part

Rework

Html: Final rework part for the html processing code

Rework

Html: Fix Lua bindings

Rework

Html: Forgot to add the internal include

Rework

Html: Further html urls rework

Rework

Html: Further rework of the tags content extraction

Rework

Html: Make parameters as a vector again

Rework

Html: Move blocks part

Rework

Html: Move images processing stuff

Rework

Html: Rework lua bindings

Rework

Html: Start html text extraction rework

Rework

Html: Start refactoring of the html tags handling

Rework

Html: Start removing of GNode stuff

Rework

Html: Start rework of the html content structure

Rework

Lua_magic: Try to detect text parts with 8bit characters for non-utf8 encodings

Rework

Move HTML url functions and rework them

Rework

Move and adopt entities handling logic

Rework

Move common and rarely used dmarc code to the library

Rework

Move compression routines outside of rspamd_util library

Rework

Move entities/tags handling

Rework

Phishing: Split from redirectors usage

Rework

Redesign html blocks propagation logic

Rework

Remove tag name string

Rework

Rename phished url to a linked url

Rework

Reorganize dmarc plugin and remove unsupported reporting code

Rework

Reputation: Use more flexible types in get/set functions

Rework

Require proper C++ environment for Rspamd build

Rework

Rework extended urls output

Rework

Rework tags parsing machine

Rework

Slightly improve old regexp API

Rework

Start conversion of the redis pool code to c++

Rework

Try to resolve failed upstreams more agressively

Rework

Use C++ utf8 library with unit tests to trim whitespaces

Rework

Use C++ version for unicode normalisation

Rework

Use C++ version of the lua threads pool

Rules

Add raw addresses to MULTIPLE_FROM options

Rules

Another fix to HTTP_TO_HTTPS rule

Rules

Do not trigger HTML_SHORT_LINK_IMG on external images

Rules

Extend FORGED_X_MAILER

Rules

Extend OLD_X_MAILER

Rules

Fix CTYPE_MIXED_BOGUS for text attachments

Rules

Fix FPs for CTYPE_MIXED_BOGUS

Rules

Fix HTTP_TO_HTTPS rule

Rules

Fix HTTP_TO_HTTPS rule

Rules

Fix zerofont rule (partially)

Rules

Micro-optimize X_PHP_EVAL

Rules

Reduce default weight for R_MISSING_CHARSET

Conf

Add R_DKIM_PERMFAIL to the metric

CritFix

Dkim: Fix simple canonicalisation if multiple signatures are presented

CritFix

Fix controller paths normalisation

Feature

Add INVALID_DATE rule

Feature

Add controller endpoint for training neural

Feature

Add sanity checks for actions thresholds

Feature

Add support of '==' and '!=' in Rspamd expressions

Feature

Composites: Improve composite atoms parser

Feature

Docker: use Debian slim variant

Feature

Elastic: Add some missing fields

Feature

Extract text from img alt attributes

Feature

Improve charset detection logic

Feature

Lua_clickhouse: Add optional row callback for large selections

Feature

Lua_dns_resolver: Add idna_convert_utf8 method

Feature

Lua_mime: Add ability to do multipattern replacement

Feature

Lua_trie: Allow to report start of the match

Feature

Multimap: support adding map values as extra options

Feature

Neural: Move PCA learning to a subprocess

Feature

RBL: support matching content/image URLs only

Feature

RBL: support use of multiple selectors

Feature

Reputation: Allow to specify ip masks

Feature

Support SMIME signed messages container

Feature

Support multiple conditions for symbols

Feature

Support ping in milter mode

Feature

Support rspamd_text in selector regexps

Feature

Use own daemonization routine

Feature

Vadesecure: Implement settings_outbound feature as recommended by Vade

Feature

`rspamadm clickhouse` command

Feature

allow hyperscan for aarch64

Fix

Allow to set priorities between post init scripts

Fix

Allow to use maps for strings that are not zero terminated

Fix

Apply max_lua_urls limit for emails as well

Fix

Arc: Fix CV check on signing

Fix

Arc: Fix signing of the broken ARC chains

Fix

Clickhouse: escape carriage return

Fix

Composites: Allow partial match

Fix

Deduct type of a table methods

Fix

Do not load errored hyperscan database

Fix

Do not process links in ignored html tags

Fix

Fix ClamAV result for cached encrypted file (#3395)

Fix

Fix canonicalisation when l= tag is presented

Fix

Fix flag shift

Fix

Fix handling of skip/skip_process http flags

Fix

Fix html attachments checks

Fix

Fix issue with pushing binary formats to Lua strings

Fix

Fix logging for rspamadm

Fix

Fix off-by-one with init check

Fix

Fix parsing of escape characters in quoted pairs

Fix

Fix pushing ucl strings with \0 inside

Fix

Fix quoted-printable soft newlines bugged case

Fix

Fix settings in case actions are set to null (#3415)

Fix

Fix several issues with auth results producing

Fix

Fix smtp comments exclusion

Fix

Fix smtp date syntax definition

Fix

Fix substring search in case if srchlen == inlen

Fix

Fix text selectors

Fix

Honour `systemd` setting when logging to console (#3514)

Fix

Html: Add entities collisions prevention logic (e.g. for mathml entities)

Fix

Lua_auth_results: Quote potentially bad values in AR header

Fix

Multimap: Fix flags usage

Fix

Multimap: Fix scoring for combined maps

Fix

Plug GList * leak in redis pool

Fix

RBL: allow for multiple matches of the same label if types are different

Fix

Rely on libev checks for file maps

Fix

Restore simple dkim canonicalisation mode

Fix

Return MimeCharset as we work with emails...

Fix

Spamassassin: Fix pcre_only flags

Fix

Spamassassin: Preserve 'pcre_only' flag when dealing with regexp replacements

Fix

Try to fix GError leak

Fix

Try to fix a mess with settings loading by adding priorities

Fix

Try to move setings initialisation to a later stage

Fix

Use dup fd in milter handler to avoid races with the proxy

Fix

Use message pointer to avoid obsolete data to be cached

Rework

Add final cleanup logic

Rework

Add preliminary support of hyperscan caching for re maps

Rework

Add stale cache removal

Rework

Clickhouse: Improve performance

Rework

Distinguish between strict config test mode

Rework

Furhter logging improvements

Rework

Milter_headers: improve extended_headers_rcpt support

Rework

Move parsers to a separate lua library

Rework

Neural: Skip composite symbols

Rework

Rbl: Rework defaults logic

Rework

Some tunes to cache saving

Rework

Track maps origins

Rework

Use full crypto hash for regexp maps

Rules

Remove broken rule

Conf

Add missing symbols

Conf

Add missing symbols

Conf

Fix fat-fingers typo

Conf

Fix wrong comment in options.inc

Conf

Neural: Fix the default name for max_trains

Conf

Register a known symbol

Conf

Spf: Add R_SPF_PERMFAIL symbol

CritFix

Arc: Fix ARC validation for chains of signatures

CritFix

Distinguish socketpairs between different fuzzy workers

CritFix

Fix IDNA dots parsing

CritFix

Fix test assertion method

CritFix

Fix usage of crypto_sign it should be crypto_sign_detached!

Feature

Add BOUNCE rule

Feature

Add controller plugins support and selectors plugin

Feature

Add maps query method

Feature

Add minimal delay to fuzzy storage

Feature

Add multiple base32 alphabets for decoding

Feature

Add preliminary support of BCH addresses

Feature

Add query_specific endpoint

Feature

Allow multiple base32 encodings in Lua API

Feature

Allow to specify nonces manually

Feature

Controller: Allow to pass query arguments to the lua webui plugins

Feature

Fuzzy_check: Add gen_hashes command

Feature

Fuzzy_check: Add weight_threshold option for fuzzy rules

Feature

Implement address retry on connection failure

Feature

Improve limits in pdf scanning

Feature

Initial support of subscribe command in lua_redis

Feature

Lua_cryptobox: Add secretbox API

Feature

Lua_text: Add encoding methods

Feature

Milter_headers: Allow to activate routines via users settings

Feature

PDF: Add timeouts for expensive operations

Feature

Preliminary maps addon for controller

Feature

Split pdf processing object and output object to allow GC

Feature

Support BLIS blas library

Feature

Support input vectorisation by recvmmsg call

Feature

Support multiple base32 alphabets

Feature

add queueid, uid, messageid and specific symbols to selectors [Minor] use only selectors to fill vars in force_actions message

Feature

allow variables in force_actions messages

Feature

extend lua api

Fix

#3249

Fix

Allow to adjust neurons in the hidden layer

Fix

Another try to fix email names parsing

Fix

Arc: Allow to reuse authentication results when doing multi-stage signing

Fix

Arc: Fix bug with arc chains verification where i>1

Fix

Arc: Sort headers by their i= value

Fix

Change neural plugin's loss function

Fix

Deal with double eqsigns when decoding headers

Fix

Default ANN names in clickhouse

Fix

Disable reuseport for TCP sockets as it causes too many troubles

Fix

Disable text detection heuristics for encrypted parts

Fix

Distinguish DKIM keys by md5

Fix

Distinguish type from flags in register_symbol

Fix

Dmarc: Unbreak reporting after cf2ae3292ac93da8b6e0624b48a62828a51803c9

Fix

Do not flag pre-result of virus scanners as least if action is reject

Fix

Do not use GC64 workaround on 32bit platforms, omg

Fix

Exclude damaged urls from html parser

Fix

Fix FREEMAIL_REPLYTO_NEQ_FROM_DOM

Fix

Fix FROM_NEQ_ENVFROM

Fix

Fix FWD_GOOGLE rule (#1815)

Fix

Fix adding of the empty archive file for gzip

Fix

Fix aliases in forged recipients and limit number of iterations

Fix

Fix authentication results insertion

Fix

Fix calling of methods in selectors

Fix

Fix clen length for hiredis...

Fix

Fix endless loop if broken arc chain has been found

Fix

Fix false - operation

Fix

Fix get_urls table invocation

Fix

Fix group based composites

Fix

Fix headers passing in rspamd_proxy

Fix

Fix incomplete utf8 sequences handling

Fix

Fix lua_next invocation

Fix

Fix lua_parse_symbol_type function logic

Fix

Fix multiple listen configuration

Fix

Fix occasional encryption of the cached data

Fix

Fix parsing boundaries with spaces

Fix

Fix passing of methods arguments

Fix

Fix poor man allocator algorithm

Fix

Fix regexp selector and add flattening

Fix

Fix rfc base32 encode ordering (skip inverse bits)

Fix

Fix rfc based base32 decoding

Fix

Fix sockets leak in the client

Fix

Fix storing of the original smtp from

Fix

Fix types check and types usage in lua_cryptobox

Fix

Fix unused results

Fix

Fuzzy_check: Disable shingles for short texts (really)

Fix

Ical: Fix identation grammar

Fix

Improve part:is_attachment logic

Fix

Mmap return value must be checked versus MAP_FAILED

Fix

One more fix to skip images that are not urls

Fix

Pdf: Support some weird objects with no newline before endobj

Fix

Rbl: Fix ignore_defaults in conjunction with ignore_whitelists

Fix

Restore support for `for` and `id` parts in received headers

Fix

Segmentation fault in contrib/lua-lpeg/lpvm.c on ppc64el

Fix

Skip spaces at the boundary end

Fix

Slashing fix: fix captures matching API

Fix

Spamassassin: Rework metas processing

Fix

Store reference of upstream list in upstreams objects

Fix

Understand utf8 in content-disposition parser

Fix

Unify selectors digest functions

Fix

Use `abs` value when checking composites

Fix

Use strict IDNA for utf8 DNS names + add sanity checks for DNS names

Fix

Use unsigned char and better support of utf8 in ragel parser

Fix

add missing selector_cache declaration

Rework

Allow to specify exat metric result when adding a symbol

Rework

Change and improve openblas detection and usage

Rework

Close listen sockets in main after fork

Rework

Further rework of lua urls extraction API

Rework

Lua_cryptobox: Allow to store output of the hash function

Rework

Lua_task: Add more methods to deal with shadow results

Rework

Modernize logging for expressions

Rework

Remove empty prefilters feature - we are not prepared...

Rework

Remove old FindLua module, disable lua fallback when LuaJIT is enabled

Rework

Rework and refactor forged recipients plugin

Rework

Rework expressions processing

Rework

Rework fuzzy commands processing

Rework

Rework url flags handling API

Rework

Rework urls extraction

Rework

Split operations processing and add more debug logs

Rework

Update zstd to 1.4.5

Rework

Use google-ced instead of libicu chardet as the former sucks

Rework

add alias util:parse_addr for util:parse_mail_address

Rework

get rid of util:parse_addr duplicating the util:parse_mail_address, replace where used

Rules

Allow prefix for bitcoin cash addresses

Rules

More fixes for bitcoin cash addresses decoding

Rules

Refactor bleach32 addresses handling

Conf

Mark Rspamd emailbl as ignore whitelist

Conf

RBL: Add missing emails = true option

Feature

Add support for scripts in fuzzy storage

Feature

Arc: Add whitelisted_signers_map option

Feature

Implement hosts file processing

Feature

Neural: Introduce classes bias that allows non-equal classes learning

Feature

Update libev to 4.33

Fix

Another brain damage html standard adoptions

Fix

Another fix for brain damaged obs-fws state

Fix

Fix flags that caused force_actions failure

Fix

Fix logging issue

Fix

Fix lua symbols scores registration when config does not define scores

Fix

Fix opaque maps logic

Fix

Fix parsing of the html tags with no spaces after attributes

Fix

Fix some corner cases in urls parsing, add limits

Fix

Fix tlds extraction if custom composition rules are used

Fix

Fix variables replacement in mempool

Fix

Improve base64 detection

Fix

Normalize dynamic scores in ANN correctly

Fix

Plug memory leak introduced by #3153

Fix

Stat_redis_backend: Fix memory leak and simplify learn path

Fix

Try hard to deal with ghost workers

Fix

metadata_exporter default formatter

Rework

Change the way to extract URLs when dealing with alternative parts

Rework

Fix various url extraction issues

Rework

Re cache: Load compiled hyperscan in the main process as well

Rework

Re cache: Load hyperscan early

Rework

Rework URL structure: adjust tld part

Rework

Rework URL structure: host field

Rework

Rework URL structure: more structure optimisations

Rework

Rework URL structure: user field

Rework

URL: Another update for urls extraction logic

Rework

Urls: Improve query urls handling

Rework

Urls: adopt html related stuff

Rework

Urls: more rework of the urls sets

Rework

Urls: process query urls in HTML urls correctly

Rework

Urls: rework urls hash structure

Rework

Urls: update lua libraries

Rework

Use multiple search tries for different url extraction types

CritFix

Fix parsing of the content type attributes

Feature

Clickhouse: Add extra columns support

Feature

Rbl: Add url_compose_map option for RBL rules

Fix

'R' flag is for all headers regexp

Fix

Allow to reset settings id from Lua (e.g. because of the priority)

Fix

Avoid collisions in mempool variables by changing fuzzy caching logic

Fix

Avoid strdup usage for symbols options

Fix

Do not trust stat(2) it lies

Fix

Filter all options for symbols to have sane characters

Fix

Fix all headers iteration

Fix

Fix allowed_settings for neural

Fix

Fix listen socket parsing

Fix

Fix maps expressions evaluation

Fix

Fix sentinel connections leak by using async connections

Fix

Fix smtp message on passthrough result

Fix

Fix tld compositon rules

Fix

Fuzzy_storage: Do not check for shingles if a direct hash has been found

Fix

Lua_mime: Do not perform QP encoding for 7bit parts

Fix

Neural: Distinguish missing symbols from symbols with low scores

Fix

Support listening on systemd sockets by name

Rework

Logger infrastructure rework

Rework

Refactor libraries structure

Rework

Rework SSL caching

Rework

Update snowball stemmer to 2.0 and remove all crap aside of UTF8

Conf

SPF is no longer a C module

Conf

Update spamtrap map path example

CritFix

Fix html entities decoding

CritFix

Fix re cache when mix of pcre and hyperscan is used

Feature

Allow milter code to deal with multiple headers

Feature

Antivirus: Add avast support

Feature

Dkim_signing: Allow to sign via milter_headers

Feature

Implement content hashes

Feature

Lua_text: Add regexp split iterator method

Feature

Lua_text: Implement flattening of the input tables

Feature

Send quit command to Redis

Feature

Speed up is_ascii function

Feature

Spf: Add external_relay option

Fix

Avoid double escaping

Fix

Fix O(N^2) algorithm

Fix

Fix arc seal validation

Fix

Fix base tag processing according to stupid HTML renderer behaviour

Fix

Fix dealing with `\0` in ucl strings and JSON

Fix

Fix gpg parts misdetection

Fix

Fix ignored symbols exporting

Fix

Fix processing of numeric url's

Fix

Fix processing of the closed tcp connections

Fix

Fix regexp type check for pcre2

Fix

Fix urls encode function

Fix

Fix urls shifting when doing decode to include separators

Fix

Fix white on white rule and add is_leaf flag

Fix

Further fixes in charset detection

Fix

Ignore diacritics in chartable module for specific languages

Fix

Limit size of symbols options by max_opts_len option

Fix

More fixes in html tag content calculations

Fix

Plug memory leak in fuzzy storage

Fix

Process high priority settings even if settings/id has been specified

Fix

Select a different upstream on last retransmit

Fix

Treat soft hyphen as zero width space

Fix

Try harder to watch the lifetime of the key_stat

Fix

Use ipv6-mapped-ipv4 addresses in radix trie

Rework

Allow to add userdata as symbols options

Rework

Change mime part specifics handling

Rework

Move LRU SPF cache from spf plugin

Rework

Rework HTML tags content attachment

Rework

Rework options hash structure

Rework

Start lua_content library

Rework

Stop using of uthash for http headers

Rework

Use faster hashing approach for memory pools variables

Rules

Add PDF related rules

Conf

Antivirus: Fix the default config

Feature

Add verdict library in lua

Feature

Allow exception when choosing upstream

Feature

Allow to disable symbols from the metric config

Feature

Allow to limit maps per specific worker

Feature

Always validate Rspamd protocol output

Feature

Antivirus: Add preliminary virustotal support

Feature

Clickhouse: Rework Clickhouse collection logic

Feature

Improve base64 usage

Feature

Shutdown timeout is now associated with task timeout

Fix

#3129 Multiple classifiers on redis working incorrectly

Fix

Allow real upstreams configuration

Fix

Another try to fix slow callbacks and timers

Fix

Check results of write message as SSL can bork them

Fix

Clickhouse: Avoid potential races in collection

Fix

Clickhouse: Fix periodic script

Fix

Fail DNS upstream on each retransmit attempt

Fix

Fix consistent hashing when upstreams are marked inactive

Fix

Fix issues found

Fix

Fix off-by-one in retries for the proxy

Fix

Fix termination

Fix

Fix upstreams exclusion logic

Fix

Fix utf8 validation for symbols options and empty strings

Fix

Oops, fix maps reload

Fix

Rbl: Allow utf8 lookups for IDN domains

Fix

Sigh, another try to fix brain-damaged openssl

Rework

Another phase of finish actions rework

Rework

Further cmake system rework

Rework

Further isolation of the controller's functions

Rework

Make cmake structure more modular

Rework

Move cmake modules to a dedicated path

Rework

Replace controller functions by any scanner worker if needed

Rework

Rework final scripts logic

Rework

Rewrite rspamd_str_make_utf_valid function

Conf

Update neural.conf

CritFix

Fix dkim verification for multiple headers listed

Feature

Add support of uudecode

Feature

Allow to explicitly set events backend

Feature

Implement configurable limits for SPF lookups

Feature

Lua_scanners: Use lua magic for inclusion/exclusion logic

Feature

Multimap: Do not check files in office archives

Feature

Neural: Add sampling when storing training vectors

Feature

SPF: Allow to disable AAAA checks in configuration

Feature

Spf: Add limits configuration support

Feature

Store etag in cached HTTP maps + better logging

Feature

Support segwit BTC addresses, fix LTC verification

Feature

Support uuencoding

Fix

Add configurable number of threads for OpenBLAS

Fix

Add workaround for ragel 7 in hyperscan related maps code

Fix

Another fix for numeric urls parsing

Fix

Correct EMA time calculations

Fix

Do not treat archives as text

Fix

Do not use strdup on data extracted from lua

Fix

Fix a failure calcuating URL reputation.

Fix

Fix crash due to constructors init order

Fix

Fix crash on parts with no cd

Fix

Fix empty prefilters that require mime structures

Fix

Fix event loop creation

Fix

Fix issues sending DMARC reports.

Fix

Fix misprint

Fix

Fix saving of the file maps

Fix

Fix size calculations when converting from utf16

Fix

Fix support of disable_monitoring in rbl

Fix

Fix use-after-free

Fix

Fix zip files check to relax requirements

Fix

Important hiredis fixes

Fix

Lot's of fixes in maps check logic

Fix

Lua_tcp: Deal with temporary fails on write

Fix

Lua_tcp: Make write errors fatal and rework error handlers

Fix

Meta: Filter some more values

Fix

Neural: Add protection agains infinities

Fix

Oops, fix math.huge invocation

Fix

Plug memory leak

Fix

Sigh, another email to string fix

Fix

Try to fix another ownership race in ssl connection

Fix

Uuencode: Fix parsing of corrupted uuencode

Fix

lua_scanners - razor rename need_check function

Rework

Require CMake 3.9 to work, remove manual lto crap

Conf

Add BROKEN_HEADERS_MAILLIST composite

Conf

Add path to greylist-whitelist-domains.inc

Conf

Clarify documentation in the config files

Conf

Introduce maps.d directories

Conf

Log settings id by default

Conf

Make LEAKED_PASSWORD_SCAM a composite rule again

Conf

Move all surbl/emails rules to rbl

Conf

Register new Spamhaus codes

Conf

Remove configs for deleted modules

Conf

Remove surbl parts, fix hash_format attribute

Conf

Show autolearn sample

Conf

Slashing: Change default stats backend to Redis

Conf

Surbl: Utilise new `check_emails` option

Conf

Update header

Conf

Use multi-prefixes RBLs in the default config

CritFix

Deal with case-sensivity in Content-Disposition parser

CritFix

Eliminate old endpoint

CritFix

Fix case sensivity when parsing Content-Type

CritFix

Fix loading of DKIM public keys

CritFix

Fix procesing of urls

CritFix

Fix whitelisting when both spf and dkim are required to be valid

CritFix

Langdet: Fix language detection where no stop words found

Feature

Add description to the groups

Feature

Add limit for number of URLs in Lua

Feature

Add logging of groups to the log_format

Feature

Add lua_smtp library

Feature

Add maps cache and type refinement

Feature

Add p0f scanner

Feature

Adopt emails module to use lua_maps

Feature

Allow options matching in composites

Feature

Allow selectors in rbl module

Feature

Allow to output group results

Feature

Asn: Allow to use bgpdump when NET::MRT is broken

Feature

Calculate tokens occurrences distribution

Feature

Clickhouse: Add authenticated user and settings id columns

Feature

Clickhouse: Store groups data

Feature

Clickhouse: Utilise LowCardinality feature

Feature

Implement Redis prefixes registration logic

Feature

Implement settings id propagation between deps

Feature

Improve AV results caching

Feature

Improve autolearning

Feature

Improve logging locking logic (remove it actually)

Feature

Improve settings processing

Feature

Langdet: Limit number of stop words to be checked

Feature

Libucl: Allow to sort keys in ucl objects

Feature

Lua_config: Extend get symbols method

Feature

Lua_maps: Allow static maps for key-value pairs

Feature

Lua_mimepart: Add function filter_words

Feature

Lua_selectors: Add `words` selector

Feature

Lua_selectors: Add sort and uniq transform functions

Feature

Lua_selectors: Allow table arguments for selectors

Feature

Lua_tcp: Add preliminary support of SSL connections

Feature

Lua_trie: More flexible API

Feature

Lua_util: Add filter_specific_url function

Feature

Lua_util: table_digest can now recursively traverse tables

Feature

Maillist: Improve detection

Feature

Maps: Allow caching for complex maps

Feature

Monitored: Support random lookups

Feature

Multimap: Add combined maps prototype

Feature

Multimap: Add dependend maps via redis keys selectors

Feature

Multimap: Allow multiple email addresses matches

Feature

Multimap: Also check detected charset when do filename checks

Feature

Output number of messages processed to proctitle

Feature

Perform clean SSL shutdown

Feature

Performance: Do not use base64 SIMD version for bad inputs

Feature

RBL: Support bit results in replies

Feature

RBL: Support type specific prefixes

Feature

Ratelimit: Consider number of SMTP recipients

Feature

Rbl: Add ability to check urls

Feature

Rbl: Add resolve_ip based RBLs

Feature

Rbl: Make config checks much more strict

Feature

Rbl: Support per-rule whitelists

Feature

Rbl: Support process script

Feature

Rbl: Support replyto addresses

Feature

SURBL: Allow to check email domains

Feature

Selectors: Add `list` generator

Feature

Selectors: Add `specific_urls` extractor

Feature

Selectors: Add flatten function

Feature

Selectors: Support filter_map and apply_map functions

Feature

Store Clickhouse data outside of lua alloc

Feature

Support caching for encrypted files and macros

Feature

Support images when extracting urls

Feature

Support more hyperscan flags

Feature

Support protocol flags

Feature

URL: Apply stringprep to hostnames to filter garbage

Feature

Upstreams: Add lazy resolving logic to all upstreams

Feature

Upstreams: Set noresolve flag on numeric upstreams

Feature

Use `scores` in apply section

Feature

Use maps logic from lua_maps for multimap

Feature

Use random monitored in rbl module

Feature

lua_scanners - add Razor support

Fix

Add another safe-guard in urls processing

Fix

Add debug to ssl, fixed write hangs

Fix

Add missing groups to C callback symbols

Fix

Add more checks for ghosts symbols

Fix

Allow to enable or add new actions via settings

Fix

Allow to set 0 size for spf/dkim caches

Fix

Another bunch of fixes towards protocol mess

Fix

Another fix to deal with bad URLs

Fix

Arc: Another bunch of fixes for arc signing

Fix

Arc: More arc signing fixes

Fix

Avoid another overflow in fpconv

Fix

Clickhouse: Fix quoting

Fix

Clickhouse: Fix retention query quoting

Fix

Distinguish empty and non-empty prefilters

Fix

Distinguish remote and local addrs parsing

Fix

Do not assert if length of sig is bad, just fail verification

Fix

Do not assert if we have broken mime boundary in the headers

Fix

Do not call implicit strlen to avoid issues

Fix

Do not count images urls when checking url regexps for compatibility

Fix

Do not output rbl suffix in symbol option

Fix

Do not use config pool to avoid issues with double reload

Fix

Do not use ephemeral string

Fix

Do not use lightuserdata for traceback

Fix

Do not use priority in metric registration

Fix

Emails: Check email sanity before testing on BL

Fix

Emails: Fix misprint in key name

Fix

Escape utf in regexp to dodge ragel/hyperscan issue

Fix

Extend task_timeout to postfilters stage

Fix

Fix ARC signing after fixing another bug in it...

Fix

Fix AV scan logic

Fix

Fix DMARC_NA behaviour in case of no valid policies

Fix

Fix LRU hash iteration logic

Fix

Fix alignment mess

Fix

Fix configuring symbols without scores

Fix

Fix disabling of the actions

Fix

Fix dkim signing exceptions

Fix

Fix embedded images linking logic

Fix

Fix events leak

Fix

Fix eviction corner case

Fix

Fix fuzzy image score calculation #2962

Fix

Fix hang in fuzzy_learn when explicit rotation is set

Fix

Fix headers propagation logic

Fix

Fix hearbeats restart issue

Fix

Fix history reset

Fix

Fix log parameter

Fix

Fix lua_ip_equal logic

Fix

Fix more issues with nested messages + tests

Fix

Fix normalization of non-alphabet based languages

Fix

Fix offsets when parsing message/rfc822 in multipart

Fix

Fix options in rbl symbols

Fix

Fix out of bound access in lua logger

Fix

Fix out-of-bound read in qp decode

Fix

Fix parent CTE propagation

Fix

Fix parsing of the received headers with empty part

Fix

Fix pending checks for events

Fix

Fix printing of NULL pointer with fixed length

Fix

Fix race condition in watcher handler

Fix

Fix read-after-end in quoted printable decoding

Fix

Fix redis sentinel support

Fix

Fix registry leak in case of DNS errors

Fix

Fix reload logic

Fix

Fix sending of large entries via HTTPS

Fix

Fix settings reload

Fix

Fix some more corner cases for fpconv

Fix

Fix trie code when there are regexps and Hyperscan is absent

Fix

Further fixes to printing of the FP numbers

Fix

Fuzzy_check: Fix timeouts

Fix

Grrr, fix empty ip case

Fix

Html: Fix processing of fjlig entity

Fix

Lang_det: Try better to distinguish Chinese and Japanese

Fix

Lua_mime: Fix reversed extensions map

Fix

Lua_task: Fix message-less API

Fix

Lua_tcp: Report connection failures

Fix

Lua_tcp: Various fixes and debugging improvements

Fix

Metadata_exporter: This plugin is idempotent not a postfilter

Fix

More fixes to extract_specific_urls

Fix

More stages fixes

Fix

Neural: Another bunch of fixes

Fix

Neural: use version in ANN key profile

Fix

Postpone lua state destruction to allow lua dtors to be used

Fix

Prefer surbl/emails rule on rbl to preserve compatibility

Fix

RBL: Fix behaviour of emails_domainonly

Fix

Ratelimit: Fix dynamic score

Fix

Rbl: Fix emailbl functions

Fix

Really fix hyperscan workaround

Fix

Set sanity limits for pcre2

Fix

Settings: Fix settings check flags

Fix

Sort keys when getting data from Lua when filling rules

Fix

Statistics: Do not query Redis tokens when there are no learns

Fix

Stop IO event on write finished in http connection

Fix

Use heuristically detected text parts data

Fix

Various fixes to QP encoding algorithm

Fix

Various fixes to SSL state machine handler

Fix

Various fixes to asn module

Fix

Workaround for empty charset in rfc2231 encoding

Rework

Allow execution of async events when hs compiles regexps

Rework

Bayes expiry: eliminate `default` expiration mode

Rework

Dkim: Remove signing code

Rework

Dkim_signing: Move sign condition to dkim_signing

Rework

Do not lowercase all data send to ClickHouse

Rework

Drop url tags

Rework

Eliminate lua_squeeze as it has shown no improvements

Rework

Eliminate virtual scan time as it is useless

Rework

Lua core: Use lightuserdata to index classes

Rework

Lua_util: Another rework for extract_specific_urls

Rework

Migrate from ip_score to reputation

Rework

Move mime modification functions to lua_mime library

Rework

Rbl: Major whitelisting logic rework

Rework

Remove deprecated plugins

Rework

Remove log helper worker

Rework

Remove rspamd.classifiers.lua

Rework

Rename filter.h to a more sane name

Rework

Reorganise selectors implementation

Rework

Replace linenoise with replxx

Rework

Reputation: Remove ipnet from the ip reputation

Rework

Reputation: Slashing - change name of symbols

Rework

Rework children operations

Rework

Rework config reload

Rework

Rework expression API

Rework

Rework image urls processing

Rework

Rework initialisation to reduce static leaks count

Rework

Rework request headers processing

Rework

Slashing: Change versioning schema - move to 2.0

Rework

Slashing: Turn off postfilters when passthrough result is set

Rework

Start moving to replxx

Rework

Stop support of signed HTTP maps to simplify code

Rework

Store ASN as UInt32 in ClickHouse

Rework

Url_redirector: Rewrite plugin

Rework

Use a dedicated library for autolearn

Rework

Use libsodium instead of hand crafted crypto implementations

Rework

Use opaque structure to store a table of mime headers

Rules

Add dedicated bitcoin addresses filter rule

Rules

Add more detection to LEAKED_PASSWORD_SCAM

Rules

Catch LTC addresses

Rules

Reduce weight of RSPAMD_EMAILBL

Rules

Rework LEAKED_PASSWORD_SCAM rule one more time

Release 1.9.4: 23 May 2019

CritFix

Fix case sensitivity when parsing Content-Type

Fix

Arc: Another bunch of fixes for arc signing

Fix

Arc: More arc signing fixes

Fix

Avoid another overflow in fpconv

Fix

Fix ARC signing after fixing another bug in it...

Fix

Fix dkim signing exceptions

Fix

Fix some more corner cases for fpconv

Fix

Further fixes to printing of the FP numbers

Fix

Ratelimit: Fix dynamic score

Release 1.9.3: 13 May 2019

Conf

Add IP_SCORE_FREEMAIL composite rule

Feature

Add cryptobox method to generate dkim keypairs

Feature

Add fast hashes to lua cryptobox hash

Feature

Add least passthrough results

Feature

Allow oversign if exists mode

Feature

Clickhouse: Modernise table initial schema

Feature

Implement IUF interface for specific fast hashes

Feature

Lua_util: Allow to obfuscate different fields

Feature

Tune memory management in Rspamd and Lua

Fix

Avoid buffer overflow when printing long lua strings

Fix

Change the default oversigning headers to a more sane list

Fix

Clickhouse: Do not store digest as it is not needed now

Fix

Clickhouse: Fix lots of storage issues

Fix

Clickhouse: Support custom actions

Fix

Deny URLs where hostname is bogus

Fix

Do not blacklist mail by SPF/DMARC for local/authed users

Fix

Fix DoS caused by bug in glib

Fix

Fix UCL parsing of the multiline strings

Fix

Fix buffer overflow when printing small floats

Fix

Fix init code for servers keypairs cache

Fix

Fix issue with urls with no tld (e.g. IP)

Fix

Fix memory in arc signing logic

Fix

Fix memory leak in language detector during reloads

Fix

Fix mixed case content type processing

Fix

Fix processing of the ip urls in file

Fix

Fix use after free

Fix

HTML: Fix `size` attribute processing

Fix

Hum, it seems that 99ff1c8 was not correct

Fix

Lua_task: Fix task:get_from method

Fix

Preserve fd when mapping file to scan

Fix

Re-use milter_headers settings when doing arc signing

Fix

Set dmarc force action as least action

Fix

Switch to GMT

Fix

allow PKCS7 signatures to be text/plain, too

Rework

Move lua_worker to a dedicated unit

Release 1.9.2: 16 Apr 2019

Conf

Allow to load users plugins from plugins.d

Conf

oversign openpgp and autocrypt headers

Feature

Add SPF FFI library for Lua

Feature

Add more verbosity for SPF caching

Feature

Antivirus: Handle encrypted files specially

Feature

Clickhouse: Slashing - add new fields to CH

Feature

Dkim_signing: Add OpenDKIM like signing_table and key_table

Feature

Dkim_signing: Allow to use new options as maps

Feature

Import fpconv library

Feature

Lua_maps: Allow static regexp and glob maps

Feature

Parse ical files

Feature

Rspamadm: Add dns_tool utility

Feature

Store SPF records digests

Feature

Use fpconv girsu2 implementation for printing floats

Fix

Clickhouse: Use integer seconds when inserting rows

Fix

Fix floating point printing

Fix

Fix processing of embedded urls

Fix

Lua_clickhouse: Fix CH errors processing

Fix

Make spf digest stable

Fix

Properly detect encrypted files in zip archives

Fix

Slashing: Store times in GMT timezone in ClickHouse

Rules

Add additional conditions to perform BTC checks

Rules

Fix pay-to-hash addresses validation

Release 1.9.1: 5 Apr 2019

Conf

Add vendor groups for symbols

Feature

Add `rspamadm template` command

Feature

Allow to add messages from settings

Feature

Allow unconnected DNS servers operations

Feature

Check limits after being set, migrate to uint64

Feature

Greylist: Allow to disable greylisting depending on symbols

Feature

Improve lua binary strings output

Feature

Mime_types: Implement user configurable extension filters

Feature

Mime_types: When no extension defined, detect it by content

Feature

Preprocess config files using jinja templates

Feature

Replies: Filter replies sender to limit whitelisting to direct messages

Feature

Treat all tags with HREF as a potential hyperlinks

Feature

Validate BTC addresses in LEAKED_PASSWORD_SCAM

Fix

Add crash safety for HTTP async routines

Fix

Another fix for Redis sentinel

Fix

Clickhouse: Fix table schema upload

Fix

Core: Fix squeezed dependencies handling for virtual symbols

Fix

Finally fix default parameters parsing in actions section

Fix

Fix ES sending logic (restore from coroutines mess)

Fix

Fix finishing script for clickhouse collection

Fix

Fix priority for regexp symbols registriation

Fix

Fix various issues found by PVS Studio

Fix

Initialize lua debugging earlier

Fix

Neural: Fix training

Fix

Rework cached Redis logic to avoid sentinels breaking

Fix

SURBL: Fix regression in surbl module

Fix

Fix double signing in the milter

Rework

Change lua global variables registration

Rework

Rework HTML content urls extraction

Rework

Start rework of aliasing in Rspamd

Release 1.9.0: 12 Mar 2019

Conf

Add missing includes

Conf

Move to options

Conf

Rbl: DWL is actually special whitelist

Conf

Relax some uribl rules

Conf

Remove abuse.ch

CritFix

Html: Entities are not valid within tag params values

Feature

Add `rspamadm mime sign` tool

Feature

Add configgraph utility

Feature

Add dedicated ZW spaces detection for URLs

Feature

Add flag to url object when visible part is url_like

Feature

Add method task:lookup_words

Feature

Add pyzor support (by crosenberg)

Feature

Allow to add upstream watchers to Lua API

Feature

Allow to set rewrite subject pattern from settings

Feature

Better escaping of unicode

Feature

Clickhouse: Allow to store subject in Clickhouse

Feature

Core: Add QP encoding utility

Feature

Core: Add libmagic detection for all parts

Feature

Core: Add support for gzip archives

Feature

Core: Allow to construct scan tasks from raw data

Feature

Core: Detect charset in archived files

Feature

Core: Ignore and mark invisible spaces

Feature

Core: Normalise zero-width spaces in urls

Feature

Core: Process data urls for images

Feature

Core: Relax quoted-printable encoding

Feature

Core: Support RFC2231 encoding in headers

Feature

Core: Support telephone URLs

Feature

Core: allow to emit soft reject on task timeout

Feature

DCC: Add bulkness and reputation checks to dcc

Feature

Elastic: Modernize plugin

Feature

Export visible part of url to lua

Feature

Fuzzy_storage: add preliminary support of rate limits

Feature

HTML: Specially treat data urls in HTML

Feature

Implement event watchers for upstreams

Feature

Implement includes tracing in Lua

Feature

Improve dkim part in configwizard

Feature

Lua_scanners: Add VadeSecure engine support

Feature

Lua_task: Add flexible method to get specific urls

Feature

Mime_types: Add MIME_BAD_UNICODE rule

Feature

Mime_types: Use detected content type as well

Feature

Plugins: Add preliminary version of the external services plugin

Feature

Query sentinel on master errors

Feature

Regexp: Allow local lua functions in Rspamd regexp module

Feature

Rspamadm: Allow to append footers to plain messages

Feature

Rspamadm: Allow to rewrite headers in messages

Feature

Selectors: Add `ipmask` processor

Feature

Settings: Allow hostname match

Feature

Settings: Allow local when selecting settings

Feature

Settings: Allow multiple selectors

Feature

Settings: Allow to inverse conditions

Feature

Support User-Agent in HTTP requests

Feature

Support ed25519 dkim keys generation

Feature

Try to filter bad unicode types during normalisation

Feature

external_services - oletools (olefy) support

Feature

lua_scanners - icap protocol support

Feature

lua_scanners - spamassassin spam scanner

Fix

Add filter for absurdic URLs

Fix

Add some more cases for Received header

Fix

Allow to disable/enable composite symbols

Fix

Arc: Use a separated list of headers for arc signing

Fix

Archive: Final fixes for 7z archives

Fix

Clickhouse: Fix database usage

Fix

Controller: Make save stats timer persistent

Fix

Core: Detect encrypted rarv5 archives

Fix

Core: Don't detect language twice

Fix

Core: Fix address rotation bug

Fix

Core: Fix content calculations for message parts

Fix

Core: Fix emails comments parsing and other issues

Fix

Core: Fix etags support

Fix

Core: Fix headers folding on the last token

Fix

Core: Fix iso-8859-16 encoding

Fix

Core: Fix log_urls flag (and encrypted logging)

Fix

Core: Fix part length when dealing with boundaries

Fix

Core: Fix parts distance calculations

Fix

Core: Fix processing of NDNs of certain type

Fix

Core: Implement logic to find some bad characters in URLs

Fix

Core: treat nodes with ttl properly in lru cache

Fix

Fix Content-Type parsing

Fix

Fix HTTP headers signing case

Fix

Fix control interface

Fix

Fix deletion of the duplicate headers

Fix

Fix emails filtering in emails module

Fix

Fix greylisting log message and logic

Fix

Fix issues with storing of the accepted addr in rspamd control

Fix

Fix maps object update race condition

Fix

Fix memor leaks and whitespace processing

Fix

Fix processing of null bytes in headers

Fix

Fix rcpt_mime and from_mime in user settings

Fix

Fix rfc2047 decoding for CD headers

Fix

Fix rfc2231 for Content-Disposition header

Fix

Fix setting of the subject pattern in config

Fix

Greylist: fix records checking

Fix

HTML: Another HTML comments exception fix

Fix

HTML: Another entities decoding logic fix

Fix

HTML: Fix HTML comments with many dashes

Fix

HTML: Fix entities in HTML attributes

Fix

HTML: Fix some more SGML tags issues

Fix

Ignore whitespaces at the end of value in DKIM records

Fix

MID module: Fix DKIM domain matching

Fix

Milter_headers: Fix remove_upstream_spam_flag and modernise config

Fix

Mime_parser: Fix issue with parsing of the trailing garbadge

Fix

Mime_parser: Fix parsing of mime parts without closing boundary

Fix

Multimap: Fix operating with userdata

Fix

Process orphaned `symbols` section

Fix

Rdns: Fix multiple replies in fake replies

Fix

Rework groups scores definitions

Fix

Set proper element when reading data from Sentinel

Fix

Set rspamd user to initialise supplementary groups on reload

Fix

Settings: Fix selectors usage

Fix

Sort data received from Sentinel to avoid constant replacing

Fix

groups.conf - filename typo

Fix

lua_scanner - oletools typos, logging

Fix

lua_scanners - actions and symbol_fail

Fix

lua_scanners - fix luacheck

Fix

lua_scanners - kaspersky - response with fname

Fix

lua_scanners - savapi redis prefix

Fix

tests - antivirus - fprot symbols

Rework

Convert rspamd-server to a shared library

Rework

Dcc: Rework DCC plugin

Rework

Enable explicit coroutines symbols

Rework

Rework telephone urls parsing logic

Rework

Rewrite RBL module

Rework

Settings: Rework settings check

Rework

Slashing: Distinguish lualibdir, pluginsdir and sharedir

Rework

Unify task_timeout

Rework

Use VEX instructions in assembly, relocate

Release 1.8.3: 03 Dec 2018

CritFix

Make flags mutually exclusive for mime parts

CritFix

Strictly deny unencoded bad utf8 sequences in headers

Feature

Add Kaspersky antivirus support

Feature

Add method to get dkim results

Feature

Add more words regexp classes

Feature

Allow to choose words format in `rspamadm mime`

Feature

Allow to get all types of words from Lua

Feature

Allow to get task flags in C expressions

Feature

Allow to require encryption when accepting connections

Feature

Ignore bogus whitespaces in the words

Feature

Implement more strict configuration tests

Feature

Improve SPF results in Authentication-Results

Feature

Support ClickHouse database

Fix

Add failsafety for utf8 regexps

Fix

Do not trigger BROKEN_CONTENT_TYPE on innocent text parts

Fix

Emit error if connection has been terminated with no stop pattern

Fix

Fix boundaries checks in embedded messages

Fix

Fix double free

Fix

Perform policy downgrade on sample out, add tests

Fix

Properly escape utf8 regexps in hyperscan mode

Fix

Selectors - attachments args condition

Fix

Some fixes for raw parts

Fix

Treat learning errors as non-fatal

Fix

Use tld when looking for DKIM domains

Release 1.8.2: 19 Nov 2018

Conf

Add DWL support in the default configuration

Conf

Disable rspamd_update by default (again)

Conf

Fix configuration sample for ratelimit

CritFix

Disable broken url tags by default

CritFix

Fix \0 processing when doing RSA sign

CritFix

Fix adding symbols to their primary groups

Feature

Add `rspamadm cookie` utility

Feature

Add specialised functions for generating encrypted cookies

Feature

Add support of cookies in replies module

Feature

Add support of words regexps

Feature

Allow to add 3rd party clang plugins

Feature

Allow to create lua regexps from glob or plain patterns

Feature

Allow to set custom limits for upstream lists

Feature

Detect orphaned parts and attach them to message

Feature

Filter tokens in bayes

Feature

Fold b= value when doing arc sealing

Feature

Ignore cookies in the future and too old in the past

Feature

Skip stop words in statistics

Feature

Store stop words and allow to query them

Feature

Support query arguments in controller's custom commands

Feature

Tune upstream limits in Rspamd proxy

Feature

Use different callback symbols for different uribls

Feature

Write DKIM selector in dkim allow/reject symbols

Fix

Add obs_fws state support to eoh state machine

Fix

Add sanity check when applying mime boundaries heuristic

Fix

Antivirus - virus names with 0 were recognized as tables

Fix

Disable headernames in bayes temporarily

Fix

Do not allow syntax errors in include files...

Fix

Do not allow to merge an object with an array (or vice versa)

Fix

Don't perform forged recipients check for missing recipients

Fix

Fix DKIM based RBLs

Fix

Fix actrie implementation (sync from upstream), fixed OOB read

Fix

Fix explicit methods call in selectors

Fix

Fix extraction of additional parts

Fix

Fix finalization for internal plugins

Fix

Fix override_defaults function

Fix

Fix squeezed symbols when using settings

Fix

Fix urls insertion in Clickhouse module

Fix

Furhter fixes to ratelimits logic

Fix

Ignore signatures when looking for boundaries

Fix

Properly set learned count

Fix

Really fix ratelimits configuration and work

Fix

Remove ambigious format flag from printf

Fix

Restore URLs exporting in ClickHouse plugin

Fix

Rework bayes calculations...

Fix

Switch from chi-square to naive for large Fisher value

Fix

Treat normal password as enable password if there is no enable password

Fix

Use proper syntax for making DNS requests

Fix

Various fixes in embedded plugins

Release 1.8.1: 16 Oct 2018

CritFix

Fix options insertion

CritFix

Fix words decay one more time (affects long messages)

CritFix

Increase default words_decay

CritFix

Plug memory leak in redis pool

Feature

Add `check_violation` feature to DKIM/ARC signing

Feature

Add only unique elements to Clickhouse url arrays

Feature

Allow `g+:` and `g-:` composite atoms

Feature

Allow dkim domains check in surbl

Feature

Allow maps with HTTP auth

Feature

Allow to disable actions by users settings

Feature

Extend whitelisting options

Feature

Store url object in images

Feature

Use verdict instead of the plain action in plugins

Fix

Allow to call fstring append with NULL string

Fix

DCC - luacheck

Fix

Do not load torch on each rspamadm invocation

Fix

Fix boundaries detection and rework stop words algorithm

Fix

Fix dependencies for DNS_SIGNED symbol

Fix

Fix errors when dealing with dynamic rates/bursts in Ratelimit

Fix

Fix groups mess

Fix

Fix groups mess

Fix

Fix parsing address with comments

Fix

Fix resolving in DMARC reports

Fix

Fix various issues with parsing of the received headers

Fix

Fix watchers issue in lua_tcp when doing no resolving

Fix

Plug memory leak in language detector (affects reloads)

Fix

Remove one letter stop words

Fix

Slashing: backport chunk logic from libucl

Fix

Stop libevent from using cached time in rspamadm

Fix

Try to fix watchers chaining

Fix

Various fixes in redis sync interface

Fix

ip_score - respect check_authed and check_local settings from config

Rework

Always create result for a task

Rework

Completely rewrite DMARC checks logic

Rework

Rework and fix whitelist plugin

Release 1.8.0: 24 Sep 2018

Feature

Add arguments schemas to processors and extractors

Feature

Add functional selectors library

Feature

Add generic selector to reputation module

Feature

Add more ratelimits: by digest, by attachments data, by filenames

Feature

Add preliminary stop words detection support

Feature

Add pure Lua debugm function

Feature

Add schema validation for Redis settings

Feature

Add selectors combine function

Feature

Add some recursion protection to lua logger

Feature

Add support for Lua API tracing

Feature

Allow to apply schema to arguments

Feature

Allow to get dkim signing data directly from HTTP headers

Feature

Allow to reuse existing authentication results

Feature

Cache selectors results in re runtime

Feature

Implement new text tokenizer based on libicu

Feature

Integrate selectors framework to multimap

Feature

Relax FORGED_RECIPIENTS

Feature

Support (almost) all html entities

Feature

Support adding and deletion of recipients in the milter block

Feature

Support gathering HTTP body from fragments in lua_http

Feature

Support multi flag in regexp and glob maps

Feature

Support selectors in ratelimit module

Feature

Support selectors in settings

Feature

Use khash in HTML parser

Feature

Use pure Lua debugm function

Fix

Add fail-safety for destroying sessions

Fix

Allow to add result-less fake DNS records

Fix

Another try to fix race conditions on config unload

Fix

Call Lua callback on DNS timeouts

Fix

Deprecate task:inc_dns_req as it is redundant

Fix

Do not allow events deletions on cleanup

Fix

Do not try to process skipped messages

Fix

Fix HTTP requests with no body

Fix

Fix another cleanup race condition

Fix

Fix bug in processing of pcre regexps

Fix

Fix byte array allocation in the pool

Fix

Fix crashes on task cleanup

Fix

Fix dynamic buckets in ratelimits

Fix

Fix endless loop when waiting for Rspamd to stop

Fix

Fix lua_util.str_split in case of delimiters set

Fix

Fix more issues with watching of async events

Fix

Fix stop words detection and loading logic

Fix

Fix various corner cases for language detection

Fix

Fix watchers in lua_tcp

Fix

Fix words decay algorithm

Fix

Implement watchers replacement to handle nested calls

Fix

Save faked code into fake dns record

Fix

Show the proper frame when using lua_util.debugm

Fix

Use fake dns records in tests

Fix

Use unicode replacements for HTML entities

Fix

fixed "cannot find dependency on symbol 1" issue when using replaced symbols in spamassassin rules

Fix

partition_id is not available in old versions of CH

Rework

Move phishtank to a DNS based service

Rework

Rework Clickhouse plugin to use the new API

Rework

Rework language detector

Rework

Rework utf content processing in text parts

Release 1.7.9: 01 Aug 2018

CritFix

Fix caseless comparison of equal length strings

Feature

Add HTTP basic auth support to elastic and clickhouse plugins

Feature

Add SPF selector to reputation

Feature

Add support of the fallback backends for HTTP maps

Feature

Allow to print full mime structure when extracting mime data

Feature

Allow to split symbols in reputation plugin

Feature

Check attachments only on AV scanners in attachments_only mode

Feature

Disable all SSL checks if ssl_no_verify flag is set

Feature

Implement parsing of scoped IPv6 addresses

Feature

Improve `rspamc counters` output

Fix

Add sanity checks when expanding SPF macros

Fix

Allow to parse SA rules with no spaces around =~ (dirty hack)

Fix

Avoid one extra byte writing

Fix

Deal with direct hash table

Fix

Detect empty text part as text, not HTML

Fix

Do not reduce map watch timeout for mixed http/file maps

Fix

Fix HTML part detection heuristic

Fix

Fix double free in redirectors cleanup

Fix

Fix legacy history handling in the controller

Fix

Fix messages insertion

Fix

Fix sending string method

Fix

Fix statconver command line arguments

Fix

Fixed argument checking for being null

Fix

Fixed issues reported by luacheck

Fix

Freeze updates queue when do actual storage update

Fix

HTTP map hash is per-backend and not per-map

Fix

Plug memory leak in fuzzy updates

Fix

Prefer 'MTA-Name' when producing authentication results

Fix

Replace bad unicode sequences instead of stopping on them

Fix

Set classifier version on learning

Release 1.7.8: 12 Jul 2018

Feature

Add more extended statistics about fuzzy updates

Feature

Add more non-conformant Received headers support

Feature

Add preliminary function to get fuzzy hashes from text in Lua

Feature

Allow to configure AV module rejection message

Feature

Implement fuzzy hashes extraction in mime tool

Feature

Improve WHITE_ON_WHITE rule

Feature

Improve integer -> string conversion

Feature

Reuse maps in multimap module more aggressively

Fix

Avoid race condition in skip map as pool lifetime is not enough

Fix

Eliminate all specific C plugins pools

Fix

Fix DKIM check rule if DNS is unavailable

Fix

Fix build where ucontext is defined in ucontext.h

Fix

Fix crash in base url handling

Fix

Fix descriptors leak in sqlite3 locking code

Fix

Fix messages quarantine

Fix

Fix padded numbers printing

Fix

Fix race condition on maps reinit

Fix

Fix regexp functions when no data is passed

Fix

Fix specific urls extraction

Fix

Fix styles propagation

Fix

Improve resetting of the limit buckets

Fix

Initialize sqlite3 properly

Fix

Work with broken resolvers in resolv.conf

Release 1.7.7: 02 Jul 2018

CritFix

Check NM part of pubkey to match it with rotating keypairs

CritFix

Do not overwrite PID of the main process

CritFix

Fix maps after reload

CritFix

Fix maps race conditions on reload

CritFix

Fix shmem leak in encrypting proxy mode

Feature

Add a concept of ignored symbols to avoid race conditions

Feature

Add ability to print bayes tokens in rspamadm mime

Feature

Add method to get statistical tokens in Lua API

Feature

Add preliminary mime stat command

Feature

Add rspamadm mime tool

Feature

Add urls extraction tool

Feature

Address ZeroFont exploit

Feature

Allow rspamadm mime to process multiple files

Feature

Allow to extract words in `rspamadm mime`

Feature

Allow to print mime part data

Feature

Allow to show HTML structure on extraction

Feature

Distinguish IP failures from connection failures

Feature

Improve output for mime command

Feature

Improve styles propagation

Feature

Main process crash will now cleanup all children

Feature

Preload file and static maps in main process

Feature

Print stack trace on crash

Feature

Process font size in HTML parser

Feature

Propagate content length of invisible tags

Feature

Read ordinary file maps in chunks to be more safe on rewrites

Feature

Support base tag in HTML

Feature

Support more size suffixes when parsing HTML styles

Feature

Support opacity style

Fix

Another fix for nested composites

Fix

Fill nm id in keypairs cache code

Fix

Fix colors alpha channel handling

Fix

Fix destruction logic

Fix

Fix double free

Fix

Fix maps preload logic

Fix

Fix nested composites process

Fix

Fix proxying of Exim connections

Fix

Fix reload crash

Fix

Fix rspamadm -l command

Fix

Update ed25519 signing schema

Release 1.7.6: 15 Jun 2018

CritFix

Fix multiple neural networks support

Feature

Add decryption function to keypair command

Feature

Add gzip compression for HTTP requests in elastic module

Feature

Add gzip methods to lua util

Feature

Add maps based on Top Level Domains

Feature

Add pubkey checks for dkim_signing

Feature

Add support of fake DNS records

Feature

Add tool to encrypt files

Feature

Allow to add symbols using settings directly

Feature

Allow to match private and public keys for DKIM signatures

Feature

Allow to set task flags via settings

Feature

Allow to specify fake DNS address from the config

Feature

Implement signatures verification using rspamadm keypair

Feature

Implement signing using `rspamadm keypair`

Feature

Improve error reporting for DKIM key access issues

Feature

Provide $HOSTNAME variable in UCL

Feature

Rework levenshtein distance computation

Feature

Split message parsing and processing

Feature

Support ED25519 DKIM signatures

Feature

Support encrypted configs in UCL

Feature

Suppress duplicate warning on very large radix tries

Feature

Use OSB to combine header names

Fix

Cleanup maps data on shutdown

Fix

Fix '~' behaviour in composites

Fix

Fix HTTP maps updates

Fix

Fix NIST signatures

Fix

Fix RFC822 comments when processing a mime address

Fix

Fix double free

Fix

Fix dynamic settings application

Fix

Fix for CommuniGate Pro maillist

Fix

Fix keypair creation method to actually create keypair...

Fix

Fix matching patterns with no paths

Fix

Fix memory leak in parsing comments

Fix

Fix parsing of urls with numeric password

Fix

Fix plugins intialisation in configwizard

Fix

Fix potential crash on reload

Fix

Fix potential race condition for a finished HTTP connections

Fix

Fix race-condition leak on processes reload

Fix

Fix signing in openssl mode

Fix

Free language detector structures

Fix

Relax alignment requirements

Fix

Send DMARC reports compressed

Fix

Try to fix leak in dmarc module

Fix

Try to plug memory leak in metric exporter

Release 1.7.5: 18 May 2018

Conf

Add MSBL proposed return codes

Conf

Add additional groups for policies

CritFix

Do not use volatile Lua strings as UCL keys

Feature

Add ability to add fuzzy hashes to headers

Feature

Add function to extract most meaningful urls

Feature

Add rule to block mixed text and encrypted parts

Feature

Allow multiple groups for symbols

Feature

Allow to disable lua squeezing logic

Feature

Allow to get multipart children in Lua

Feature

Allow to insert multiple headers from milter headers

Feature

Allow to print scores in subject and further extensions

Feature

Be more error-prone in squeezed rules

Feature

Support multiple return codes in emails module

Feature

Use EMA for calculating averages

Feature

Use common jit cache for all regexps

Feature

support for CommuniGate Pro self-generated messages

Fix

Allow to have multiple values for headers as arrays

Fix

Do not open sockets for disabled workers

Fix

Fix AuthservId

Fix

Fix base64 folding in Lua API

Fix

Fix build on non-x86 platforms

Fix

Fix cached maps logic

Fix

Fix compatibility with old maps query logic

Fix

Fix crash if skip_map is used

Fix

Fix importing static maps from UCL

Fix

Fix parsing of unix sockets

Fix

Fix raw_mime regexp on HTML part with no text content

Fix

Fix tables logging

Fix

Fix vertical tab handling in libucl

Fix

Try to fix frequency counters

Fix

Use better sharding for ip_score

Fix

Use multiple results from SURBL DNS reply

Fix

When doing AV scan select a different server for retransmit

Release 1.7.4: 01 May 2018

Conf

Major stock config updates:

Conf

Make more sane fuzzy_check default settings

CritFix

Fix ucl escape for bad symbols

Feature

Add failure symbol for AV module

Feature

Add lazy expiration mode for new classifier schema

Feature

Add preliminary version of maps stats plugin

Feature

Allow to block fuzzy requests from specific networks

Feature

Allow to change `expire` of live statistics

Feature

Distinguish AV failure from clean result

Feature

Further improvements of language detector by using khash

Feature

Further optimization of the lang_detection

Feature

Implement cluster-aware bayes expiry

Feature

Implement exclude patterns in rspamc

Feature

Implement glob maps in addition to regexp maps

Feature

Implement map statistics function for lua API

Feature

Implement stop symbols for Clickhouse collection

Feature

Support recipients separated by commas

Feature

Try harder to upload scripts to the Redis server

Feature

Upgrade t1ha distribution

Feature

use_domain_sign_inbound

Feature

Use scores from maps if `symbols_set` is not defined

Fix

Add resolving version of radix map helper

Fix

Check URL before adding implicit prefix

Fix

Do not check pid/state when using PRNG

Fix

Fix CentOS logrotate script for systemd

Fix

Fix slash + dot in urls

Fix

Fix systemd version of the logrotate script

Fix

Propagate key when import implicit array from Lua

Fix

Strip spaces from map keys and values

Fix

Try to fix a specific case when processing milter protocol

Fix

Try to fix crash when a tcp connection cannot be set

Fix

Typo use_domain_local --> use_domain_sign_local

Fix

Various fixes to once_received module

Release 1.7.3: 10 Apr 2018

CritFix

Plug bad memory leak in protocol reply

Feature

Add avx2 codec for base64

Feature

Add method to receive all URL flags from Lua API

Feature

Allow to fold headers on stop characters

Feature

Allow to set lua_cpath from options

Feature

Allow to specify custom rejection message in milter

Feature

Deal with unnormalised Unicode obfuscation

Feature

Do not detect language twice for relative parts

Feature

Implement oversigning feature

Feature

Implement silent logging level to minimize noise in logs

Feature

Improve URL_IN_SUBJECT rule

Feature

Use hashing to reduce redis attack surface

Fix

Add oversigning for the most important headers

Fix

add 'rewrite subject' to History dropdown

Fix

Another fix in folding algorithm

Fix

Do not call multimap addr for parts of addr if filter is presented

Fix

Do not clean hostname on generic reset

Fix

Do not create pid file in no-fork mode

Fix

Fix fold_after case to preserve multiple spaces

Fix

Fix folding and folding tests

Fix

Fix hostname usage in milter mode

Fix

Fix lua RSA verify and its tests

Fix

Fix metadata exporter send_mail backend (#2124)

Fix

Fix processing of '\v' in libucl

Fix

Fix shemaless URLs detection

Fix

Fix support of multiple headers in sign_header

Fix

Fix usage of util.parse_mail_address

Fix

Fix weights of dynamic squeezed rules

Fix

Leak from bucket before checking the burst

Fix

Stop using own localtime as DST could be messy in many cases

Fix

Treat unnormalised URLs as obscured

Rework

Restore leaky bucket model in ratelimit plugin

Release 1.7.2: 23 Mar 2018

Feature

Store emails in Clickhouse

Feature

Support single quotes in config

Feature

Use templates when publishing CH schema

Feature

Improve Docker image

Fix

Add rounding when printing a lot of FP variables

Fix

Allow to disable certain actions by assigning null to them

Fix

Disable results caching

Fix

Fix disabling of squeezed symbols

Fix

Fix scan time set

Fix

Rework logic of actions setting

Fix

Try to fix various Lua stack issues

Release 1.7.1: 20 Mar 2018

CritFix

Fix lowercase comparison

CritFix

Timezone defines seconds WEST UTC not East

Feature

Add filename to log format

Feature

Add lua rules squeezing

Feature

Add related symbols analysis to rspamd_stats

Feature

Remove upstream `X-Spam: Yes` header by default

Feature

rspamd_stats: Output progress info on STDERR

Feature

Whitelist for emails module

Fix

Do not allow dependencies on self

Fix

Do not cache metric result

Fix

Do not trust all issuers as a client certificate

Fix

Fix dependencies in lua squeeze

Fix

Fix enabling/disabling squeezed rules

Fix

Fix enabling/disabling symbols

Fix

Fix external dependencies

Fix

Fix processing of a single compressed file

Fix

Fix some typos

Fix

Fix various modules in case of empty message

Fix

Handle callbacks that returns table of options

Fix

Improve cached action interaction

Fix

Make dynamic conf more NaN aware

Fix

Never hide actions from WebUI `configuration` tab

Release 1.7.0: 12 Mar 2018

Conf

Add bayes_expiry as explicit module

Conf

Adjust names and weights for neural network plugin

Conf

Change updates url

Conf

Default statistics is stored in Redis now

Conf

Disable fann_redis module by default

Conf

Fix default elastic configuration

Conf

Fix double quote position

Conf

Massive config rework for new structure of symbols and scores

Conf

Rename Rambler BLs as they are now Rspamd's ones

Conf

Use dedicated rspamd.com subdomains

Conf

Use more data from rspamd.com fuzzy storage

CritFix

Add sanity guards for badly broken HTML

CritFix

Another errors path handling fix

CritFix

Another portion of tokenization fixes

CritFix

Do not send reject messages after set reply

CritFix

Fix ARC chain verification

CritFix

Fix crash in milter errors handler

CritFix

Fix memory leak in spf caching logic

CritFix

Fix milter commands pipelining

CritFix

Fix newlines detection

CritFix

Fix semicolons parsing in the content type

CritFix

Plug memory leak in zstd protocol compression

Feature

Add ability to match score in force_actions module

Feature

Add aes-rng PRF to libottery

Feature

Add 'composites' debug module

Feature

Add concept of experimental modules

Feature

Add DKIM trace symbol

Feature

Add EBL to the default config

Feature

Add expected ip check for emails plugin

Feature

Add framework to manage Redis scripts

Feature

Add framing for the new reputation generic plugin

Feature

Add function to show plugins stat

Feature

Add gzip compression support for clickhouse module

Feature

Add gzip compression support for rspamd controller

Feature

Add gzip support when sending lua http requests

Feature

Add json output for rspamd_stats

Feature

Add method to do a synchronous Redis connection

Feature

Add method to get all content-type attributes in Lua

Feature

Add `-m` flag to configdump to show modules states

Feature

Add mime types to extensions map

Feature

Add more features to rescore utility

Feature

Add more gtube like patterns to test other spam actions

Feature

Add more metafunctions, improve logging

Feature

Add more text attributes

Feature

Add new configwizard command to rspamadm

Feature

Add new tooling for stats conversation

Feature

Add old groups migration tool

Feature

Add plugins state variable

Feature

Add preliminary ecdsa keys support in DKIM

Feature

Add preliminary support of idempotent symbols

Feature

Add Redis server wizard

Feature

Add routine to convert old style stats to a new one

Feature

Add some sanity checks for actions and controller

Feature

Add statistic convertation module to configwizard

Feature

Add sugestions logic to mempool allocator

Feature

Add support of config transform in Lua

Feature

Add timeout to rspamc when doing corpus test

Feature

Add tooling to convert bayes schemas

Feature

Add torch conditional to configuration

Feature

Add torch-decisiontree package

Feature

Add torch-optim contrib package

Feature

Add TTL autodetection

Feature

Add urls reputation to the reputation framework

Feature

Allow floating and negative values in expressions limits

Feature

Allow multiple CTs in full extensions map

Feature

Allow multiple fann rules

Feature

Allow randomly select User-Agent from a list

Feature

Allow rspamadm commands to export methods in Lua

Feature

Allow rule specific min_bytes in fuzzy check

Feature

Allow to adjust symbols scores from Lua

Feature

Allow to attach stat signature to messages

Feature

Allow to change SMTP from via milter headers

Feature

Allow to configure monitored

Feature

Allow to create directories in Lua API

Feature

Allow to disable torch and skip train samples for ANN

Feature

Allow to discard messages dynamically

Feature

Allow to enable/disable languages from the detector

Feature

Allow to generate DKIM keys from rspamadm API

Feature

Allow to get CPU flags from Lua

Feature

Allow to have high precision timestamps in logs

Feature

Allow to insert headers into specific position

Feature

Allow to limit redirector requests per task

Feature

Allow to load and use dynamic ANNs with torch

Feature

Allow to quarantine rejected messages using milter interface

Feature

Allow to receive signing keys from mempool vars

Feature

Allow to reserve elements in libucl

Feature

Allow to reuse signal handlers chains

Feature

Allow to set custom mempool variables from settings

Feature

Allow to set headers from settings

Feature

Allow to set Settings-Id for all connections

Feature

Allow to skip real action and add a header instead

Feature

Allow to skip specific hashes in fuzzy storage

Feature

Allow to spawn asynchronous processes from Lua

Feature

Allow to specify number of threads for ANN learning

Feature

Allow to use global lua maps in settings

Feature

Allow to use postfilters in composites

Feature

Allow to verify signatures from HTTP headers in maps

Feature

Antivirus: ordered pattern matches

Feature

Authentication-Results: support hiding usernames

Feature

Automatically create tables in clickhouse

Feature

Catch next-to-last bad extension

Feature

Check cached maps more frequently

Feature

Check groups sanity

Feature

Deal with obscured URLs with @ symbols

Feature

Enhance task:store_in_file method

Feature

Export password encryption routines to Redis

Feature

Filter nan and inf when adding scores

Feature

Finalize 7zip files support

Feature

Further improvements in language detection

Feature

Further improvements in language detection algorithm

Feature

Generic key name expansion for Redis keys

Feature

Hash whitelist for fuzzy_check

Feature

Implement bayes signatures storage

Feature

Implement buckets for Redis backend

Feature

Implement DKIM reputation adjustments

Feature

Implement forked workers children monitoring

Feature

Implement headers flags in mime parser

Feature

Implement l1/l2 regularization against the current weights

Feature

Implement manual ANN train mode

Feature

Implement per-user ANN support

Feature

Implement torch based ANN learning

Feature

Implement upstreams logic for clickhouse exporter

Feature

Import torch to Rspamd...

Feature

Improve allocation policy when interacting with Lua

Feature

Improve Lua/C interaction in history_redis

Feature

Improve multiple fuzzy results combining

Feature

Improve parsing of DKIM keys: parse algorithm

Feature

Improve subprocesses termination handle

Feature

Improve symbol type parsing in Lua API

Feature

Metadata Exporter: e-Mail Alerts: support multiple recipients; alerting senders/recipients/users (#1600)

Feature

Milter headers: support adding/removing arbitrary headers from config

Feature

More metatokens

Feature

Multimap: checking of symbol options

Feature

Multimap: template URL filter

Feature

New bayes expiry plugin

Feature

Periodically save rspamd stats to disk

Feature

Preliminary import of the elasticsearch module

Feature

Ratelimit: allow full addresses in whitelisted_rcpts

Feature

Ratelimit: support fetching limits from Redis

Feature

RBL: received: filtering by position & flags

Feature

Read global maps for lua

Feature

Redis settings: support checking multiple keys

Feature

Rework fann plugin to be a normal post-filter

Feature

Rework logging configuration for rspamadm case

Feature

Rework short hashes generation to avoid FP

Feature

Save real ucl types when exporting to Lua

Feature

Set TCP_NODELAY for milter sockets

Feature

Setup DKIM signing from configwizard

Feature

Skip certain symbols from ANN classify

Feature

Store plugins state

Feature

Support etag for HTTP maps

Feature

Support Expires header when using HTTP maps

Feature

Support sending given header multiple times in lua_http

Feature

Support sha512 in DKIM signatures

Feature

Try to detect HTML messages better

Feature

Use array instead of queue to reduce memory fragmentation

Feature

Use controller port by default when connecting to local IP

Feature

Use rdtsc where possible

Fix

Actively load skip hashes map in fuzzy storage

Fix

Add another workaround to display history properly

Fix

Add definition for old glib compatibility method

Fix

Add missing rspamadm control options to help

Fix

Add workaround for IPv6 in sendmail

Fix

Add workaround for system with non-XSI compatible tzset

Fix

Allow oversigning in DKIM signatures

Fix

Allow to check negative scores in force_actions

Fix

Allow to have negative actions limits

Fix

Allow to set any layers number for fann rules

Fix

Another fix for rdtcs

Fix

Another fix to lua xmlrpc

Fix

Another try to deal with #1998

Fix

Another try to fix #1998

Fix

Another try to fix threading in torch

Fix

Apply language detection when adding fuzzy hashes

Fix

ARC: Fix Lua 5.3 compatibility; timestamp should be integer

Fix

Authentication Results: Fix SPF smtp.mail_from

Fix

Auth-Results: Multiple DKIM signatures

Fix

Avoid changing content-transfer-encoding header's value

Fix

Better handling of the legacy protocol

Fix

Check decoded headers sanity (e.g. by excluding \0)

Fix

Check for magic when checking for an archive

Fix

Cleanup mess with groups

Fix

Clickhouse: Insertion in the symbols table

Fix

Crash in URL processing

Fix

Deal with another case when processing exceptions

Fix

Deal with deeply nested messages more aggressively

Fix

Deal with nan and inf encoding in json/ucl

Fix

Deal with non-key arguments in lua_redis.exec_script

Fix

Deal with unknown weight

Fix

Deal with URLs with no slashes after protocol

Fix

Deal with URLs wrapped in [] in text parts

Fix

Deal with zero scores symbols

Fix

Default monitoring domain for surbl plugin

Fix

Delay upstream re-resolving when one upstream is defined

Fix

Detection of maillist optimized and fixed

Fix

DKIM signing: allow for auth_only to be false

Fix

DMARC: require report_settings for sending reports only

Fix

Do not allow garbadge when checking url domain

Fix

Do not cache SPF records with PTR elements

Fix

Do not constantly re-resolve failed upstreams with a single element

Fix

Do not crash if no words defined

Fix

Do not crash on empty subtype

Fix

Do not expose spamtrap messages to SMTP reply

Fix

Do not fail rbl plugin when there are no received or emails

Fix

Do not ignore short words

Fix

Do not include idempotent/nostat symbols to checksum

Fix

Do not override groups when converting metrics

Fix

Do not override unix socket group when group comes before owner

Fix

Do not skip the last character

Fix

Do not spawn too many workers by default

Fix

Do not stop monitored on dns errors

Fix

Do not stop parsing headers on bad IP header

Fix

Do not strip last character in the last word

Fix

Do not treat script content as text

Fix

Do not try to connect to non-supported addresses

Fix

Do not try to dereference last character

Fix

Do not try to sign unknown domains

Fix

Don't use whitelist/greylist maps as regexp, but as map

Fix

Erase unknown HTML entities

Fix

Exim Received header protocol parsing

Fix

First load selector_map and path_map. And only return false when domain not found if try_fallback is false

Fix

Fix a lot of FP in chartable in mixed languages

Fix

Fix ANN checks

Fix

Fix ANN loading logic

Fix

Fix another tokenization issue

Fix

Fix autolearn parameters reading

Fix

Fix bad archive characters stripping

Fix

Fix bad extension check

Fix

Fix bayes schema conversion

Fix

Fix blacklists and DMARC in whitelist

Fix

Fix brain-damaged torch build system

Fix

Fix build on FreeBSD

Fix

Fix clickhouse exporter

Fix

Fix clickhouse schema

Fix

Fix comparision

Fix

Fix composites processing

Fix

Fix connecting to a unix socket in rspamadm statconvert

Fix

Fix couple of warnings

Fix

Fix crashes in the rspamd_control path

Fix

Fix deletion from hash

Fix

Fix DKIM forgeries via multiple headers

Fix

FIx dynamic conf plugin

Fix

Fix emails detection

Fix

Fix empty headers simple canonicalization

Fix

Fix empty threshold check in greylisting module

Fix

Fix encrypted legacy reply in fuzzy storage

Fix

Fix enormous scores for R_WHITE_ON_WHITE

Fix

Fix exceptions list in surbl

Fix

Fix *_EXCESS_BASE64 rules

Fix

Fix expire rounding

Fix

Fix extra hits in PCRE mode for regular expressions

Fix

Fix format strings

Fix

Fix get_content method

Fix

Fix groups override when defining symbols

Fix

Fix learned count in new schema

Fix

Fix learn errors propagation

Fix

Fix loading of per-user redis backend for statistics

Fix

Fix logging buffer corruption in case of repeated messages

Fix

Fix lua cached elements invalidation

Fix

Fix merging of the implicit arrays

Fix

Fix mime_types scoring

Fix

Fix multiple headers in DKIM headers list

Fix

Fix null callee case in clang plugin

Fix

Fix obscured url in format user@@example.com

Fix

Fix parsing of the per-user script

Fix

Fix priorities in rspamd_update, disable rules execution

Fix

Fix processing of closed tags

Fix

Fix processing of idempotent rules when autolearn fails

Fix

Fix processing of multipart parts with no headers

Fix

Fix processing of skip-hashes in fuzzy storage

Fix

Fix PTR processing in SPF

Fix

Fix pushing country to clickhouse asn table

Fix

Fix random forests module

Fix

Fix real IP parsing for some strange Exim received

Fix

Fix Redis timeout setup

Fix

Fix reload crash when hyperscan is enabled

Fix

Fix reusing of redis connection after exec

Fix

Fix sanity checks on macro value

Fix

Fix setting of path and cpath for Lua

Fix

Fix setting of signals when spawning a thread

Fix

Fix text splitting: stack overflow (too many captures)

Fix

Fix ticks processing

Fix

Fix upstream addrs updating

Fix

Fix urls/emails distinguishing found in queries

Fix

Fix user settings check

Fix

Fix variable increment

Fix

Fix various issues in stat_convert

Fix

F-PROT Antivirus infection string for all known occurences

Fix

F-PROT Antivirus: only check return code to determine infection

Fix

Further fixes around floating point expressions

Fix

Further fixes to ANN module

Fix

Further fixes to rescore tool

Fix

Further fixes to support ES 6

Fix

Further tokenization fixes

Fix

Greylisting set phase is not idempotent

Fix

Handle proxy copy errors

Fix

Header checks: Fix get_raw_header method

Fix

Header checks: REPLYTO_UNPARSEABLE rule

Fix

Kill spawned processes on termination

Fix

Load skip map from all processes as shared cache is unavailable

Fix

Lowercase HTTP headers to make them searchable from Lua

Fix

Lowercase words

Fix

Lua_http: freeing

Fix

Lua: lpeg to be loaded with rspamd_lua_add_preload, to avoid "rspamd_config_read: rcl parse error: cannot init lua file […] module 'lpeg' not found"

Fix

Map absence is not an error

Fix

Metadata exporter: check IP sanity

Fix

Milter headers: custom headers: removing headers

Fix

Milter headers: skip_local / skip_authenticated settings

Fix

Milter headers: X-Spamd-Result header if X-Virus ran first

Fix

mime_types: fix next-to-last extension length check

Fix

More hacks to deal with old configs

Fix

Move composites second pass to the dedicated stage

Fix

Multimap: received: filtering of artificial header

Fix

Multiple fixes in torch based ANN plugins

Fix

Once more (#1879) fix bad extension check

Fix

Optimize rspamd_fstring_t reallocations

Fix

options.local_networks setting

Fix

Parse HREF urls without explicit prefix

Fix

Plan new event on HTTP errors

Fix

Plug another possible memory leak

Fix

Plug memory leak

Fix

Plug memory leak in lua_tcp

Fix

Plug memory leak when setting email addresses from Lua

Fix

Propagate learn/stat errors more precisely

Fix

Ratelimit: fix whitelisted_rcpts matching

Fix

Ratelimit: lowercase email addresses

Fix

RBL: received: deal with missing data (#1965)

Fix

Rebalance and slightly rework MX check plugin

Fix

Redis key expansion: EVAL: deal with strings

Fix

Redis script loading in DMARC; URL tags; URL reputation

Fix

Reject invalid bh for DKIM signatures earlier

Fix

Relax pem signature detection

Fix

Relax unicode properties requirements for chartable module

Fix

Remove extra noise from dkim and arc signing

Fix

Remove hop-by-hop headers in proxy

Fix

Remove incorrect method `task:set_metric_subject`

Fix

Replace space like characters in headers with plain space

Fix

Restore old style ratelimits support

Fix

Rework elasticsearch plugin

Fix

Rewriting subjects via force actions module

Fix

RPM postinstall

Fix

Sanitize IP in history redis

Fix

Select the correct signature when doing simple canon

Fix

Set CLOEXEC flag on files opened

Fix

Setting check_local / check_authed in plugins (#1954)

Fix

Settings: avoid checking invalid IP (#1981)

Fix

Settings: header: deal with multiple settings (#1988)

Fix

Skip checks if both extensions are not bad

Fix

Skip nostat tokens when get number of tokens

Fix

Some more fixes towards emails detection

Fix

SpamAssassin: Fail check_freemail_header if regexp didn't match

Fix

Stop using of g_slice...

Fix

Switch rspamadm logging to message level

Fix

Symbol 'FANNR_SPAM' has its score defined..

Fix

Table parameter for rspamd_config:add_doc()

Fix

Treat 'rewrite subject' as spam action

Fix

Try harder in passing IPv6 addresses

Fix

Try harder to find rfc822 notifications

Fix

Try harder to find urls

Fix

Use decoded values when parsing mime addresses

Fix

Use full URL when making an HTTP request

Fix

Use greylisting threshold in greylisting module

Fix

Use n_words attribute from ngramms

Fix

Use raw urls when sending requests to redirector

Fix

Use the right boolean operator on error check

Fix

Use weight from map for fuzzy scoring

Fix

Various fixes to elastic plugin

Fix

Various fixes to fann_redis instantiation

Fix

Various improvements in language detection

Fix

Virus infection string for F-PROT Antivirus

Fix

Virus infetction string for F-PROT Antivirus

Fix

WebUI: use relative path for savemap (#1943)

Fix

WHITE_ON_WHITE: Ensure score is matched to part that fired the rule

Fix

Write configuration changes as UCL config

Rework

Add frame for new reputation based IP score module

Rework

Continue stat_convert rework task

Rework

Implement new version of fuzzy replies

Rework

Improve readability of xmlrpc API

Rework

Kill metrics!11

Rework

Ratelimit module

Rework

Rename fann_redis to neural plugin

Rework

Reorganize mime_types module

Rework

Rework rescore utility

Rework

Rewrite model and learning logic for rescore

Rework

Run post-loads when all initialization is completed

Rework

Simplify lua path initialization

Rework

Start major stat_convert rework

Rework

Start mempool fragmentation reduce project

Rework

Start moving of fann redis to torch

Rework

Stop embedding rspamadm scripts into C

Rework

Use floating point arithmetics in Rspamd expressions

Rework

Use frequencies distribution in language detector

Rules

Penalise R_BAD_CTE_7BIT for utf8 messages

Release 1.6.6: 16 Feb 2018

CritFix

Add sanity guards for badly broken HTML

CritFix

Another errors path handling fix

CritFix

Fix ARC chain verification

CritFix

Fix crash in milter errors handler

Feature

Allow to insert headers into specific position

Feature

Allow to receive signing keys from mempool vars

Feature

Authentication-Results: support hiding usernames

Fix

Another try to deal with #1998

Fix

Another try to fix #1998

Fix

Better handling of the legacy protocol

Fix

Check decoded headers sanity (e.g. by excluding \0)

Fix

Deal with nan and inf encoding in json/ucl

Fix

Deal with URLs wrapped in [] in text parts

Fix

DKIM signing: allow for auth_only to be false

Fix

Do not crash on empty subtype

Fix

Do not fail rbl plugin when there are no received or emails

Fix

Do not skip the last character

Fix

Do not try to dereference last character

Fix

Do not try to sign unknown domains

Fix

Exim Received header protocol parsing

Fix

First load selector_map and path_map. And only return false when domain not found if try_fallback is false

Fix

Fix bad archive characters stripping

Fix

Fix comparision

Fix

Fix connecting to a unix socket in rspamadm statconvert

Fix

Fix empty headers simple canonicalization

Fix

Fix extra hits in PCRE mode for regular expressions

Fix

Fix parsing of the per-user script

Fix

Fix processing of skip-hashes in fuzzy storage

Fix

Fix Redis timeout setup

Fix

Fix sanity checks on macro value

Fix

Fix text splitting: stack overflow (too many captures)

Fix

Fix urls/emails distinguishing found in queries

Fix

F-PROT Antivirus: only check return code to determine infection

Fix

Metadata exporter: check IP sanity

Fix

Multimap: received: filtering of artificial header

Fix

Plan new event on HTTP errors

Fix

Plug another possible memory leak

Fix

Remove hop-by-hop headers in proxy

Fix

Sanitize IP in history redis

Fix

Setting check_local / check_authed in plugins (#1954)

Fix

Settings: avoid checking invalid IP (#1981)

Fix

Try harder in passing IPv6 addresses

Fix

WebUI: use relative path for savemap (#1943)

Release 1.6.5: 22 Oct 2017

CritFix

Another portion of tokenization fixes

CritFix

Fix memory leak in spf caching logic

CritFix

Fix milter commands pipelining

CritFix

Fix newlines detection

Feature

Filter nan and inf when adding scores

Feature

Implement headers flags in mime parser

Feature

Support Expires header when using HTTP maps

Fix

Actively load skip hashes map in fuzzy storage

Fix

Add workaround for IPv6 in sendmail

Fix

Authentication Results: Fix SPF smtp.mail_from

Fix

Check for magic when checking for an archive

Fix

Deal with another case when processing exceptions

Fix

Deal with URLs with no slashes after protocol

Fix

Do not allow garbadge when checking url domain

Fix

Do not ignore short words

Fix

Do not strip last character in the last word

Fix

Do not treat script content as text

Fix

Erase unknown HTML entities

Fix

Fix another tokenization issue

Fix

Fix DKIM forgeries via multiple headers

Fix

Fix emails detection

Fix

Fix empty threshold check in greylisting module

Fix

Fix enormous scores for R_WHITE_ON_WHITE

Fix

Fix loading of per-user redis backend for statistics

Fix

Fix multiple headers in DKIM headers list

Fix

Fix obscured url in format user@@example.com

Fix

Further tokenization fixes

Fix

Load skip map from all processes as shared cache is unavailable

Fix

Lowercase words

Fix

Milter headers: skip_local / skip_authenticated settings

Fix

Milter headers: X-Spamd-Result header if X-Virus ran first

Fix

Ratelimit: fix whitelisted_rcpts matching

Fix

Some more fixes towards emails detection

Fix

SpamAssassin: Fail check_freemail_header if regexp didn't match

Fix

Use greylisting threshold in greylisting module

Release 1.6.4: 10 Sep 2017

Feature

Add method to get all content-type attributes in Lua

Feature

Add some sanity checks for actions and controller

Feature

Allow randomly select User-Agent from a list

Feature

Deal with obscured URLs with @ symbols

Feature

Milter headers: support adding/removing arbitrary headers from config

Fix

Add another workaround to display history properly

Fix

Add missing rspamadm control options to help

Fix

Auth-Results: Multiple DKIM signatures

Fix

Crash in URL processing

Fix

Default monitoring domain for surbl plugin

Fix

Detection of maillist optimized and fixed

Fix

Do not cache SPF records with PTR elements

Fix

Fix blacklists and DMARC in whitelist

Fix

Fix exceptions list in surbl

Fix

Fix processing of closed tags

Fix

Fix PTR processing in SPF

Fix

Lowercase HTTP headers to make them searchable from Lua

Fix

options.local_networks setting

Fix

Ratelimit: lowercase email addresses

Fix

Rebalance and slightly rework MX check plugin

Fix

Redis script loading in DMARC; URL tags; URL reputation

Fix

Reject invalid bh for DKIM signatures earlier

Fix

Remove incorrect method `task:set_metric_subject`

Fix

Rewriting subjects via force actions module

Fix

RPM postinstall

Fix

Treat 'rewrite subject' as spam action

Fix

Try harder to find urls

Fix

Use full URL when making an HTTP request

Fix

Use raw urls when sending requests to redirector

Fix

Use weight from map for fuzzy scoring

Rules

Penalise R_BAD_CTE_7BIT for utf8 messages

Release 1.6.3: 26 Jul 2017

CritFix

Fix semicolons parsing in the content type

Feature

Add EBL to the default config

Feature

Allow to configure monitored

Feature

Allow to skip specific hashes in fuzzy storage

Feature

Multimap: checking of symbol options

Feature

Redis settings: support checking multiple keys

Fix

ARC: Fix Lua 5.3 compatibility; timestamp should be integer

Fix

Avoid changing content-transfer-encoding header's value

Fix

Don't use whitelist/greylist maps as regexp, but as map

Fix

Fix get_content method

Fix

Header checks: Fix get_raw_header method

Fix

Header checks: REPLYTO_UNPARSEABLE rule

Fix

Lua_http: freeing

Fix

Milter headers: custom headers: removing headers

Fix

Parse HREF urls without explicit prefix

Fix

WHITE_ON_WHITE: Ensure score is matched to part that fired the rule

Release 1.6.2: 08 Jul 2017

Conf

Remove Rambler email bl for now

Conf

Switch RAMBLER_URIBL to a locally managed source

CritFix

Switch from ragel to C for Content-Type parsing

Feature

Add `-e` option for lua_repl

Feature

Add per-domain emails normalisation rules

Feature

Add sessions cache to debug dangling sessions

Feature

Add short_text_direct_hash for fuzzy check module

Feature

Add text_part:get_stats function

Feature

Allow to add custom processing script for surbl

Feature

Allow to check reply-to email

Feature

Allow to customize spam header, remove existing spam headers

Feature

Allow to disable specific workers in the config

Feature

Allow to discard messages instead of rejection

Feature

Allow to specify custom delimiter in emails plugin

Feature

Allow to specify custom User-Agent for rspamc

Feature

Allow to store symbols data in Clickhouse

Feature

Allow to use HTTPS when connecting to Clickhouse

Feature

Enable sessions cache tracking for milter connections

Feature

Implement per-line mode in lua_repl (like `perl -p`)

Feature

Implement rdns-curve plugin based on rspamd cryptobox

Feature

Improve maps cached data lifetime

Feature

Improve maps checking frequency

Feature

Improve monitored timeouts logic

Feature

milter_headers: add `extended_headers_rcpt` option

Feature

Milter headers: Add X-Spam-Flag to rmilter-compatibility headers

Feature

Milter headers: remove-header routine

Feature

Multimap: received filters for extracting TLDs from hostnames

Feature

Normalize email aliases in emails module

Feature

Re-add rambler email bl (as hashed list)

Feature

Reload file maps more frequently

Feature

Rework newlines strip parser one more time

Feature

Skip updates for messages scanned via controller

Feature

Split long DKIM public keys

Feature

Store more data when stripping newlines

Feature

Support SPF macros transformations

Feature

Support suppressing DMARC reports for some domains

Fix

Add missing `break` statement

Fix

Allow modifiers in SPF macros

Fix

DKIM sign tools: edge-cases around use_esld

Fix

Do not cache SPF records with macros

Fix

Do not overwrite score when setting pre-action

Fix

Fix comparison logic

Fix

Fix DKIM base64 folding for milter flagged messages

Fix

Fix emails module configuration

Fix

Fix folding for arc headers when milter interface is used

Fix

Fix gmail dots removal

Fix

Fix rspamc detection in greylist module

Fix

Fix some more issues with HTTP maps

Fix

Milter sessions can live forever

Fix

Normalize fuzzy probability better

Fix

Plug memory leak

Fix

RBL: Fixed hashed email address lookups

Fix

Try to deal with brain-damaged milter behaviour

Fix

Use `\n` to fold headers for milter

Rework

Allow to use custom callback for monitored checks

Rework

Further steps towards one process monitoring

Rework

Send health checks from a single worker

Release 1.6.1: 14 Jun 2017

Fix

Allow to init resolver without rspamd_config

Fix

Do not crash when resolver failed to initialize

Fix

Fix abstract context layout

Fix

Fix CGP helper reply parsing

Fix

Fix crashes when socket write errors occur

Fix

Fix parsing IPv6 nameservers in resolv.conf

Fix

Milter: Don't defer on "greylist" action

Release 1.6.0: 12 Jun 2017

Conf

Add rspamd_proxy to the default configuration set

Conf

Add sample arc module config

Conf

Do away with systemd specifics completely

Conf

Increase min_bytes to avoid FP

Conf

Remove ratelimits from default configuration

CritFix

Fix accepting on IPv6 sockets

CritFix

Fix corruption when multiple fuzzy are defined

CritFix

Fix learn condition in fuzzy check

CritFix

Fix memory leak in fuzzy check

CritFix

Fix memory leak in maps scheduling

CritFix

Paese the last character in DKIM signature correctly

CritFix

Zero fill sockaddr_un

Feature

Add ability to add doc strings by example

Feature

Add API to verify DKIM (and ARC) signatures

Feature

Add compression/decompression to proxy

Feature

Add count to url structure

Feature

Add initial support of the new protocol reply

Feature

Add Lua plugin spamtrap

Feature

Add `monitored_address` for rbls

Feature

Add new schema for bayes tokens

Feature

Add preliminary ARC support to dkim code

Feature

Add preliminary support of ARC signing

Feature

Add rules to detect bad 8bit characters in From and To

Feature

Add scanning support for milter protocol

Feature

Add support for bidirectional symbols in rspamd_stats

Feature

Add support for static maps

Feature

Add support of maps with multiple regexps matches

Feature

Add `text_multiplier` param

Feature

Add the preliminary ARC plugin

Feature

Add top redirector targets rank

Feature

Allow async events to be registered from LUA rules

Feature

Allow storing bayes tokens in Redis

Feature

Allow to exclude specific domains from mx check

Feature

Allow to have a stack of watcher finalisers

Feature

Allow to pass hostname to `-i` flag in Rspamc

Feature

Allow to set custom user agent in url redirector

Feature

Allow to use custom callback when parsing resolv.conf

Feature

Allow to use domain from authenticated user

Feature

Bayes expiry plugin

Feature

Check dkim sign keys for modifications

Feature

DKIM signing: sign_networks/local address specific use_domain settings

Feature

DMARC: Support excluding domains from sampling

Feature

Expire processing items for URL redirector aggressively

Feature

Fix surbl monitored for IP lists, add `monitored_domain` option

Feature

Implement caching for dkim body hashes

Feature

Implement milter protocol scan reply

Feature

Improve omograph phishing detection

Feature

Initial support of self-scan in Rspamd proxy

Feature

Keep track of headers in milter interface

Feature

Milter headers: better controls for local/authenticated

Feature

Multimap: email:domain:tld filter

Feature

Preliminary DMARC reporting implementation

Feature

Reuse stemmers in the cache

Feature

Rework confighelp to load Lua plugins

Feature

Rework hfilter to use hyperscan if possible

Feature

Rework lua RSA API

Feature

Rmilter_headers: approximate rmilter's extended_spam_headers

Feature

Start integration of milter support in proxy

Feature

Store average words length and short words count

Feature

Store hash of headers order and names

Feature

Support MTA name header

Feature

Support multiple types of dkim signing in Lua

Feature

Support numeric arguments for Redis requests

Feature

Use headers hash in bayes metatokens

Feature

Use normal resolv.conf rules of rotation in Rspamd

Feature

Use version 2 proto for checking messages

Fix

Allow to follow symlinks when safe

Fix

Append MX name for authentication results as required

Fix

Change default text multiplier from 0.5 to 2.0

Fix

Check min_bytes for images as well

Fix

Deal with 7bit charsets properly

Fix

Deal with 8bit characters in email addresses

Fix

Deal with unpaired <a> tags

Fix

Detect confighelp in plugins initialisation

Fix

Disable certain checks for utf spoof detection

Fix

DKIM Signing: avoid nil index when From header is missing

Fix

Do not add exact hashes from different parts

Fix

Do not check DMARC if SPF or DKIM were not checked

Fix

Do not check URLs that are resolved to be redirected

Fix

Do not set bayes probability if we don't use it

Fix

Do not stop on illegal unicode points - replace them

Fix

Fix another race condition in arc checks

Fix

Fix arc count logic

Fix

Fix ARC signing

Fix

Fix brain-damaged spamc protocol for now

Fix

Fix calling for peak functions

Fix

Fix couple of issues in FORWARDED rule

Fix

Fix CTE propagation from parent containers to children parts

Fix

Fix errors processing in the controller

Fix

Fix format string in milter

Fix

Fix issues in SPF macros parsing

Fix

Fix logging format string

Fix

Fix logic of cached passwords check

Fix

Fix lowercasing of stemmed words

Fix

Fix LRU elements removal

Fix

Fix memory leak when accepting from unix sockets

Fix

Fix milter connections persistence

Fix

Fix objects merging in UCL

Fix

Fix order of operations to avoid race condition

Fix

Fix parsing of long regexp types

Fix

Fix passing data to log helper when many symbols defined

Fix

Fix pools management for milter session

Fix

Fix processing of the watchers

Fix

Fix queue id macro in milter

Fix

Fix R_BAD_CTE_7BIT rule

Fix

Fix Redis timeout set

Fix

Fix REPLYTO_UNPARSEABLE rule

Fix

Fix setting of email address

Fix

Fix some more issues about duplicated fuzzy requests

Fix

Fix spamc support in rspamd proxy

Fix

Fix syntax error in spamtrap plugin

Fix

Fix url counts for href urls

Fix

Fix url handling in the protocol

Fix

Multimap: Received IP filters with Redis

Fix

Oops, fix d9d0fa5e86db2f4470d34395a233b450478b2f60

Fix

Parse rgb[a](x,x,x[,x]) css colors

Fix

Phishing: strict_domains

Fix

Reduce maps aggressiveness

Fix

Reresolve upstreams even if there is a single server there

Fix

Rspamadm grep: Disable Lua patterns in string search by default

Fix

Skip text parts when checking binary parts in fuzzy check

Fix

Support v2 checks in controller

Fix

Treat empty address as valid

Fix

Try harder to detect CTE

Fix

Try to deal with v4 mapped to v6 addresses on accept

Fix

Use dkim signing callback properly

Fix

Use non-volatile memory for storing data

Fix

Use static maps instead of ugly hack for radix_from_config

Fix

Use the same pool for related sessions

Rework

Continue modularisation for lua library

Rework

Initial milter protocol support

Rework

Make log pipes worker agnostic, add scanners API

Rework

Move authentication results generation to a separate routine

Rework

Move common DKIM functions to a separate lua module

Rework

Move global functions to a separate directory

Rework

Prepare dkim module for ARC checks

Rework

Propagate ucl variables from the command line

Rework

Remove multiple metrics support from Rspamd

Rework

Stop using name 'rmilter' for the modern protocol

Rework

Use LFU algorithm in LRU cache

Rules

Fix received TLS rules

Rules

Improve URL_COUNT_ODD rule

Release 1.5.9

Conf

Increase min_bytes to avoid FP

Conf

Remove ratelimits from default configuration

CritFix

Fix accepting on IPv6 sockets

CritFix

Zero fill sockaddr_un

Feature

Add `text_multiplier` param

Fix

Check min_bytes for images as well

Fix

Do not add exact hashes from different parts

Fix

Fix memory leak when accepting from unix sockets

Fix

Fix some more issues about duplicated fuzzy requests

Fix

Phishing: strict_domains

Fix

Skip text parts when checking binary parts in fuzzy check

Release 1.5.8

CritFix

Fix memory leak in fuzzy check

CritFix

Fix memory leak in maps scheduling

Feature

Multimap: email:domain:tld filter

Fix

DKIM Signing: avoid nil index when From header is missing

Fix

Do not set bayes probability if we don't use it

Fix

Do not stop on illegal unicode points - replace them

Fix

Fix brain-damaged spamc protocol for now

Fix

Fix Redis timeout set

Fix

Fix spamc support in rspamd proxy

Fix

Multimap: Received IP filters with Redis

Fix

Parse rgb[a](x,x,x[,x]) css colors

Fix

Reresolve upstreams even if there is a single server there

Fix

Treat empty address as valid

Fix

Try harder to detect CTE

Fix

Try to deal with v4 mapped to v6 addresses on accept

Release 1.5.7

CritFix

Fix corruption when multiple fuzzy are defined

CritFix

Fix learn condition in fuzzy check

Feature

Add rules to detect bad 8bit characters in From and To

Feature

DKIM signing: sign_networks/local address specific use_domain settings

Feature

Support numeric arguments for Redis requests

Fix

Deal with 8bit characters in email addresses

Fix

Fix couple of issues in FORWARDED rule

Fix

Fix passing data to log helper when many symbols defined

Fix

Fix R_BAD_CTE_7BIT rule

Fix

Fix REPLYTO_UNPARSEABLE rule

Fix

Fix setting of email address

Fix

Rspamadm grep: Disable Lua patterns in string search by default

Release 1.5.6

Feature

Add unigramms support in bayes

Feature

Allow configurable sign headers for DKIM

Feature

Allow to add unigramm metatokens from Lua

Feature

DKIM Signing: envelope match exception for local IPs

Feature

UCL: register parser variables from Lua

Fix

Always try to adjust filename

Fix

Do extra copy to ensure that original content is never touched

Fix

Fix SPOOF_REPLYTO rule

Fix

Ignore Rmilter added Received

Fix

More fixes for hashed email dnsbls

Fix

Plug memory leak in chartable module

Release 1.5.5

CritFix

Fix classifier learning with Redis backend

CritFix

Fix issue when parsing encoded rfc822/messages

Feature

Add escaped version of lua_ucl import

Feature

Add task:headers_foreach function

Feature

Allow to process filenames from content type

Feature

Allow to query hashed emails

Feature

Ignore bayes with mostly metatokens or with too few text

Feature

Probabilistically skip metatokens

Feature

Retrieve all virus names from SAVAPI

Feature

Rework classifiers lua metatokens

Feature

Store headers order

Feature

Store text tokens inside bayes tokens

Feature

Use cached shingles keys

Fix

Add missing score normalisation for HFILTER_URL_ONLY

Fix

Avoid lookup in absent hash

Fix

Check return values from Lua functions called from C

Fix

Do not count sending and loading time in rspamc

Fix

Escape json strings for controller rejplies from Lua

Fix

Fix archive scans for savapi

Fix

Fix domain_only emails RBL

Fix

Fix ip_score map configuration

Fix

Fix JSON output for history_redis

Fix

Fix one character length substrings search

Fix

Fix parsing of non-RFC compatible Exim received

Fix

Fix parsing of options for workers with the same type

Fix

Fix processing of small tokens vectors

Fix

Fix rfc2047 tokenization

Fix

Fix typo

Fix

More fixes for inplace decoding

Fix

Try to avoid modifications of the original data

Fix

URL redirector: Fix call to is_redirector

Rework

Set token data as uint64_t instead of chars array

Release 1.5.4

Conf

Add history_redis default configuration

Feature

Add spoofed rules

Feature

Add URL_IN_SUBJECT rule

Feature

Allow to get task's subject

Feature

Allow to specify maximum number of shots for symbols

Feature

Distinguish URLs found in Subject

Feature

Memoize LPEG grammars

Feature

Parse else parts in SA rules

Feature

Process subject for mixed characters

Feature

Resolve url chains in url_redirector module

Feature

Stat greylisted messages as greylisted not soft-rejected

Feature

Support checking for redirector in Lua SURBL

Feature

Support tag_exists SA function

Feature

Work with broken rfc2047 tokens

Fix

Check all watcher's dependencies

Fix

Do not compile hyperscan with no SSSE3 support

Fix

Do not crash if cannot decode qp encoded part

Fix

Fix dependencies of DKIM when multiple signatures are found

Fix

Fix lists in whitelist plugin

Fix

Fix one-shot symbols weight calculations

Fix

Fix options and shots match

Fix

Fix order of symbol options

Fix

Fix parsing of dot at the end of the address

Fix

Fix parsing of lua table arguments

Fix

Fix processing of subject words

Fix

Fix string split memoization

Fix

Fix templates grammar usage

Fix

Fix various issues related to Lua stack manipulation

Fix

Force actions: Use postfilter if we have honor_action / require_action

Fix

Further fixes to avoid PHISHING FP

Fix

Preserve order of options in symbols

Fix

Rspamadm grep: deal with unusually-formatted logs

Fix

Use hostname suffix when dealing with history

Rework

Remove outdated SA rules

Release 1.5.3

Conf

Add composite for hacked wordpress phishing

CritFix

Fix base64 decoding when there are unparseable characters

Feature

Additional symbol metadata in metadata exporter

Feature

Add method to get protocol reply from Lua

Feature

Add symbols when tagged rcpt/sender are normalised

Feature

Add task:get_symbols_all() function

Feature

Allow multiple formats of DKIM signing key

Feature

Allow to cache and use flexible protocol reply

Feature

Allow to set one_shot flag from register_symbol

Feature

Allow to skip certain types of hashes when learning fuzzy

Feature

Cache and insert scan time into the protocol

Feature

Detect newlines in rspamc --mime

Feature

DKIM signing: support use of maps

Feature

Greylist: Support excluding low-scoring messages from greylisting

Feature

Implement lua history in controller

Feature

Implement redis history querying

Feature

Preliminary implementation of redis history plugin

Feature

Support using request headers in settings

Fix

Change default template to deal with non-ASCII characters

Fix

Deal with lists of maps in whitelist module

Fix

DKIM signing: use domain-specific signing key

Fix

Do not reallocate completed zstd buffer

Fix

Do not use local_addrs in proxy

Fix

Fix crash when resolver is undefined

Fix

Fix double free when closing lua_tcp connections

Fix

Fix for lua 5.3

Fix

Fix freeing of arrays iterators

Fix

Fix issue with task:get_symbol and symbols with no metric

Fix

Fix log line duplication in `rspamadm grep`

Fix

Fix memory corruption on termination

Fix

Fix out-of-bound access in base64 decode

Fix

Fix ratelimit + greylisting

Fix

Fix subject rewriting

Fix

Fix task:set_recipients function

Fix

Fix URI_COUNT_ODD rule

Fix

Follow the traditional symbols conventions in RCPT_COUNT rule

Fix

Greylist: Suppress greylist action for whitelisted hosts too

Fix

Metadata exporter: use rule-specific settings for emails

Fix

Properly set missing fields in exporter

Fix

Proxy: max_retries option

Fix

RCPT_COUNT fixes

Fix

Rework HAS_X_PRIO rule to match symbols conventions

Fix

Update issues in ac-trie

Fix

Use optimised base64 decoding in DKIM

Release 1.5.2

Conf

Add default config for spamassasssin plugin

Conf

Add default configuration for antivirus module

Conf

Add dkim signing docs

Conf

Add mx_check default config

Conf

Add replies config

Conf

Add trie default config

Feature

Add heuristic to find text parts in files

Feature

Add rule to detect broken content type

Feature

Allow to extract CTE in Lua API

Feature

Allow to set from address for a lua_task

Feature

Allow to set recipients of a task from Lua

Feature

Enchance text_part:get_content method

Feature

Remove + aliases from emails

Feature

Support rmilter block and dkim signature in CGP helper

Feature

Support running event loop from Lua

Fix

Antivirus: use scanner-specific redis prefix

Fix

Couple of fixes for DKIM signing module

Fix

Distinguish missing and broken mandatory headers

Fix

Do more heuristical detection for missing CTE

Fix

Do not resort cache on each check

Fix

Fix CGP escaping

Fix

Fix MISSING_MIME_VERSION rule for plain messages

Fix

Fix parsing of cte in expressions

Fix

Fix partial matches in rspamadm grep

Fix

Fix setting class on style field

Release 1.5.1

CritFix

Fix processing of stop_patterns with `\0` character

CritFix

Fix setting of raw key for signing

Fix

Fix lua exports from plugins during reload

Fix

Fix prefilters action scores

Fix

Fix symbols processing order

Release 1.5.0

Conf

Add configurations for asn, clickhouse and dcc

Conf

Add default config for url redirector plugin

Conf

Add the default config for greylist module

Conf

Allow to edit all local maps from WebUI by default

CritFix

Deal with absent headers in DKIM

CritFix

Do not trust remote shingles count

CritFix

Fix bad memory leak in TLS certificates validation

CritFix

Fix critical memory issues with radix maps

CritFix

Fix descriptors leak on reload

CritFix

Fix headers selection in DKIM verification

CritFix

Fix parsing of boundaries that end with `--`

CritFix

Repair PTR_ARRAY_FOREACH macro

Feature

Add CORS support to the controller

Feature

Add FROM_NAME_EXCESS_SPACE rule

Feature

Add REPLYTO_EMAIL_HAS_TITLE rule

Feature

Add `caseless_hash` method to `lua_util`

Feature

Add `rip` keyword to ratelimit module

Feature

Add a simple benchmark for content type parsing

Feature

Add boundaries parsing in content type

Feature

Add charset detection for text parts

Feature

Add content disposition parser

Feature

Add fallback if too many updates are failing

Feature

Add function to convert struct tm to time using timezone

Feature

Add function to normalize HTTP paths

Feature

Add fuzzy collection plugin

Feature

Add fuzzy logic for images

Feature

Add gmime parser to mime_tool

Feature

Add heuristic to detect broken messages

Feature

Add heuristic to find displayed URLs

Feature

Add heuristic to process broken email addresses

Feature

Add images normalization

Feature

Add mechanism for disabling composites (Fixes #1270)

Feature

Add method to create regexp from a glob pattern

Feature

Add mime encoding manipulation routines

Feature

Add mime tool to explore messages

Feature

Add more meta tokens from received headers

Feature

Add neighbours option to support Rspamd cluster in WebUI

Feature

Add new function to parse mime addresses

Feature

Add new methods for lua_tcp

Feature

Add own headers decoding routine

Feature

Add own routine to generate a message id

Feature

Add parser for SMTP date

Feature

Add per-task lua cache to reuse 'heavy' objects

Feature

Add plugins list path in WebUI

Feature

Add preliminary multipart support

Feature

Add preliminary version of DKIM signing module

Feature

Add profiling support in client output

Feature

Add rfc2047 grammar

Feature

Add rfc2047 variant for QP decoding

Feature

Add rmilter_headers module (Fixes #1227)

Feature

Add sse42 version of base64 decoding

Feature

Add ssse3 and avx2 base64 decoders

Feature

Add support of libgd

Feature

Add the preliminary version of redirects resolver in Lua

Feature

Add ucl_object_iterate_full function

Feature

Add url encoding function

Feature

Allow SOA requests in lua dns

Feature

Allow custom parse types in lua ucl

Feature

Allow plugins to register webui handlers

Feature

Allow to add options explicitly to symbols

Feature

Allow to call a callback when symbol frequency is on peak

Feature

Allow to call redirector script from SURBL

Feature

Allow to create variable length dkim keys

Feature

Allow to have module specific options for Redis in plugins

Feature

Allow to pass sign key directly from Lua

Feature

Allow to register configuration docs from Lua API

Feature

Allow to return options as a table

Feature

Allow to set peak callbacks from Lua

Feature

Allow to specify custom method for a message

Feature

Allow to store dkim keys in Redis

Feature

Allow to store messages in files

Feature

Apply DCT using AAN for fuzzy signature

Feature

Avira SAVAPI support

Feature

Cache and simplify DCT and jpeg decode

Feature

Cache libicu converters

Feature

Detect URLs with suspicious omographs

Feature

Do not increase score for duplicate options

Feature

Do not trust CTE, check base64 and qp strictly

Feature

Dynamic reputation in URL reputation plugin

Feature

Extend redis lock when learning spawned

Feature

Filter non-utf chars from all decoded headers

Feature

Fix phishing detection for IDNA urls

Feature

Ignore bad symbols on base64 decoding

Feature

Ignore too wide elements in SPF

Feature

Implement fuzzy collection mode

Feature

Implement helo maps in multimap

Feature

Implement human readable buckets configuration

Feature

Implement min-hash shingles for DCT data from images

Feature

Implement new algorithm for fuzzy hashes of images

Feature

Implement new unicode normalizer

Feature

Implement quoted printable decoding

Feature

Implement received headers flags

Feature

Implement rspamdgrep tool

Feature

Implement sane checksum for config file

Feature

Implement url tags concept

Feature

Improve detection of omographs using libicu

Feature

Improve url redirector module

Feature

Multimap: Received header processing

Feature

Multiple improvements in the maps

Feature

New URL filters in multimap

Feature

Plugin to force actions on selected symbols

Feature

RBL module: support hashing for emails and helo RBL

Feature

Reuse URL tags in SURBL module

Feature

Rework RRD ds count, add conversion path

Feature

Rework surbl module to avoid extra redirector calls

Feature

Send config id to the WebUI

Feature

Simplify HTTPCrypt client support

Feature

Skip processing for large images

Feature

Start collection only mode implementation for fuzzy storage

Feature

Start import of the optimized base64 decode

Feature

Store all received headers in lua

Feature

Store relational order of all headers in a message

Feature

Support DKIM signing in Lua plugins

Feature

Support HTTPCrypt client in lua_http

Feature

Support setting SMTP message in multimap

Feature

Support setting metric subject from Lua

Feature

Support setting subject in force actions module

Feature

Treat v6 mapped addresses as v4 addresses

Feature

URL reputation plugin

Feature

Use Redis instead of memcached in URLs redirector

Feature

Use Rspamd rfc2047 decoder instead of gmime one

Feature

Use a different normalization for fuzzy images

Feature

Use normalized images in fuzzy hashes

Feature

Use own code for parsing of date

Feature

Use shingles for images fuzzying

Feature

Use t1ha for hashes, allow inlining

Feature

Use t1ha instead of metrohash and xxhash32

Feature

Various new features in metadata exporter module

Feature

rmilter_headers: authentication-results (#78)

Fix

Add additional check to mark redis connection inactive

Fix

Add packed attribute for protocol structure

Fix

Adopt OMOGRAPH_URL rule

Fix

Allow static maps

Fix

Allow to disable classifiers checks using settings and conditions

Fix

Another try to fix 0 length maps

Fix

Another try to fix corruption during maps reload

Fix

Another try to fix descriptors leak

Fix

Another try to fix reload and logger

Fix

Antivirus module: register virtual symbols for patterns

Fix

Avoid extensive reallocs

Fix

Avoid mempool leak in SA plugin on reload

Fix

Avoid race condition on saving cache and reload

Fix

Avoid reusing g_error (Fixes #1262)

Fix

Break pool connection on fatal redis errors

Fix

Check for NaN properly

Fix

Couple of fixes for date parsing

Fix

Date header timezone adjustments (#1279)

Fix

Deal with EOF properly

Fix

Decode filename in content disposition

Fix

Disable fuzzy images by default

Fix

Disable zero-copy mode for text parts to avoid crashes

Fix

Do not destroy session when not all finish scripts are done

Fix

Do not greyscale images

Fix

Do not leave parent-less workers processes on fatal errors

Fix

Do not lowercase Content-Disposition to perform decoding

Fix

Do not penalize characters just after numeric prefix

Fix

Do not refork workers that are intended to die

Fix

Do not set pre-result and update records for no `Queue-ID` messages

Fix

Do not skip post-filters when pre-filters have set some results

Fix

Do not stop symbols planning if async events are pending

Fix

Do not try to set keys for unencrypted requests in proxy

Fix

Encode URLs according to rfc3986

Fix

Encode URLs before sending them to the protocol

Fix

Filter bad characters from message id

Fix

Fix CTE detection heuristic

Fix

Fix Content-Type in HTTP requests

Fix

Fix IDN eslds phishing checks

Fix

Fix adding maps from config in Lua

Fix

Fix another reload memory issue

Fix

Fix argument returned on redis backend errors

Fix

Fix assertion in graph handling

Fix

Fix body trie matching

Fix

Fix build

Fix

Fix byte array expansion during toutf8 conversion

Fix

Fix charset normalisation

Fix

Fix checking of DKIM bodies that needs just `\n` to be added

Fix

Fix couple of cornercases with email addresses

Fix

Fix couple of issues

Fix

Fix dependencies tracking for callback symbols

Fix

Fix detection of jpeg size

Fix

Fix errors handling in fuzzy backend initialization

Fix

Fix fuzzy hashes count

Fix

Fix globbing and convert lists to arrays in fuzzy_check

Fix

Fix heuristical CTE detection for QP encoding

Fix

Fix ignoring of bad text parts

Fix

Fix indexes in array access, interleave loop

Fix

Fix int64 -> double conversion

Fix

Fix invalid memory access on reload

Fix

Fix issues with empty updates

Fix

Fix issues with quoted-printable encoding

Fix

Fix keys names

Fix

Fix lots of issues in mime parser code

Fix

Fix lua maps load

Fix

Fix macro name

Fix

Fix mas group score calculations

Fix

Fix matching of the same patterns from different tries

Fix

Fix memory corruprtion and leak

Fix

Fix memory leak in HTTP maps

Fix

Fix memory leak in expression destroying

Fix

Fix memory leak in parsing of mime names

Fix

Fix memory leak in safe ucl iterators

Fix

Fix memory leak on reload in plugins

Fix

Fix modules reconfigure on reload

Fix

Fix monitored setup fro URLBLs with IP addresses

Fix

Fix name of var

Fix

Fix new rrd updates

Fix

Fix out of bounds access

Fix

Fix parsing messages with no body

Fix

Fix parsing of '=' character in headers

Fix

Fix parsing of messages with no content type

Fix

Fix plugins callbacks in webui

Fix

Fix possible memory corruption in redis pool

Fix

Fix probability calculations for fuzzy redis backend

Fix

Fix processing errors in lua_tcp

Fix

Fix processing of emails with name only

Fix

Fix processing of non-multipart messages

Fix

Fix processing of parts with no valid content type

Fix

Fix race condition in SIGUSR2 handler

Fix

Fix redis options parsing when no redis servers are defined

Fix

Fix reload and hyperscan ready event

Fix

Fix reload memory issue

Fix

Fix rra_ptr conversion

Fix

Fix rrd file conversion

Fix

Fix setting of content-type attributes

Fix

Fix signing headers creation in DKIM

Fix

Fix stddev calculations

Fix

Fix surbl plugin to work with composite maps

Fix

Fix timezones parsing

Fix

Fix tokens usage

Fix

Fix urls and emails hashes

Fix

Fix usage of unsafe ucl iterators

Fix

Fix work with broken utf8 tokens

Fix

Fix writing of user to roll history

Fix

Forgotten worker

Fix

Further memory leaks fixes

Fix

Ignore lua metatokens in bayes for now

Fix

Improve OMOGRAPH_URL rule

Fix

Lua IP from string should be invalid if parsing failed

Fix

Miltiple fixes to new lua_tcp, add debugging

Fix

More fixes for iterators cleanup

Fix

More fixes to logger initialization

Fix

More heuristic fixes for phishing detection

Fix

More leaks eliminated

Fix

More leaks...

Fix

More random fixes for reload...

Fix

Multimap: Fixes for email filters

Fix

Multiple fixes for fann module

Fix

Multiple memory corruption fixes

Fix

Normalize path in HTTP router

Fix

Plug memory leak

Fix

Plug memory leak in adding radix trees

Fix

Plug memory leak in configuration parser

Fix

Plug memory leak in expressions parsing during reload

Fix

Plug memory leak in learning fuzzy storage

Fix

Plug memory leak in lua_tcp

Fix

Plug reload leaks

Fix

Plug termination memory leaks

Fix

Really increase lock lifetime

Fix

Replies module: fix symbol weight

Fix

Restore content type params related functions

Fix

Set task's subject from mime subject

Fix

Sigh, one more reload leak

Fix

Simplify images shingles

Fix

Some more memory issues are fixed

Fix

Stop hardcoding of lua in C

Fix

Stop processing of bad parts as text parts

Fix

Strictly filter bad characters when emittin json

Fix

Strings returned from lua are ephemeral

Fix

Support unix sockets for lua redis

Fix

Try to fix issues with reloading config

Fix

Try to fix race condition in redis_pool

Fix

Use checksum to avoid intersection between different ANNs

Fix

Use rspamd hashes in embedded ucl

Fix

Use sane default rewrite subject (*** SPAM *** %s)

Fix

Various collection mode fixes

Fix

Various fixes to mime parser

Fix

Various reload leak fixing

Fix

Whitelist certain extensions from archive checks

Rework

Add preliminary implementation of the mime parser

Rework

Adopt code for the new options

Rework

Change logger setup interface

Rework

Composite configuration (#1270)

Rework

Finally remove gmime dependency from Rspamd

Rework

Further fixes to symbols frequencies

Rework

Implement content type parser for mime

Rework

Kill all InternetAddressList usages

Rework

Multiple fixes for symbols cache statistics

Rework

Refactor struct names

Rework

Rework images fuzzy hashes algorithm

Rework

Rework lua_tcp to allow TCP dialog

Rework

Start massive rework to get rid of gmime

Rework

Start new approach for multiparts parsing

Rework

Start rework of mime addresses

Rework

Start rework of symbols cache updates

Rework

Start switching to libicu

Rework

Use a special structure for stats tokens

Rework

Use hash tables for symbols options

Rework

Use libicu instead of iconv for conversions

Rework

Use new scheme to parse mime parts

Release 1.4.2

CritFix

Deal with absent headers in DKIM

CritFix

Do not trust remote shingles count

CritFix

Fix headers selection in DKIM verification

Feature

Add EXT_CSS rule

Feature

Add toggle for disabling SURBLs

Feature

Extend redis lock when learning spawned

Feature

Parse <link> HTML tags

Fix

Avoid reusing g_error (Fixes #1262)

Fix

Do not reset loaded ANN when learning is requested

Fix

Fix another issue with external deps in SA

Fix

Fix body trie matching

Fix

Fix checking of DKIM bodies that needs just `\n` to be added

Fix

Fix fuzzy hashes count

Fix

Fix keys names

Fix

Fix length calculations for url encoded urls

Fix

Fix matching of the same patterns from different tries

Fix

Fix name of var

Fix

Fix parsing of URLs with spaces and other bad chars

Fix

Fix probability calculations for fuzzy redis backend

Fix

Fix signing headers creation in DKIM

Fix

Plug memory leak

Fix

Really fix chained SA dependencies

Fix

Really increase lock lifetime

Fix

Use checksum to avoid intersection between different ANNs

Fix

Use rspamd hashes in embedded ucl

Fix

Yet another change for testing external deps

Release 1.4.1

Feature

ASN support in Clickhouse module

Feature

Add clickhouse plugin

Feature

Add generic tool to add universal maps for lua modules

Feature

Add logger.debugm to debug lua modules

Feature

Allow to register metrics symbols using register_symbol

Feature

Allow to specify prefix for fann_redis

Feature

Clickhouse: support different masks for IPv4/IPv6

Feature

Support forcing action in antivirus plugin

Fix

Add handling of regexp maps

Fix

Allow backslashes in http urls

Fix

Avoid mapping of empty files

Fix

Do not load tld file to speed up rspamadm

Fix

Do not resolve numeric IP addresses due to ipv6 insanity

Fix

Filter incorrect training data

Fix

Fix Fuzzyconvert tool when password or DB is given

Fix

Fix build with custom glib/gmime

Fix

Fix converting of learn count from sqlite to redis

Fix

Fix crashes with invalid received and task:set_from_ip

Fix

Fix external dependencies for SA module

Fix

Fix fann_redis when number of scores has been changed

Fix

Fix hyperscan usage for non compatible platforms

Fix

Fix loading of maps from UCL objects

Fix

Fix memory leak for task-less redis requests

Fix

Fix mid module with new maps syntax

Fix

Fix parsing of URLs with username

Fix

Fix re cache initialisation

Fix

Fix replacements to sanitize '%' character

Fix

Fix set and regexp like static maps

Fix

Fix some issues in redis settings

Fix

Fix static IP maps

Fix

Fix total learns counter for redis stats

Fix

Fix usage of config during reload

Fix

Fix various warnings and issues

Fix

Invalidate ANN if training data is incorrect

Fix

Miltiple fixes to fann_redis module

Fix

More fixes for URLs with backslashes

Fix

Properly get options for ip_score module

Fix

Relax requirements for Received as gmail cannot RFC

Fix

Remove or fix hyperscan incompatible regexps

Fix

Settings: correctly read redis config

Rework

Rework lua logger interface slightly

Rework

Use new maps add function

Release 1.4.0

CritFix

Add guards for inactive redis connections

CritFix

Another fix for proxying files using rspamd_proxy

CritFix

Cleanup inactive redis connections

CritFix

Do not sometimes try to exec posfilters before classification

CritFix

Fix application of IPv6 mask

CritFix

Fix chunked encoding when reading messages

CritFix

Fix file mode for rspamd_proxy

CritFix

Fix hyperscan compilation on regexp change

CritFix

Fix issue with finding of end of lines pointers

CritFix

Fix iteration over headers array (introduced in 1.4)

CritFix

Fix processing of learned tokens count for redis backend

CritFix

Fix race condition in checking of cached maps

CritFix

Fix workers scripts by sharing workers configs

CritFix

Introduce raw content to text parts

CritFix

Plug memory leak and potential memory corruption

Feature

Adaptive ratelimits

Feature

Add ASN -> rbldnsd script for asn.rspamd.com

Feature

Add DMARC_NA symbol

Feature

Add F-Prot support to antivirus module

Feature

Add HTTP backend to metadata exporter

Feature

Add Lua API module for monitored objects

Feature

Add R_DKIM_NA / R_SPF_NA / AUTH_NA symbols

Feature

Add R_DKIM_PERMFAIL symbol

Feature

Add R_SPF_PERMFAIL symbol

Feature

Add Sophos antivirus support

Feature

Add ZSTD compression to Lua API

Feature

Add `mid` Lua module

Feature

Add `one_param` flag for metric symbols

Feature

Add a generic lua classifier

Feature

Add a very basic interface to access workers data from on_load

Feature

Add ability to delete a hash by its data to fuzzy_check plugin

Feature

Add ability to enable/disable symbols via dynamic_conf

Feature

Add ability to lookup settings by key

Feature

Add common way to disable Lua modules

Feature

Add compression support to rspamd client

Feature

Add condition to do antiviral check

Feature

Add configuration for lua classifiers

Feature

Add configuration knobs for the errors circular buffer

Feature

Add decompression support in rspamd client

Feature

Add errors exporter to the controller

Feature

Add expected value for monitored DNS resources

Feature

Add exporter from error ringbuf to ucl

Feature

Add extended version for fann creation function

Feature

Add ffi friendly version of process_regexp function

Feature

Add frequency and time display to webui

Feature

Add fuzzy_delhash command to rspamc client

Feature

Add implementation of redis connections pool

Feature

Add latency and offline time monitoring

Feature

Add learning support for lua classifiers

Feature

Add max-size and timeout options to CGP helper

Feature

Add method to enable/disable symbols in config

Feature

Add methods to get metric's actions and symbols from Lua

Feature

Add mmap support to lua_text

Feature

Add monitored object for surbl plugin

Feature

Add more exceptions to surbl whitelist

Feature

Add more meta-tokens to bayes

Feature

Add neural net classifier to fann_scores module

Feature

Add neural net serialization/deserialization

Feature

Add new dynamic conf module

Feature

Add periodic events support for lua_config

Feature

Add plugin to check MX'es for the sender's domain

Feature

Add preliminary monitored module

Feature

Add preliminary support of dynamic conf updates in Redis

Feature

Add preliminary version of clamav plugin

Feature

Add redis cache to asn module

Feature

Add replies compression

Feature

Add spamhaus DROP dnsbl

Feature

Add support for dictionary in client compression

Feature

Add support for fuzzy learn and unlearn from lua

Feature

Add support for input encryption

Feature

Add support of min_learns to neural net classifier

Feature

Add termination callbacks for workers

Feature

Add user-agent for rspamc

Feature

Add utility to perform classifier tests

Feature

Add zstd compression library

Feature

Allow HTTPS requests in lua_http

Feature

Allow conditions for pre and postfilters

Feature

Allow custom functions for ratelimits

Feature

Allow for excluding messages from AV scanning based on size

Feature

Allow for getting worker stats from Lua

Feature

Allow getting task UID from Lua

Feature

Allow parsing of mailbox messages from the commandline

Feature

Allow plugins to publish their lua API via rspamd_plugins

Feature

Allow to compare other systems with Rspamd

Feature

Allow to execute Lua scripts by controller

Feature

Allow to have a function to set custom greylist message

Feature

Allow to iterate over multiple tags

Feature

Allow to pass extra data from plugins to log helper

Feature

Allow to plan new periodics at different time

Feature

Allow to reset hashes

Feature

Allow to run rspamadm lua just as a lua interpreter

Feature

Allow to store settings in redis

Feature

Allow to update dynamic conf in Redis

Feature

Allow to use dictionaries for compression

Feature

Allow to use md5, sha1, sha256, sha384 and sha512 hashes in Lua

Feature

Allow whitelisting by IP for greylisting plugin

Feature

Antivirus: Support whitelists & pattern-matching sig names

Feature

Backport pack/unpack routines from Lua 5.3

Feature

Check settings with equal priopities in alphabetical order

Feature

Compress neural net in redis

Feature

Consider more tags when doing WHITE_ON_WHITE rule

Feature

Descriptive options for DMARC failure symbols

Feature

Descriptive options for RBL symbols

Feature

Enable configuration for monitored objects

Feature

Execute on_load scripts with ev_base ready

Feature

Fann scores now uses metadata from a message

Feature

Implement FANN threaded learning

Feature

Implement classifying for lua classifiers

Feature

Implement finish scripts for worker processes

Feature

Implement monitoring for DNS resources

Feature

Implement real priorities for pre and post filters

Feature

Insert two symbols: FANN_HAM and FANN_SPAM instead of one

Feature

Module to push metadata/messages to redis pubsub

Feature

Monitor RBL records

Feature

Move fann_classifier to a separate plugin

Feature

Normalize all ANN inputs

Feature

Preliminary version of metric exporter module

Feature

Preserve decompression context between tasks

Feature

Ratelimit: Support dynamic bucket size/leak rate

Feature

Relax FORGED_RECIPIENTS: allow senders to BCC themselves

Feature

Remove symbols weights on composites processing

Feature

Return symbol scores when getting resulting symbols

Feature

Rework lua tcp module

Feature

Rule to detect some obvious X-PHP-Originating-Script forgeries

Feature

Rule to identify some X-PHP-Script forgeries

Feature

Rules for scoring Google Message-ID fixes

Feature

Send hashes values to reply

Feature

Set expire for dmarc reports

Feature

Stop using cymru zone as it is unstable

Feature

Stop using of GLists for headers, improve performance

Feature

Store `for` in task:get_received_headers

Feature

Store `for` part in received headers

Feature

Store enabled flag for webui session

Feature

Store error messages in ring buffer

Feature

Support compressed maps

Feature

Support excluding selected users from ratelimits

Feature

Support looking up NS records in lua_dns

Feature

Support modern style SURBL configuration

Feature

Support multiple hashes in delhash path

Feature

Support new messages in rspamc

Feature

Support requests without reads in lua_tcp

Feature

Support setting task message from Lua

Feature

Track visibility of HTML elements

Feature

Try to add CRLF when checking DKIM

Feature

Try to guess line endings when folding headers

Feature

Try to improve normalization function for bayes

Feature

Use FFI to optimize SA module

Feature

Use length based arguments for redis, allow lua_text as arg

Feature

Use more layers for fann and another normalization

Feature

User-defined ratelimits

Feature

Utility to convert fuzzy storage from sqlite to redis

Feature

Yield DMARC_DNSFAIL on lookup failure

Fix

Adopt fuzzy storage for flexible backends

Fix

Allow plain IP addresses in Rspamd maps

Fix

Another fix for brain-damaged hiredis

Fix

Another fix for rdns write errors

Fix

Another fix for rdns_make_request_full invocation

Fix

Another fix in DKIM canonicalization

Fix

Another memory leak plugged

Fix

Another try to deal with posix idiotizm

Fix

Another try to fix RDNS events processing logic

Fix

Avoid double frees in HEAD requests

Fix

Avoid extra symbols for RBLs

Fix

Banish table.maxn from Lua parts

Fix

Check for socket error before connection in lua_tcp

Fix

Correctly propagate redis timeouts to Lua

Fix

Do not add extra newline in MIME mode

Fix

Do not be cheated by system hiredis

Fix

Do not classify when a message has not enough tokens

Fix

Do not crash on redis errors

Fix

Do not distinguish NXDOMAIN and NOREC for monitored

Fix

Do not replan retransmits if merely one server is defined

Fix

Do not use headers to calculate messages digests

Fix

Don't force action in replies module for authenticated users/local networks

Fix

Explicitly ban default passwords in webui

Fix

Finally fix ambiguity between parsed and resolved spf elts

Fix

Fix 'decoded' value in task:get_header_full()

Fix

Fix DKIM calculations

Fix

Fix DKIM signing for messages with no newline at the end

Fix

Fix DNS request in monitored

Fix

Fix DNS write errors processing

Fix

Fix HTTP methods other than GET and POST

Fix

Fix PERMFAIL for v6/v4 ambiguities

Fix

Fix absurdic scores for HFILTER_URL_ONLY

Fix

Fix actions in rolling history

Fix

Fix actrie patterns

Fix

Fix applying of lua dynamic confg

Fix

Fix autolearning errors and redis cache

Fix

Fix bayes learn_condition

Fix

Fix build with the recent OpenSSL

Fix

Fix caching and compressed maps

Fix

Fix check plain text part

Fix

Fix crash on OpenBSD in `url_email_start`

Fix

Fix double free in SPF

Fix

Fix extraction of shingles from redis fuzzy storage

Fix

Fix false sharing for symbols in the cache

Fix

Fix float usage in util:get_time

Fix

Fix folding algorithm to deal with empty tokens

Fix

Fix format string

Fix

Fix format string usage in controller errors handling

Fix

Fix handling of '\0' in lua_tcp

Fix

Fix handling of HTTP HEAD methods

Fix

Fix hash creation

Fix

Fix hiredis stupidity

Fix

Fix implicit settings module settingsup

Fix

Fix interaction with lua GC to avoid craches

Fix

Fix ip_score module registration

Fix

Fix issue with empty messages and dkim

Fix

Fix issues with CGP helper

Fix

Fix issues with the recent SPF changes

Fix

Fix key name to load ANN correctly

Fix

Fix lua tcp module by saving `do_read` in callback data

Fix

Fix memory leak in client when using compression

Fix

Fix min_learns option

Fix

Fix on_finish scripts and async handlers

Fix

Fix options for SPF dnsfail symbol

Fix

Fix parsing includes and redirects in SPF

Fix

Fix parsing of lua comments with empty lines

Fix

Fix parsing of unquoted HTML attributes

Fix

Fix periodic events and redis

Fix

Fix processing of fuzzy learns from Lua

Fix

Fix processing of redirect in SPF includes

Fix

Fix processing of symbols when reject limit is reached

Fix

Fix refcounts when map is specified by IP

Fix

Fix rspamd{session} class in Lua API

Fix

Fix setting ratelimit key for 'ip' bucket

Fix

Fix some cases of TLD urls detector

Fix

Fix statconvert tool

Fix

Fix stats for backend-less classifiers

Fix

Fix training script for fann_redis

Fix

Fix variable in ann module

Fix

Fix various errors in lua dynamic conf plugin

Fix

Forget old ANN when max_usages is reached to avoid overtrain

Fix

Further canonicalization fixes

Fix

Further fixes for fann_redis prefixes

Fix

Handle failures for inactive pooled connections

Fix

Improve multimap info message

Fix

More fixes in ANN loading

Fix

More fixes to fann_redis

Fix

More issues in fann_redis

Fix

More spaces fix in DKIM signature

Fix

Multiple fixes to asn script, add IPv6 support

Fix

Multiple issues in fann_redis

Fix

No greylist rejected messages

Fix

One more attempt to fix lua_redis

Fix

One more check for readdir...

Fix

Params should be treated as a hash

Fix

Plug memory leak in regexp desctructor

Fix

Process headers only once

Fix

Properly handle nil values in ratelimit plugin

Fix

Really fix redis shingles check

Fix

Remove fann with incorrect layers count

Fix

Remove mentions of deleted include

Fix

Remove some incompatible functions

Fix

Settings: fix `authenticated` parameter (#886)

Fix

Skip MX check for authenticated users and local networks

Fix

Slightly fix ANN routines

Fix

Stop caching records with DNS failures

Fix

Treat all errors in redis_pool as fatal errors for a connection

Fix

Try avoid false-positives in HEADER_FORGED_MDN rule

Fix

Try to avoid race condition when using rrd

Fix

Try to reload redis scripts if they are missing

Fix

Unbreak once_received skipping for local networks

Fix

Unlock ANN on error

Fix

Use memmove for overlapping regions

Fix

Use real size instead of displayed for core limits

Fix

Use the correct macro to get the size of control

Fix

Various fixes for errors ringbuffer

Fix

Yield R_SPF_DNSFAIL if lookup of included record fails

Fix

mid: fix map initialization

Fix

mid: handle incorrect rgexps in the map

Rework

Add extract training data function to fann_redis

Rework

Add preliminary train tests

Rework

Add redis storage feature to fann_redis

Rework

Adopt fuzzy storage for abstract backend

Rework

Adopt plugins

Rework

First reiteration on fann scores

Rework

Implement loading/invalidating

Rework

Make lua_redis task agnostic

Rework

Make rspamd protocol messages useful

Rework

Massive removal of legacy code

Rework

More cleanup actions

Rework

Remove legacy code never used for classifiers

Rework

Remove outdated and unused lua_session module

Rework

Reorganize fuzzy backend structure

Rework

Reorganize the internal backend structure

Rework

Restore old fann_scores, move common parts

Rework

Rework and simplify rbl plugin

Rework

Rework parsing of DMARC records

Release 1.3.4

Feature

ASN module; support matching ASN/country in multimap

Feature

Add SPF method in spf return result

Feature

Add Yandex and Mail.ru forwarding rules

Feature

Add mempool maps in multimap

Feature

Add rule for identifying mail sent by eval()'d PHP code

Feature

Add support of stub DNSSEC resolver to rdns

Feature

Add task:get_digest method

Feature

Allow for more fine-grained scoring for ip_score

Feature

Allow to get digest of a mime part from lua

Feature

Allow to print message digest in logs

Feature

Fold DKIM-Signature header

Feature

Implement encrypted logs

Feature

Log URLs encrypted if we have log encryption pubkey

Feature

Pass authenticated bit to lua

Feature

Read redis backend statistics configuration from global section

Feature

Show the exact value matched for multima symbols

Feature

Store task checksum

Fix

Avoid setting limits when required elements are missing

Fix

DMARC: Fix alignment checking for subdomains

Fix

DMARC: deal with missing and spurious spaces

Fix

Defer insertion of results in ip_score to avoid skewing stats

Fix

Disable DMARC for local/authorized mail

Fix

Fix handling of proxied headers in controller

Fix

Fix hex printing of strings

Fix

Fix issue with spaces in maps

Fix

Fix parsing of forwarded IP

Fix

Fix reload in some plugins and workers

Fix

Fix reloading on SIGHUP

Fix

Fix some border cases for DKIM canonicalization

Fix

Fix url maps

Fix

Make dnssec configurable option disabled by default for now

Fix

rspamadm statconvert: force db to be a string

Fix

rspamadm statconvert: use db/password for learn cache

Rework

Rework flags in rspamd logger

Release 1.3.3

CritFix

Check hyperscan cache sanity before loading

CritFix

Fix setting of fuzzy keys (completely breaks fuzzy storage)

Feature

Add SARBL (sarbl.org) uribl

Feature

Add `--search-pattern` option to rspamd_stats

Feature

Add some sanity check for very long from/to log elements

Feature

Allow to create hashes from string in a single step

Feature

Fix order of pre and postfilters

Feature

Improve lua URLs API

Feature

Improve message about fuzzy rules

Feature

Pre-calculate blake2 digest for all parts

Feature

Print radix duplicate keys as IP addresses

Feature

Simple mechanism for disabling RBLs in local.d/rbl.conf

Feature

Use faster hash function for fuzzy storage

Feature

rspamd_stats: support log directory reading

Fix

Add sanity check for url filters

Fix

Do not show rmilter section as a fake metric in rspamc

Fix

Fix URL filters

Fix

Fix a stupid mistake in util.strequal_caseless

Fix

Fix blake2b hash of the string "rspamd"

Fix

Fix filename maps filter

Fix

Fix finding tld in util.get_tld

Fix

Fix multimap content filters

Fix

Fix returning boolean from Lua

Fix

Fix returning of REDIS_NIL

Fix

Try to deal with multiple workers terminated

Fix

Use forced DNS request when calling for lua_http

Rework

Rework multimap filters, add redis maps

Release 1.3.2

Feature

Add a special symbol for SPF DNS errors: R_SPF_DNSFAIL

Feature

Add correlations report in fuzzy stats

Feature

Add experimental CGP integration

Feature

Add method to get urls length in a text part

Feature

Add new methods to lua_html to access HTML tags

Feature

Allow all types of symbols to be added via __newindex method

Feature

Allow to create settings for authenticated users

Feature

Allow to get block content for HTML tags

Feature

Improve DNS failures when dealing with SPF

Feature

Properly implement R_WHITE_ON_WHITE rule

Feature

Remove old ugly rules

Feature

Rspamc can now add dkim signature in mime mode

Feature

Store content length for HTML tags

Feature

Support reacher set of HTML colors

Feature

Try to avoid FP for low contrast fonts detection

Fix

Add missing HTML colors

Fix

Add spaces to dkim signature to allow folding

Fix

Avoid returning NaN as score on scan

Fix

Decode entitles in href parts

Fix

Do not cache SPF records with DNS errors

Fix

Do not crash on cyclic depends

Fix

Do not insert HELO/HOSTNAME unknown when they are not passed

Fix

Do not set absent hostname to "unknown"

Fix

Do not stress redis with KEYS command (#791)

Fix

Fix DMARC_BAD_POLICY symbol

Fix

Fix HFILTER_URL module

Fix

Fix HFILTER_URL_ONELINE rule

Fix

Fix buffering in CGP integration

Fix

Fix colors propagation from parent nodes

Fix

Fix confusing OpenSSL API usage of i2d_RSAPublicKey

Fix

Fix dependencies id sanity check

Fix

Fix folding for semicolon separated tokens

Fix

Fix largest possible TLD behaviour

Fix

Fix last token folding

Fix

Fix length calculations in white on white rule

Fix

Fix multiple request headers structure

Fix

Fix multiple values headers freeing

Fix

Fix parsing of background color

Fix

Fix printing from field in log_urls

Fix

Fix processing of last element of DMARC policies

Fix

Further fixes for HTML colors

Fix

Further fixes for multiple values headers

Fix

Further fixes for white on white rule

Fix

Further fixes in HTML tags parsing

Fix

Ignore content type/subtype case

Fix

Increase score of R_WHITE_ON_WHITE

Fix

Parse CGP envelope data

Fix

Propagate colors in HTML

Fix

Restore multiple values headers in protocol

Fix

Restore multiple values in headers processing

Fix

Some more changes to tag's content length calculations

Fix

Some more fixes for low contrast fonts detector

Fix

SpamAssassin plugin: support check_freemail_header('EnvelopeFrom', [..])

Fix

Trigger HTML_SHORT_LINK_IMG on any external image

Fix

rspamd_stats: remove deprecated defined(@array)

Release 1.3.1

CritFix

Fix catena passwords validation

CritFix

Fix crash when the first received is faked

Feature

Add DMARC_BAD_POLICY symbol when DMARC policy was invalid

Feature

Allow for matching hostnames in multimap (#773)

Feature

Allow for setting action based on DMARC disposition

Feature

Allow limiting of the inbound message size

Feature

Allow maps with multiple symbols and scores

Feature

Allow regexps in the emails maps

Feature

Allow to register metric symbols from multimap

Feature

Allow to reset redis tokens instead of appendig values

Feature

Allow to store strings in radix maps

Feature

Check UTF validity when there are utf regexps in a map

Feature

Correctly work when there is no hard reject action

Feature

Implement dependencies for maps

Fix

Another effort to unbreak sqlite locking

Fix

Avoid crash when closing mmapped file

Fix

Do not break history on NaN in required score

Fix

Ensure that hyperscan cache written is written properly

Fix

Filter NaN from scores in history

Fix

Fix DNSBL maps

Fix

Fix another locking issue in sqlite

Fix

Fix another locking issue with mapped files

Fix

Fix deadlock in mmaped file stats

Fix

Fix dependencies in multimap plugin

Fix

Fix emails module configuration

Fix

Fix greylist plugin (#755)

Fix

Fix greylisting plugin variable usage

Fix

Fix installed permissions for rspamd_stats

Fix

Fix locking in mmapped statistics

Fix

Fix paths in tests

Fix

Fix prefilter mode for multimap

Fix

Forgot to commit leftover changes

Fix

Really fix local.d includes

Fix

Restore selective greylisting behaviour

Fix

Set max size on per connection basis

Fix

Use temporary storage for hyperscan cache

Rework

Remove systemd socket activation

Release 1.3.0

CritFix

Fix SA rawbody processing - exclude top part

CritFix

Fix decoding of UTF HTML entitles

CritFix

Fix encrypted fuzzy requests

CritFix

Fix leak of shared memory fds and files

CritFix

Fix levenshtein distance calculations

CritFix

Fix mime headers processing

CritFix

Fix parsing of URLs in texts

CritFix

Fix parsing of missing classes

CritFix

Fix redis structure by adding {NULL, NULL} member

CritFix

Fix some more URL detector issues

CritFix

Fix systemd sockets activation

CritFix

Fix unencrypted passwords processing in the controller

CritFix

Fix writing CDPs to the database

CritFix

Fix writing of encrypted HTTP requests

CritFix

Plug memory leak in dkim module

CritFix

Plug memory leak in headers getting code

CritFix

Pre-filters and post-filters were completely broken

CritFix

Properly support SA body regexps

CritFix

Really skip filters in case of pre-result set

CritFix

Restore the intended pre-filters behaviour

Rework

Adopt new maps code

Rework

Compile ragel sources when building rspamd

Rework

Finish rework for the rest of places that use HTTP

Rework

Fix DKIM headers canonicalization

Rework

Fix lua maps API

Rework

Import linenoise for line editing

Rework

Include config structure to all rcl handlers

Rework

Make chartable module useful

Rework

Move http internal structures to a private header

Rework

Partly fix controller

Rework

Remove dedicated images list

Rework

Rename http proxy to rspamd proxy

Rework

Rename mime parts structures

Rework

Rework HTTP code

Rework

Rework exceptions and newlines processing

Rework

Rework pre and postfilters system

Rework

Separate method to close backend connections

Rework

Start the complete maps rework

Rework

Use dynamically generated ragel C sources

Feature

Add 'blacklist' and 'strict' modes for whitelists

Feature

Add 'symbols_enabled' and 'groups_enabled' to settings

Feature

Add ESMTPSA received type

Feature

Add a simple script to evaluate rspamd rules in the logs

Feature

Add a simple tool to generate DKIM keys

Feature

Add a trivial heuristic for codepages

Feature

Add and use mumhash for non-crypto hashing

Feature

Add better method to check lua userdata types

Feature

Add better zip files search algorithm

Feature

Add catena PBKDF function

Feature

Add configuration knobs for in and out parser scripts

Feature

Add content filtering support to multimap

Feature

Add different timeouts for proxy connections

Feature

Add dot commands for lua REPL:

Feature

Add execution of lua global functions script

Feature

Add function for pretty printing of inet addresses

Feature

Add function to convert fstring_t to c string

Feature

Add function to create temporary shared memory mapping

Feature

Add function to generate random hex string

Feature

Add generic fucnction to parse IP maps

Feature

Add initial version of HTTP lua repl

Feature

Add learn condition to the default configuration

Feature

Add learn conditions for classifiers

Feature

Add limit for dkim signatures to be checked

Feature

Add locking routines for lua_util

Feature

Add lua API for getting info from archives

Feature

Add lua utility to decode URL encoding

Feature

Add method to copy message from http connection

Feature

Add mirrors feature

Feature

Add more algorithms for shingles generation

Feature

Add more domains to redirectors list

Feature

Add more encodingsto cryptobox hash API

Feature

Add more file utilities to lua_util

Feature

Add more functions to extract data from text parts

Feature

Add more methods to get headers from a task

Feature

Add more methods to init http message body

Feature

Add more options for redis config parsing function

Feature

Add new representation of email address

Feature

Add new symbols to filter bad extensions in messages

Feature

Add new utility methods to mimepart object

Feature

Add openphish support to rspamd phishing module

Feature

Add parsers for SMTP address in ragel

Feature

Add parsing of mirror hosts for fuzzy storage

Feature

Add parsing scripts for master connection as well

Feature

Add preliminary greylist plugin

Feature

Add preliminary phishtank support

Feature

Add preliminary rarv5 support

Feature

Add preliminary version of ssl toolbox

Feature

Add protection against open files limit and accepting sockets

Feature

Add rar v4 support

Feature

Add reading scripts for master connection

Feature

Add replies plugin

Feature

Add results parsing code

Feature

Add routines to compare and check pubkeys

Feature

Add simpler versions of refcounts

Feature

Add some time manipulation functions for lua APi

Feature

Add support for non-standard BATV signatures

Feature

Add support of address with at-domain list

Feature

Add support to search archives by magic

Feature

Add task:get_rawbody method

Feature

Add test to check shared memory support sanity

Feature

Add the initial version of LUA repl to rspamadm

Feature

Add throughput graph for RRD backend to WebUI

Feature

Add universal function to make a proper redis request

Feature

Add universal function to parse redis servers for plugins

Feature

Add util.unlink function

Feature

Add utility function to return random number from 0 to 1

Feature

Add utility method to convert ftok to C string

Feature

Add utility to map shared memory segments

Feature

Add versions to fuzzy storage

Feature

Add workaround for legacy clients in rspamd proxy

Feature

Add workaround for systems without sane shmem support

Feature

Add xoroshiro+ fast rng for non-crypto purposes

Feature

Adopt plugins for new maps API

Feature

Allow SPF to be checked for empty tasks

Feature

Allow binary patterns in lua_trie

Feature

Allow catena encrypted passwords in controller

Feature

Allow client ip match in the settings

Feature

Allow easy adding and overriding of fuzzy rules

Feature

Allow empty tasks to be processed

Feature

Allow hostnames in IP maps

Feature

Allow https maps

Feature

Allow multiple PBKDF types in `rspamadm pw`

Feature

Allow named fuzzy rules

Feature

Allow non zero terminated patterns in multipattern

Feature

Allow partial hash updates

Feature

Allow pipelining for redis.make_request

Feature

Allow sending empty requests using client

Feature

Allow setting fuzzy flag by symbol not by value

Feature

Allow setting scores and actions from lua

Feature

Allow shared memory simple http client

Feature

Allow static lua files in any parts of rspamd sources

Feature

Allow to change flag from fuzzy learn condition

Feature

Allow to check rspamd_text using maps

Feature

Allow to disable composite rules from settings

Feature

Allow to disable some modules from common redis setup

Feature

Allow to extract ucl_object from lua using common API

Feature

Allow to get settings and settings id hash from lua_task

Feature

Allow to have specific settings for mirrored traffic

Feature

Allow to open message from a shared memory segment

Feature

Allow to parse pubkeys from the rcl config

Feature

Allow to pass extradata from rspamd to rmilter

Feature

Allow to query storage about number of fuzzy hashes stored

Feature

Allow to read logs without symbols scores

Feature

Allow to read password from console for rspamc

Feature

Allow to set ciphers and CA paths in config

Feature

Allow to skip some initialization phases to speed up rspamadm

Feature

Allow underscore separated names in settings

Feature

Allow versioning for sqlite databases

Feature

Always allow to terminate rspamd

Feature

Better deal with backend errors

Feature

Better lua_redis logging

Feature

Configure CA path and ciphers

Feature

Create a dedicated parser to strip newlines

Feature

Deduplicate the same urls in multimap module

Feature

Distinguish luajit from lua

Feature

Do not print garbadge in --compact output

Feature

Dynamically detect if a CPU is incompatible with hyperscan

Feature

Enable forced resolving for some lua plugins

Feature

Enable rrd by default

Feature

Enable workaround for exim

Feature

Fix task functions to work without rspamd_config

Feature

Further improvements to chartable module

Feature

Further micro-optimizations for hashing and shingles

Feature

Further relax parser

Feature

Humanize numbers in stats widgets

Feature

Implement HTTPS client

Feature

Implement SSL support in http client

Feature

Implement body rules for the trie plugin

Feature

Implement braced regexp quantifiers

Feature

Implement compare scripts for mirrors results

Feature

Implement compare scripts setup

Feature

Implement composites policies

Feature

Implement conditional learning for classifiers

Feature

Implement constructing of map from UCL

Feature

Implement dkim signing in dkim check plugin

Feature

Implement fuzzy storage updates

Feature

Implement fuzzy updates push protocol

Feature

Implement https maps

Feature

Implement inter-process maps caching

Feature

Implement mapping of remote flags to local flags

Feature

Implement mirroring in rspamd proxy

Feature

Implement multi-flags fuzzy replies

Feature

Implement multiple-sources fuzzy storage

Feature

Implement order of pre/post filters

Feature

Implement partial deleting for multi-flags

Feature

Implement pipelining for redis async interface

Feature

Implement ragel parser for received headers

Feature

Implement reading of messages to shared memory

Feature

Implement refcount for messages

Feature

Implement retransmits for master connection

Feature

Implement zero-copy mode for HTTP reading

Feature

Improve SPF domain detection logic

Feature

Improve config:register_symbol function

Feature

Improve error report for type mismatch in lua

Feature

Improve fstrings API

Feature

Improve getting SMTP data from lua_task

Feature

Improve levenshtein distance function

Feature

Improve logging in proxy and add refcounts

Feature

Improve logging lua types

Feature

Improve master/slave logging

Feature

Improve phishing plugin

Feature

Improve phishtank and openphish support

Feature

Improve ragel build target

Feature

Improve statistics script

Feature

Initialize ssl library to use SSL connections

Feature

Interpolate points sent to webui

Feature

Limit logging of elements that could have too many items

Feature

Lock ANN file when loading

Feature

New abstract hashing API in cryptobox

Feature

Normalize quoted addresses

Feature

Now cryptobox lua API accepts rspamd text as input

Feature

Optimize alignment to speed up hashing

Feature

Parse received date and ESMTPA proto

Feature

Parse received timestamp

Feature

Pass settings id to log helper

Feature

Pass settings id to lua script from log helper

Feature

Perform files expansion on proxying

Feature

Preliminary implementation of fuzzy master/slave updates

Feature

Print userdata using tostring if possible

Feature

Propagate HTTP result string when using proxy

Feature

Properly implement unweighted round-robin algorithm

Feature

Reduce number of timers queries

Feature

Rework and improve fuzzy storage

Feature

Rework dns resolving API for lua, add 'forced' option

Feature

Rework fann module to understand settings

Feature

Rework listening system to allow multiple socket types per worker

Feature

Rework ratelimit module to set expiration

Feature

Save bayes probability in memory pool var

Feature

Save settings id hash for convenience

Feature

Search for SSL_set_tlsext_host_name support

Feature

Send DKIM signature to protocol reply

Feature

Show DKIM signature in rspamc client

Feature

Show symbols description in scan output

Feature

Sign message merely after DKIM check

Feature

Simplify machines by assuming that headers are unfolded

Feature

Sort symbols when displaying them in log

Feature

Split main connection from mirrored connections

Feature

Start moving to the new email address structure

Feature

Store HTTP headers in a hash table

Feature

Store more information about compressed files

Feature

Store raw headers value to use them in DKIM

Feature

Store text parts content with newlines stripped

Feature

Support DKIM signing

Feature

Support EXIF jpeg images

Feature

Support archive files list extraction

Feature

Support archives when matching patterns in multimap

Feature

Support premium/academic feed for openphish

Feature

Support rspamd_updates via https

Feature

Supprort FQDNs in phishing module maps

Feature

Try to read on fuzzy timeout to avoid fake timeouts

Feature

Try to select the optimal possible function for input

Feature

Unescape and unquote smtp addresses

Feature

Update fuzzy timestamp when adding value

Feature

Update mumhash

Feature

Use -flto if possible when optimizations are enabled

Feature

Use extended map types in lua map, unify code

Feature

Use file lock in logger to avoid deadlocks

Feature

Use generic global string split function

Feature

Use metrohash as well

Feature

Use mumhash by default for incremental hashing

Feature

Use mumhash for words hashing

Feature

Use new ip parsing API

Feature

Use new maps API for local addrs

Feature

Use new ragel parser in message parsing code

Feature

Use new received parser instead of old one

Feature

Use new redis API in DMARC plugin

Feature

Use new redis API in greylist plugin

Feature

Use new redis API in ip_score plugin

Feature

Use new redis API in ratelimit plugin

Feature

Use new redis API in replies plugin

Feature

Use new version of register_symbol in rspamd plugins

Feature

Use offset when passing shm to deal with encrypted requests

Feature

Use one pass to remove newlines and store their positions

Feature

Use rspamd specific type checks for userdata

Feature

Use shared memory storage for http maps

Feature

Use universal redis definitions in rspamd plugins

Feature

Various improvements in greylist module

Feature

Wait for sqlite if locked when switching to WAL mode

Fix

Add filenames sanity filtering for mime types

Fix

Add guards for empty parts

Fix

Add missing types

Fix

Add more priority for config file symbols registered from UCL

Fix

Add sanity checks when compiling regexp maps

Fix

Add spaces instead of newlines to the normalized content

Fix

Add workaround for ancient openssl

Fix

Add workaround for gmime CTE stupidity

Fix

Add workaround for hex digits

Fix

Adjust MISSING_MIMEOLE score

Fix

Adjust body/body_buf when stealing encrypted message

Fix

Adopt lua task API for the new email addresses structure

Fix

Allow for disabling DMARC reporting when Redis is configured

Fix

Always register openphish and phishtank virtual symbols

Fix

Always use shmem on linux

Fix

Another change of newlines policy

Fix

Another d3evolution update

Fix

Another fix for exim workaround

Fix

Another fix for legacy clients

Fix

Another fix for maps scheduling

Fix

Another fix for marking upstreams inactive

Fix

Another fix for postfilters

Fix

Another fix for redis timeouts

Fix

Avoid `table.getn` method as it has been removed in lua 5.3

Fix

Avoid double hashing for images

Fix

Avoid linking with actrie if hyperscan is enabled

Fix

Check copy result when sending message to mirrors

Fix

Cleanup message when assiging body

Fix

Cleanup stack from global vars

Fix

Correctly parse query type

Fix

Disable all symbols if enable_groups is found in settings

Fix

Disable fts as it is completely broken in bloody linux

Fix

Disable multiple autolearn checks

Fix

Disallow updates by default

Fix

Do not abort when cannot load a map

Fix

Do not check recursion for non-DNS SPF record types

Fix

Do not delete uninitialized events

Fix

Do not die if shmem_mkstemp fails

Fix

Do not die when no metrics defined

Fix

Do not even try pcre in case of regexp maps

Fix

Do not greylist messages if redis has failed somehow

Fix

Do not greylist on rejection

Fix

Do not leave temporary maps cached

Fix

Do not output meaningless errors

Fix

Do not send NaN in json

Fix

Don't mix hyperscan and pcre processing within a same task

Fix

Finally rework and simplify redis backend for statistics

Fix

Fix Exim shutdown patch

Fix

Fix JIT compilation for PCRE2 expressions

Fix

Fix JIT usage for PCRE2

Fix

Fix REPL output

Fix

Fix SMTP address parsing machine

Fix

Fix UTF8 mode in PCRE2

Fix

Fix a stupid misprint in word 'phishing'

Fix

Fix accepting the first update when local idx is -1

Fix

Fix adding maps from ucl

Fix

Fix adding upstream to an active queue

Fix

Fix and rescore R_PARTS_DIFFER logic

Fix

Fix body rules in SA plugin

Fix

Fix body start position

Fix

Fix border case in urls detector

Fix

Fix border cases for incremental hashing

Fix

Fix caseless uthash application

Fix

Fix chartable issue with starting digits

Fix

Fix client_ip in users settings

Fix

Fix compilation issue

Fix

Fix conditional learning

Fix

Fix crash on empty maps

Fix

Fix creating of URLs from LUA

Fix

Fix creating of temporary shmem segment

Fix

Fix creation of mmapped statfiles

Fix

Fix descriptors leak on shmem detaching

Fix

Fix detaching of shared memory segments

Fix

Fix detection of URLs in text parts

Fix

Fix directories processing for rspamc

Fix

Fix displaying of rewrite subject in WebUI

Fix

Fix dkim private keys operations

Fix

Fix dkim signing

Fix

Fix dynamic scoring of subject symbols

Fix

Fix email address build

Fix

Fix encrypted proxy requests

Fix

Fix errors counting in upstreams

Fix

Fix errors handling in the proxy

Fix

Fix event bases for IO events

Fix

Fix events handling when scheduling map wacth

Fix

Fix fann rewrite

Fix

Fix files fallback for shmem transfer

Fix

Fix fuzzy adding in webui

Fix

Fix fuzzy storage encrypted mirroring

Fix

Fix fuzzy storage sync replies

Fix

Fix handling of the same words

Fix

Fix inserting values to the sources list

Fix

Fix ipv6 mask application

Fix

Fix issue with missing recipients

Fix

Fix issues with multiple returns from lua

Fix

Fix learning for non-existent backend

Fix

Fix legacy clients support in proxy

Fix

Fix length calculations for shared memory segments

Fix

Fix listening on UDP sockets

Fix

Fix loading of file maps

Fix

Fix long regexp flags (e.g. {sa_body})

Fix

Fix lua compare function init

Fix

Fix maps descriptions

Fix

Fix maps locking

Fix

Fix max_train setup in ANN module

Fix

Fix memory corruption

Fix

Fix memory leak in unsigned maps reading

Fix

Fix misprints for lto usage

Fix

Fix more issues with scripts processing

Fix

Fix next-to-last extension length check

Fix

Fix openssl initialization

Fix

Fix order of arguments

Fix

Fix order of initialization

Fix

Fix parser

Fix

Fix parsing of binary tries

Fix

Fix parsing of braced IPv6 addresses

Fix

Fix parsing of nested braces in SMTP comments

Fix

Fix parsing of rarv5 archives

Fix

Fix periodic scheduling when a map is not modified

Fix

Fix possible FP in TRACKER_ID rule

Fix

Fix post-filters processing

Fix

Fix potential NULL dereference

Fix

Fix processing of <br> and <hr> tags

Fix

Fix processing of addresses in protocol

Fix

Fix processing of messages without received headers

Fix

Fix proxying issue for unconnected sessions

Fix

Fix proxying of the encrypted messages

Fix

Fix race condition with shared memory by refcounts

Fix

Fix ratelimit initialization

Fix

Fix redis set request in replies plugin

Fix

Fix redis timeout events handling

Fix

Fix redis timeouts processing logic

Fix

Fix refcounts in lua_redis

Fix

Fix results checking if no master connection is active

Fix

Fix return value for couple of lua functions

Fix

Fix round-robin selection when upstreams have no weight

Fix

Fix rows calculation in graph

Fix

Fix rspamd_redis_make_request syntax in replies plugin

Fix

Fix scheduling of locked map events

Fix

Fix scores detection

Fix

Fix searching for newline positions

Fix

Fix secure_ip setting in controller

Fix

Fix sending data to graph command

Fix

Fix setting of score for parts differ

Fix

Fix setting of the lua top

Fix

Fix setting path for lua

Fix

Fix setting path for phishtank

Fix

Fix settings application

Fix

Fix shm_open call as described in POSIX

Fix

Fix size of length in fuzzy mirror wire protocol

Fix

Fix smtp grammar issues

Fix

Fix some issues with redis API

Fix

Fix some issues with retries in the proxy

Fix

Fix stack growing

Fix

Fix start of body detection in DKIM

Fix

Fix state on timeout

Fix

Fix stats script

Fix

Fix substring search when there are '\0' in strings

Fix

Fix symbol name for spf soft fail

Fix

Fix symbol type's check

Fix

Fix symbols registration and execution

Fix

Fix the case of multiple values keys

Fix

Fix the default symbol names according to metric

Fix

Fix timeout setup on learning

Fix

Fix timeouts in redis cache processing

Fix

Fix timeouts processing in lua_redis

Fix

Fix upstreams interaction for rspamd proxy

Fix

Fix usage of rdns reply structure

Fix

Fix varargs loop

Fix

Fix whitelists and blacklists in SA rules

Fix

Fix write servers setup for redis

Fix

Fix writing of HTTP messages

Fix

Force rspamd to upgrade fuzzy db on opening

Fix

Free the correct pointer

Fix

Further fixes for lto and static linking

Fix

Further fixes for surbl extensions map

Fix

Further fixes in maps code

Fix

Further improvements to error messages in fuzzy check

Fix

Further tweaks to redis garbadge collection

Fix

Groups are now case insensitive

Fix

Handle log pipe read errors

Fix

Handle nested dependencies in SpamAssassin plugin

Fix

Implement new automata to skip empty lines for dkim signing

Fix

Improve error messages on fuzzy add

Fix

Improve lua redis handling

Fix

Improve phishing module logging

Fix

Improve printing of fuzzy errors

Fix

Improve rrd diagnostic errors

Fix

Improve strcase hash used in uthash

Fix

Include fuzzy key to distinguish storages with different keys

Fix

Include slave cluster name into http request

Fix

Include some more information about archives

Fix

Indicate upstream error on timeout

Fix

Initialize hash tables array to avoid crashes

Fix

Initialize parser scripts properly

Fix

Initialize vars to avoid warnings

Fix

Inverse logic for saving ANN

Fix

Link lpeg to rspamd lua library

Fix

Make extension checks case-insensitive

Fix

Mark expired hashes as not found and not as zero flag

Fix

Match archive name as well

Fix

More and more fixes to redis states

Fix

More fixes about shared memory in proxy

Fix

More fixes for redis refcounts

Fix

More fixes to end of headers detection

Fix

More fixes to events logic

Fix

More fixes to multi-flag fuzzy storage

Fix

More fixes to parts distance calculations

Fix

More guards for redis free

Fix

One more fix in redis destructor

Fix

One more try to fix redis

Fix

PIE is required for static build

Fix

Partial fix for mmap'd statistics tests

Fix

Plug memory leak in proxy

Fix

Properly detect end of headers position

Fix

Properly init and free session structures

Fix

Reduce PRECEDENCE_BULK rule weight

Fix

Reduce the default thresholds for learning

Fix

Remove Type=forking from systemd unit file (#709)

Fix

Remove bad FANN file to save computational resources

Fix

Remove event before closing of fd to avoid race conditions

Fix

Remove parsing of 'from' variable in redis backend

Fix

Remove some bad domains from whitelists

Fix

Repair optional dependencies

Fix

Reset master connection when retransmitting scan request

Fix

Restore ONCE_RECEIVED symbol

Fix

Restore compatibility with old lua API behaviour

Fix

Restore redis runtime state

Fix

Reverse options when received

Fix

Send updates to mirrors only if we have some changes

Fix

Set host attribute properly when making HTTP request from lua

Fix

Set terminated state before calling of async free

Fix

Simplify MISSING_MIMEOLE rule

Fix

Simplify state machine by ignoring multiple spaces

Fix

Skip setting RPATH for static builds

Fix

Slightly reduce weights of rules with high FP rate

Fix

Some fixes to libmagic initialization

Fix

Some more fixes to ratelimit plugin

Fix

Strip '\r\n' properly

Fix

Switch hashes to mumhash

Fix

Treat NaN values properly in graph command

Fix

Try to avoid FP when checking for phished URLs

Fix

Try to avoid recursive events deletions

Fix

Try to fix false positive URL detections in text parts

Fix

Try to fix issue in redis stats backend when task is closed

Fix

Try to fix proxying of stupid spamc protocol to HTTP mirrors

Fix

Try to fix redis crashes

Fix

Try to fix upstreams with one element

Fix

Try to handle multiline history in a more sane way

Fix

Unbreak build on gcc < 4.9

Fix

Update RPM spec/sources (#700)

Fix

Update d3evolution version

Fix

Update mumhash implementation

Fix

Use custom error function for pre and post filters

Fix

Use new postfilters and prefilters API in the plugins

Fix

Use non-blocking mode for systemd sockets

Fix

Use shared memory merely for local backends in the proxy

Fix

Use watchers for spf plugin

Fix

Varioud fixes to the maps code

Release 1.2.8

Release 1.2.7

Release 1.2.6

Release 1.2.5

Release 1.2.4

Release 1.2.3

Release 1.2.2

Release 1.2.1

Release 1.2.0

Release 1.1.4

Release 1.1.3

Release 1.1.2

Release 1.1.1

Release 1.1.0

Release 1.0.11

Release 1.0.10

Release 1.0.9

Release 1.0.8

Release 1.0.7

Release 1.0.6

Release 1.0.5

Release 1.0.4

Release 1.0.3

Release 1.0.2

Release 1.0.1

Release 1.0.0

Release 0.9.10

Release 0.9.9

Release 0.9.8

Release 0.9.7

Release 0.9.6

Release 0.9.5

Release 0.9.4

Release 0.9.3

Release 0.9.2

Release 0.9.1

Release 0.9.0

Release 0.8.3

Release 0.8.2

Release 0.8.1

Release 0.8.0

Release 0.7.6

Release 0.7.5

Release 0.7.4

Release 0.7.3

Release 0.7.2

Release 0.7.1

Release 0.7.0

Release 0.6.8

Release 0.6.7

Release 0.6.6

Release 0.6.5

Release 0.6.4

Release 0.6.3

Release 0.6.2

Release 0.6.1

Release 0.6.0

Release 0.5.6

Release 0.5.5

Release 0.5.4

Release 0.5.3

Release 0.5.2

Release 0.5.1

Release 0.5.0

Release 0.3.2

Release 0.3.1