We are excited to introduce Rspamd 3.7.1, the latest iteration of our advanced spam filtering system. Rspamd continues to evolve as a robust and efficient spam filtering solution. This release is packed with new features, enhancements, and crucial bug fixes designed to elevate Rspamd’s performance, versatility, and security to new heights. Let’s delve into the key changes in this version:

Critical fixes:

• Addressed a critical memory leak in the gzip function, ensuring more efficient resource management.

New Features:

• Added support for ICAP Content-Type and Filename, expanding Rspamd’s capabilities for content analysis.
• Introduced the logging->task_max_elts option, providing greater control over logging.
• Added a utility for splitting strings within C++ code, simplifying string manipulation.
• You can now set HTTP authentication parameters for maps, enhancing security.
• Improved configuration error checking during configtest, facilitating the identification and resolution of plugin configuration issues.
• Introduced the known_senders plugin, assisting in the management and recognition of known senders.
• Transitioned to using backward-cpp instead of manual libunwind handling, enhancing code quality and reliability.
• Enhanced RBL (Real-time Blackhole List) support to check numeric URLs individually, enhancing the accuracy of spam detection.

Fixes:

• Addressed various issues in CMakeLists.txt to prevent test failures related to whitespace.
• Corrected the location of the Date: header to conform with RFC standards.
• Ensured the correct format pattern for RE tree tempfile names.
• Fixed format string and length issues, contributing to code stability.
• Improved grammar definitions for content-disposition attributes.
• Enhanced the logic for Redis parameters in the Lua schema enrichment process.
• Resolved a Lua stack corruption issue when logging large tables.
• Fixed the utility for merging tables.
• Ensured the correct output of non-RSA DKIM keys.
• Addressed various corner cases related to parsing single-host URLs.
• Corrected several issues in the url_redirector plugin, improving its functionality.
• Prevented DNSWL (DNS-based Whitelist) sabotage, enhancing spam filtering.
• Resolved dependency registration issues in the RBL plugin when using symbols prefixes.
• Improved the behavior of the rspamadm mime command to avoid conflicts with arguments starting with the letter ‘t’.
• Prevented the matching of messages from the Android GMail app in the MISSING_MIMEOLE check.

Rework

• Reverted to semantic versioning (semver) to provide better versioning consistency.
• Transferred RCL (Rspamd Configuration Language) logic to C++, streamlining configuration processing.

Rspamd 3.7.1 represents another significant stride forward in our ongoing commitment to providing an efficient, reliable, and feature-rich spam filtering solution. We encourage you to upgrade to this latest version to take advantage of these improvements and ensure the continued security of your email communications.

We are excited to present the latest version of Rspamd - version 3.6. This release brings a host of new features, enhancements, and fixes to improve further the performance, flexibility, and security of the Rspamd email scanning system. With additions like language detection configuration, dynamic multimap symbol registration, and enhanced fuzzy storage options, Rspamd continues to evolve as a powerful and reliable solution for filtering and classifying messages.

New Features:

• Added one_shot option to specific multimap rules for improved rule behavior
• Introduced language detection configuration and associated attributes
• Added sentinel_password option to enhance Redis sentinel password protection
• Enabled denial of specific fuzzy flags by default for better control over fuzzy storage
• Implemented a controller endpoint to retrieve fuzzy hashes from messages
• Added extra symbol for URL redirector when reaching nested limit for easier identification
• Included a function to transliterate utf8 to ascii with normalization for text processing
• Added html parsing limit and set order to urls structure for improved handling
• Expanded functionality of lua_rsa library with additional functions
• Enabled fuzzy workers to exchange blocked information
• Allowed weak flags in fuzzy storage for more versatile fuzzy matching
• Enabled reading options from maps in the multimap plugin for dynamic configuration
• Provided alternative methods when fasttext detection is enabled
• Enabled counting stats per key per flag for better statistics tracking
• Completed implementation of dynamic composites for more flexible rule composition
• Improved processing of HTML parts before text ones for better text extraction
• Reorganized struct rspamd_url for reduced memory footprint
• Implemented saving fuzzy ratelimit buckets for rate control
• Added ip_map strategy to external_relay plugin for more versatile IP handling
• Implemented on_load support for maps to perform actions on map loading

Fixes:

• Addressed race condition between config new/free using a counter to ensure stability
• Enhanced fasttext language model with pre-tokenized words for improved detection
• Fixed issues with rspamd_has_only_html_part for accurate HTML detection
• Resolved order of destruction race between Redis pool and Lua for stable behavior
• Addressed parsing of invalid mask values for proper configuration handling
• Adjusted header parsing to include the last character when no value is present
• Addressed various issues with fuzzystat for accurate fuzzy storage behavior
• Corrected counter usage for more accurate counting
• Implemented measures to clean pending bucket and remove bad hyperscan files
• Updated stats before encryption to ensure accurate data representation
• Improved DMARC grammar by allowing spaces before ;
• Fixed registration issue in RBL plugin when using symbols_prefixes
• Removed obsolete files related to rspamd-redirector

Project Enhancements:

• Enabled dynamic registration of multimap symbols for flexible rule management
• Implemented fasttext language detection for efficient language classification
• Refactored default max shots to avoid interfering with options
• Rewrote dkim keygen tool in Lua for better performance and functionality
• Added thread hijacking composite rule for improved rule handling

Please note that this is not an exhaustive list of changes and other minor improvements, bug fixes and optimizations have also been included in this release.

We are excited to announce the release of Rspamd 3.5, packed with new features, improvements, and fixes. This version brings enhancements to configuration, critical fixes, and added functionalities to the Rspamd project. Here’s an overview of what you can expect in this release:

New Features:

• Added SURBL hashbl support
• Introduced the thresholds field to the scan result
• Added the ability to execute Lua scripts for blocked fuzzy clients
• Added preliminary support for external maps in the multimap plugin
• Enabled the building of maps by combining tuples of selectors
• Added query support for external maps for settings
• Introduced selector_alias in map definitions
• Enabled MIME part filters on the antivirus module
• Improved rate limit Redis scripts
• Added the specific_urls_filter_map extractor in Selectors
• Reworked the selectors framework

Critical Fixes:

• Deserialized Hyperscan to page-aligned space to prevent alignment issues
• Filled path field in Hyperscan notice command

Fixes:

• Multiple fixes related to Hyperscan, Redis configuration, Ratelimit, RBL, and URL reputation plugin
• Fixed off-by-one error in CSS tokenizer and issues with boundaries containing only dashes
• Restored strict_domains support and replaced broken strict_domains with phishing_exceptions
• Reworked list applications and added external maps support
• Improved handling of hostnames with no dots

Rework:

• Stopped reporting soft reject in history
• Converted the chartable plugin to C++ for convenience
• Changed the approach for customization of settings

Rules:

• Added the MID_END_EQ_FROM_USER_PART rule to the Mid section

Upgrade notes

In addition to the numerous improvements in Rspamd 3.5, this release introduces some notable changes to the supported platforms. We are excited to announce the provision of arm64 packages, extending Rspamd’s compatibility to a wider range of devices. However, as part of our commitment to providing up-to-date and secure software, we have removed support for outdated and end-of-life (EOL) Debian distributions, specifically Ubuntu Bionic and Debian Buster. This decision ensures that our users are running Rspamd on well-maintained platforms with active security updates. For more context on this change and guidance on upgrading your distribution, please refer to the following [document]((/packages_support_policy.html)

It is essential to carefully review the upgrading implications to ensure a smooth transition to Rspamd 3.5. These changes allow us to focus on delivering the best possible email filtering solution while promoting the use of secure and up-to-date platforms.

We have released Rspamd 3.4 today. This is a bugfix release with no incompatible changes. Several new features have also been added. Here are the most important changes in this version explained.

Main changes

Sharing hyperscan database among Rspamd processes

Hyperscan databases are now shared between all Rspamd processes reducing memory footprint, especially when multiple worker processes are running.

Critical fix in the compatibility with the integrations and headers alterations

There was a critical compatibility issue, caused by the change in the milter_headers reply block that prevents some Rspamd integrations to be functional. In this release that issue has been fixed, and the compatibility with the previous output format has been restored.

Fix additional fields in the Redis schema

Some fields were no longer accepted in Redis settingsissue. Now it works correctly.

All significant changes

Here is the list of the important changes:

• [Feature] Milter_headers: Add x-rspamd-action routine
• [Feature] Share hyperscan database among processes
• [Fix] Another corner case in url parsing
• [Fix] Another fix for the enable password
• [Fix] Another try to fix close method in lua_tcp
• [Fix] Fix emoji joiner FP
• [Fix] Fix favicon.ico Content-Type header
• [Fix] Fix hang when close is used
• [Fix] Lua_tcp: Sigh, another try to fix close invocation
• [Fix] Mx_check: Cache the fact of a missing MX record
• [Fix] Try to fix parsing of the unencoded > characters in html attributes
• [Fix] Try to fix the case where password == enable_password
• [Project] (Re)implement hyperscan caching
• [Project] Rework cleanup
• [Project] Synchronize hyperscan caches via the main process
• [Rework] Convert multipattern to use hyperscan tools
• [Rework] Make http normalize path function a generic function
• [Rework] Split locked and unlocked files, as mmap does not need flock normally
• [Rework] Start movement of the hyperscan related routines into a single unit
• [Rework] Store the current worker, so other libraries could use this information
• [Rework] Use blocking socket for IPC between main and workers
• [Rework] Use more predictable size for commands buffers
• [Rules] Do not insert ONCE_RECEIVED_STRICT on RDNS missing
• [Rules] Reduce score of HTTP_TO_HTTPS - subject to remove completely

We have released Rspamd 3.3 today. There are incompatible changes in this release, so please get familiar with the upgrade guide.

Here are the most important changes in this version explained.

Main changes

Reworked and redesigned symbols cache

Symbols cache is responsible for rules exectuion and planning. In this release, there was a major rework of it’s logic and functionality. For example, it can now keep track of timeouts, plan fast events before slow and implement real passthrough for the rules that define such a behaviour. It is useful, when you want some rule to be executed as quick as possible to block or pass evident spam/ham without wasting network/cpu resources. The major drawback of such a rework is that passthrough rules are now really passthrough and can prevent other rules from being executed (that is expected from the design, but it could be not the case before).

Critical fix in the neural network module

There was a regression introduced in the version 3.2 that prevented old keys in Redis to be cleaned that caused infinite Redis database growth. This is fixed in the release 3.3 and the mitigation of this bug are described in the upgrade guide.

DKIM parser now ignores unknown tags

By standard, DKIM checker must ignore unknown tags for forward compatibility. Rspamd will now behave properly and ignore unknown tags as specified in RFC.

Upstreams support in lua_http and lua_tcp modules

It is possible now to use the functionality of the upstreams directly in Lua modules that use lua_http and lua_tcp libraries. It allows better support of the names resolution, IPv6 support for resolving the hostnames and internal handling of the upstreams logic by C code automatically.

CNAME records support in the DNS resolver

Rspamd DNS resolver now supports querying and parsing of the CNAME records. This technique might be useful for fighting some specific spam patterns.

Various memory leaks detected and plugged

In this release, we have found and fixed a good bunch of memory leaks and memory corruptions in the code.

All significant changes

Here is the list of the important changes:

• [Conf] Add missing groups for whitelist module symbols
• [CritFix] Neural: Fix keys regression after #3968
• [Feature] Accept upstream in lua_tcp
• [Feature] Add ability to statically maintain disabled/enabled patterns
• [Feature] Add function to store upstreams for HTTP urls
• [Feature] Allow augmentations set in Lua API
• [Feature] Allow lua_http module to accept upstreams
• [Feature] Allow to limit write access to fuzzy storage by key
• [Feature] Allow to sort symbols output
• [Feature] Check content for binary stuff before dumping it to Lua
• [Feature] Implement symbols augmentations
• [Fix] Add missing flags
• [Fix] Add more sanity checks for rua in dmarc_report
• [Fix] Adjust length of the fuzzy checks for short text parts
• [Fix] Another try to fix add headers compatibility logic
• [Fix] Another try to fix race condition in the runtime destruction
• [Fix] Avoid cyclic references in symcache and fix memory leaks
• [Fix] Avoid overriding IP with Sender IP
• [Fix] BAD_REP_POLICIES did not trigger when message was classified as spam by Bayes
• [Fix] Bind AF_UNIX DGRAM client connection to annonymous address
• [Fix] Disable IPv6 lookups for Blocklist.de RBL
• [Fix] Distinguish dynamic and static items
• [Fix] Dkim: Ignore unknown DKIM kv pairs as stated in RFC
• [Fix] Dmarc report: Use local timezone instead of GMT
• [Fix] Do not exclude authenticated users from URIBL lookups
• [Fix] Empty envelopes should not be emitted as arrays (json+messagepack) when populated envelopes are objects. This greatly complicates decoding in strictly typed languages.
• [Fix] External_relay: Restore the originating hostname check
• [Fix] Fix DKIM keys with spaces still allowing errors on invalid base64
• [Fix] Fix copying of sockaddr_un addresses
• [Fix] Fix crash with cname replies
• [Fix] Fix dependencies propagation
• [Fix] Fix iteration over milter headers
• [Fix] Fix ordering when sorting symcache
• [Fix] Fix reading of the cached maps
• [Fix] Fix several issues with the HTTP keepalive parsing
• [Fix] Fix stack smashing
• [Fix] Fix synchronous auth/select in lua_redis
• [Fix] Fix various symcache issues
• [Fix] Ignore all (I hope) unknown DKIM signature KV pairs
• [Fix] Ignore directories in RarV5 archives
• [Fix] Libucl: avoid memory leak on objects merging
• [Fix] Lua_tcp: Another try to fix closing logic
• [Fix] Mempool: Fix alloc_array function to actually multiply nmembers by size
• [Fix] Only check allowed fuzzy worker update ips for non-unix sockets
• [Fix] Plug memory leak in regexp destruction with pcre2
• [Fix] Properly check the original email flag
• [Fix] Properly deal with get_symbol/get_metric_symbol ambiguity
• [Fix] Properly parse expressions atoms
• [Fix] Properly set Host in rspamd_proxy
• [Fix] Rbl: Fix received positioned checks
• [Fix] Remove check for a score with no symbol being registered
• [Fix] Same fix for lua_tcp
• [Fix] Skip cname records when processing SPF records
• [Fix] Skip sending dmarc reports in no-opt mode fixes https://github.com/rspamd/rspamd/issues/4241
• [Fix] Stop slow timer on task destruction
• [Fix] Symcache: Do not use C style comparators in C++ sorts
• [Fix] Try to avoid a corner case for @ pattern
• [Fix] Try to fix dkim reputation adjustements
• [Fix] Try to fix passthrough results processing logic
• [Fix] Try to fix the mess with read only flag
• [Fix] Upstreams: Don’t ignore revive_time config option
• [Fix] Use proper format string, sigh…
• [Fix] Use space category in ragel automata to resolve space characters
• [Fix] Zstd: Fix compression with the new Zstd API
• [Fix] milter_headers: Header fields may be inserted at wrong position.
• [Project] Rework symbols cache
• [Rework] Rewrite rspamc in C++