Back to the topic of Suricata performance and memory usage…
I have dug down a bit into memory usage and specifically Suricata memory usage. The RAM and especially swap usage seems to be heavily dependent on the size of the set of rules.
From what I understand, the full set of alert (~10k) and drop (~30k) rules are loaded and used both for updating and in the actual suricata engine, independent of whether the inline IPS is actually enabled or not.
Commands to get counts...
grep -Ec '^[[:space:]]*alert[[:space:]]' \
/var/lib/suricata/rules/suricata.rules
grep -Ec '^[[:space:]]*drop[[:space:]]' \
/var/lib/suricata/rules/suricata.rules
So, basically, from my understanding, for only IDS enabled both the updater and the engine is pushing the used RAM upward and leaving not much margin, and growing heavily into swap, for absolutely no reason at all. Edit: Or is the drop set of rules also actually used to trigger the downstream non-inline/reactive IPS blocks?
I have experimented and shaved off half of the rules (temporary addition to disable.conf that gets overwritten by toggling IDS/IPS off and on again, via ips-rule-policy.sh). Anyway, just experimenting to see the effect on memory pressure. Going from ~40k to ~20k rules lowered the swap usage substantially, or even made the swap being unused…though margins for free/available RAM is still very low.
So, 1) for the IDS-only case, in order for speeding up the updater as well as minimizing RAM and swap usage both for the updated and the engine, would it be possible to shave off the entire list of ~30k drop rules before updating and starting the engine? Work with only with alert rules there. Edit: Given that drop rules are redundant, otherwise disregard that.
Also, 2) as the updater runs for ~5 minutes (give or take some, I haven’t timed it), would it actually be possible to remove the updater to run from starting IDS/IPS via UI toggle? It feels like it is running too often. Remove it from the orchestration script (suricatad.sh), or at least gate it (better) and only allow update if is actually needed, as the updater runs on Cron schedule nightly anyway.
Then, for the IPS inline enabled mode: 3) is the alert based firewall rule blocking still active?, 4) would there be a compromise/tradeoff between level of RAM+swap usage and a decent enough set of rules? Let’s cleverly shave off rules to allow for some blocking to be handled by the downstream Alta reactive IPS and some really serious drop rules would be handled by the inline IPS. Let’s make the memory fit within RAM and avoid swap, while also getting a very decent IDS alert, inline IPS and reactive IPS.