Suricata NS7 template and errors

Hi,

I was just looking into the configuration file (suricata.yaml) and the log files of suricata on Nethserver 7…

I don’t know much about suricata, so please apologize my questions if they are nonsense… :wink:

First, I find only one file in the e-smith template of suricata containing the whole configuration file (isn’t that very strange for the template system of Nethserver?). And variables are very rarely used in this file…
in particular, for example the port numbers are hard coded, e.g. the ssh port is set to port 22 regardless whether you have changed it…!
Furthermore, I wonder if the value of http ports is set correctly (currently it is only “80” but I guess it should be “[80,443]” - however this is only a suggestion… perhaps it is right as it is!?). I haven’t checked the whole file for further problems yet.

Furthermore, I am getting lots of errors (after each restart) in the /var/log/suricata/suricata.log which are very similar or even identical to @flatspin’s errors described in the post: No more IPS report panel in V7?.
for example, 2/6/2017 -- 14:06:02 - <Error> - [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - "http_header" keyword seen with a sticky buffer still set. Reset sticky buffer with pkt_data before using the modifier. 2/6/2017 -- 14:06:02 - <Error> - [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "drop tcp $EXTERNAL_NET $HTTP_PORTS -> $HOME_NET any (msg:"MALWARE-CNC Win.Trojan.Sodebral HTTP Response attempt"; flow:to_client,established; file_data; dsize:<194; content:"BRASIL"; depth:6; content:!"Content-Length"; http_header; content:"Transfer-Encoding: chunked"; http_header; metadata:impact_flag red, policy balanced-ips drop, policy security-ips drop, ruleset community, service http; reference:url,www.virustotal.com/en/file/e0290c3900445dc00ca24888924e37fa6ac17ecaddc60591e32b81536b9f5ef7/analysis/; classtype:trojan-activity; sid:32608; rev:1;)" from file /etc/suricata/rules/suricata.rules at line 25409 2/6/2017 -- 14:06:02 - <Error> - [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - Signature combines packet specific matches (like dsize, flags, ttl) with stream / state matching by matching on app layer proto (like using http_* keywords). 2/6/2017 -- 14:06:02 - <Error> - [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "drop tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"MALWARE-CNC Win.Agent.BHHK variant outbound connection"; flow:to_server,established; dsize:136; urilen:1; content:"GET / HTTP/1.1|0D 0A|User-Agent: Mozilla/4.0 (compatible|3B| MSIE 7.0|3B| Windows NT 6.0)|0D 0A|Host: windowsupdate.microsoft.com|0D 0A|Connection: Close|0D 0A 0D 0A|"; fast_pattern:only; content:!"Accept"; http_header; metadata:impact_flag red, policy balanced-ips drop, policy security-ips drop, ruleset community, service http; reference:url,www.virustotal.com/en/file/cab1fffe7a34b5bb7dab2cacd406cf15628d835ab63502d28df78c2faeaad366/analysis/1421677054/; classtype:trojan-activity; sid:33227; rev:2;)" from file /etc/suricata/rules/suricata.rules at line 25500 2/6/2017 -- 14:06:02 - <Error> - [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - Signature combines packet specific matches (like dsize, flags, ttl) with stream / state matching by matching on app layer proto (like using http_* keywords). 2/6/2017 -- 14:06:02 - <Error> - [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "drop tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"MALWARE-CNC Win.Trojan.FileEncoder IP geolocation checkin attempt"; flow:to_server,established; dsize:214; urilen:1; content:"GET / HTTP/1.1|0D 0A|User-Agent: Mozilla/4.0 (compatible|3B| MSIE 6.0|3B| Windows NT 5.1|3B| SV1|3B| .NET4.0C|3B| .NET4.0E|3B| .NET CLR 2.0.50727|3B| .NET CLR 3.0.4506.2152|3B| .NET CLR 3.5.30729)|0D 0A|Host: ip-addr.es|0D 0A|Cache-Control: no-cache|0D 0A 0D 0A|"; fast_pattern:only; metadata:impact_flag red, policy balanced-ips drop, policy security-ips drop, ruleset community, service http; reference:url,www.virustotal.com/en/file/17edf82c40df6c7268191def7cbff6e60e78d7388018408800d42581567f78cf/analysis/; classtype:trojan-activity; sid:33449; rev:1;)" from file /etc/suricata/rules/suricata.rules at line 25507 2/6/2017 -- 14:06:02 - <Error> - [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - previous keyword has a fast_pattern:only; set. Can't have relative keywords around a fast_pattern only content 2/6/2017 -- 14:06:02 - <Error> - [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "drop tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"MALWARE-CNC Win.Trojan.GateKeylogger initial exfiltration attempt"; flow:to_server,established; content:"/gate.php"; fast_pattern:only; content:"pc="; nocase; http_client_body; content:"&admin="; distance:0; nocase; http_client_body; content:"&os="; distance:0; nocase; http_client_body; content:"&hid="; distance:0; nocase; http_client_body; content:"&arc="; distance:0; nocase; http_client_body; content:"User-Agent|3A 20|"; http_header; pcre:"/User-Agent\x3a\x20[A-F0-9]{32}\x0d\x0a/H"; metadata:impact_flag red, policy balanced-ips drop, policy security-ips drop, ruleset community, service http; reference:url,www.virustotal.com/en/file/77c802db1731fa8dae1b03d978f89b046309adfa1237b1497a69ccb9c2d82c16/analysis/1459520578/; classtype:trojan-activity; sid:38562; rev:2;)" from file /etc/suricata/rules/suricata.rules at line 25723 2/6/2017 -- 14:06:02 - <Error> - [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - "http_stat_code" keyword seen with a sticky buffer still set. Reset sticky buffer with pkt_data before using the modifier. 2/6/2017 -- 14:06:02 - <Error> - [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "drop tcp $EXTERNAL_NET $HTTP_PORTS -> $HOME_NET any (msg:"MALWARE-CNC Win.Trojan.GateKeylogger fake 404 response"; flow:to_client,established; file_data; content:"200"; http_stat_code; content:"OK"; http_stat_msg; content:">404 Not Found<"; fast_pattern:only; content:" requested URL / was not found "; metadata:impact_flag red, policy balanced-ips drop, policy security-ips drop, ruleset community, service http; reference:url,www.virustotal.com/en/file/77c802db1731fa8dae1b03d978f89b046309adfa1237b1497a69ccb9c2d82c16/analysis/1459520578/; classtype:trojan-activity; sid:38563; rev:1;)" from file /etc/suricata/rules/suricata.rules at line 25724
sorry if this is a bit too much code…

Despite these errors, suricata seems to work fine however…

How could these errors be resolved? I can of course send you the whole log or make a gist if you wish…

and how to improve the template?? @dev_team

1 Like

Seen this, still investigating. It’s harmless, I’ll try to fix it, low priority.
For a far more advanced suricata setup see
https://wiki.nethserver.org/doku.php?id=userguide:web_interface_for_suricata&s[]=suricata

2 Likes

thanks for pointing me to the howto in the wiki…
I tried it today. unfortunately, I encountered several problems but solved most of them by myself (I edited the wiki a bit for more clarity).

However, I don’t know how to reach the webgui of scirius… I tried http://myip:8000 (of course with the IP of the nethserver installation) but it didn’t work… any suggestions?
Furthermore, I get an error message in the dashboard tab of kibana that it could not locate that index-pattern-field (id: event_type.raw). Perhaps there is simply not enough data on the server?!?

btw, the wiki article should perhaps be updated since it uses elasticsearch 2.2 - and the current version is 5.4, same for kibana and logstash… (I used however the old versions as in the wiki).

Honestly, I have no ideas.
I agree that the howto is probably outdated, ELK is evolving rapidly.
Do you see any error?

yesterday, I didn’t get any error…
however I have just installed django which is probably needed for scirius, and now I get errors when trying python manage.py runserver. yesterday this command did nothing at all (besides that I had to press ctrl+c to exist it - but neither a login command nor any error message were displayed). what would be the right output from it?
After installing django, I first got an error stating that no module named south could be found… I googled it and found that I have to downgrade django to version 1.6. what I did… but now I get the following errors:scirius]# python manage.py runserver Validating models... Unhandled exception in thread started by <function wrapper at 0x1d17488> Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/django/utils/autoreload.py", line 93, in wrapper fn(*args, **kwargs) File "/usr/lib/python2.7/site-packages/django/core/management/commands/runserver.py", line 102, in inner_run self.validate(display_num_errors=True) File "/usr/lib/python2.7/site-packages/django/core/management/base.py", line 310, in validate num_errors = get_validation_errors(s, app) File "/usr/lib/python2.7/site-packages/django/core/management/validation.py", line 34, in get_validation_errors for (app_name, error) in get_app_errors().items(): File "/usr/lib/python2.7/site-packages/django/db/models/loading.py", line 196, in get_app_errors self._populate() File "/usr/lib/python2.7/site-packages/django/db/models/loading.py", line 78, in _populate self.load_app(app_name) File "/usr/lib/python2.7/site-packages/django/db/models/loading.py", line 99, in load_app models = import_module('%s.models' % app_name) File "/usr/lib/python2.7/site-packages/django/utils/importlib.py", line 40, in import_module __import__(name) File "/etc/scirius/rules/models.py", line 23, in <module> from django.contrib.contenttypes.fields import GenericForeignKey, GenericRelation ImportError: No module named fields

I installed fields via pip but it didn’t changed anything…

do you have any idea?

Update: I have just again run the command pip install -r requirements.txt which upgraded the version of django to 1.8.18… and now the errors disappeared!
the output of python manage.py runserver is now:

python manage.py runserver
Performing system checks...
System check identified no issues (0 silenced).
June 06, 2017 - 21:14:20
Django version 1.8.18, using settings 'scirius.settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.

however, I still cannot reach the gui… I checked the firewall logs but nothing to see.

Furthermore, I cannot use suri-reload:

scirius]# suri_reloader -p /path/to/rules  -l /var/log/suri-reload.log  -D
-bash: suri_reloader: command not found

any ideas?

nothing changed…! Does anyone have any idea?

Ive been trying that as well, I installed the ELK stack and webui for suricata but if you only want 1 sensor, such as home use or small business try out aanval. :8000 doesnt work but you can still get eve and kibana, though imo kibana isnt a very good front end as its more for data analysis on a larger scale, while we need simply something parsing suricata logs. You can do that in kibana but its not very intuitive. My suricata log simply is full of “invalid rule” entries, nothing shows up in DPI, and nothing is blocked. Its not pulling rules for some reason. Not something an average person would notice or need really but its a thing. Ive set up snort/pulledpork, all that from scratch before on debian so I feel like maybe pulledpork isnt working as intended? Its harder to know whats going on since the gui set it up and I did not personally. So my log looks a lot like yours, though I do have rules in /etc/suricata/rules

1 Like

Guys, I’m working on suricata in my spare time in these days. I have a plan:

  1. use evebox to view alerts (thanks @jasonish)
  2. remove snort community rules, as I don’t get any alert from them (only syntax errors)
  3. change the profiles in the UI to one or two presets keeping the Expert option for manual setup
  4. provide enough instructions to refine the setup looking at evebox

I’m collecting logs from my sensors and developing the presets (I’m thinking of something like alert only, alert on some rules and drop on others).

Installing evebox is a joke, it’s a really nice software.
Just give me some days to build nethserver-evebox.rpm to install it from the software center.

We still miss a user interface to enable/disable/drop single rules.
Do you think we could be fine with an interface that works on whole rules categories? I maybe able to reuse the web content filter interface.

3 Likes

Yes! though, evebox is really easy to install. Im going to do a clean ns7 install here sometime and get rid of everything but evebox, ive been tinkering and installing stuff to test and now I cant remember what to uninstall :stuck_out_tongue:

it would be to time consuming to make an interface to drop or add rules dont you think? If you know what suricata is and are thinking about rules you probably can do it yourself through ssh? Just a thought. I mean, a gui would be pretty :smiley: if you think it is doable, but the way suricata imports rules im not sure. Heard of aanval? https://www.aanval.com/ its free for a single sensor, Ive got that installed currently and am in the process of figuring out how to tie it into suricata.

Can I move your post into a new discussion? I’d like to give more visibility to proposals like that.
So that more people interested can chime in.

1 Like