Crowdsec the next fail2ban generation

crowdsec will be the next generation of fail2ban, I am still watching it, maybe one day :wink:

But https://www.lesechos.fr/pme-regions/innovateurs/crowdsec-veut-devenir-le-waze-de-la-cybersecurite-1255803

It is a startup, wait and see what it could become
They burn money sometimes

2 Likes

https://crowdsec.net/

1 Like

May i suggest a topic for chatting about goods and bads of crowdsec approach?

4 Likes

The main feature is to detect and ban IP on servers and to store IP in a centralized database, an attaquer will be banned on all servers and not only in the server that he has been banned. I suppose the startup wants to sell the access to the database.

The access to the database I am not sure it worths it because the IP are used and abandoned quickly but when you have an architecture of NethServer in a datacenter for example, it makes sense that an IP could be banned on all your server. For that retrieving IP locally and use it on all your crowdsec instances the idea is pleasant.

2 Likes

It’s power
 is also it’s weakness.
This “db” crowd-created is “more rule” and so “more CPU power” used for retrieving, processing, checking and “let the buggers drop” when they try to connect.

On the other side, missing login/knocks on the server could save CPU power, but it’s a lot of balancing.

This might be useful for avoiding emerging botnets, well known buccaneers, but
 buccaneers may be already available/stored into threatshield. Which is, in my humble opinion, the nice complement for Fail2ban.

Being dependent on Crowdsec means, for avoiding useless duplications, traffic and cpu-power consumption, complete removal of ThreadShield and Fail2ban.

This startup created a really interesting and well-saleable business plan.
Free product for gaining “antennas” for gathering data. AI development about analysis for create the “pro set”, which might be refined, smaller and weighted for have the biggest defense/load balance.
Costs? Few developers for creating the product (payed list), data centers for distributing the payed version and keeping syncing between “free antennas” for delivering the free-not-refined version, allowing lower data consumption on their infrastructure.

At least, by my perception. Maybe it will be a real gamechanger, but until now is a “nicer repack” of something known, with people trying to create a business about that.

2 Likes

Like 99.9999999999999 percent of startup, they try to sell something sometime not helpfull, they hopefully know a time where free money can boost there projects.

Put .com to your name and you will see a lot of gogo ask you please take my money

The idea of having some built in common list of known bad actors in one package is very appealing. For example I have had some dedicated love from a VPS spam my end point and moved down their IP block 1 at a time. Some ability to analyze this behavior and correct for it/inform others automatically seems nice but I also acknowledge the potential for false positives and abuse.

The business model they have in mind is explained in the FAQ (and in some other sites where they have spread the word, like on dev.to)

Don't know the current state of things but there were some attempts with fail2ban

GitHub - stintel/vallumd: Centralize or distribute IPset blacklists
GitHub - buanzo/fail2ban-zmq-tools: A zeromq-based fail2ban clustering solution
Centralize Fail2ban Database to share IP addresses with other systems
Sharing of fail2ban banned IPs - Server Fault
How to Configure Distributed Fail2Ban: Actionable Threat Feed Intelligence - Black Hills Information Security

Not the same but AbuseIPDB can feed from fail2ban from the “crowd”.
Don’t know if someone here has used/evaluated OSSEC-HIDS (some years back it was considered a Swiss army knife). Anyway
 going a bit off-topic.

Looks like crowdsec could provide something with more features and easier to implement without being full SIEM solution.

2 Likes

Yes the project is interesting, I still need to check again however the installation was not so easy (obviously at the time I checked). Instead to install the database on each host you split the installation. One database hosted on a server, then on each host you install the equivalent log reader and jail.

Fail2ban is simple, easy to understand for a sysadmin, do a regex, set a log to watch, and it is over.

2 Likes

Also, Fail2ban can be contextual.
The data processed is
 .coherent with the offenders that are trying to break into the box. Threatshield might (or might not) be a little more proactive, if you’re not into the first, second and third group attack, but the size of the data to be checked is far bigger.

Downside of this approach: users if not forced, will still use
 fakewords, instead of passwords.

1 Like

CrowdSec Community Webinar | Take the community to the next level
Tuesday, December 14 2021 at 11:00 am (CET)

https://app.livestorm.co/crowdsec/crowdsec-community-webinar-1?type=detailed

I registered, I am interested :smiley:

4 Likes

Honestly I would say that a lot of the reason why you see fail2ban as easy is the power of habit :slight_smile:

When you install CrowdSec agent and bouncer common services are detected and configured automatically. If you need to install new scenarios it’s done easily with cscli - so is updating them. Do you need to modify scenarios? No problem, everything it YAML and GROK. No crazy regexp here :slight_smile: Also docs are superb.

Disclaimer: I am head of community at CrowdSec and this was my first post. Thanks for the interest :slight_smile:

6 Likes

You’re somewhat right. All elements of CrowdSec communicate via http rest api. This means that it can be fully distributed and centralised. Bouncers can be on dumb endpoints, log parsing can happen where the log is currently placed - and the LAPI can be completely centralised so you only need one in your enterprise environment.
Also, thinking that CrowdSec is ‘just a f2b replacement’ is wrong. The fail2ban use case is easy to understand so it’s a story we often tell. But CrowdSec is so much more. It’s intelligence, so to speak, is much more versatile than f2b as it can detect all sorts of resource abuse like L7 DDoS, scalping, credit card stuffing and data exfiltration. Also traffic can be blocked on L3 via host firewalls (iptables/nfttables or pf) or L7 firectly in applications like nginx, Wordpress, in any PHP app, in node.js, Magento CMS, in Cloudflare, in Caddy, in Traefik and much more.
Literally the sky is the limit - is very much about getting the right idea for a use case.

1 Like

@klausagnoletti

Hi

And welcome to the NethServer community!

I do like the idea of the “Unban me” button at the bottom of your page!

The idea is good.

It all makes or breaks with:

A) crappy lists
B) bad availability of lists

A lot of issues with threatshield are with badly available / maintained lists.

My 2 cents
Andy

1 Like

Hi all,

Taking a look at CrowdSec: Installation & Example Scenario: Taking a look at CrowdSec: Installation & Example Scenario - YouTube.

Michel-André

2 Likes

Hi and thanks :slight_smile:

You’re right. Fortunately we’ve thought a lot about this.

We have two datlakes: smoke and fire. All intel is sent into the smoke datalake where the consensus system assess it based on a trust ranking system. Very shortly put all agents who reports data to us gets a trust ranking from 1 to 99 based on for how long they’ve reported intel and how trustworthy and stable they’ve been. In order for an ip to be deemed ‘bad’ it needs a certain amount of votes and points based on that trust ranking. That means that it’s really expensive and hard to poison the fire datalake. And on top of that there’s ips that can’t ever be banned such as CDN networkds, SEO bots etc. Finally all ips live in the fire datalake for 72 hrs. After that they will have to be resubmitted (=bad actor needs to still be active) to remain there.

All in all I think it’s a pretty good design. Of course there’s details I have omitted in this short write-up but these are the basics.

What do you think?

2 Likes

Excuse me, what? So, once on the fire datalake, all it takes for bad actor to be withdrawn from fire DL is to become silent ( ceasse all activity ) for 72 hrs? After 72 hrs they can become " vocal " again?

Seriously flawed


You must be kidding !!

Why? What are these details?

Why? If its wrong, than what we should think CS tries to be?

But, we already have well established, proven, working mechanism for detection and prevention of DDoS and other attacks. Its called CloudFlare; we dont need yet another clone.

For every serious security admin, thats huge NO-NO

GROK = REGEX


Public-facing one or within cooperate network? If public facing than - unless its fully secured - its huge design flaw; if that server is accessed internally than ok.

True; just to add :: f2b is well-established.

Thats why you try to defend CS.

Now, regarding project’s README.md::

CrowdSec is not a SIEM, storing your logs (neither locally nor remotely). Your data are analyzed locally and forgotten. Signals sent to the curation platform are limited to the very strict minimum: IP, Scenario, Timestamp. They are only used to allow the system to spot new rogue IPs, rule out false positives or poisoning attempts.

So: not storing logs / data, but, at, the same time, sending data somewhere? Some thoughts:

  • where is this " curation platform " hosted ?,
  • if whole project is OSS, than we. should be able to have ( at least read ) access to this " curation platform ", dont you think?
  • user’s IP is PII so storing it without user consent is illegal ( GDPR ),
  • any guarantee that " curated platform " is leak-proof?
  • if you say that other installations of CS rely on this " curated platform " than how do you know what other people do with data downloaded from this platform?
  • whats the data retention period on this platform?
  • if CS is not SIEM, than what it is? From what one can read, as well as how you present CS, its clear that CS is ( or at least tries to be ) fully fledged SIEM software
1 Like

Netherver-fail2ban does it, we enable jails when we found the relevant logs that is a pretty feature.
Obviously you can enable a jail manually also

you mean that nethserver-fail2ban enable jails with default config?

We customize software to be enabled and started with securized configurations, this is the dna of NethServer

2 Likes