DPI and Proxy not working together

squid
dpi
webproxy
ndpi
v7

(Harold Maponde Shamanoh) #1

NethServer 7.3 7.3
Module: DPI and Web Proxy

I am really struggling to block websites with nethserver. I have been a linux guru since 2009 with ebox which is now zentyal administration experience. The firewall implementation is so different and i can’t work out how it integrates with the proxy. In previous installations of zentyal being setup as a gateway and firewall with the proxy module enabled. All traffic went through the firewall before being passed to squid proxy. So basically if i blocked all http traffic over the the firewall even though am an authenticated user, i never had access to the site. Which is a different issue with nethserver. ndpi and firewall seems to be dormant no matter how i set it up. its as good as not having a firewall on the system installed. But nonetheless, i really like the idea of dpi so my question is anyone know to set it up so that it works in conjunction with the proxy.
Thanks a lot,
Harold.


Firewall doubts
How can i Block whatsapp on mobile phones on DPI firewall rule
(Ralf Jeckel) #2

To setup a rule for dpi please go to firewall rules an create a new rule like this:

When you select service, you can search the protocol i.e. facebook.

A list of protocols is in “Status/DPI”


(Harold Maponde Shamanoh) #3

Hi Flatspin,
My point exactly. none of these rules work because i have tried to setup up firewall rules in that instance. I have used network objects and host groups as i don’t really want to block everyone from accessing sites like facebook but only blocj students. And it still goes through. How does this work if the proxy is on?


(Harold Maponde Shamanoh) #4

Please see this…


(Ralf Jeckel) #5

Please don’t get me wrong, but only a few questions to ensure things:

Do you have Proxy “Transparent with SSL” enabled?
Shorewall is running?
Do you have webfilter module instlled? If yes, do you have enabled filters?
How did you configure the Source (IP range student_computer)?


(Marc) #6

The manual has some notes on the limitations when using ndpi. Here some information about the problem:
https://community.nethserver.org/t/firewall-tests-on-ns7b2/4475/11?u=dnutan


(Harold Maponde Shamanoh) #7

Thank you for your prompt feedback,
Here are my settings:


(Harold Maponde Shamanoh) #8


(Ralf Jeckel) #9

You choose “Authenticated”. AFAIK https-filtering only works with “Transparent with SSL”.
@filippo_carletti can you please tell us how this is related to DPI? I don’t use “Authenticated”.
Unfortunately it is not mentioned here: http://docs.nethserver.org/en/latest/firewall.html?highlight=DPI

In devel manual I found:

Assumptions and limitations

1. All nDPI traffic is marked in forward chain. When a nDPI protocol is found, the whole connection is marked.
2. Priority rules are in post chain and can use nDPI markers. If a priority rule uses a role (interface) as source, 
    the rule can’t be added to postrouting chain since the packet is already natted: Shorewall will move the rule
    on top of forwarding chain.
3. nDPI rules can’t block the http/https traffic if web proxy is enabled in transparent mode.
4. All nDPI markers are read from /proc/net/xt_ndpi/proto and shifted by 8 bits.
5. Divert rules can’t be used with nDPI, because an established TCP connection can’t be moved between 
    providers.
6. Prerouting table is reserved by Shorewall for handlind the multi wan scenario.

(Harold Maponde Shamanoh) #10

So basically there is no need for me to have shorewall installed then… I can’t use it. Could we possibly rewrite how the firewall operates? so connections are not terminated at firewall level


(Harold Maponde Shamanoh) #11

sorry so connections are not terminated at webproxy level


(Filippo Carletti) #12

DPI works on connections traversing the firewall, ie going from a pc to the net.
When there is a proxy, there are two connections and none of them traverses the firewall:

  1. pc to nethserver
  2. nethserver to net

The rules are simple:

  • want to block a web site (http)? Use the proxy
  • want to block a protocol (torrent)? use the firewall

If you want to identify http traffic with DPI techniques you must “take it out” from the proxy. A common scenario is windows update, which is usually put on low priority. Configure as follows:

  1. add the list of windows update domains in Domains without proxy (use google to find the list)
  2. add a rule to set ndpi windowsupdate to low prio in firewall rules

This sentence should be slightly reworded to clarify the limitations.


(Harold Maponde Shamanoh) #13

This is really nice filippo thank you. But it can block traffic in authenticated mode?


(Harold Maponde Shamanoh) #14

Can in turn then use both firewall and proxy simultaneously?


(Michael Träumner) #15

Hi Ralf,
I’m using squid in manual mode with blocking https. It works. The only behavior is, that there is a certificate-error if squid/ufdbguard has to redirect https sites due filtering it.


(Eliezer Croitoru) #16

@Harold_Maponde_Shama DPI and proxy do not play together if they overlap with their rules.
For example facebook and google are mainly a web service and there for overlap the http proxy functions and rules.
Technically you can use them both together but the results would be very weird(I havn’t seen the exact iptables rules to confirm) and might lead to a situation which traffic is not being blocked at all or being blocked anyway.
The proxy works in a much higher layer of the connection and if you need to block sites or traffic which is related to HTTP or HTTPS do this with a proxy and not DPI unless you have a very specific demands such as blocking facebook or google or youtube only but let all other traffic pass by.
The main issue is that you cannot technically identify content using DPI until the connection was initiated with the origin server or the proxy while on the other side the proxy intercepts\interpret the connection before any traffic is identified by DPI.
So chicken and egg, what first? is this specific case.


(Harold Maponde Shamanoh) #17

Nice one… thank you


(Harold Maponde Shamanoh) #18

Is possible to block extension files and certain mime types? I still have students in my school getting away with downloading certain media content. I still feel the proxy or the firewall is not quite as solid. maybe there is something that i’m just missing.


(Eliezer Croitoru) #19

@Harold_Maponde_Shama You will need to make sure what are these downloads…
These can be HTTP or another protocol downloads.
The first thing I can recommend you to try and see if someone complains is to use the firewall in order to block any other traffic then to ports 53,80 and 443.
When these are there with a proxy which implements SSL-BUMP you will maybe get complains about something not working but you will have very fast answers.
Depends on the size of the school you might want to use DPI in such a way that will help you to find-out what is the main network traffic that passes the firewall.
I don’t know how exactly you would be able to set it all up but the general idea is to line of a set of DPI rules from top to bottom(since first hits…) and for example http and proxy traffic would be one of the last ones.
Maybe with DPI you will find out that most traffic is directed towards windows updates for example.