Transparent HTTPS proxy

v7-rc3

(kai) #41

good morning, i activate the cachemgr.cgi but not sure what points are interesting to see if there is a problem.

my general ram using is after a day now that way

from cache manager it looks like


(kai) #42

after restart the squid manually i got

and it is without lags.


(Ralf Jeckel) #43

Hello Kai,

how many clients are on this machine? I’m asking, because I can’t get my memory-usage up on my machine with only 1 user. I’m still on the same as yesterday.


(Michael Träumner) #44

I have installed squid in manual mode with cache and my memory usage is high too.

At this moment there are 5 people using squid and two people using sogo calendar.
But for all that I don’t have any problems.


(And) #45

over 200 users
18 day without restart server

Transparent SSL mode :slight_smile:


(kai) #46

at my Server are 12 User. hmm, so i guess everything is normal? Cause i have the idea the response of some websites after a day are very slow.


Transparent proxy with ssl (NS7) solved
(Giacomo Sanchietti) #47

I just started some tests, for now I can’t reproduce memory usage problems but I will keep you informed.

The main issue I found it’s that some resources are inaccessible, mostly images or scripts from CDN.
You can find these kind of lines inside /var/log/squid/cache.log:

2016/12/09 09:44:18 kid1| SECURITY ALERT: on URL: avatars0.githubusercontent.com:443
2016/12/09 09:44:18 kid1| SECURITY ALERT: Host header forgery detected on local=151.101.60.133:443 remote=192.168.5.22:40950 FD 166 flags=33 (local IP does not match any domain IP)

In this case, when accessing github, the avatars won’t be displayed by the browser, and you can fin a “Timeout error” for the not loaded images.

i guess you’re facing the same problem. Could try to verify it with this command?

grep "Host header forgery detected" /var/log/squid/cache.log

There is no real work fix for this but a couple of workaround:

  • do not use 8.8.8.8 as your external DNS (I didn’t see any difference even with other dns servers)
  • make sure all clients use the same DNS (this is effective)
  • configure all clients to explicitly use the proxy (this works great)

You can find more information here:


Internet is very slow with proxy
(Ralf Jeckel) #48

I can confirm this:

2016/12/09 08:24:23 kid1| SECURITY ALERT: Host header forgery detected on local=40.77.229.108:443 remote=192.168.0.10:13                                                                      54 FD 55 flags=33 (local IP does not match any domain IP)
2016/12/09 08:25:21 kid1| SECURITY ALERT: Host header forgery detected on local=216.58.214.131:443 remote=192.168.0.10:1                                                                      381 FD 81 flags=33 (local IP does not match any domain IP)
2016/12/09 08:25:21 kid1| SECURITY ALERT: Host header forgery detected on local=216.58.214.131:443 remote=192.168.0.10:1                                                                      382 FD 82 flags=33 (local IP does not match any domain IP)
2016/12/09 10:12:12 kid1| SECURITY ALERT: Host header forgery detected on local=151.101.36.133:443 remote=192.168.0.10:2                                                                      202 FD 33 flags=33 (local IP does not match any domain IP)
2016/12/09 10:12:12 kid1| SECURITY ALERT: Host header forgery detected on local=151.101.36.133:443 remote=192.168.0.10:2                                                                      206 FD 45 flags=33 (local IP does not match any domain IP)

I’m using DNS of lokal ISP (A1).


(kai) #49

Good Morning,
yes, i also can confirm the problem:

2016/12/09 11:26:17 kid1| SECURITY ALERT: Host header forgery detected on local= 216.58.212.142:443 remote=192.168.100.137:33844 FD 330 flags=33 (local IP does n ot match any domain IP)
2016/12/09 11:26:30 kid1| SECURITY ALERT: Host header forgery detected on local= 216.58.212.142:443 remote=192.168.100.137:33845 FD 283 flags=33 (local IP does n ot match any domain IP)
2016/12/09 11:26:35 kid1| SECURITY ALERT: Host header forgery detected on local= 216.58.212.142:443 remote=192.168.100.137:33846 FD 312 flags=33 (local IP does n ot match any domain IP)
2016/12/09 11:26:48 kid1| SECURITY ALERT: Host header forgery detected on local= 216.58.212.142:443 remote=192.168.100.137:33847 FD 294 flags=33 (local IP does n ot match any domain IP)
2016/12/09 11:27:01 kid1| SECURITY ALERT: Host header forgery detected on local= 216.58.212.142:443 remote=192.168.100.137:33848 FD 293 flags=33 (local IP does n ot match any domain IP)
2016/12/09 11:27:06 kid1| SECURITY ALERT: Host header forgery detected on local= 216.58.212.142:443 remote=192.168.100.137:33849 FD 300 flags=33 (local IP does n

i dont use the google dns.
my clients use the transparent proxy all with the same configuration.

at my side i recognize after a restart of squid the problem with ram etc. is fixed for a day or so. and also the speed is good after the restart. it is like something happen after one day or so.


(Ralf Jeckel) #50

Just for information:
I had the problem that certain webseites of Fiat-servers didn’t work. They need IE. With ‘detect proxysettings automaticaly’ enabled in IE-options / lan-settings, so IE reads the wpad file, the problem was gone.


(Giacomo Sanchietti) #51

I can also confirm @hucky fear: squid is really memory hungry but I didn’t see any side effect or slow down on the machine. :wink:
It takes about 1.5GB of swap and 1.5GB of RAM.

This is our production firewall:

  • 4 x Intel® Atom™ CPU C2518 @ 1.74GHz
  • 4GB RAM
  • 4GB swap
  • 20 PC
  • 4 or 5 public services
  • about 30 other devices (smartphone mostly)

Shortly I will do some tests also with ufdbguard.


(kai) #52

Thx for testing Giacomo. I have the idea that with the transparent ssl proxy it need much more time to connect through websites after a day. but maybe it is only at my side… will do some new checks again.


(João Ferreira) #53

I installed version
7.3.1611 activating Transparent with SSL after 5 minutes the pages take
to load arriving or opening many of the times.

Memory: 2655/7857 MB
2 x Intel ® Pentium ® CPU G3260 @ 3.30GHz
Swap
Usage: 0/7936 MB


(Filippo Carletti) #54

Do you see host header forgery messages in /var/log/squid/cache.log?
Have you followed squid instructions here?
http://wiki.squid-cache.org/KnowledgeBase/HostHeaderForgery


(jay) #55

I do not think then it resolves the main issue of pure transparent proxy and we are back to square one where we need to make the browser aware of the proxy in one or the other way or we should remove that security commit done at 3.2 x squid (definitely be at risk ) to least keep this working in a way we want with in an intranet at least

Going through all the firewalls I found only UNTANGLE the ONLY one able to resolve this FULLY where definitely it is paid (the top end features) which even the commercial utm do not provide.


(Marc) #56

3 posts were split to a new topic: How can I block youtube


(Vuk Cetkovic) #57

Hello friends!

Does this mean that squid clamAV will not inspect https traffic for viruses?

Best Regards.


(Marc) #58

That’s correct. See note from documentation:

Web browsing can be checked for malicious content, but only for clear text HTTP protocol. If the proxy is configured in SSL transparent mode (SSL Proxy), content downloaded via HTTPS will not be scanned.