SSL ContentFilter

Feature
,
#1

I’m looking at this

so far I’ve got it installed in docker with

docker run -d --name="e2guardian" \
    -v /opt/docker/volumes/e2g/config:/config \
    -v /opt/docker/volumes/e2g/log:/app/log \
    -p 8081:8080 \
    -p 8675:80 \
    -e FILEBROWSER=on \
    -e FILEBROWSER_PORT=80 \
  -e TZ="Australia/Brisbane" \
   -p 8676:8999 \
    -e NWEB=on \
    -e NWEB_PORT=8999 \
    -e PUID=1234 \
    -e PGID=1234 \
    --dns=1.1.1.1 \
    --dns=1.0.0.1 \
    --restart=unless-stopped \
    beechfuzz/e2guardian

it works but while I can get youtube.com blocked and allow

(e.g., https://www.youtube.com/watch?v=lGoVy44C3cA)

I’m yet to get it to load the video with the embed code

(e.g., https:// www.youtube.com/embed/lGoVy44C3cA)

after I get it all working ill release an install script also while this does work with squid (with configuration) the default doesn’t use squid so atm it’s just acting as a content filter

from what I’ve understood it’s just a matter of finding the right list to add the embed link as it’s technically a referral link

1 Like
Allow only certain videos from youtube
#2

so the steps I need to fix and polish with the above would be

* use let’s encrypt cert instead of Self Signed certs not possible

  • get antivirus working
  • cache proxy set to tiny proxy or squid (this is more for script)
  • make a web gui to automate the entries instead of editing list file (maybe caddy and have it set the certs could also have settings and optional install of cache proxy on it or specify existing proxy)
  • setup authentication with ad and hopefully SSO

I’m thinking when it’s all working nicely (and when I learn more about podman) I could convert this into a NS8 module

#3

Unfortunately after searching for a solution for let’s encrypt I found out it’s not possible

The Let's Encrypt CA on your system does NOT include the key, it is only the certificate. You can't make your own certificates without the key.
Let's Encrypt automatically signs requests only if your request can pass validation. Since you don't control the domains or sites in question, you could never pass the validation and thus could never obtain a certificate from Let's Encrypt for those sites.

The only way you can do MITM is with your own self-signed CA installed on every device/browser.
#4

We had implemented MITM in NS6, but then we abandoned it just for the limitation you pointed out: you need to install a certificate in every client.

Also it is very very bad for the user privacy :wink:

1 Like
#5

When I read the explanation I found I had a realisation that it makes sense in a way I should have already realised as I knew what the mitm does it was one of those brain disconnect moments I guess :slightly_smiling_face:

1 Like
#6

I agree but was willing to compromise since it would be my server doing the filtering so I guess I’ll get a script to run as a logon script to auto import the as cert into chrome for set groups

1 Like
#7

Although this makes me wonder what their thinking with qwac’s that seems like asking for trouble

#8

heres how to auto install Self Signed cert to mac os (tested on high sierra only) over ssh (can be adjusted to logon script but as it only needs to be done once maybe add as script to auto run on mac join ad)

ssh -t -t <username>@<remote computer> \
'sudo bash -c "security add-trusted-cert -d -r trustRoot \
-k /Library/Keychains/System.keychain \
-d <(echo \"<your CA certificate in PEM format>\")"'

you do have to enter the password twice but I’m sure it can be automated (or maybe have run on web gui that asks for pwd and runs cmd)

I’ve got the steps to auto set proxy in RPI will post when I’m back in front of my laptop still working on auto import of self signed cert once that works should be the same for Ubuntu as well

Looking at the steps for windows looks easier seams you can just add it to gpo

the script for pi (which has not been tested…yet)
is

#!/usr/bin/env bash

echo -e
echo  "What is your proxy IP address?"
echo -e
read -r proxyipaddress
sleep 0.2s

echo -e
echo  "What is your proxyport?"
echo -e
read -r proxyport
sleep 0.2s

read -r -p "Does the Proxy require Authentication? [Y/n] " input
 
case $input in
      [yY][eE][sS]|[yY])

echo -e
echo  "What is your Username?"
echo -e
read -r username
sleep 0.2s

echo -e
echo  "What is your Password?"
echo -e
read -r password
sleep 0.2s

echo 'export http_proxy="http://$username:$password@$proxyipaddress:$proxyport"
export https_proxy="http://$username:$password@$proxyipaddress:$proxyport"
export no_proxy="localhost, 127.0.0.1"
' >> /etc/environment

grep -qxF 'Defaults env_keep+="http_proxy https_proxy no_proxy' /etc/sudoers || echo 'Defaults env_keep+="http_proxy https_proxy no_proxy' >> /etc/sudoers

sudo reboot now
;;
            
      [nN][oO]|[nN])
echo 'export http_proxy="http://$proxyipaddress:$proxyport"
export https_proxy="http://$proxyipaddress:$proxyport"
export no_proxy=" localhost, 127.0.0.1"' >> /etc/environment;

grep -qxF 'Defaults env_keep+="http_proxy https_proxy no_proxy' /etc/sudoers || echo 'Defaults env_keep+="http_proxy https_proxy no_proxy' >> /etc/sudoers

sudo reboot now
            ;;
      *)
            echo "Invalid input..."
            exit 1
            ;;
esac
1 Like
#9

ok here’s a script to run on Raspberry Pi (Tested on bullseye) under user to be using said proxy

proxycert.sh 
#!/usr/bin/env bash

echo -e
echo  "What is the IP of your Proxy?"
echo -e
read -r proxyip
sleep 0.2s

wget http://$proxyip:8676/cacertificate.crt

sudo apt-get update && sudo apt-get install -y p11-kit libnss3
find / -type f -name "libnssckbi.so" 2>/dev/null | while read line; do
    sudo mv $line ${line}.bak
    sudo ln -s /usr/lib/x86_64-linux-gnu/pkcs11/p11-kit-trust.so $line
done

sudo mv cacertificat.crt /usr/local/share/ca-certificates
sudo update-ca-certificates```

obviously you could simply add the ipaddress in the script and run

for windows gpo take a look here How to add a trusted CA certificate to Chrome and Firefox

#10

to force safe search for google and youtube go to http://proxyip:8675/files/lists/sslsiteregexplist
edit to look like this

http://proxyip:8675/files/lists/sslsiteregexplist 
#SSL site modifying Regular Expressions
#
# The format is: "extended regular expression"->"replacement straight string"
# E.g. "bannedword"->"censored" would replace all occurrences of bannedword in any case.
# Far more complicated matches are possible.  See other sources for examples
# of extended regular expressions.
#
# Users are pointed at the replaced site transparently.
# This is used to 'cname' ssl sites and avoids having to adjust DNS
# and allows option of switching off/on depending on filter group.

# This list applies to the site only (not full URL) for 
#   HTTPS sites (regardless of SSL MITM configuration).

# Do not use patterns with full urls in this list as they can never
#   match the site name.

# Enforce restricted mode in YouTube
#
"(^https://www.youtube.com)"->"https://restrict.youtube.com"
"(^https://m.youtube.com)"->"https://restrict.youtube.com"
"(^https://youtubei.googleapis.com)"->"https://restrict.youtube.com"
"(^https://youtube.googleapis.com)"->"https://restrict.youtube.com"
"(^https://www.youtube-nocookie.com)"->"https://restrict.youtube.com"
#
# Enforce restricted mode in Google
#
"(^https://www\.google\.[a-zA-Z0-9_.]*$)"->"https://forcesafesearch.google.com"
"(^https://.*\.gstatic\.com$)"->"https://forcesafesearch.google.com"
http://proxyip:8675/files/lists/bannedsearchlist 
#Banned Search Words
#
#Words must be in alphabetic order within a single line
# and separated by a '+' sign.
#All combinations of the words will be blocked
# e.g.   girl+naughty
#     will block  naughty+girl as well as girl+naughty
http://proxyip:8675/files/lists/bannedsitelist 
#domains in banned list
#Don't bother with the www. or the http://

#The bannedurllist is for blocking PART of a site
#The bannedsitelist is for blocking ALL of a site

#NOTE: Sites using just IP should be put into bannedsiteiplist

#You can include
#.tld so for example you can match .gov for example

#The 'grey' lists override the 'banned' lists.
#The 'exception' lists override the 'banned' lists also.
#The difference is that the 'exception' lists completely switch
#off *all* other filtering for the match.  'grey' lists only
#stop the URL filtering and allow the normal filtering to work.

#An example of grey list use is when in Blanket Block (whitelist)
#mode and you want to allow some sites but still filter as normal
#on their content

#Another example of grey list use is when you ban a site but want
#to allow part of it.

#To include additional files in this list use this example:
#.Include</config/anotherbannedurllist>

#You can have multiple .Includes.

# Time limiting syntax:
# #time: <start hour> <start minute> <end hour> <end minute> <days>
# Example:
##time: 9 0 17 0 01234
# Remove the first # from the line above to enable this list only from
# 9am to 5pm, Monday to Friday.

# List categorisation
#listcategory: "Banned Sites"

#List other sites to block:

# badboys.com

# NOTE: From v5 Blanket blocks are now implimented using Storyboarding 
# WARNING: Old style Blanket blocks in this file will be silently ignored


# The squidGuard advert domain/URL lists are now included by default.
# To work with advanced ad blocking & the logadblocks option, advert
# phrase/site/URL lists should have the string "ADs" in their listcategory.
# .Include</config/lists/blacklists/ads/domains>

#Remove the # from the following and edit as needed to use a stock
#squidGuard/urlblacklists collection.
#.Include</config/lists/blacklists/adult/domains>
#.Include</config/lists/blacklists/aggressive/domains>
#.Include</config/lists/blacklists/artnudes/domains>
#.Include</config/lists/blacklists/audio-video/domains>
#.Include</config/lists/blacklists/beerliquorinfo/domains>
#.Include</config/lists/blacklists/beerliquorsale/domains>
#.Include</config/lists/blacklists/chat/domains>
#.Include</config/lists/blacklists/childcare/domains>
#.Include</config/lists/blacklists/clothing/domains>
#.Include</config/lists/blacklists/culinary/domains>
#.Include</config/lists/blacklists/dialers/domains>
#.Include</config/lists/blacklists/drugs/domains>
#.Include</config/lists/blacklists/entertainment/domains>
#.Include</config/lists/blacklists/forums/domains>
#.Include</config/lists/blacklists/frencheducation/domains>
#.Include</config/lists/blacklists/gambling/domains>
#.Include</config/lists/blacklists/government/domains>
#.Include</config/lists/blacklists/hacking/domains>
#.Include</config/lists/blacklists/homerepair/domains>
#.Include</config/lists/blacklists/hygiene/domains>
#.Include</config/lists/blacklists/jewelry/domains>
#.Include</config/lists/blacklists/jobsearch/domains>
#.Include</config/lists/blacklists/kidstimewasting/domains>
#.Include</config/lists/blacklists/mail/domains>
#.Include</config/lists/blacklists/news/domains>
#.Include</config/lists/blacklists/onlineauctions/domains>
#.Include</config/lists/blacklists/onlinegames/domains>
#.Include</config/lists/blacklists/onlinepayment/domains>
#.Include</config/lists/blacklists/personalfinance/domains>
#.Include</config/lists/blacklists/pets/domains>
#.Include</config/lists/blacklists/porn/domains>
#.Include</config/lists/blacklists/proxy/domains>
#.Include</config/lists/blacklists/publicite/domains>
#.Include</config/lists/blacklists/redirector/domains>
#.Include</config/lists/blacklists/ringtones/domains>
#.Include</config/lists/blacklists/sportnews/domains>
#.Include</config/lists/blacklists/sports/domains>
#.Include</config/lists/blacklists/vacation/domains>
#.Include</config/lists/blacklists/violence/domains>
#.Include</config/lists/blacklists/virusinfected/domains>
#.Include</config/lists/blacklists/warez/domains>

# You will need to edit to add and remove categories you want
youtube.com
  • To allow part of a banned website (I.e., domain.tld/gooodpart ) add the url exception here
http://proxyip:8675/files/lists/exceptionurllist 
#URLs in exception list
#Don't bother with the www. or
#the http://
#
#These are parts of sites that filtering should
#be switched off for.
#
#These should not be domains, i.e. entire sites,
#they should be a domain with a path.
#
#For example 'foo.bar' is no good, you need
#to just have 'foo.bar/porn/'.
#
#Another example:
#generallybadsite.tld/partthatsok/

*Please Note it doesn’t allow youtube videos to work while blocking youtube yet I think there is another list you need to add it to as they (youtube links) are referrer links will update when I’ve found the answer It does work with youtube videos but not embedded

  • This following list works by banning searches in google if it matches 2 of the banned words (i.e., it won’t ban tiny butt but it will ban if the search includes another banned word (i.e., boob).
    It can be good to ban search phrases that attempt to get around filtering while allowing normally band words to be accepted for appropriate reasons (i.e., someone searches blue tit for a school project)
http://proxyip:8675/files/lists/phraselists/googlesearches/banned 
# Image Search Site Blocks
# 05-08-03 JFH r4

#listcategory: "Google Searches"

<nude>,<nude>,<nude>,<nude>,<nude>,<sex>,<girls>
<?q=nude>
<?q=naked+women&>
<?q=naked+girls&>

tested with yahoo.com bing.com aol.com ask.com baidu.com all allowed tiny butt but blocked tiny butt boobs So they work not just for google (decided to test with the least offensive words I could see incase to avoid awkward questions if I get called out and my kids “accidentally” go into my office )

1 Like
#11

also for icap
docker run --rm -d -p 1344:1344 --name scanner packetchef/icap-clamav-server:1.0 havent tested fully yet