Build web filter modules for ARM

it’s so pity: after putting the raspi in the new housing I cracked the SD card .

Today morning I was surprised that the raspi won’t start. I checked all parts and the reason was clear :frowning:

Now I can start again from beginning because yesterday I didn’t made a backup…

This command is automatically run when you save the configuration, see /etc/e-smith/events/actions/nethserver-squid-conf.
The problem is that the ARM squid package has a different path:

  1. x86 = /usr/lib64/squid/ssl_crtd
  2. arm = /usr/lib/squid/ssl_crtd

Luckily, the nethserver-squid-conf is the only place where ssl_crtd is called, we may have a fix trying both path.
I’ll create a pull request, it may be accepted.

My mistake, I fixed the double quotes issue only partially.
PR: Fix quotes in profiles categories names. NethServer/dev#5171 by filippocarletti · Pull Request #9 · NethServer/nethserver-squidguard · GitHub

Thank you very much for your findings, really helpful.

1 Like

@denis.robel, may I ask you a favor when you’ll have your RPI online again?
Instead of running /usr/lib/squid/ssl_crtd to create the cert dir, modify /etc/squid/squid.conf where it says:
https_port 3130 intercept ssl-bump generate-host-certificates=on ...
with
https_port 3130 intercept ssl-bump generate-host-certificates=off ...

I’ve tested it on Intel x86 and it seems to work, even if squid manual says the opposite (but someone has filed a documentation bug).

1 Like

Okay I’ve got a new SD-Card :slight_smile: and I’ll try it again.

Before I’m compiling hostapd from source: is there a RPM for Centos7 with systemd startup script available? The same is necessary for ufdbguard
I would be more consistent to start hostapd and ufdbguard via systemd instead of old init scripts.

Can somebody provide me the systemd unit scripts or assist me to create the scripts.
Main problem is to define the start dependencies for the services…

Because I know that my project will working I’m writing a documentation parallel to my installation so that the doc team can use it for a Howto.

with warm regards

Denis

2 Likes

hostapd comes from epel repo, so you should simply type:
yum install hostpad
unless epel for arm still doesn’t contain the rpm. We may signal hostapd maintainer.
I’ll try to rebuild my RPI after we release NethServer 7 final.

Regarding ufdbguard, we could ask to the author (Marcus), but I’m not sure he wants to maintain one more package.
You should be able to build from the srpm, I can help.

There is a src rpm available for Centos7

wget -c https://www.urlfilterdb.com/files/downloads/ufdbGuard-1.32.4.src.rpm

but there is an old init script used not a systemd service. I was able to compile the src rpm that’s no problem.

But as i know in Centos 7 systemd is the new standard…

For hostapd will check if I can compile from src rpm…

Don’t worry about systemd, it can handle old sysv initscripts.
I agree with you that a systemd unit file would be better, we could probably build one as a future improvement.

I know that systemd can handle sysv init scripts …
But I’m a German and all things should be in a clear structure :wink:

2 Likes

well I’m edited /etc/e-smith/templates/etc/squid/squid.conf/40ports as you suggested and now it’s working well.

okay here I’m again with my documentation to setup a nethserver 7 based router with raspi3:

  1. Install CentOs7 on Raspi follow the Nethserver Howto:

http://wiki.nethserver.org/doku.php?id=ns_raspi2

ON FIRST BOOT CHANGE ROOT PASSWORD!!!

  1. Install MC --> I’m not a fan of vi vim

yum install mc

  1. change CPU frequency to 1000 Mhz - for faster compiling

mcedit /boot/config.txt

systemctl reboot

  1. Install Nethserver

yum -y update
yum clean all

new nsswitch.conf is created as nsswitch.conf.rpmnew

cp /etc/nsswitch.conf.rpmnew /etc/nsswitch.conf
yum localinstall http://mirror.framassa.org/nethserver-arm/nethserver-release-7arm.rpm
nethserver-install

  1. Install Development Tools

yum groupinstall "Development Tools"
yum install nethserver-devtools

  1. Enable wifi as decribed in /root/README

curl --location https://github.com/RPi-Distro/firmware-nonfree/raw/master/brcm80211/brcm/brcmfmac43430-sdio.bin > /usr/lib/firmware/brcm/brcmfmac43430-sdio.bin
curl --location https://github.com/RPi-Distro/firmware-nonfree/raw/master/brcm80211/brcm/brcmfmac43430-sdio.txt > /usr/lib/firmware/brcm/brcmfmac43430-sdio.txt

systemctl reboot

  1. Install Hostapd:

yum install libnl3-devel
yum install openssl-devel

download hostapd-src from epl

wget -c http://dl.fedoraproject.org/pub/epel/7/SRPMS/h/hostapd-2.4-3.el7.src.rpm
rpmbuild --rebuild hostapd-2.4-3.el7.src.rpm

yum install /root/rpmbuild/RPMS/armv7hl/hostapd-2.4-3.el7.centos.armv7hl.rpm

edit /etc/hostapd/hostapd.conf

########################################################################################################### begin hostapd.conf 
#
# This will give you a minimal, insecure wireless network.
# 
# DO NOT BE SATISFIED WITH THAT!!!
#
# A complete, well commented example configuration file is
# available here:
#
#       /usr/share/doc/hostapd/hostapd.conf
#
# For more information, look here:
#
#       http://wireless.kernel.org/en/users/Documentation/hostapd
#

ctrl_interface=/var/run/hostapd
ctrl_interface_group=wheel

# Some usable default settings...
macaddr_acl=0        # macaddr_acl will be managed from dhcp ...
auth_algs=1
ignore_broadcast_ssid=0

# Uncomment these for base WPA & WPA2 support with a pre-shared key
wpa=3
wpa_key_mgmt=WPA-PSK
wpa_pairwise=TKIP
rsn_pairwise=CCMP

# DO NOT FORGET TO SET A WPA PASSPHRASE!!
wpa_passphrase=YOURSECRETPASSWORD

# Most modern wireless drivers in the kernel need driver=nl80211
driver=nl80211

# Customize these for your local configuration...
interface=wlan0
hw_mode=g
channel=1
ssid=YOURSSID
########################################################################################################## end hostapd.conf 

enable hotapd.service:

systemctl enable hostapd.service

systemctl start hostapd

now you should see your wifi but you wont get an IP address because dhcp is not ready yet

  1. Build Nethserver-WebContentfilter stuff

download src rpms:

wget -c http://mirrorlist.nethserver.org/nethserver/7.3.1611/base/Source/SPackages/nethserver-squid-1.5.0-1.ns7.src.rpm
wget -c http://mirrorlist.nethserver.org/nethserver/7.3.1611/base/Source/SPackages/nethserver-squidguard-1.6.0-1.ns7.src.rpm
wget -c https://www.urlfilterdb.com/files/downloads/ufdbGuard-1.32.4.src.rpm

nethserver-squid:
rpmbuild --rebuild nethserver-squid-1.5.0-1.ns7.src.rpm
yum install /root/rpmbuild/RPMS/noarch/nethserver-squid-1.5.0-1.el7.centos.noarch.rpm

ufdbguard:
yum install bzip2-devel
create user ufdb - without htis user the RPM will not be packed - why???
useradd -r ufdb -d /var/ufdbguard -M -s /sbin/nologin
rpmbuild --rebuild ufdbGuard-1.32.4.src.rpm
yum install /root/rpmbuild/RPMS/armv7hl/ufdbGuard-1.32.4-CentOS7.armv7hl.rpm

nethserver-squidguard:
rpmbuild --rebuild nethserver-squidguard-1.6.0-1.ns7.src.rpm
yum install /root/rpmbuild/RPMS/noarch/nethserver-squidguard-1.6.0-1.el7.centos.noarch.rpm

9.login to nethserver web surface

set wlan0 to green network
set eth0 to red network

configure dhcp for wlan0

configure web-proxy
configure web-contenfilter

3 Likes

okay I tested a little bit the filter:

ufdbguard does not working correctly. I’m using shalla as blacklist and no listed domain will be blocked.

There must be a problem deeper inside. I’ll investigate it later…

Wow I love your work and I’d like to involve also the @arm_team in this discussion!

Okay I checked the urfdbguard reference manual but it’s not so easy to find a start point for debugging. Some hints would be helpful…

At the moment I’m using squid manually with debug level 5 and debugging for ufdbguard is not clear. So I’m not able to check if there are data in blacklist database etc…

Hi @denis.robel, great work!

unfortunately there is no official epel for arm, here are unsigned packaged build by the centos arm-interest group.
http://armv7.dev.centos.org/repodir/epel-pass-1/hostapd/

that is odd, I will try to build it in mock at Christmas

I don’t know much about urfdbguard, so can’t help you right now…

Do not kwon if it helps, I see more configuration files pointing to /usr/lib64/…

okay the path problem seems to be a general problem for arm …

But maybe the config template for ufdbguard ist for a wrong version?
I’m using urfdbguard 1.32.4 and the nethserver-squidguard is working with which version of urfbguard???

Is there any chance to check that the import from shalla.tar.gz into the urfdbguard database is done?

It seems for me that the blacklist database is empty…

For my understanding: urfdbguard import the blacklist form SquidgGuard like from shalla into it’s own database right?

When I’m creating an own category the blocking mechanism is working well. So my thesis of empty blacklist database could be true.

Technically a problem with 32 bit architecture, on 64 bit arm (aarch64) this problem does not exist. Although I normally advise against it you could make a simlink /usr/lib64/ > /usr/lib/ for testing.

In the x86_64 nethserver repository is urfdbguard 1.32.4

Can not help you with questions regarding urfdbguard/SquidgGuard

Right. The conversion is done every night after the updated lists are downloaded.
All logs are in /var/ufdbguard/logs/.
In my logs I find:
2016-12-07 15:27:10 [9783] loading URL category defaulttable with creation date 20161207.0420
2016-12-07 15:27:10 [9783] loading URL table from “/var/squidGuard/blacklists/gamble/domains”
2016-12-07 15:27:10 [9783] loading URL category defaulttable with creation date 20161207.0420
2016-12-07 15:27:10 [9783] loading URL table from “/var/squidGuard/blacklists/science/chemistry/domains”

And:
2016-12-05 15:54:35 [7471] received TERM signal
2016-12-05 15:54:35 [7471] statistics: TERM
2016-12-05 15:54:35 [7471] statistics: 287 URL lookups (252 https). 115 URLs blocked. 0 tunnels detected. 0 safe searches. 0 Youtube
edufilter. 172 uncategorised URLs. 3 clients. 1 users.
2016-12-05 15:54:35 [7471] statistics: category gamble was matched 0 times and blocked 0 times

Both refer to auth, the outcome is that squid in authenticated mode will not work on RPI, unless the path is changed in squid configuration.