Using expressions matching to Web Proxy & Filter

Support
#1

NethServer Version: 7.9.2009
Module: Web Proxy & Filter
Hi,
I’m using the ‘Université Toulouse’ blacklist in Web Proxy & Filter.
While trying to add some expression filtering in ufdbGuard and I found this post:

How and where specify an "expression matching on URL"

However it seems to be outdated and it’s not working anymore. I also tried to add some expressions in /etc/e-smith/templates-custom/var/squidGuard/blacklists/custom/blacklist/expressions/30myexp, but after issuing signal-event nethserver-squidguard-save the expressions file is not created under /var/squidGuard/blacklists/custom/blacklist/ and /etc/ufdbguard/ufdbguard.conf doesn’t include the command to load the expressions file.
Which should be the proper way to acomplish this?

#2

Hi @Juan_Martin,

Please do a backup before doing the steps, I didn’t test them and I’m not 100% sure if it is right.

I’ve searched a little bit at my system and found the following file:

/var/squidguard/blacklists/custum/files/expressions

I think this is the file you need. It looks like you can change it without a custom template, only content is

^$

The file is loaded at

/etc/ufdbguard/ufdbGuard.conf

at the following lines

category “files” {
expressionlist /var/squidGuard/blacklists/custom/files/expressions
}

Last you have to convert the file to a .ufdb file

ufdbConvertDB /var/squidGuard/blacklists/custom/files/

But perhaps @giacomo has some more infos.

1 Like
#3

It should do the job, but I agree with Michael: do a backup of the files because I could miss something too :slight_smile:

#4

Thanks Michael and Giacomo.
I saw that possibility, but isn’t that expressions file used for the “blocked file extensions”? Isn’t there any other way add the expressionlist option to a custom category or to the global nh_blacklist category?
If I follow your recommendation, I guess that I should do it via the corresponding file in /etc/e-smith/templates-custom right? If I edit ufdbGuard.conf and the expressions files directly, wouldn’t be all the changes lost if I make some changes from the GUI?

#5

Yes, you are right.

I have a new idea, but somebody else has to help, because I can’t expand the template and I don’t know why.

[root@groupware custom]# expand-template /etc/ufdbguard/ufdbGuard.conf          ERROR in /etc/e-smith/templates-custom//etc/ufdbguard/ufdbGuard.conf/19MyExpressions: Program fragment delivered error <<Illegal division by zero at /etc/e-smith/templates-custom//etc/ufdbguard/ufdbGuard.conf/19MyExpressions line 2.>> at template line 1
ERROR: Template processing failed for //etc/ufdbguard/ufdbGuard.conf: 1 fragment generated errors
 at /sbin/e-smith/expand-template line 45.
[root@groupware custom]# nano /etc/e-smith/templates-custom/etc/ufdbguard/ufdbGuard.conf/19MyExpressions
  • Create a folder myexpressions at /var/squidGuard/blacklists/custom

mkdir /var/squidGuard/blacklists/custom/myexpressions

  • Create a file expressions at the myexpressions folder. I use nano for it.

nano /var/squidGuard/blacklists/custom/myexpressions/expressions

  • Enter your expressions line by line and safe the file.

  • Now create a custom template for loading your expressions

mkdir /etc/e-smith/templates-custom/etc/ufdbguard
mkdir /etc/e-smith/templates-custom/etc/ufdbguard/ufdbGuard.conf
nano /etc/e-smith/templates-custom/etc/ufdbguard/ufdbGuard.conf/19MyExpressions

  • The content of the file is the following:
category "myexpressions" {
expressionlist /var/squidGuard/blacklists/custom/myexpressions/expressions
}
  • For using it, we have to expand the template

expand-template /etc/ufdbguard/ufdbGuard.conf

cc @giacomo @support_team

1 Like
#6

Can @dev_team help here?
I also tried the path at the template with " "

category "myexpressions" {
expressionlist "/var/squidGuard/blacklists/custom/myexpressions/expressions"
}
#7

I did a quick review of the package and I think it’s not possible to do what you’re asking for without deep modifications.
I do not have a valid solution right now, sorry.

But I can advice you against using expressions because in my experience it can lead to overblocking.

#8

Ok, thanks for your help @m.traeumner and @giacomo.

#9

@giacomo or somebody else who understands it
Can you help me to understand the following template fragment: I have a problem by understanding why file gets an expressionlist and the other categories not.

{
    use File::Find;
    use File::Basename;
    my $exprstatus = $squidguard{'Expressions'} || 'disabled';
    our %categories;
    our @files;
    
    sub wanted() {
        my %tmp;
        if ( -d $File::Find::name ) {
            my $domains = $File::Find::name."/domains";
            my $expressions = $File::Find::name."/expressions";

            if ( -e $domains ) {
               $tmp{'domains'} = $domains; 
            }
            if ( -e $expressions ) {
               $tmp{'expressions'} = $expressions; 
            }
            if ( -e $domains || -e $expressions ) {
                my $category = $_;
                if ( $File::Find::name eq '/var/squidGuard/blacklists/custom/whitelist') {
                    $category = 'nh_whitelist';
                }
                if ( $File::Find::name eq '/var/squidGuard/blacklists/custom/blacklist') {
                    $category = 'nh_blacklist';
                }
                $categories{$category} = \%tmp;
            }
        }
    }

    find (\&wanted, "/var/squidGuard/blacklists/");

    while (($key, $value) = each(%categories)) {
        $OUT.="category \"$key\" {\n";
        if (defined($value->{'domains'})) {
            $OUT .= "      domainlist ".$value->{'domains'}."\n";
        }
        if (defined($value->{'expressions'}) && ( $exprstatus eq 'enabled' || $key eq 'files')) {
            $OUT .= "      expressionlist ".$value->{'expressions'}."\n";
        }
        $OUT.="}\n";
    }
}

It’s from

/etc/e-smith/etc/templates/ufdbguard/ufdbGuard.conf/20categories

At

there is a config with expressionlists, so it could be possible, but like @giacomo said, probably with deep modifications.

ufdbGuard Regex
#10

The template outputs the expressionslist directive if the category has a file named expressions (eg. /var/squidGuard/blacklist/news/expressions) and the Expressions prop is enabled.

1 Like
#11

Thanks @giacomo,
this could be a solution for @Juan_Martin

1 Like