Gitea A painless self-hosted Git service

With a small data set (3 small repo’s)

  1. survived this

  2. (A) deleted data from one repo . put it back and restarted gitea : oke
    (B) removed repo (nethserver-gitea) in web-gui > danger zone; albeit all warnings this can’t be undone :open_mouth:
    created new empty repo in web gui (=nethserver-gitea)
    restored the nethserver-gitea repo files to /tmp/nethserver-gitea

    git clone /tmp/nethserver-gitea nethserver-gitea
    git remote remove origin
    git remote add origin https://_hostname_/git/_user_/nethserver-gitea.git
    git push -u origin master

all commits and code resorted, missing is other data such as issues

  1. rm -rf /var/lib/mysql/gitea/* ; gitea dies :open_mouth:
    systemctl stop gitea
    restored gitea.dump (/var/lib/nethserver/backup_2018-08-23/mysql)
    need to drop (remove) the database and recreate it
    mysql --defaults-file=/root/.my.cnf -e "DROP DATABASE gitea"
    mysql --defaults-file=/root/.my.cnf -e "CREATE DATABASE IF NOT EXISTS gitea;"
    mysql --defaults-file=/root/.my.cnf gitea < /var/lib/nethserver/backup_2018-08-23/mysql/gitea.dump
    systemctl start gitea
    up and running again :grinning:

  2. still to do

2 Likes

Hmmm. Rolled back the test VM to a clean state, built nethserver-gitea-0.0.4, and installed that (along with dependencies). Now can’t log in at all. Not with gitea/gitea, not with a local user on the system.

Edit: Yeah, must be some difference in how authentication is handled between 0.0.3 and 0.0.4. When I rolled the VM back to a clean state and reinstalled 0.0.3, I could log in with gitea/gitea. When I repeated, but installed 0.0.4, I get this when I try:


Not seeing anything in the logs yet, but I’m sure I haven’t looked everywhere. Authentication attempts don’t seem to hit /var/log/gitea/gitea.log, even when I set the log level to Debug in app.ini.

Cannot reproduce it yet, will try to do so!

Biggest change between 0.0.3 and 0.0.4 is how the database is populated with tables.
A bit of background: The workflow of a “normal” non-nethserver install is to configure it in the web-gui on first run. This means the user need to provide the the password of the database, create a admin user and configure a login source.
To face this chalange the database is populated with tables up front, a (local) admin user is added and a record for PAM login source is directly written in the database.

Until 0.0.3 the data base was populated with a dump of the structure of the db from a (once) running instance. In 0.0.4 during setup (conf) gitea is started up for 2 seconds to populate the db with tables. The advantage is the “0.0.3” method is a maintaining nightmare, for new installs the version of provided db structure must be inline with the current gitea version.

In short: most likely at your install the db is not properly populated with tables the following commands have failed:

# populate database with tables on the fly by running gitea for 2s
# and add a local admin user

pushd /var/lib/nethserver/gitea
timeout 2s su gitea -c \
"GITEA_WORK_DIR='/var/lib/nethserver/gitea' /usr/bin/gitea web --config /etc/gitea/app.ini"
su gitea -c \
"GITEA_WORK_DIR='/var/lib/nethserver/gitea' /usr/bin/gitea admin create-user --name gitea --password 'gitea' --email gitea@example.com --admin --config /etc/gitea/app.ini"
popd

# add PAM login source
epoch=$(date +%s)
/usr/bin/mysql --defaults-file=/root/.my.cnf -e \
"INSERT INTO gitea . login_source (id, type, name, is_actived, is_sync_enabled, cfg, created_unix, updated_unix) 
VALUES ('1', '4', 'Nethserver', '1', '0', '{\"ServiceName\":\"password-auth\"}', '$epoch', '$epoch');"

EDIT(2): thinking out loud now; to speed up testing i use an image with nethserver-mysql already installed. maybe that’s why it works for me, will tests this! Nope can’t reproduce yet.

Thanx for the report!

clean install with only ldap account provider all is fine, journalctl also shows the above mentioned provisioning succeed:

... usermod[2865]: change user 'gitea' home from '/var/lib/gitea' to '/var/lib/nethserver/gitea'
...  esmith::event[2858]: /var/lib/nethserver/gitea ~
... su[2886]: (to gitea) root on none
... su[2886]: pam_unix(su:session): session opened for user gitea by root(uid=0)
... esmith::event[2858]: 2018/08/25 09:19:43 [T] AppPath: /usr/bin/gitea
... esmith::event[2858]: 2018/08/25 09:19:43 [T] AppWorkPath: /var/lib/nethserver/gitea
... esmith::event[2858]: 2018/08/25 09:19:43 [T] Custom path: /var/lib/nethserver/gitea/custom
... esmith::event[2858]: 2018/08/25 09:19:43 [T] Log path: /var/log/gitea/
... esmith::event[2858]: 2018/08/25 09:19:44 Serving [::]:3000 with pid 2887
... su[2886]: pam_unix(su:session): session closed for user gitea
... esmith::event[2858]:
... esmith::event[2858]: Session terminated, killing shell...2018/08/25 09:19:45 Exiting pid 2887.
... esmith::event[2858]:  ...killed.
... su[2905]: (to gitea) root on none
... su[2905]: pam_unix(su:session): session opened for user gitea by root(uid=0)
... esmith::event[2858]: 2018/08/25 09:19:47 [I] XORM Log Mode: File(Warn)
... esmith::event[2858]: New user 'gitea' has been successfully created!
... su[2905]: pam_unix(su:session): session closed for user gitea
... esmith::event[2858]: ~
... esmith::event[2858]: Action: /etc/e-smith/events/nethserver-gitea-update/S30nethserver-gitea-conf SUCCESS [4.508758]
... systemd[1]: Reloading.
  • usermod “home dir” oke
  • start and killing gitea after 2s (this is the “hack” to populate the db with tables) oke
  • tables are in place, otherwise New user 'gitea' has been successfully created! would have failed.

“Inserting” of record for PAM login source is not logged ( it’s on my todo list now) but did work for me.

Strange :thinking::thinking:

I don’t know systemd well at all, so didn’t know about journalctl. So, back to baseline on the test VM, scp over the gitea and nethserver-gitea RPMs, and yum install *.rpm while running journalctl -f in another terminal. It looks like this is the relevant portion:

Aug 25 07:33:31 neth-test.familybrown.org esmith::event[2873]: Action: /etc/e-smith/events/nethserver-gitea-update/S00initialize-default-databases SUCCESS [0.380144]
Aug 25 07:33:31 neth-test.familybrown.org esmith::event[2873]: expanding /etc/gitea/app.ini
Aug 25 07:33:31 neth-test.familybrown.org esmith::event[2873]: expanding /etc/httpd/conf.d/default-virtualhost.inc
Aug 25 07:33:31 neth-test.familybrown.org esmith::event[2873]: expanding /etc/httpd/conf.d/zz_gitea.conf
Aug 25 07:33:31 neth-test.familybrown.org esmith::event[2873]: expanding /etc/pki/gitea/certs/gitea.crt
Aug 25 07:33:31 neth-test.familybrown.org esmith::event[2873]: expanding /etc/pki/gitea/private/gitea.key
Aug 25 07:33:31 neth-test.familybrown.org esmith::event[2873]: expanding /etc/sysconfig/gitea
Aug 25 07:33:31 neth-test.familybrown.org esmith::event[2873]: Action: /etc/e-smith/events/actions/generic_template_expand SUCCESS [0.34837]
Aug 25 07:33:31 neth-test.familybrown.org usermod[2880]: change user 'gitea' home from '/var/lib/gitea' to '/var/lib/nethserver/gitea'
Aug 25 07:33:31 neth-test.familybrown.org esmith::event[2873]: /var/lib/nethserver/gitea ~
Aug 25 07:33:32 neth-test.familybrown.org su[2901]: (to gitea) root on none
Aug 25 07:33:32 neth-test.familybrown.org su[2901]: pam_unix(su:session): session opened for user gitea by root(uid=0)
Aug 25 07:33:32 neth-test.familybrown.org esmith::event[2873]: 2018/08/25 07:33:32 [T] AppPath: /usr/bin/gitea
Aug 25 07:33:32 neth-test.familybrown.org esmith::event[2873]: 2018/08/25 07:33:32 [T] AppWorkPath: /var/lib/nethserver/gitea
Aug 25 07:33:32 neth-test.familybrown.org esmith::event[2873]: 2018/08/25 07:33:32 [T] Custom path: /var/lib/nethserver/gitea/custom
Aug 25 07:33:32 neth-test.familybrown.org esmith::event[2873]: 2018/08/25 07:33:32 [T] Log path: /var/log/gitea/
Aug 25 07:33:34 neth-test.familybrown.org su[2901]: pam_unix(su:session): session closed for user gitea
Aug 25 07:33:34 neth-test.familybrown.org esmith::event[2873]: 
Aug 25 07:33:36 neth-test.familybrown.org esmith::event[2873]: Session terminated, killing shell... ...killed.
Aug 25 07:33:36 neth-test.familybrown.org su[2916]: (to gitea) root on none
Aug 25 07:33:36 neth-test.familybrown.org su[2916]: pam_unix(su:session): session opened for user gitea by root(uid=0)
Aug 25 07:33:36 neth-test.familybrown.org esmith::event[2873]: 2018/08/25 07:33:36 [I] XORM Log Mode: File(Warn)
Aug 25 07:33:36 neth-test.familybrown.org esmith::event[2873]: CreateUser: Error 1146: Table 'gitea.email_address' doesn't exist
Aug 25 07:33:36 neth-test.familybrown.org su[2916]: pam_unix(su:session): session closed for user gitea
Aug 25 07:33:36 neth-test.familybrown.org esmith::event[2873]: ~
Aug 25 07:33:36 neth-test.familybrown.org esmith::event[2873]: ERROR 1146 (42S02) at line 1: Table 'gitea.login_source' doesn't exist
Aug 25 07:33:36 neth-test.familybrown.org esmith::event[2873]: Action: /etc/e-smith/events/nethserver-gitea-update/S30nethserver-gitea-conf FAILED: 1 [4.39021]
... esmith::event[2873]: 2018/08/25 07:33:32 [T] Log path: /var/log/gitea/
... su[2901]: pam_unix(su:session): session closed for user gitea

vs my log

... esmith::event[2858]: 2018/08/25 09:19:43 [T] Log path: /var/log/gitea/
... esmith::event[2858]: 2018/08/25 09:19:44 Serving [::]:3000 with pid 2887
... su[2886]: pam_unix(su:session): session closed for user gitea

gitea does not come in a full running state Serving [::]:3000 with pid 2887

question is why?

you slould be able to “save” your install with

systemctl stop gitea
pushd /var/lib/nethserver/gitea

su gitea -c \
"GITEA_WORK_DIR='/var/lib/nethserver/gitea' /usr/bin/gitea admin create-user --name gitea --password 'gitea' --email gitea@example.com --admin --config /etc/gitea/app.ini"

popd

export epoch=$(date +%s)

/usr/bin/mysql --defaults-file=/root/.my.cnf -e \
"INSERT INTO gitea . login_source (id, type, name, is_actived, is_sync_enabled, cfg, created_unix, updated_unix) 
VALUES ('1', '4', 'Nethserver', '1', '0', '{\"ServiceName\":\"password-auth\"}', '$epoch', '$epoch');"

systemctl start gitea

But this is unacceptable!

Maybe ease the time gitea is running to build up the database (2 second now) could be an answer although i cant believe a RPI is faster than what-ever-other-environment.

Yes, that fixes it:

[root@neth-test ~]# systemctl stop gitea
[root@neth-test ~]# pushd /var/lib/nethserver/gitea
/var/lib/nethserver/gitea ~
[root@neth-test gitea]# su gitea -c \
> "GITEA_WORK_DIR='/var/lib/nethserver/gitea' /usr/bin/gitea admin create-user --name gitea --password 'gitea' --email gitea@example.com --admin --config /etc/gitea/app.ini"
2018/08/25 08:02:13 [I] XORM Log Mode: File(Warn)
New user 'gitea' has been successfully created!
[root@neth-test gitea]# popd
~
[root@neth-test ~]# export epoch=$(date +%s)
[root@neth-test ~]# /usr/bin/mysql --defaults-file=/root/.my.cnf -e \
> "INSERT INTO gitea . login_source (id, type, name, is_actived, is_sync_enabled, cfg, created_unix, updated_unix) 
> VALUES ('1', '4', 'Nethserver', '1', '0', '{\"ServiceName\":\"password-auth\"}', '$epoch', '$epoch');"
[root@neth-test ~]# systemctl start gitea
[root@neth-test ~]# 

I’m now able to log in both as gitea and also as a Neth user.

That would seem like the obvious culprit, but I’ve run into this on three separate installations on a decently-resourced VM. If two seconds is adequate on a Pi, it would seem like at least one of these ought to work.

if you are whiling and have the time you could try to rebuild nethserver-gitea after changing the timeout time to something ridiculously long as 10s

timeout 10s su gitea -c \

/root/etc/e-smith/events/actions/nethserver-gitea-conf

1 Like

With that change, it works. I built 0.0.4-2 with (only) that change, rolled back the VM to a clean state, installed gitea and nethserver-gitea, and I’m able to log in to the installation both as gitea and as a Neth user.

2 Likes

thanx for helping me out here! :+1:

1 Like

Now back to checking the backup/restore scenarios…

Edit: clone and push via HTTPS and SSH continue to work.

1 Like

nice!

BTW until a better solution is found, an if loop until the tables are available comes in to mind
i’ve set the time out to even 20s better save than sorry; it’s only done once on new installs…

Is there a configuration to make Gitea use its own credentials? Like my Dokuwiki install, I wouldn’t really expect that other users with Neth accounts on my system (my family, in my case) would have be editing or adding content, but other folks could. It’d be helpful if it could do its own authentication.

Yes there are a lot of possibilities, including AD/Ldap bind using group permisions.
Still need to figure out the right group filter syntax to accomplish this.

https://docs.gitea.io/en-us/authentication/

you can simply deactivate the PAM authentication.
as admin user (gitea) in drop down menu Site Administration > Authentication Sources > Edit (PAM) > uncheck “is active”

this is a partial solution because everybody can register…

In my local wip branch this configuration option in the app.ini at [service] section is already moved to the esmith-db. DISABLE_REGISTRATION: false

https://docs.gitea.io/en-us/config-cheat-sheet/

EDIT: if you want to bind to ad/ldap i can share my finding so far
EDIT2 or make a pam unit for a group

Some brainstorming:

  • lengthy timeout: OK for most cases, unless on very very slow server.
  • monitor database threads/transactions: difficult, impractical (if even possible).
  • mysqlshow (or alike) loop until tables available: OK on most cases;
    • case: unlikely to happen on first setup, but is it possible that all tables are created but there are pending transactions (many or big INSERTs, UPDATEs) when the process is killed?
  • monitor creation of some file or content of a log: don’t know order of gitea events
  • monitor output of gitea web command or connection status to localhost:3000 (see below): OK, if assumption is correct, until upstream changes command behaviour:

$ GITEA_WORK_DIR=‘/var/lib/nethserver/gitea’ /usr/bin/gitea web --config /etc/gitea/app.ini
2018/08/26 10:22:37 [T] AppPath: /usr/bin/gitea
2018/08/26 10:22:37 [T] AppWorkPath: /var/lib/nethserver/gitea
2018/08/26 10:22:37 [T] Custom path: /var/lib/nethserver/gitea/custom
2018/08/26 10:22:37 [T] Log path: /var/log/gitea/
2018/08/26 10:22:40 Serving [::]:3000 with pid 4204
[Macaron] 2018-08-26 10:22:43: Started GET / for 127.0.0.1
[Macaron] 2018-08-26 10:22:43: Completed GET / 200 OK in 921.834µs

I think the database has been fully populated either when gitea shows the remarked line or when localhost:3000 accepts connections (tried with curl, hence the last two lines).

If the output of the command can be piped or redirected to your script (or curl OK response), I think it can help…

# su gitea -c "GITEA_WORK_DIR='/var/lib/nethserver/gitea' /usr/bin/gitea web --config /etc/gitea/app.ini" >1
2018/08/26 10:21:47 Serving [::]:3000 with pid 4088
^C
Session terminated, killing shell...2018/08/26 10:21:51 Exiting pid 4088.
 ...killed.
1 Like

Thanx, did not think about waiting until the the web interface is responsive!
Think this is the most promising method to pursiut first!

  1. worked after some rework (should have known better: use abs paths in bash-script!) :grinning:

It took a while to realize the local repository entry in /etc/yum.repos.d/ is set to disabled during the recovery event. To be able to simulate a full test hacked the settings a bit by exchanging my local settings in NethFrog.repo.
A simple workaround for the above is simply installing nethserver-gitea before issuing the restore-data on the command-line.

Thanx this was indeed a clever way to go and is implemented!

cc/ @danb35

1 Like

Just tested this one, and this is the result on login:

Specifics:

[root@neth-test ~]# rpm -qa | grep gitea
gitea-1.5.0-1.ns7.x86_64
nethserver-gitea-0.0.5-3.ns7.noarch

…built from your unmodified .spec files.

Procedure:

  • Start with clean Neth 7.5 install (off the ISO).
  • scp the above two RPMs to the machine, then yum install *.rpm.
  • Log in to server-manager, restore configuration.
  • Log back in to server-manager (required since IP changed after restoring configuration), install updates through software center.
  • Reboot since kernel was updated.
  • restore-data
  • Reboot again just to be on the safe side.
  • Browse to neth-test/git and log in as a user. Get the message above.

gitea.log shows:

[root@neth-test gitea]# cat gitea.log 
2018/08/30 21:41:54 [...itea/routers/init.go:60 GlobalInit()] [E] Failed to initialize ORM engine: dial tcp 127.0.0.1:3306: getsockopt: connection refused
2018/08/30 21:41:56 [...itea/routers/init.go:64 GlobalInit()] [E] Failed to initialize OAuth2 support: dial tcp 127.0.0.1:3306: getsockopt: connection refused
2018/08/30 21:41:58 [...itea/routers/init.go:60 GlobalInit()] [E] Failed to initialize ORM engine: dial tcp 127.0.0.1:3306: getsockopt: connection refused
2018/08/30 21:43:48 [...routers/user/auth.go:162 SignInPost()] [E] UserSignIn: User not known to the underlying authentication module

Thanks for reporting, will try to reproduce.

First glimpse looking at the log you supplied: there are issues with (accessing) the database.

If you still have this testing machine could you check if the mysql password match ?

cat /var/lib/nethserver/secrets/gitea
cat /etc/gitea/app.ini | grep PASSWD

The later this gives a few entry’s for “PASSWD” one of then should match the hash from the first command, or just look in /etc/gitea/app.ini at the [database] section.

( obvious no need to share the password)

If they do not match signal-event nethserver-gitea-update may correct this…

Followed this to the letter (on x86_64) and all good for me :thinking: