Gitea A painless self-hosted Git service

module

(Mark Verlinde) #1

I’am working an a new feature Gitea, which is nice to have and fun to run on a RPI. :grinning:

It not so easy to setup from the command-line because most of the initialization is done at the first visit to the web interface. I figured out how to do it any way but encounter on problem which i just can’t solve.

in short the problem is the order of setup which can be done manual but can figure out how to do it with e-smith:

  1. create a new mysql database and load tables in to it
  2. write record to database with Pam login source
  3. expand templates with the mysql credentials
  4. run (bash) script to add local admin user
  5. start service

in bash scrips doing it manually it looks like this:

#!/bin/bash

# create new data base
password=`perl -e "use NethServer::Password; print NethServer::Password::store('giteaPass');"`

/usr/bin/mysql --defaults-file=/root/.my.cnf -e \
"CREATE DATABASE IF NOT EXISTS gitea DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;"
/usr/bin/mysql --defaults-file=/root/.my.cnf -e \
"CREATE USER gitea@localhost IDENTIFIED BY '$password';" 
/usr/bin/mysql --defaults-file=/root/.my.cnf -e \
"GRANT ALL PRIVILEGES ON gitea .* TO gitea@localhost;"

# write tables to it
mysql --defaults-file=/root/.my.cnf gitea < gitea-mysql-sample.sql

# Add PAM login source
epoch=$(date +%s)
/usr/bin/mysql --defaults-file=/root/.my.cnf -e \
"INSERT INTO gitea . login_source (id, type, name, is_actived, is_sync_enabled, cfg, created_unix, updated_unix) 
VALUES ('1', '4', 'Nethserver', '1', '0', '{\"ServiceName\":\"password-auth\"}', '$epoch', '$epoch');"

now (3) expand templates, then (4)

#!/bin/bash

# Add local admin user
pushd /var/lib/gitea
su gitea -c "/bin/gitea/gitea admin create-user --name gitea --password 'gitea' --email gitea@havak.lan --admin --config /etc/gitea/app.ini"
popd

A little bit to much information, my curllpit is to do (4) run a script after expanding templates.

Any directions? :confused:


We Are NethServer - Community Overview - Sep 18
(Marc) #2

I’m not the best one to give advice on this but if using templates2expand:

order of implicit actions:
You should normally link your action scripts in the range S10 to S80 so that they occur after templates2expand and before services2adjust.
The generic_template_expand action is currently run at S05 and adjust-services is run at S90.

Or as part of an action script:


(Mark Verlinde) #3

Nevertheless you opened my eyes and solved it :grinning:

Stupid of me not to think about an expand-template in the bash-conf-script


(Mark Verlinde) #4

It’s still rough around the edges but we have a runner.

You can have a sneak preview on nethsever-arm 7.5.1804 (32 bit) :grinning:

yum --enablerepo=nethserver-testing install nethserver-gitea

local admin = gitea
password = gitea

nethersever (ldap) users must login in with the short username and their own password.

Known to work:

  • creating repos
  • cloning / pushing over https and ssh
  • creating tags (releases)
  • migrate (clone) directly from github

Still to DO:

  • mailer
  • backup

(Dan) #5

Do you have the .src.rpms available? It’d be interesting to see how hard it would be to port over to Intel architecture.


(Mark Verlinde) #6

It’s work in progress, still need to check if all dir permissions are as they should be.
It runs fine on x86_64 to

Also wip : mailer and backup…

So if you have remarks, happy to hear them from you!

EDIT:
srpm : https://nethserver.globalcortex.net/mirror/nethserver-arm/7.5.1804/testing/Source/


(Stéphane de Labrusse) #7

you have a typo at https://github.com/markVnl/gitea-SPEC/blob/master/gitea.service#L5


(Stéphane de Labrusse) #8

nice work mark :slight_smile:


(Mark Verlinde) #9

thank you, fixed on github.
updating arm repo as we speak!


(Dan) #10

Thanks. I tried changing the spec file to build 1.5.0. It built but didn’t run–tons of mysql errors in the logs. Time for some more digging, I expect.

I’d think, in app.ini, it would make sense to use the existing server cert/key rather than a separate one for gitea. You could template that, or just take advantage of the fact that the existing system (for some reason) copies the configured cert/chain to /etc/pki/tls/certs/localhost.crt and the key to /etc/pki/tls/private/localhost.key, and use those values in app.ini.


(Mark Verlinde) #11

Thanks Dan!

There is a lot of room for improvement setting up the data base. Its hacked now by coping a schema in. Also see https://discourse.gitea.io/t/install-provision-from-command-line/461

It’s a result of my interpretation off :

A certificate consumer daemon should expand those templates to its own certificate paths, by installing the proper configuration under /etc/e-smith/templates.metadata.
http://docs.nethserver.org/projects/nethserver-devel/en/latest/certificate_management.html

In fact it are copies of the default (NSRV) certificate/key updated every certificate-update event.


(Stéphane de Labrusse) #12

Just some direction for newer development. Gitea is done to host a gitlab like at home and I like the idea but what is done to save the git repositories. Of course if you host your sources you must be sure to not loose them with your data … It could be horrible.


(Mark Verlinde) #13

Sure, backup is very important. That’s the main reason why its work in progress.

Did not decide how to hook it up in the nethserver data back-up, mainly because I need to get familiar with it. I play with the idea to move the home- and repository directories to/var/lib/nethserver/gitea.

If i understand correctly it will be included in the data-backup by doing this, right?
then there is a third chunk of data: the mysql database, which is also included automatically in the data-backup, right?

The nice thing about gittea it uses default (linux) git; the repository data is exactly the same as you find in .git in your local clone. And all your project history / data is present in this .git folder , in a local clone you actually have all the project data 2 times.
Meaning you can recover a git repo / project without even running gitea: copy the content of the repo/project you want to recover to some_project/.git then git stash and git stash drop and your project is “rebuild”. This needs to be tested a lot :grinning:

Another way to make a backup-data set is gitea --dump it packs all data (workdir, repo-dir and database) in one zip. But do not like this because you can’t make incremental backups this way…


(Stéphane de Labrusse) #14

you described the full process, you need to do some actions now

  • home to /var/lib/nethserver/gitea
  • backup mysql by the backup-data
  • post-restore action with git stash and git stash drop for each repo or globally
  • restart if needed gitlea

(Mark Verlinde) #15

This is not needed, the repositories do not have to be rebuild for gitea.

This seem to be needed:
Do you have an example how to hook the restart in to a “post-restore-action”


(Stéphane de Labrusse) #16

check nethserver-mattermost, you have a good example of what you need, actions to backup postgresql and restart by the createlinks


(Mark Verlinde) #17

Found the issue:
The version record is not populated/written, gitea can not update/migrate the database to a new schema because it does not know the current version

Still working on a solution though

EDIT:
And there is a little bit more to it. :disappointed_relieved:
To accommodate special characters I setup the database with utf8mb4 character set; not realizing the mariadb version we are running limits the max field length for this encoding to 190 chars.


(Mark Verlinde) #18

Did some work on it:

Bumped the minimum version of Gitea to 1.5.0

Solved the database update issues for the future;
unfortunately there no update path from the former package.

Al the data and the (mysql) database are included in the default data-backup; (thanx @stephdl)
Did just one restore: it really needs more testing.

Enabled a basic mailer by default (sendmail) meaning if there is no mail-server installed email ends up in/var/mail/

Please note it is still work in progress, but coming close to an alpha state

cc/ @danb35


(Dan) #19

Cool, I’ll have to check it out under x64.

Edit: built gitea and nethserver-gitea and installed them on a clean Neth 7.5 test VM. I’m able to import a repo from github, clone it to a local machine via HTTPS, make/commit edits and push them back to gitea. Clone via SSH asks for a password for gitea, even though I’ve already put in an SSH public key for this user. Still playing, but it’s looking positive so far.

Edit 2: Think I might have identified the what’s keeping SSH from working. According to /etc/passwd, gitea's home directory is in /var/lib/gitea. But the gitea web GUI says the “root path” is /var/lib/nethserver/gitea/.ssh, and that’s where the user’s SSH public key was written. That would seem to explain why the system asked for a password when I tried to clone via SSH–it wasn’t looking for authorized_keys in the right place.


(Mark Verlinde) #20

I think you nailed it! :+1:
My bad, moved it to /var/lib/nethserver/gitea for easy back-up inclusion

I’ll gues we need to do usermod -m -d /var/lib/nethserver/gitea gitea

EDIT: Confirmed thanx @danb35
The usermod as described above fixes the issue (you need to stop / start gitea to do this). Although on existing installs it throws a warning the directory already exists.