QA/Testing Team basic HOWTO

qa
howto

#1

The document is moved here but you can still improve the document replying to this discussion

https://wiki.nethserver.org/doku.php?id=howto:qa_testing_team_basic_howto_ns7


QA Jam Session 2016_02
Squidguard's group based profile creation broke Squid and shared folders permissions
Failed to get machine PTY: No such file or directory
Nethserver community
Option to leave visible folder on the share samba
Call for a QA Jam Session
Suricata rules download error
Call for action from QA team 2016_05
Rc2 QA/testing campaign
So, what are you working on? - 29 May 17
Feedback on New OpenVPN tunnels
Spring QA Jam Session
NethServer Guides Index
(Alessio Fattorini) #2

That’s a great piece of work, @vcc, @Adam, @medworthy, @fasttech, @mabeleira, @GG_jr, do you have any suggestion?


(Jose "Martin" Abeleira. AKA Marto) #3

I think the how to is excellent, and very detailed. the only thing that we
might add is standard test database, or at least a list of tests performed
in order to have the same tests for each tester and on each version.


(Alessio Fattorini) #4

@dz00te Helpful verification template! Just used twice today :slight_smile: :clap: :clap: :clap:


#5

This week-end was a really rainy, and I worked a lot on the wiki.

I saw a little stuff in the user guides, and another stuff in the developer howtos.
This situation let me a strange feeling with the Testing Team…:unamused:

I would like to make few suggestions to valorize the testing team:

  • Rename the “Testing team” in “Quality Team”.
    These guys aren’t KAMIKAZES only, they are writing bug reports, they give return about working stuff too… They participe entirely in the NethServer quality.
    The team name must show this…
  • Reorganize the stuff in the wiki to make the Quality Team at the right valorized place, at the same level of User Guides and Developer HowTo in place to be hidden in userguides and developer howtos.
    This stuff must appear in the wiki left menu, and they must have their onw wiki page, where explain better their work, the bug report procedures, and all quality stuff in general. this part is really important too…

The Quality Team must organize it, and everybody have to participate :wink:


Failed to update to nextcloud 12.0.1 through Nextcloud updater
(Alessio Fattorini) #6

I behind this! @dz00te @mabeleira, @medworthy, @fasttech, @GG_jr, @Adam what do you think?

Totally agree, @dz00te would you like to lead this movement? :smile: You already wrote many helpful stuff!


#7

sorry for the delay (also on other thread about similar topic) but the last month was a little “strange” and and I hope it’s all behind… anyway, i must admit that writing docs is not my favorite sport :slight_smile: but while ns7.2 is in alpha i would like to write about some basic installation/base tests. i hope to have something new before the end of the week…
it’s ok for me the name change and the wiki change proposed by @Jim


#8

http://wiki.nethserver.org/doku.php?id=quality

Is already done.
We have to fill it…:wink:

Take a look at user guides and developer howto to have inspiration.


(Alessio Fattorini) #9

Welcome to the new Quality Team
http://community.nethserver.org/groups/quality_team/members


#10

some update on QA/testing with Github instead of Redmine
(hoping to have soon the time to update the wiki…)
for all the @quality_team and of course to all who want to join us in testing

development process:
http://docs.nethserver.org/projects/nethserver-devel/en/v7b/development_process.html

useful link on Github
open issues in testing: https://github.com/NethServer/dev/labels/testing

open issues (sort by recently updated): https://github.com/NethServer/dev/issues?q=is%3Aissue+is%3Aopen+sort%3Aupdated-desc

open issues: https://github.com/NethServer/dev/issues


(Alessio Fattorini) #11

Now that github is our official bug tracker, how can we adapt this doc?


#12

it’s only an adaption of the previous post for GitHub, there are more to change/enhance, but no more time for now… comments, hints, suggestions and corrections are welcomed


Nethserver Testing Team - Why join Testing Team
The main feature of Nethserver is the modular design, this means that there are many different setups as people has different interests. As member of Testing Team you could act as a contact person for a specific area to help developers fix the bugs you care about, and is the best way of making sure it does what you need it to!

some useful links on QA:
http://blog.smartbear.com/programming/14-ways-to-contribute-to-open-source-without-being-a-programming-genius-or-a-rock-star/
Nethserver Dev Manual:
http://docs.nethserver.org/projects/nethserver-devel/en/latest/development_process.html#qa-team-member-testing
or more in general to better understand the development process:
http://docs.nethserver.org/projects/nethserver-devel/en/latest/development_process.html

Who can join
Requirements for new members:

  • Verified at least an issue or one verified bug report
  • A GitHub account
    Anyone who is interested in improving the quality on his/her setup and is willing to communicate with developers and other testers.

How do you join
Simply send a PM to me (@dz00te) or to @alefattorini

How do you test:

GENERAL WORKFLOW

  1. send your GitHub accont name to QA Team Coordinator and @davidep to be enabled
  2. devs put issue on QA (blue “testing” label)
    https://github.com/NethServer/dev/issues
  3. select the issue you want to test and assign it to you (select “assign yourself”)
  4. compile template (follow after) with the status of the box, and the package/groups currently installed
  5. first, if possible (if it’s a bug) try to replicate the bug
  6. then try to install the new packages (during installation check /var/log/messages or with journalctl for strange error/fail messages)
  7. follow test cases (if no tests cases are present feel free to ask more info writing a comment to the issue)
  8. if you have some ideas for other tests cases, add them to list in template and test them
  9. repeat testing steps used when replicating the bug
  10. at every step control /var/log/messages or use journalctl
  11. write resulst on GitHub
  • if all is ok put the issue in verified state: select the “verified label” and deselect the “testing” label. Then write a comment with results of your test possibly using the template
  • otherwise comment the issue with the results of your tests and deselect the “testing” label
  • if in doubt use write a comment

VERIFICATION TEMPLATE for GitHub

*System and  Package Version installed*
...
Package Installed: ...
Other Package installed: ...

*Test Original Problem*
...

*Install Updated Package*
`yum --enablerepo=nethserver-testing update ... `


*Test Results after update*
...

*Verified Or Reopen*
...

*Note*
...

EXAMPLE of compiled verification Template

*System and Package Version installed*
[description of the test system (version, installation methods, upgrade history. etc).
Package Installed: [rpm -qa <package name>]
Other Package installed: [yum grouplist | sed  '1,/Installed/d;/Available/,$d']

*Test Original Problem*
[Reproduce bug if you can, showing steps taken]

*Install Updated Package*
[Update to new package, show steps taken]
` yum --enablerepo=nethserver-testing update ...`

*Test Results after update*
[Repeat steps carried out under TESTING above.]

*Verified Or Reopen*
[Problem fixed, then VERIFIED - not fixed, then leave a new Comment]

*Note*
[Documentation Impact]
[Other]
note: maybe to comply the point 4 of the workflow, it would be nice that when a new bug is opened it contains 
1. status of test box and list of packages/groups currently installed
2. howto replicate problem in detail

useful link on Github
open issues in testing: https://github.com/NethServer/dev/labels/testing

open issues (sort by recently updated): https://github.com/NethServer/dev/issues?q=is%3Aissue+is%3Aopen+sort%3Aupdated-desc


Add allowed ip range for phpmyadmin/urbackup/phpldapadmin
Call for a QA Jam Session - April 2017
OpenVPN net2net slave server won't connect to master server after reboot
Failed to get machine PTY: No such file or directory
(Alessio Fattorini) #13

@dz00te can I ask you to edit the first post and eventually move it on the wiki too?


(Davide Principi) #14

I want to note here that github now supports templates for replies and issues description!

I should study this feature: we could add @dz00te template to our tracker :wink:


(Alessio Fattorini) #15

Ping @dz00te I’d like to have a unique link to point out to newcomers that want help us


(Tony Boston) #16

Are bug reports for NS7 going into this NS6 Project really? Or do you have a bug tracker for NS7 too?
Also, the links are not up to date.


#17

please see post 12 in this thread

i’ll try to find time this weekend to reorganize infos here and on wiki, sorry


(Giacomo Sanchietti) #18

If a bug is found both on NS 6 and NS 7, we can track it on github and do a backport.
If the bug affects only NS 6, we should open it on dev.nethserver.org, otherwise if the bug affects only NS 7, we should open it on github.


(Alessio Fattorini) #19

We need you man :slight_smile: there are a lot of “quality” people around


#20

added on wiki the basic howto fo ns7 on github
https://wiki.nethserver.org/doku.php?id=howto:qa_testing_team_basic_howto_ns7
as always, comments, hints, suggestions and corrections are welcomed
tnx