add qa test plans

Signed-off-by: szaimen <szaimen@e.mail.de>

.github/ISSUE_TEMPLATE/Bug_report.md

@@ -17,9 +17,9 @@ labels: bug, 0. Needs triage
 2.
 3.
 
-### Expected behaviour <!--- Tell us what should happen -->
+### Expected behavior <!--- Tell us what should happen -->
 
-### Actual behaviour <!--- Tell us what happens instead -->
+### Actual behavior <!--- Tell us what happens instead -->
 
 
 ### Host OS <!--- (the host OS on which you are trying to install AIO on) -->

develop.md

@@ -14,6 +14,9 @@ nextcloud/all-in-one:develop
 And you are done :)
 It will now also select the developer channel for all other containers automatically.
 
+## How to publish new releases?
+Simply use https://github.com/nextcloud/all-in-one/issues/180 as template.
+
 ## How to build new containers
 
 Go to https://github.com/nextcloud-releases/all-in-one/actions/workflows/repo-sync.yml and run the workflow that will first sync the repo and then build new container that automatically get published to `develop` and `develop-arm64`.

tests/QA/001-initial-setup.md

@@ -0,0 +1,10 @@
+# Initial setup
+
+- [ ] Verify that after starting the test container, you can access the AIO interface using https://internal.ip.address:8080
+- [ ] After clicking the self-signed-certificate warning away, it should show the setup page with an explanation what AIO is and the initial password and a button that contains a link to the AIO login page
+- [ ] After copying the password and clicking on this button, it should open a new tab with the login page
+- [ ] The login page should show an input field that allows to enter the AIO password and a `Log in` button
+- [ ] After pasting the new password into the input field and clicking on this button button, you should be logged in
+- [ ] You should now see the containers page and you should see three sections: one general section which explains what AIO is, one `New AIO instance` section and one section that allows to restore the whole AIO instance from backup.
+
+You can now continue with [002-new-instance.md](./002-new-instance.md) or [010-restore-instance.md](./010-restore-instance.md).

tests/QA/002-new-instance.md

@@ -0,0 +1,31 @@
+# New instance
+
+For the below to work, it is important that you have a domain that you point onto your testserver and open port 443 in your router/firewall.
+
+- [ ] The `New AIO instance` section should show an input field that allows to enter a domain that will be used for Nextcloud later on as well as a short explanation regarding dynamic DNS
+- [ ] Now test a few examples in the input box:
+    - [ ] Entering `djfslkklk` should report that DNS config is not set or the domain is not in a valid format
+    - [ ] Entering `https://sdjflkjk.cpm` should report that this is not a valid domain
+    - [ ] Entering `10.0.0.1` should report that ip-addresses are not supported
+    - [ ] Entering `nextcloud.com` should report that the domain does not point to this server
+    - [ ] Entering the domain that does point to your server e.g. `yourdomain.com` should finally redirect you to the next screen (if you did not configure your domain yet or did not open port 443, it should report that to you)
+- [ ] Now you should see a button `Start containers` and an explanation which points out that clicking on the button will start the containers and that this can take a long time.
+- [ ] Below that you should see a section `Optional addons` which shows a checkbox list with addons that can be enabled or disabled.
+    - [ ] Collabora and Nextcloud Talk should be enabled, the rest disabled
+    - [ ] Unchecking/Checking any of these should insert a button that allows to save the set config
+    - [ ] Checking OnlyOffice and Collabora at the same time should show a warning that this is not supported and should  not saving the new config
+    - [ ] Recommended is to uncheck all options now
+    - [ ] Clicking on the save button should reload the page and activate the new config
+- [ ] Clickig on the `Start containers` button should finally reveal a big spinning wheel that should block all elements on the side of being clicked.
+- [ ] After waiting a few minutes, it should reload and show a new page
+    - [ ] On top of the page should be shown which channel you are running
+    - [ ] Below that, it should show that containers are currently starting
+    - [ ] Below that it should show a section with Containers: Apache, Database, Nextcloud and Redis and that your containers are up-to-date
+    - [ ] On the bottom should be the Optional addons section shown but with disabled checkboxes (not clickable)
+    - [ ] A automatic reload every 5s should happen until all Containers are started (as long as this window is focused)
+- [ ] After waiting a bit longer it should instead of the advice that your containers are currently running a details tag that is expandable with the initial Nextcloud credentials (username, password) and below that a button that allows to open the Nextcloud interface in a new tab
+- [ ] Clicking on that button should open the Nextcloud interface in a new tab and you should be able to log in using the provided credentials
+- [ ] Below the Containers section it should show a `Stop containers` button
+- [ ] Below the Containers section and above the Optional Addons section, you should see a Backup and restore section and an AIO password change section
+
+You can now continue with [003-automatic-login.md](./003-automatic-login.md).

tests/QA/003-automatic-login.md

@@ -0,0 +1,8 @@
+# Automatic login
+
+- [ ] After you log in to Nextcloud using the provided initial credentials, open https://yourdomain.com/settings/admin/overview
+- [ ] There you should see a Nextcloud AIO section and a button that allows to log into the AIO interface.
+- [ ] Clicking on this button should open the AIO interface in a new tab and should automatically log you in
+- [ ] All sessions in other tabs that are currently open should be closed (you can verify by reloading all other AIO tabs)
+
+You can now continue with [004-initial-backup.md](./004-initial-backup.md).

tests/QA/004-initial-backup.md

@@ -0,0 +1,17 @@
+# Initial backup
+
+- [ ] In the Backup and restore section, you should now see and input box where you should type in the path where the backup should get created and some explanation below
+    - [ ] Enter `/` which should send an error
+    - [ ] Enter `/mnt/` or  `/media/` or `/host_mnt/` or `/var/backups/` should send an error as well
+    - [ ] Accepted should be `/mnt/backup`, `/media/backup`, `/host_mnt/c/backup` and `/var/backups`.
+    - [ ] The side should now reload
+- [ ] In the Backup restore section you should now see a Backup information section with important info like the encryption password, the backup location and more.
+- [ ] Also you should see a Backup cretion section that contains a `Create backup` button.
+- [ ] Clicking on the `Create backup` button should open a window prompt that allows to cancel the operation.
+- [ ] Canceling should return to the website, confirming should reveal the big spinner again which should block the website again.
+- [ ] After a while you should see the information that Backup container is currently running
+- [ ] Below the Containers section you should see the option to `Start containers` again.
+- [ ] After a while and a few automatic reloads (as long as the side is focused), you should be redirected to the usual page and seen in the Backup and restore section that the last backup was successful.
+- [ ] Below thhat you should see a details tag that allows to reveal all backup options
+
+You can now continue with [020-backup-and-restore.md](.//020-backup-and-restore.md)

tests/QA/010-restore-instance.md

@@ -0,0 +1,19 @@
+# Restore instance
+
+For the below to work, you need a backup archive of an AIO instance and the location on the test machine and the password for the backup archive. You can get one here: [backup-archive](./assets/backup-archive/)
+
+- [ ] The section that allows to restore the whole AIO instance from backup should show two input fields: one that allows to enter a location where the backup archive is located and one that allows to enter password of the archive. It should also show a short explanation regarding the path requirements
+- [ ] Entering an incorrect path and/or password should let you continue and test your settings in the next step
+    - [ ] Clicking on the test button should after a reload bring you back to the initial screen where it should say that the test was unsuccessful.
+    - [ ] You should now see the input box again where you can change the path and password, confirm it and test the changes with a button on the bottom of the screen. 
+- [ ] Entering the correct path to the backup archive and the correct password here (in the first try) should:
+    - [ ] Should reload and should hide all options except the option to test the path and password
+    - [ ] After the test you should see the options to check the integrity of the backup and a list of backup archives that you can choose from to restore your instance
+    - [ ] Clicking on either option should show a window prompt that lets you cancel the operation
+    - [ ] Clicking on the integrity check option should check the integrity and report that the backup integrity is good after a while which should then only show the option to choose the backup archive that should be restored
+    - [ ] Choosing the restore option should finally restore your files. 
+    - [ ] After waiting a while it should reload the page and should show the usual container interface again with the state of your containers (stopped) and the option to start and update the containers again.
+- [ ] Clicking on `Start and update containers` should show a window prompt that you should create a backup. Canceling should cancel the operation, confirming should reveal the big spinner again.
+- [ ] After waiting a bit, all containers should be green and your instance should be fully functional again
+
+You can now continue with [020-backup-and-restore.md](./020-backup-and-restore.md)

tests/QA/020-backup-and-restore.md

@@ -0,0 +1,11 @@
+# Backup and restore
+
+- [ ] Expanding all backup options in the Backup and restore sectioin should reveal a Backup information section, Backup creation section, Backup check section, Backup restore section and Daily backup creation section.
+- [ ] The backup restore section should list all available backup archives and list them from most recent to least recent.
+- [ ] Clicking on either option of Create backup, Check backup integrity or Restore selected backup should run the corresponding action and report after a while in the last check, backup or restore was successful.
+- [ ] Daily backup creatio should allow to enter a time in 24h format e.g. `04:00` should be accepted, `24:00` or `dfjlk` not.
+- [ ] Submitting a time here that is only a minute away should reload the page and reveal at the same place the option to delete the setting again.
+- [ ] When the time of the automatic backup has come, it should automatically log you out (you can verify by reloading) and after you log in again you should see that the automatic backup is currently running.
+- [ ] After a while you should see that your container are starting and in the Backup and restore section you should see that the backup was successful
+
+You can now continue with [030-aio-password-change.md](./030-aio-password-change.md)

tests/QA/030-aio-password-change.md

@@ -0,0 +1,12 @@
+# AIO password change
+
+- [ ] In the AIO password change section you should see two input fields. And below the requirements for a new password
+- [ ] When entering nothing it should report that you need to enter your current aio password
+- [ ] When entering a false password, it should report that to you
+- [ ] After entering your current password and leaving the new password empty it should report that you need to enter a new password
+- [ ] After entering a new passwort shorter than 24 characters or not allowed characters, it should report that the password requirements are not met.
+- [ ] `sdfjlksj` should not be accepted
+- [ ] `jdsfklöjiroewoäsadjkfölk` should not be accepted
+- [ ] `sdjlfj SDJFLK 32489 sdjklf` should which should reload the page
+
+You can now continue with [040-login-behavior.md](./040-login-behavior.md)

tests/QA/040-login-behavior.md

@@ -0,0 +1,7 @@
+# Login behavior
+
+- [ ] When opening the AIO interface in a new tab while the apache container is running, it should report on the login page that Nextcloud is running and you should use the automatic login
+- [ ] When the apache container is stopped, you should see here an input field that allows you to enter the AIO password which should log you in
+- [ ] Starting and stopping the containers multiple times should every time produce a new token that is used in the admin overview in Nextcloud as link in the button to log you into the AIO interface. (see [003-automatic-login.md](./003-automatic-login.md))
+
+You can now continue with [050-optional-addons.md](./050-optional-addons.md)

tests/QA/050-optional-addons.md

@@ -0,0 +1,10 @@
+# Optional addons
+
+- [ ] At the bottom of the page in the AIO interface, you should see the optional addons section
+- [ ] You should be able to change optional addons when containers are stopped and not change them when containers are running
+- [ ] Enabling either of the options should start a new container with the same or comparable name and should also list them in the containers section
+- [ ] After all containers are started with the new config active, you should verify that the options were automatically activated/deactivated.
+    - [ ] ClamAV by trying to upload a testvirus to Nextcloud https://www.eicar.org/?page_id=3950
+    - [ ] Collabora by trying to open a .docx or .odt file in Nextcloud
+    - [ ] Nextcloud Talk by opening the Talk app in Nextcloud, creating a new chat and trying to join a call in this chat. Also verifying in the settings that the HPB and turn server work.
+    - [ ] Onlyoffice by trying to open a .docx file in Nextcloud

tests/QA/assets/backup-archive/borg/.gitattributes

@@ -0,0 +1 @@
+data/** filter=lfs diff=lfs merge=lfs -text

tests/QA/assets/backup-archive/borg/README

@@ -0,0 +1,2 @@
+This is a Borg Backup repository.
+See https://borgbackup.readthedocs.io/

tests/QA/assets/backup-archive/borg/config

@@ -0,0 +1,20 @@
+[repository]
+version = 1
+segments_per_dir = 1000
+max_segment_size = 524288000
+append_only = 0
+storage_quota = 0
+additional_free_space = 2G
+id = 341f36bcb3656a000339ea3fb75f9e1645bbad6ee5299b38a9fd9d9e9412125b
+key = hqlhbGdvcml0aG2mc2hhMjU2pGRhdGHaAZ4HfIjdJek4CCc15EJdZ5IPdhMgSKu1RDimS1
+	C6Xj/zZtbmLXp9wTzDJzVq2DAhM9Vl2wZo766gyywzz5oUuVocbIlLD2JNEHsWL08W8QYC
+	mEjufkvRPCgupuz/ZOjfJmtKzKm56t1K4jUf0oaYtt3pKlEY7RQStdYTq43JNYFI23bNH4
+	u+OnGDEYa/Y4XZkl6K4yDZgc1jpDf5uHzNTX1MiJpCWJlBQbJQC03i1bVqBlZpAo+x23Z3
+	4Hg5Is5n24f2+AhFtdeD4NY6CzoHa+HQR7K67ZGIA6hUOHo7xPKaKKBlBvORgWppItq7j6
+	Ch+K1Z+/N1MBn4ltCSNtTmsCe2QT+4Xz7je7DzUNhmt3yx9Kuxe/e8NBSXXRb3eMuyGovT
+	Cs8JCWwQStDMgt+4s88wwaTfbt93f5d9W65tw+Z7x0gNUz0bZAzM3wld8vFqk3Vc2EQ5t0
+	azBuOzCFTL7sdyNqns+sgIs83sEfpyi872+IRX+bMBrXPwGL+J1+yaLvTuShjuq3Wv1Q9l
+	wqFMDUMB/ELSPYWS8Bj+Lix0dqmkaGFzaNoAINdDmXejBcvEcHa7qq0m3IzSyjhSDwpjgg
+	i7o64njD73qml0ZXJhdGlvbnPOAAGGoKRzYWx02gAgAYrIc2bmhIDmLmiQi2oFFrmQ3TlH
+	AW0UIpa4AtxkdXindmVyc2lvbgE=
+

tests/QA/assets/backup-archive/borg/data/0/1

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0a2e55f51c5d5125dfd8d4677af5c80d5d366b65ff70838d7ced6ef3bc152361
+size 17

tests/QA/assets/backup-archive/borg/data/0/2

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:565329299be8c85dc70d1231f365d210829785b67ff5edefee8ec37db324ad86
+size 217476296

tests/QA/assets/backup-archive/borg/data/0/3

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0a2e55f51c5d5125dfd8d4677af5c80d5d366b65ff70838d7ced6ef3bc152361
+size 17

tests/QA/assets/backup-archive/borg/data/0/4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0a2e55f51c5d5125dfd8d4677af5c80d5d366b65ff70838d7ced6ef3bc152361
+size 17

tests/QA/assets/backup-archive/borg/hints.4

@@ -0,0 +1 @@
+„§version¨segments<74>ÍR§compact<63>)±storage_quota_useÎöl—

tests/QA/assets/backup-archive/borg/nonce

@@ -0,0 +1 @@
+0000000020000082

tests/QA/assets/backup-archive/readme.md

@@ -0,0 +1,6 @@
+# Backup archive
+
+This is a backup archive only for testing.
+
+You need to copy the archive onto your test machine.
+The password is `65e5feb92b9765f7de003151b59fd1fc46f5bdc1f16c0c48`.

tests/QA/readme.md

@@ -0,0 +1,7 @@
+# QA test plans
+
+In this folder are manual test plans for QA located that allow to manually step through certain features and make sure that everything works as expected.
+
+For a test instance, you should make sure that all potentially breaking changes are merged, build new containers by following https://github.com/nextcloud/all-in-one/blob/main/develop.md#how-to-build-new-containers, stop a potential old instance, remove it and delete all volumes. Afterwards start a new clean test instance by following https://github.com/nextcloud/all-in-one/blob/main/develop.md#developer-channel.
+
+Best is to start testing with [001-initial-setup.md](./001-initial-setup.md).