Sunet Drive - Continuous Testing at a Global Scale
Lightning talk
17/09/2023 04:55 - 17/09/2023 05:00 (HEE) (5 minutes)

Richard Freitag Project Manager

Richard Freitag is project leader at Sunet. A physicist by training, with a diploma from Humboldt-University Berlin, he transitioned to work in the automation industry for Siemens Corporate Research in Princeton, New Jersey. At a Munich-based SME, he led the product development for medical device integration, specialized on dental clinics. Later, at Uppsala University, Richard led the development of a data storage solution that became the template for Sunet Drive (Uppsala University Dataportal Allvis), and managed a portfolio of academic support systems, including BIOVIA Electronic Lab Notebooks.
Since 2020, he leads the development of Sunet Drive, a federated storage solution for research data currently used by 16 Swedish Universities. This includes collaboration and integration into the ScienceMesh, as well as leading efforts to port RDS (Research Data Services) to Sunet Drive/NextCloud, for the publication of data to public repositories.

Sunet Drive is a national file storage infrastructure for universities and research institutions in Sweden. It is based on a Nextcloud Global Scale setup and is comprised of 54 nodes, one prepared for each institution. This setup ensures data sovereignty while being part of a larger federation, including the ScienceMesh for international collaboration. The setup is duplicated in a test environment which is also used for staging and development. In total, around 300 virtual servers are deployed, and a setup using k8s is in preparation.

One step in ensuring the functional stability of such a large setup is rigorous and automated testing. This covers basic tests of the status.php sites, but also OCS tests for user lifecycle management and app-consistency, WebDAV tests, as well as more sophisticated Selenium testing for login, TOTP, Collabora for file editing and more. Automated testing currently generates around 25000 daily test points, ensuring the stability and consistency of the deployed solution. Scalability and load testing is done using a setup that can simulate the load from many users, and has been used to test for up to 6000 users.

The lightning talk will highlight the core-elements of the tests and test setup using Jenkins with multiple workers and X virtual framebuffer (Xvfb), how it supports regression testing especially after updates, and how they complement conventional infrastructure monitoring with solutions such as Thruk.