Reproducible graduate theses in GIScience

Basic data for this talk

Information about the event

Abstract

INTRODUCTION: Reproducible research practices and concepts introduced at early stages of education could have a broad impact on scientific practices. These skills are relevant today for early career researchers to build a career. We report on preliminary results of an initiative to enhance awareness about good reproducibility practices through self-study and self-evaluation in graduate thesis projects. OBJECTIVES: Expose master-level students to challenges of and methods for more reproducible research through two questionnaires and three self-study lessons to increase their understanding and the reproducibility of their theses. If students reflect on reproducible research practices, they could improve their methodology. METHODS: Participants answered the first questionnaire at the start of their thesis. They were provided with an introductory 5-minute video lecture to the initiative, a 20-minute video lecture on reproducibility, and three self-paced assignments to introduce methods and tools ( https://osf.io/j97zp/). Participants were invited to self-assess the level of reproducibility of their master theses and add a short statement as the last sentence of their abstract. The statement includes the levels for five criteria for reproducibility (Nüst et al. 2018). Finally, participants answered a second questionnaire after thesis submission. RESULTS: The first questionnaire showed that participants (20+) claimed to have previous knowledge on scholarly openness (open source/data/access) and repositories. Terms connected to reproducibility (mostly from the pragmatic School of Thought, see Fecher & Friesike (2014)) such as digital notebooks, reproducibility packages, containers, or computational essays, were in general poorly known. In the second questionnaire, the results showed an overall increase in understanding of all terms, especially notable for some terms related to reproducibility. We found that the self-assessment was generally too optimistic, suggesting a poor understanding of the different levels of the criteria. However, we observed an awareness of reproducibility in all theses that included the self-assessment. DISCUSSION: We report on the first iteration of a potential long-term study on the impact of MSc-level exposure to reproducibility. It should be investigated if the self-study and self-assessment can influence the degree of reproducibility compared to earlier years or other study programmes. While there is no evidence yet on the original claim, we want to elicit feedback from the broader reproducible research community on the approach. We also invite more GIScience study courses to participate, such as members of AGILE in Europe. In the long term, the academic careers of students may be followed to investigate the impact on their careers. References Fecher, B, and S Friesike. 2014. "Open Science: One Term, Five Schools of Thought." In Opening Science, 17-47. Springer, Cham. doi: 10.1007/978-3-319-00026-8_2. Nüst, D, C Granell, B Hofer, M Konkol, FO Ostermann, R Sileryte, and V Cerutti. 2018. "Reproducible Research and Giscience: An Evaluation Using Agile Conference Papers." PeerJ 6: e5072. doi: 10.7717/peerj.5072.

Speakers from the University of Münster

Nüst, Daniel

Professur für Geoinformatik (Prof. Pebesma)