Considering the Development Workflow to Achieve Reproducibility with Variation

Michael Mercier, Adrien Faure, Olivier Richard

SC 2018 - Workshop: ResCuE-HPC

pdf url

Abstract

The ability to reproduce an experiment is fundamen-tal in computer science. Existing approaches focus onrepeatability, but this is only the first step to repro-ducibility: Continuing a scientific work from a previ-ous experiment requires to be able to modify it. Thisability is called reproducibility with Variation.In this contribution, we show that capturing theenvironment of execution is necessary but not suffi-cient; we also need the environment of development.The variation also implies that those environmentsare subject to evolution, so the whole software devel-opment lifecycle needs to be considered. To take intoaccount these evolutions, software environments needto be clearly defined, reconstructible with variation,and easy to share. We propose to leverage functionalpackage managers to achieve this goal.

Bibtex

@inproceedings{mercier:hal-01891084,
  TITLE = {Considering the Development Workflow to Achieve Reproducibility with Variation},
  AUTHOR = {Mercier Michael and Faure Adrien and Richard Olivier},
  URL = {https://hal.inria.fr/hal-01891084},
  BOOKTITLE = {SC 2018 - Workshop: ResCuE-HPC},
  ADDRESS = {Dallas, United States},
  PAGES = {1-5},
  YEAR = {2018},
  MONTH = {Nov},
  PDF = {https://hal.inria.fr/hal-01891084/file/ws_rescue102s1-file1.pdf},
  HAL_ID = {hal-01891084},
  HAL_VERSION = {v1},
  abstract = {The ability to reproduce an experiment is fundamen-tal in computer science. Existing approaches focus onrepeatability, but this is only the first step to repro-ducibility: Continuing a scientific work from a previ-ous experiment requires to be able to modify it. Thisability is called reproducibility with Variation.In this contribution, we show that capturing theenvironment of execution is necessary but not suffi-cient; we also need the environment of development.The variation also implies that those environmentsare subject to evolution, so the whole software devel-opment lifecycle needs to be considered. To take intoaccount these evolutions, software environments needto be clearly defined, reconstructible with variation,and easy to share. We propose to leverage functionalpackage managers to achieve this goal.},
}