A Study of Computational Reproducibility Using URLs Linking to Open Access Datasets and Software

Authors

Lamia Salsabil, Old Dominion UniversityFollow
Jian Wu, Old Dominion UniversityFollow
Muntabir Hasan Choudhury, Old Dominion UniversityFollow
William A. Ingram
Edward A. Fox
Sarah M. Rajtmajer
C. Lee Giles
Frédérique Laforest (Ed.)
Raphaël Troncy (Ed.)
Lionel Médini (Ed.)
Ivan Herman (Ed.)

Document Type

Conference Paper

Publication Date

2022

DOI

10.1145/3487553.3524658

Publication Title

WWW'22: Companion Proceedings of the Web Conference 2022

Pages

784-788

Conference Name

WWW'22: The ACM Web Conference 2022, April 25-29, 2022, Virtual Event, Lyon France

Abstract

Datasets and software packages are considered important resources that can be used for replicating computational experiments. With the advocacy of Open Science and the growing interest of investigating reproducibility of scientific claims, including URLs linking to publicly available datasets and software packages has become an institutionalized part of research publications. In this preliminary study, we investigated the disciplinary dependency and chronological trends of including open access datasets and software (OADS) in electronic theses and dissertations (ETDs), based on a hybrid classifier called OADSClassifier, consisting of a heuristic and a supervised learning model. The classifier achieves the best F1 of 0.92. We found that the inclusion of OADS-URLs exhibited a strong disciplinary dependence and the fraction of ETDs containing OADS-URLs has been gradually increasing over the past 20 years. We developed and share a ground truth corpus consisting of 500 manually labeled sentences containing URLs from scientific papers. The dataset and source code are available at https://github.com/lamps-lab/oadsclassifier.

Rights

© 2022 Copyright held by the owner/authors.

Link included in accordance with publisher policy.

Original Publication Citation

Salsabil, L., Wu, J., Choudhury, M. H., Ingram, W. A., Fox, E. A., Rajtmajer, S. M., & Giles, C. L. (2022). In Frédérique Laforest, Raphaël Troncy, Lionel Médini, & Ivan Herman (Eds.), WWW'22: Companion Proceedings of the Web Conference 2022 (pp. 784-788). Association for Computing Machinery. https://doi.org/10.1145/3487553.3524658

Repository Citation

Salsabil, L., Wu, J., Choudhury, M. H., Ingram, W. A., Fox, E. A., Rajtmajer, S. M., & Giles, C. L. (2022). In Frédérique Laforest, Raphaël Troncy, Lionel Médini, & Ivan Herman (Eds.), WWW'22: Companion Proceedings of the Web Conference 2022 (pp. 784-788). Association for Computing Machinery. https://doi.org/10.1145/3487553.3524658

ORCID

0000-0002-6162-2896 (Salsabil), 0000-0003-0173-4463 (Wu), 0000-0002-9318-8844 (Choudhury)