Document Type
Conference Paper
Publication Date
2018
DOI
10.17605/OSF.IO/EV42P
Publication Title
iPRES 2018 The 15th International Conference on Digital Preservation
Pages
10 pp.
Conference Name
iPRES 2018 The 15th International Conference on Digital Preservation, September 24-27, 2018, Boston, Massachusetts
Abstract
Web archives, a key area of digital preservation, meet the needs of journalists, social scientists, historians, and government organizations. The use cases for these groups often require that they guide the archiving process themselves, selecting their own original resources, or seeds, and creating their own web archive collections. We focus on the collections within Archive-It, a subscription service started by the Internet Archive in 2005 for the purpose of allowing organizations to create their own collections of archived web pages, or mementos. Understanding these collections could be done via their user-supplied metadata or via text analysis, but the metadata is applied inconsistently between collections and some Archive-It collections consist of hundreds of thousands of seeds, making it costly in terms of time to download each memento. Our work proposes using structural metadata as an additional way to understand these collections. We explore structural features currently existing in these collections that can unveil curation and crawling behaviors. We adapt the concept of the collection growth curve for understanding Archive-It collection curation and crawling behavior. Using the growth curves, we can see if most of the mementos in the collection are skewed earlier or later. We also introduce several seed features to describe the diversity and types of seeds present in an Archive-It collection. With these seed features, we come to an understanding of the diversity of resources that make up a collection and the depth of those resources within their seed websites, indicating whether the curator chose to preserve the top-level page or something more specific within a site. Finally, we use the descriptions of each collection to identify four semantic categories of Archive-It collections. Using the identified structural features, we reviewed the results of runs with 20 classifiers and are able to predict the semantic category of a collection using a Random Forest classifier with a weighted average F1 score of 0.720, thus bridging the structural to the descriptive. Our method is useful because it saves the researcher time and bandwidth. They do not need to download every resource in the collection in order to identify its semantic category. Identifying collections by their semantic category allows further downstream processing to be tailored to these categories.
Rights
© The Authors.
This is an open-access article published under the Creative Commons Attribution 4.0 International (CC BY 4.0) License.
Original Publication Citation
Jones, S., Nelson, M., Nwala, A., & Weigle, M. (2018). 205.3 The many shapes of Archive-It. https://doi.org/10.17605/OSF.IO/EV42P
Repository Citation
Jones, S., Nelson, M., Nwala, A., & Weigle, M. (2018). 205.3 The many shapes of Archive-It. https://doi.org/10.17605/OSF.IO/EV42P
ORCID
0000-0002-4372-870X (Jones), 0000-0003-3749-8116 (Nelson), 0000-0002-2787-7166 (Weigle)
Powerpoint Presentation