Reading list
-
Research software engineering
Good enough practices in scientific computing
Abstract
Computers are now essential in all branches of science, but most researchers are never taught the equivalent of basic lab skills for research computing. As a result, data can get lost, analyses can take much longer than necessary, and researchers are limited in how effectively they can work with software and data. Computing workflows need to follow the same practices as lab projects and notebooks, with organized data, documented steps, and the project structured for reproducibility, but researchers new to computing often don't know where to start. This paper presents a set of good computing practices that every researcher can adopt, regardless of their current level of computational skill. These practices, which encompass data management, programming, collaborating with colleagues, organizing projects, tracking work, and writing manuscripts, are drawn from a wide variety of published sources from our daily lives and from our work with volunteer organizations that have delivered workshops to over 11,000 people since 2010.
BibTeX
@article{wilson2017, author = {Wilson, G. and others}, title = {Good enough practices in scientific computing}, journal = {PLOS Computational Biology}, year = {2017}, volume = {13}, number = {6}, doi = {10.1371/journal.pcbi.1005510}, keywords = {Research software engineering}, abstract = {Computers are now essential in all branches of science, but most researchers are never taught the equivalent of basic lab skills for research computing. As a result, data can get lost, analyses can take much longer than necessary, and researchers are limited in how effectively they can work with software and data. Computing workflows need to follow the same practices as lab projects and notebooks, with organized data, documented steps, and the project structured for reproducibility, but researchers new to computing often don't know where to start. This paper presents a set of good computing practices that every researcher can adopt, regardless of their current level of computational skill. These practices, which encompass data management, programming, collaborating with colleagues, organizing projects, tracking work, and writing manuscripts, are drawn from a wide variety of published sources from our daily lives and from our work with volunteer organizations that have delivered workshops to over 11,000 people since 2010.} }Citation
Wilson, G., et al. (2017). Good enough practices in scientific computing. PLOS Computational Biology, 13(6). https://doi.org/10.1371/journal.pcbi.1005510
-
Research software engineering
Ten simple rules for making research software more robust
Abstract
Software produced for research, published and otherwise, suffers from a number of common problems that make it difficult or impossible to run outside the original institution or even off the primary developer's computer. We present ten simple rules to make such software robust enough to be run by anyone, anywhere, and thereby delight your users and collaborators.
BibTeX
@article{taschuk2017, author = {Taschuk, M. and Wilson, G.}, title = {Ten simple rules for making research software more robust}, journal = {PLOS Computational Biology}, year = {2017}, volume = {13}, number = {4}, doi = {10.1371/journal.pcbi.1005412}, keywords = {Research software engineering}, abstract = {Software produced for research, published and otherwise, suffers from a number of common problems that make it difficult or impossible to run outside the original institution or even off the primary developer's computer. We present ten simple rules to make such software robust enough to be run by anyone, anywhere, and thereby delight your users and collaborators.} }Citation
Taschuk, M., & Wilson, G. (2017). Ten simple rules for making research software more robust. PLOS Computational Biology, 13(4). https://doi.org/10.1371/journal.pcbi.1005412
-
Research software engineering
Ten simple rules for helping newcomers become contributors to open projects
Abstract
To survive and thrive, a community must attract new members, retain them, and help them be productive. As openness becomes the norm in research, software development, and education, knowing how to do this has become an essential skill for principal investigators and managers alike. A growing body of knowledge in sociology, anthropology, education, and software engineering can guide decisions about how to facilitate this. What exactly do we mean by community? In the case of open source and open science, the most usual meaning is a community of practice. As defined by Lave and Wenger, groups as diverse as knitting circles, oncology researchers, and web designers share three key characteristics: Participants have a common product or purpose that they work on or toward. They are mutually engaged, i.e., they assist and mentor each another. They develop shared resources and domain knowledge. Brown specializes this to define a community of effort as ...a community formed in pursuit of a common goal. The goal can be definite or indefinite in time, and may not be clearly defined, but it is something that (generally speaking) the community is aligned on. People working to preserve coral reefs in the face of global climate change are an example of such a community. No central organization coordinates their work, but the scientists who study coral reefs, the environmentalists who work to protect them, and the citizens who support them financially and politically are aware of each other’s efforts, collaborate in ad hoc ways, and are conscious of contributing toward a shared purpose. Open-source software projects are also communities of effort. E.g., the Mozilla Firefox includes a mix of paid professionals, highly involved volunteers, and occasional contributors who not only create software, documentation, and tutorials but also organize events, answer questions in online forums, mentor newcomers, and advocate for open standards. Every community of effort has unique features, but they have enough in common to profit from one another’s experience. The 10 rules laid out below are based on studies of such communities and on the authors’ experience as members, leaders, and observers. Our focus is on small and medium-sized projects, i.e., ones that have a handful of to a few hundred participants and are a few months to a few years old but may not (yet) have any formal legal standing, such as incorporation as a nonprofit.
BibTeX
@article{sholler2019, author = {Sholler, D. and others}, title = {Ten simple rules for helping newcomers become contributors to open projects}, journal = {PLOS Computational Biology}, year = {2019}, volume = {15}, number = {9}, doi = {10.1371/journal.pcbi.1007296}, keywords = {Research software engineering}, abstract = {To survive and thrive, a community must attract new members, retain them, and help them be productive. As openness becomes the norm in research, software development, and education, knowing how to do this has become an essential skill for principal investigators and managers alike. A growing body of knowledge in sociology, anthropology, education, and software engineering can guide decisions about how to facilitate this. What exactly do we mean by community? In the case of open source and open science, the most usual meaning is a community of practice. As defined by Lave and Wenger, groups as diverse as knitting circles, oncology researchers, and web designers share three key characteristics: Participants have a common product or purpose that they work on or toward. They are mutually engaged, i.e., they assist and mentor each another. They develop shared resources and domain knowledge. Brown specializes this to define a community of effort as ...a community formed in pursuit of a common goal. The goal can be definite or indefinite in time, and may not be clearly defined, but it is something that (generally speaking) the community is aligned on. People working to preserve coral reefs in the face of global climate change are an example of such a community. No central organization coordinates their work, but the scientists who study coral reefs, the environmentalists who work to protect them, and the citizens who support them financially and politically are aware of each other’s efforts, collaborate in ad hoc ways, and are conscious of contributing toward a shared purpose. Open-source software projects are also communities of effort. E.g., the Mozilla Firefox includes a mix of paid professionals, highly involved volunteers, and occasional contributors who not only create software, documentation, and tutorials but also organize events, answer questions in online forums, mentor newcomers, and advocate for open standards. Every community of effort has unique features, but they have enough in common to profit from one another’s experience. The 10 rules laid out below are based on studies of such communities and on the authors’ experience as members, leaders, and observers. Our focus is on small and medium-sized projects, i.e., ones that have a handful of to a few hundred participants and are a few months to a few years old but may not (yet) have any formal legal standing, such as incorporation as a nonprofit.} }Citation
Sholler, D., et al. (2019). Ten simple rules for helping newcomers become contributors to open projects. PLOS Computational Biology, 15(9). https://doi.org/10.1371/journal.pcbi.1007296
-
Research software engineering
Why modern open source projects fail
Abstract
Open source is experiencing a renaissance period, due to the appearance of modern platforms and workflows for developing and maintaining public code. As a result, developers are creating open source software at speeds never seen before. Consequently, these projects are also facing unprecedented mortality rates. To better understand the reasons for the failure of modern open source projects, this paper describes the results of a survey with the maintainers of 104 popular GitHub systems that have been deprecated. We provide a set of nine reasons for the failure of these open source projects. We also show that some maintenance practices---specifically the adoption of contributing guidelines and continuous integration---have an important association with a project failure or success. Finally, we discuss and reveal the principal strategies developers have tried to overcome the failure of the studied projects.
BibTeX
@inproceedings{coelho2017, author = {Coelho, J. and Valente, M. T.}, title = {Why modern open source projects fail}, booktitle = {Proceedings of ESEC/FSE 2017}, publisher = {ACM}, year = {2017}, doi = {10.1145/3106237.3106246}, keywords = {Research software engineering}, abstract = {Open source is experiencing a renaissance period, due to the appearance of modern platforms and workflows for developing and maintaining public code. As a result, developers are creating open source software at speeds never seen before. Consequently, these projects are also facing unprecedented mortality rates. To better understand the reasons for the failure of modern open source projects, this paper describes the results of a survey with the maintainers of 104 popular GitHub systems that have been deprecated. We provide a set of nine reasons for the failure of these open source projects. We also show that some maintenance practices---specifically the adoption of contributing guidelines and continuous integration---have an important association with a project failure or success. Finally, we discuss and reveal the principal strategies developers have tried to overcome the failure of the studied projects.} }Citation
Coelho, J., & Valente, M. T. (2017). Why modern open source projects fail. In Proceedings of ESEC/FSE 2017. ACM. https://doi.org/10.1145/3106237.3106246
-
Archaeological software development
Open Archaeology, Open Source? Collaborative practices in an emerging community of archaeological software engineers
Abstract
In this article, we investigate modes of collaboration in this emerging community of practice using 'open-archaeo', a curated list of archaeological software, and data on the activity of associated GitHub repositories and users. We conduct an exploratory quantitative analysis to characterise the nature and intensity of these collaborations and map the collaborative networks that emerge from them. We document uneven adoption of open source collaborative practices beyond the basic use of git as a version control system and GitHub to host source code. Most projects do make use of collaborative features and, through shared contributions, we can trace a collaborative network that includes the majority of archaeologists active on GitHub. However, a majority of repositories have 1-3 contributors, with only a few projects distinguished by an active and diverse developer base. Direct collaboration on code or other repository content - as opposed to the more passive, social media-style interaction that GitHub supports – remains very limited. In other words, there is little evidence that archaeologists' adoption of open-source tools (git and GitHub) has been accompanied by the decentralised, participatory forms of collaboration that characterise other open-source communities. On the contrary, our results indicate that research software engineering in archaeology remains largely embedded in conventional professional norms and organisational structures of academia.
BibTeX
@article{batist2024, author = {Batist, Z. and Roe, J.}, title = {Open Archaeology, Open Source? Collaborative practices in an emerging community of archaeological software engineers}, journal = {Internet Archaeology}, year = {2024}, volume = {67}, doi = {10.11141/ia.67.13}, keywords = {Archaeological software development}, abstract = {In this article, we investigate modes of collaboration in this emerging community of practice using 'open-archaeo', a curated list of archaeological software, and data on the activity of associated GitHub repositories and users. We conduct an exploratory quantitative analysis to characterise the nature and intensity of these collaborations and map the collaborative networks that emerge from them. We document uneven adoption of open source collaborative practices beyond the basic use of git as a version control system and GitHub to host source code. Most projects do make use of collaborative features and, through shared contributions, we can trace a collaborative network that includes the majority of archaeologists active on GitHub. However, a majority of repositories have 1-3 contributors, with only a few projects distinguished by an active and diverse developer base. Direct collaboration on code or other repository content - as opposed to the more passive, social media-style interaction that GitHub supports – remains very limited. In other words, there is little evidence that archaeologists' adoption of open-source tools (git and GitHub) has been accompanied by the decentralised, participatory forms of collaboration that characterise other open-source communities. On the contrary, our results indicate that research software engineering in archaeology remains largely embedded in conventional professional norms and organisational structures of academia.} }Citation
Batist, Z., & Roe, J. (2024). Open Archaeology, Open Source? Collaborative practices in an emerging community of archaeological software engineers. Internet Archaeology, 67. https://doi.org/10.11141/ia.67.13
-
Archaeological software development
Recommendations for the review of archaeological research software
Abstract
Motivated by numerous discussions around the increasing use of research software in the field of archaeology, this article outlines some aspects for its review. The evaluation of software is a complex topic, since its field of application and development context has a considerable influence. In addition, there are many very different use cases, ranging from students wanting a quick solution for an exercise to project developers, who have to integrate a software package into an existing infrastructure for continuous operation. Although this discussion paper is based on equal contributions from archaeology and applied computer science, the focus is on evaluation criteria for software in the field of archaeology. A major goal of this paper is to alert future reviewers to the complexity of software evaluation. A software review should enable a professional archaeological audience to make a quick, critical and fair, to both the product and the developers assessment of the software. Priority recommendations include a description of the context in which the review was written and the requirements for specific use cases. In addition, a short tabular overview should enable a quick assessment of the technical, financial and legal aspects. The need for future adaptations of this guideline was identified at the outset since both software development and its evaluation in the digital age are expected to remain very dynamic.
BibTeX
@article{homburg2021, author = {Homburg, T. and others}, title = {Recommendations for the review of archaeological research software}, journal = {Archäologische Informationen}, year = {2021}, doi = {10.11588/AI.2020.1.81423}, keywords = {Archaeological software development}, abstract = {Motivated by numerous discussions around the increasing use of research software in the field of archaeology, this article outlines some aspects for its review. The evaluation of software is a complex topic, since its field of application and development context has a considerable influence. In addition, there are many very different use cases, ranging from students wanting a quick solution for an exercise to project developers, who have to integrate a software package into an existing infrastructure for continuous operation. Although this discussion paper is based on equal contributions from archaeology and applied computer science, the focus is on evaluation criteria for software in the field of archaeology. A major goal of this paper is to alert future reviewers to the complexity of software evaluation. A software review should enable a professional archaeological audience to make a quick, critical and fair, to both the product and the developers assessment of the software. Priority recommendations include a description of the context in which the review was written and the requirements for specific use cases. In addition, a short tabular overview should enable a quick assessment of the technical, financial and legal aspects. The need for future adaptations of this guideline was identified at the outset since both software development and its evaluation in the digital age are expected to remain very dynamic.} }Citation
Homburg, T., et al. (2021). Recommendations for the review of archaeological research software. Archäologische Informationen. https://doi.org/10.11588/AI.2020.1.81423
-
Archaeological software development
Tool-driven revolutions in archaeological science
Abstract
There is an argument in philosophy of science that revolutions in science are either idea-driven or tool-driven. We explore this debate in light of recent efforts by many scientific disciplines to embrace methods to improve the reproducibility of their research. One of the most profound changes driven by this concern for reproducibility and transparency is from analysing data using tools dependent on point-and-clicking with a mouse in closed source software, to tools based on writing scripts in open source programming languages and making them openly available. We present bibliometric evidence for this change in ecology and in archaeology to test if the adoption of these new tools is revolutionary or transformational. We identify a positive citation effect for papers that use the open source programming language R. We discuss how computational approaches to improving reproducibility and transparency in archaeology are mediated and transformed by the use of R code.
BibTeX
@article{schmidt2020, author = {Schmidt, S. C. and Marwick, B.}, title = {Tool-driven revolutions in archaeological science}, journal = {Journal of Computer Applications in Archaeology}, year = {2020}, volume = {3}, number = {1}, doi = {10.5334/jcaa.29}, keywords = {Archaeological software development}, abstract = {There is an argument in philosophy of science that revolutions in science are either idea-driven or tool-driven. We explore this debate in light of recent efforts by many scientific disciplines to embrace methods to improve the reproducibility of their research. One of the most profound changes driven by this concern for reproducibility and transparency is from analysing data using tools dependent on point-and-clicking with a mouse in closed source software, to tools based on writing scripts in open source programming languages and making them openly available. We present bibliometric evidence for this change in ecology and in archaeology to test if the adoption of these new tools is revolutionary or transformational. We identify a positive citation effect for papers that use the open source programming language R. We discuss how computational approaches to improving reproducibility and transparency in archaeology are mediated and transformed by the use of R code.} }Citation
Schmidt, S. C., & Marwick, B. (2020). Tool-driven revolutions in archaeological science. Journal of Computer Applications in Archaeology, 3(1). https://doi.org/10.5334/jcaa.29
-
Culture and history of FOSS
Two bits: The cultural significance of free software
Abstract
In Two Bits, Christopher M. Kelty investigates the history and cultural significance of Free Software, revealing the people and practices that have transformed not only software but also music, film, science, and education. Free Software is a set of practices devoted to the collaborative creation of software source code that is made openly and freely available through an unconventional use of copyright law. Kelty explains how these specific practices have reoriented the relations of power around the creation, dissemination, and authorization of all kinds of knowledge. He also makes an important contribution to discussions of public spheres and social imaginaries by demonstrating how Free Software is a “recursive public”—a public organized around the ability to build, modify, and maintain the very infrastructure that gives it life in the first place. Drawing on ethnographic research that took him from an Internet healthcare start-up company in Boston to media labs in Berlin to young entrepreneurs in Bangalore, Kelty describes the technologies and the moral vision that bind together hackers, geeks, lawyers, and other Free Software advocates. In each case, he shows how their practices and way of life include not only the sharing of software source code but also ways of conceptualizing openness, writing copyright licenses, coordinating collaboration, and proselytizing. By exploring in detail how these practices came together as the Free Software movement from the 1970s to the 1990s, Kelty also considers how it is possible to understand the new movements emerging from Free Software: projects such as Creative Commons, a nonprofit organization that creates copyright licenses, and Connexions, a project to create an online scholarly textbook commons.
BibTeX
@book{kelty2008, author = {Kelty, C. M.}, title = {Two bits: The cultural significance of free software}, publisher = {Duke University Press}, year = {2008}, url = {https://www.dukeupress.edu/two-bits}, keywords = {Culture and history of FOSS}, abstract = {In Two Bits, Christopher M. Kelty investigates the history and cultural significance of Free Software, revealing the people and practices that have transformed not only software but also music, film, science, and education. Free Software is a set of practices devoted to the collaborative creation of software source code that is made openly and freely available through an unconventional use of copyright law. Kelty explains how these specific practices have reoriented the relations of power around the creation, dissemination, and authorization of all kinds of knowledge. He also makes an important contribution to discussions of public spheres and social imaginaries by demonstrating how Free Software is a “recursive public”—a public organized around the ability to build, modify, and maintain the very infrastructure that gives it life in the first place. Drawing on ethnographic research that took him from an Internet healthcare start-up company in Boston to media labs in Berlin to young entrepreneurs in Bangalore, Kelty describes the technologies and the moral vision that bind together hackers, geeks, lawyers, and other Free Software advocates. In each case, he shows how their practices and way of life include not only the sharing of software source code but also ways of conceptualizing openness, writing copyright licenses, coordinating collaboration, and proselytizing. By exploring in detail how these practices came together as the Free Software movement from the 1970s to the 1990s, Kelty also considers how it is possible to understand the new movements emerging from Free Software: projects such as Creative Commons, a nonprofit organization that creates copyright licenses, and Connexions, a project to create an online scholarly textbook commons.} }Citation
Kelty, C. M. (2008). Two bits: The cultural significance of free software. Duke University Press. https://www.dukeupress.edu/two-bits
-
Culture and history of FOSS
Coding freedom: The ethics and aesthetics of hacking
Abstract
Who are computer hackers? What is free software? And what does the emergence of a community dedicated to the production of free and open source software--and to hacking as a technical, aesthetic, and moral project--reveal about the values of contemporary liberalism? Exploring the rise and political significance of the free and open source software movement in the United States and Europe, Coding Freedom details the ethics behind hackers' devotion to F/OSS, the social codes that guide its production, and the political struggles through which hackers question the scope and direction of copyright and patent law. In telling the story of the F/OSS movement, the book unfolds a broader narrative involving computing, the politics of access, and intellectual property. E. Gabriella Coleman tracks the ways in which hackers collaborate and examines passionate manifestos, hacker humor, free software project governance, and festive hacker conferences. Looking at the ways that hackers sustain their productive freedom, Coleman shows that these activists, driven by a commitment to their work, reformulate key ideals including free speech, transparency, and meritocracy, and refuse restrictive intellectual protections. Coleman demonstrates how hacking, so often marginalized or misunderstood, sheds light on the continuing relevance of liberalism in online collaboration.
BibTeX
@book{coleman2012, author = {Coleman, E. G.}, title = {Coding freedom: The ethics and aesthetics of hacking}, publisher = {Princeton University Press}, year = {2012}, doi = {10.1515/9781400845293}, keywords = {Culture and history of FOSS}, abstract = {Who are computer hackers? What is free software? And what does the emergence of a community dedicated to the production of free and open source software--and to hacking as a technical, aesthetic, and moral project--reveal about the values of contemporary liberalism? Exploring the rise and political significance of the free and open source software movement in the United States and Europe, Coding Freedom details the ethics behind hackers' devotion to F/OSS, the social codes that guide its production, and the political struggles through which hackers question the scope and direction of copyright and patent law. In telling the story of the F/OSS movement, the book unfolds a broader narrative involving computing, the politics of access, and intellectual property. E. Gabriella Coleman tracks the ways in which hackers collaborate and examines passionate manifestos, hacker humor, free software project governance, and festive hacker conferences. Looking at the ways that hackers sustain their productive freedom, Coleman shows that these activists, driven by a commitment to their work, reformulate key ideals including free speech, transparency, and meritocracy, and refuse restrictive intellectual protections. Coleman demonstrates how hacking, so often marginalized or misunderstood, sheds light on the continuing relevance of liberalism in online collaboration.} }Citation
Coleman, E. G. (2012). Coding freedom: The ethics and aesthetics of hacking. Princeton University Press. https://doi.org/10.1515/9781400845293