Experiments in computing share many characteristics with the traditional experimental method, but also present significant differences from a practical perspective, due to their aim at producing software artifacts and the central role played by human actors and organizations involved in the software development process. By analyzing some of the most significant experiments in the subfield of software engineering, we aim at showing how the conceptual framework that supports experimental methodology in this context needs an extension in a socio-technical perspective.

One might poetically muse that computers have the essence both of logic and machines. Through the case of the history of Separation Logic, we explore how this assertion is more than idle poetry. Separation Logic works because it merges the software engineer’s conceptual model of a program’s manipulation of computer memory with the logical model that interprets what sentences in the logic are true, and because it has a proof theory which aids in the crucial problem of scaling the reasoning (...) task. Scalability is a central problem, and some would even say the central problem, in appli- cations of logic in computer science. Separation Logic is an interesting case because of its widespread success in verification tools. For these two senses of model—the engineering/conceptual and the logical—to merge in a genuine sense, each must maintain their norms of use from their home disciplines. When this occurs, both the logic and engineering benefit greatly. Seeking this intersection of two different senses of model provides a strategy for how computer scientists and logicians may be successful. Furthermore, the history of Separation Logic for analysing programs provides a novel case for philosophers of science of how software engineers and computer scientists develop models and the components of such models. We provide three contributions: an exploration of the extent of models merging that is necessary for success in computer science; an introduction to the technical details of Separation Logic, which can be used for reasoning about other exhaustible resources; and an introduction to the problems, process, and results of computer scientists for those outside the field. (shrink)

Software-intensive science challenges in many ways our current scientific methods. This affects significantly our notion of science and scientific interpretation of the world, driving at the same time the philosophical debate. We consider some issues prompted by SIS in the light of the philosophical categories of ontology and epistemology.

Artefacts do not always do what they are supposed to, due to a variety of reasons, including manufacturing problems, poor maintenance, and normal wear-and-tear. Since software is an artefact, it should be subject to malfunctioning in the same sense in which other artefacts can malfunction. Yet, whether software is on a par with other artefacts when it comes to malfunctioning crucially depends on the abstraction used in the analysis. We distinguish between “negative” and “positive” notions of malfunction. A negative malfunction, (...) or dysfunction, occurs when an artefact token either does not or cannot do what it is supposed to. A positive malfunction, or misfunction, occurs when an artefact token may do what is supposed to but, at least occasionally, it also yields some unintended and undesirable effects. We argue that software, understood as type, may misfunction in some limited sense, but cannot dysfunction. Accordingly, one should distinguish software from other technical artefacts, in view of their design that makes dysfunction impossible for the former, while possible for the latter. (shrink)

The mechanistic approach in philosophy of science contributes to our understanding of experimental design. Applying the mechanistic approach to experimentation in computing is beneficial for two reasons. It connects the methodology of experimentation in computing with the methodology of experimentation in established sciences, thereby strengthening the scientific reputability of computing and the quality of experimental design therein. Furthermore, it pinpoints the idiosyncrasies of experimentation in computing: computing deals closely with both natural and engineered mechanisms. Better understanding of the idiosyncrasies, which (...) manifest in terms of a nonstandard role for experimentation, are interesting both for computer scientists and for philosophers of science. Computer scientists can think more clearly about their experimental choices. The role of experimentation elucidated by computer science merits further study from philosophers of science generally, as it highlights a role for experimentation hitherto unrecognized by philosophers: demonstration that activities exist. (shrink)

Phyllis Illari and Jon Williamson propose a formulation for a general mechanistic account, the purpose of which is to capture the similarities across mechanistic accounts in the sciences. Illari and Williamson extract insight from mechanisms in astrophysics—which are notably different from the typical biological mechanisms discussed in the literature on mechanisms—to show how their general mechanistic account accommodates mechanisms across various sciences. We present argumentation that demonstrates why an amendment is necessary to the ontology referred to by the general mechanistic (...) account provided by Illari and Williamson. The amendment is required due to the variability of some components in computing mechanisms: the very same component serves as either entity or activity, both between levels and within the same level of the explanatory hierarchy. We argue that the proper ontological account of these mechanistic components involves disambiguation via explicitly indexing them as entities or activities. (shrink)

This paper takes part in the methodological debate concerning the nature and the justification of hypotheses about computational systems in software engineering by providing an epistemological analysis of Software Testing, the practice of observing the programs’ executions to examine whether they fulfil software requirements. Property specifications articulating such requirements are shown to involve falsifiable hypotheses about software systems that are evaluated by means of tests which are likely to falsify those hypotheses. Software Reliability metrics, used to measure the growth of (...) probability that given failures will occur at specified times as new executions are observed, is shown to involve a Bayesian confirmation of falsifiable hypotheses on programs. Coverage criteria, used to select those input values with which the system under test is to be launched, are understood as theory-laden principles guiding software tests, here compared to scientific experiments. Redundant computations, fault seeding models and formal methods used in software engineering to evaluate test results are taken to be instantiations of some epistemological strategies used in scientific experiments to distinguish between valid and non-valid experimental outcomes. The final part of the paper explores the problem, advanced in the context of the philosophy of technology, of defining the epistemological status of software engineering by conceiving it as a scientifically attested technology. (shrink)