Abstract

Artificial neural networks and supervised learning have become an essential part of science. Beyond using them for accurate input-output mapping, there is growing attention to a new feature-oriented approach. Under the assumption that networks optimised for a task may have learned to represent and utilise important features of the target system for that task, scientists examine how those networks manipulate inputs and employ the features networks capture for scientific discovery. We analyse this approach, show its hidden caveats, and suggest its legitimate use. We distinguish three things that scientists call a ‘feature’: parametric, diagnostic, and real-world features. The feature-oriented approach aims for real-world features by interpreting the former two, which also partially rely on the network. We argue that this approach faces a problem of non-uniqueness: there are numerous discordant parametric and diagnostic features and ways to interpret them. When the approach aims at novel discovery, scientists often need to choose between those options, but they lack the background knowledge to justify their choices. Consequentially, features thus identified are not promised to be real. We argue that they should not be used as evidence but only used instrumentally. We also suggest transparency in feature selection and the plurality of choices.