Abstract

The article examines the question of how learning multiple tasks interacts with neural architectures and the flow of information through those architectures. It approaches the question by using the idealization of an artificial neural network where it is possible to ask more precise questions about the effects of modular versus nonmodular architectures as well as the effects of sequential versus simultaneous learning of tasks. A prior work has demonstrated a clear advantage of modular architectures when the two tasks must be learned at the same time from the start, but this advantage may disappear when one task is first learned to a criterion before the second task is undertaken. Indeed, in some cases of sequential learning, nonmodular networks achieve success levels comparable to those of modular networks. In particular, if a nonmodular network is to learn two tasks of different difficulty and the more difficult task is presented first and learned to a criterion, then the network will learn the second, easier one without permanent degradation of the first one. In contrast, if the easier task is learned first, a nonmodular task may perform significantly less well than a modular one. It seems that the reason for this difference has to do with the fact that the sequential presentation of the more difficult task first minimizes interference between the two tasks. More broadly, the studies summarized in this article seem to imply that no single learning architecture is optimal for all situations