Number facts are commonly assumed to be verbally stored in an associative multiplication fact retrieval network. Prominent evidence for this assumption comes from so-called operand-related errors (e.g. 4 × 6 = 28). However, little is known about the development of this network in children and its relation to verbal and non-verbal memories. In a longitudinal design, we explored elementary school children from grades 3 and 4 in a multiplication verification task with the operand-related and -unrelated distractors. We examined the contribution (...) of multiplicative fact retrieval by verbal and visuo-spatial short-term and working memory. Children in grade 4 showed smaller reaction times in all conditions. However, there was no significant difference in errors between grades. Contribution of verbal and visuo-spatial working memory also changed with grade. Multiplication correlated with verbal working memory and performance in grade 3 but with visuo-spatial working memory and performance in grade 4. We suggest that the relation to verbal working memory in grade 3 indicates primary linguistic learning of and access to multiplication in grade 3 which is probably based on verbal repetition of the multiplication table heavily practiced in grades 2 and 3. However, the relation to visuo-spatial semantic working memory in grade 4 suggests that there is a shift from verbal to visual and semantic learning in grade 4. This shifting may be induced because later in elementary school, multiplication problems are rather carried out via more written, i.e., visual tasks, which also involve executive functions. More generally, the current data indicates that mathematical development is not generally characterized by a steady progress in performance; rather verbal and non-verbal memory contributions of performance shift over time, probably due to different learning contents. (shrink)