In the scholarship on Wittgenstein’s later philosophy of mathematics, the dominant interpretation is a theoretical one that ascribes to Wittgenstein some type of ‘ism’ such as radical verificationism or anti-realism. Essentially, he is supposed to provide a positive account of our mathematical practice based on some basic assertions. However, I claim that he should not be read in terms of any ‘ism’ but instead should be read as examining philosophical pictures in the sense of unclear conceptions. The contrast here is that basic assertions that frame philosophical ‘isms’ are propositional such that they are subject to normal argumentative evaluation, while pictures in Wittgenstein’s sense are non-propositional—they lack a clear truth condition. They, therefore, need clarification rather than argumentation. In this paper, I provide a detailed analysis of Wittgenstein’s treatment of philosophical pictures with special focus on his argument on contradiction. I begin by explaining the problem with this trend of theoretical interpretation, taking Steve Gerrard’s otherwise excellent interpretation as a representative example and pointing out why it is problematic. Next, I will argue that those problems do not arise if we take Wittgenstein’s task as the clarification of philosophical pictures. I do this, first, by explaining Wittgenstein’s method using his argument concerning the Augustinian Picture in Philosophical Investigations and then pointing out that the same method can be identified in the crucial arguments in his philosophy of mathematics. Finally, in order to connect my interpretation with the current scholarship, I will explain the relation of my interpretation with those of New Wittgensteinian scholars.