This paper presents an argument against A D Smith’s Direct Realist theory of perception, which attempts to defend Direct Realism against the argument from illusion by appealing to conscious perceptual states that are structured by the perceptual constancies. Smith’s contention is that the immediate objects of perceptual awareness are characterised by these constancies, which removes any difficulty there may be in identifying them with the external, or normal, objects of awareness. It is here argued that Smith’s theory does not provide an adequate defence of Direct Realism because it does not adequately deal with the difficulties posed by the possibility of perceptual illusion. It is argued that there remain possible illusory experiences where the immediate objects of awareness, which in Smith’s account are those characterised by perceptual constancies, cannot be identified with the external objects of awareness, contrary to Direct Realism. A further argument is offered to extend this conclusion to all non-illusory cases, by adapting an argument of Smith’s own for the generalising step of the Argument from Illusion. The result is that Smith’s theory does not provide an adequate Direct Realist account of the possibility of perceptual illusion.