Albert Casullo (2000, 2003) and Shane Oakley (2011) argue that dilemma arguments against epistemic naturalism, such as those offered by Laurence BonJour (1998) and Harvey Siegel (1984), are such that, whatever strength they have against naturalism applies equally to moderate rationalist accounts of a priori justification. They conclude that dilemma arguments are, therefore, insufficient for establishing an advantage for moderate rationalism over naturalized epistemology. I argue that both Casullo's and Oakley's criticisms depend on an illicit assumption, namely, that dilemma arguments presuppose a meta-justificatory demand that naturalists provide non-circular support for their basic sources of evidence. I argue that this assumption is not necessary for the force of dilemma arguments, and I construct a version that avoids this misreading. I conclude that, although there remain strategies for responding to dilemma arguments, they constitute a challenge naturalists must take seriously.