Symmetry-based inferences have permeated many discussions in philosophy of physics and metaphysics of science. It is claimed that symmetries in our physical theories would allow us to draw metaphysical conclusions about the world, a view that I call ‘symmetry inferentialism’. This paper is critical to this view. I claim that (a) it assumes a philosophically questionable characterization of the relevant validity domain of physical symmetries, and (b) it overlooks a distinction between two opposing ways through which relevant physical symmetries become (...) established. My conclusion is that symmetry inferentialism loses persuasive force when these two points are taken into consideration. (shrink)