Should we let personal responsibility for health-related behavior influence the allocation of healthcare resources? In this paper, we clarify what it means to be responsible for an action. We rely on a crucial conceptual distinction between being responsible and holding someone responsible, and show that even though we might be considered responsible and blameworthy for our health-related actions, there could still be well-justified reasons for not considering it reasonable to hold us responsible by giving us lower priority. We transform these philosophical considerations into analytical use first by assessing the general features of health-related actions and the corresponding healthcare needs. Then, we identify clusters of structural features that even adversely affected people cannot reasonably deny constitute actions for which they should be held responsible. We summarize the results in an analytical framework that can be used by decision-makers when considering personal responsibility for health as a criterion for setting priorities.