This paper reports early results from a new collaboration among philosophers of science, psychometricians, mental health clinicians, and young people. Our aim is to help healthcare service providers who screen and treat young people aged 12 to 24 years make responsible and informed use of patient-reported data obtained from psychometric questionnaires, such as the PHQ-9 depression questionnaire and the K-10 psychological distress scale. We describe some of the challenges facing youth mental health measurement in the clinical context, report on a pilot training we recently delivered to 23 youth mental health clinicians in British Columbia, and discuss the philosophical implications of our findings. Our project reveals a special interdependence between the validity and fitness for purpose of measures of mental health in clinical settings. What and how mental healthcare providers measure depend to a large extent on why they are measuring in a given clinical context. While validation is meant to secure that one is in fact measuring what one set out to measure, this paper argues that attention to the fitness for the clinical purpose of a measure, both in terms of its use (e.g., screening, tracking, etc.) and whether it fits the specific clinical population, is partly constitutive of its validity. This is especially true in the case of psychiatry, where factors constitutive of different syndromes, as well as the presentation of different symptoms, can vary across different clinical populations. The fit with a given population can only be determined through an interaction with the population itself, and this is constitutive of that measure’s validity.