The Substance Use Risk Profile Scale (SURPS), a widely used self-report questionnaire, assesses four personality traits which predict risk for substance use (i.e., anxiety sensitivity, hopelessness, impulsivity, and sensation seeking). Given its use in research and clinical settings, as well as potential utility, this study aimed to provide a comprehensive psychometric evaluation of the SURPS. Undergraduate participants (N = 718; 69% White; 26% Hispanic, aged 18-25 years, M = 19.00, SD = 1.33) completed a battery of measures, including the SURPS. Tests of measurement invariance, convergent and criterion validity, and internal consistency were conducted, as well as item response theory analyses and a treatment assignment simulation. Several items were removed before partial measurement invariance across gender was established with little information lost. Despite removing several SURPS items, the proposed factor structure was not empirically supported. More work is necessary to determine the predictive utility of assessing these personality traits to predict substance-related outcomes.