Rational emotive behaviour therapy (REBT) does not possess a measure of rational and irrational beliefs that meets internationally recognised standards for acceptable psychometric properties. Without such a measure the theory/practice of REBT cannot be rigorously evaluated, thus undermining its scientific veracity. The current study investigates the validity and reliability of a recently developed measure of rational and irrational beliefs: the Attitudes and Belief Scale 2-Abbreviated Version (ABS-2-AV). University students from three countries completed the ABS-2-AV (N = 397). An alternative models framework using confirmatory factor analysis indicated that a theoretically consistent eight-factor model of the ABS-2-AV provided the best fit of the data. A number of post hoc modifications were required in order to achieve acceptable model fit results, and these modifications revealed important methodological limitations with the ABS-2-AV. Results indicated that the validity of the ABS-2-AV was undermined due to items measuring both the psychological process of interest (rational and irrational beliefs) and the context in which these beliefs processes are presented. This is a serious methodological limitation of the ABS-2 and all questionnaires derived from it, including the ABS-2-AV. This methodological limitation resulted in the ABS-2-AV possessing poor internal reliability. These limitations are discussed in relation to the broader REBT literature and the impact such problems have on research and practice. A call is made for REBT researchers to come together to develop a “gold standard” method of assessing rational and irrational beliefs that meets international standard for psychometric excellence.