In the present study, a range of regional climate models have been used to test approaches to Bayesian model averaging (BMA), particularly the quantification of model weights/Bayesian priors. The results of skill assessments were used to inform probabilistic future projections of Irish climate using a BMA approach in order to evaluate how different approaches to skill assessment, based on representation of climate means, or of a large-scale driver (the NAO), or a combination thereof, may influence the final climate projection. Results indicate that meansbased skill assessments may not always provide a useful indication of model skill and that further analyses are required to assess a model’s ability to simulate the dynamics of the climate system. While this research illustrates that the use of metrics derived from the model predicted NAO impacts on the regional projection, it also supports the inclusion of other large-scale model diagnostics. When used to weight model projections to produce ensemble climate projections, the choice of skill metric may have an impact on the shape of the probability distribution and the most probable outcome of future climate predictions. The present study demonstrates that when working with probabilistic outputs of ensemble climate modelling experiments, awareness of the approaches used to evaluate models and the techniques used to combine them to formulate ensemble projections are integral in enabling robust responses to the potential changes in climate represented by models.