Thermodynamic potentials relevant to the microcanonical, the canonical and the grand canonical ensembles, associated with rotating black holes in D-dimensions, are analysed and compared. Such black holes are known to be thermodynamically unstable, but the instability is a consequence of a subtle interplay between specific heats and the moments of inertia and it manifests itself differently in the different ensembles. A simple relation between the product of the specific heat and the determinant of the moment of inertia in both the canonical and the grand canonical ensembles is derived. Myers-Perry black holes in arbitrary dimension are studied in detail. All temperature extrema in the microcanonical ensemble are determined and classified. The specific heat and the moment of inertia tensor are evaluated in both the canonical and the grand canonical ensembles in any dimension. All zeros and poles of the specific heats, as a function of the angular momenta, are determined and the eigenvalues of the isentropic moment of inertia tensor are studied and classified. It is further shown that many of the thermodynamic properties of a Myers-Perry black hole in D − 2 dimensions can be obtained from those of a black hole in D dimensions by sending one of the angular momenta to infinity.