Migration of the inter-tropical convergence zone, driven by changes in seasonal insolation and high northern latitude temperatures, is the primary control on tropical rainfall on geologic timescales. We test this paradigm using the timing of growth of stalagmites from southwest Madagascar to infer the timing of expansion of the ITCZ to the south at its southern limit. Over the past 117 ky, speleothems grew in the study area primarily when two conditions are met: summer insolation greater than the mean and relatively high Southern Hemisphere temperatures as indicted by maxima in Antarctic ice core oxygen isotope ratios. We observe little influence of Northern Hemisphere, millennial scale temperature variability on the pluvial periods. Further, we observe periods during which the ITCZ simultaneously expands or contracts in both hemispheres. Because Antarctic isotope maxima are periods of increased atmospheric CO2, our results have implications for how tropical rainfall in the Southern Hemisphere might respond to global warming.