Recent paleoclimate reconstructions have challenged the traditional view that Northern Hemisphere insolation and associated feedbacks drove synchronous global climate and ice-sheet volume during the last glacial cycle. Here we focus on the response of the Patagonian Ice Sheet, and demonstrate that its maximum expansion culminated at 28,400 ± 500 years before present (28.4 ± 0.5 ka), more than 5,000 years before the minima in 65 °N summer insolation and the formally-defined Last Glacial Maximum (LGM) at 21,000 ± 2,000 years before present. To investigate the potential drivers of this early LGM (eLGM), we simulate the effects of orbital changes using a suite of climate models incorporating prescribed and evolving sea-ice anomalies. Our analyses suggest that Antarctic sea-ice expansion at 28.5 ka altered the location and intensity of the Southern Hemisphere storm track, triggering regional cooling over Patagonia of 5 °C that extends across the wider mid-southern latitudes. In contrast, at the LGM, continued sea-ice expansion reduced regional temperature and precipitation further, effectively starving the ice sheet and resulting in reduced glacial expansion. Our findings highlight the dominant role that orbital changes can play in driving Southern Hemisphere glacial climate via the sensitivity of mid-latitude regions to changes in Antarctic sea-ice extent.