The underlying causes for Earths temperature variations during glacial cycles observed in the late Pleistocene are still not fully understood. To contribute to this understanding we here compile various climate records in order to calculate the direct effect of various processes on Earths radiative budget and thus on global annual mean near surface temperature over the last 800,000 years. The importance of orbital variations, of the greenhouse gases CO2, CH4 and N2O, of the albedo of land ice sheets, sea ice area and vegetation, and of the radiative perturbation of mineral dust in the atmosphere are investigated in detail, while potential impacts from variable solar activity are only briefly discussed. Altogether we can explain with these processes a global cooling of 3 K for the Last Glacial Maximum directly out of theradiative budget. These are about two thirds of the reconstructed temperature anomaly and thus a significant improvement to earlier estimates. Increased temperatures during warmer than Holocene interglacials in Marine Isotope Stages 5.5, 7.5, 9.3 and 11.3 as recorded in Antarctic ice cores are largely unexplained. Other processes and feedbacks in the climate system are necessary to explain the still existing bias between temperature reconstructionsand our compilation.