Environmental conditions during development can affect later-life health, but the evolutionary mechanisms underpinning these observations remain debated. The silver spoon (SS) hypothesis proposes that poor developmental conditions adversely affect development, leading to lower survival and fertility and increased metabolic disease risk, independent of later-life conditions. Meanwhile, the predictive adaptive response (PAR) hypothesis proposes that metabolic disease results from selection for development which is plastic with regard to environmental conditions, under which survival and fertility are maximised where conditions match in later life. If conditions change, metabolic disease results, but because the benefits of developmental plasticity exceed the costs, metabolic disease is not selected against. The observation that humans conceived during famine but raised in affluent conditions show later health problems is consistent with this idea. However, the proposed evolutionary mechanisms for putative developmental plasticity have rarely been empirically tested in humans. In particular, there are few studies examining the fitness consequences of conditions during development in varying later-life conditions. We tested the effects of early-life environmental conditions on later survival and fertility using data collected from several pre-industrial Finnish populations. We tested effects of early-life conditions on fitness during (i) varying later-life conditions, and (ii) adverse environmental conditions (a severe famine). Our results suggest that adverse early conditions decrease later fitness irrespective of later-life conditions, and that individuals born in poor conditions have lower survival and fertility during later-life famine. Our results do not support the PAR hypothesis, but are consistent with predictions of the SS hypothesis, which suggests that the early environmental conditions influence development in a manner which may persist into later life.