Management strategies aimed at reducing nutrient enrichment of surface waters may be hampered by nutrient legacies that have accumulated in the landscape. Here, we apply the Net Anthropogenic Phosphorus Input (NAPI) model to reconstruct the historical phosphorus (P) input trajectories for the province of Ontario, which encompasses the Canadian portion of the drainage basin of the Laurentian Great Lakes (LGL). NAPI considers P inputs from detergent, human and livestock waste, fertilizer inputs, and P outputs by crop uptake. During the entire time period considered, from 1961 to 2016, Ontario experienced positive annual NAPI values. Despite a generally downward NAPI trend since the late 1970s, the lower LGL, especially Lake Erie, continue to be plagued by algal blooms. When comparing NAPI results and river monitoring data for the period 2003 to 2013, P discharged by Canadian rivers into Lake Erie only accounts for 12.5% of the NAPI supplied to the watersheds' agricultural areas. Thus, over 85% of the agricultural NAPI is retained in the watersheds where it contributes to a growing P legacy, primarily as soil P. The slow release of legacy P therefore represents a long-term risk to the recovery of the lake. To help mitigate this risk, we present a methodology to spatially map out the source areas with the greatest potential of erosional export of legacy soil P to surface waters. These areas should be prioritized in soil conservation efforts.