Contemporary wisdom suggests that inclusion of legumes into crop rotations benefit subsequent cereal crop yields. To investigate whether this maxim was generically scalable, we contrast summer soybean-winter wheat (SW) with summer maize-winter wheat (MW) rotation systems in an extensive field campaign in the North China Plain (NCP). We identify heretofore unseen interactions between crop rotation, synthetic N fertilizer application, and stored soil water. In the year with typical rainfall, inclusion of soybean within rotation had no effect on wheat ear number and yield, while N fertilization penalized wheat yields by 6-8%, mainly due to lower dry matter accumulation after anthesis. In contrast, in dry years prior crops of soybean reduced the rate and number of effective ears in wheat by 5-27 and 14-17%, respectively, leading to 7-23% reduction in wheat yield. Although N fertilization increased the stem number before anthesis in dry years, there was no corresponding increase in ear number and yield of wheat in such years, indicating compensating reduction in yield components. We also showed that N fertilization increased wheat yield in MW rather than SW as the former better facilitated higher dry matter accumulation after flowering in dry years. Taken together, our results suggest that soybean inclusion reduced soil available water for subsequent wheat growth, causing yield penalty of subsequent wheat under drought conditions. We call for more research into factors influencing crop soil water, including initial state, crop water requirement, and seasonal climate forecasts, when considering legumes into rotation systems.