Estimating crop model uncertainty in agricultural systems: a comprehensive examination of APSIM for potato production in Australia

Crop growth models are essential tools which are used to simulate crop growth and production at different spatial scales. These models are used in many fields of research such as: climate change impact assessment; land use; food security; environmental impact assessments; and market dynamics. Although these models are increasingly used and are very important, various uncertainties exist in the outputs provided by these models. The major sources of these uncertainties are model inputs, model structure and model parameters. It is important to carefully quantify and address the sources of uncertainties in crop models, to identify priority areas for model development and to provide information on reliability of the outputs to users.
The vast knowledge on the existing approaches for assessing, quantifying and decomposing crop model uncertainty were yet to be reviewed. In order to better understand and estimate uncertainty in crop models, a systematic review (Chapter 2) was conducted to investigate (a) how uncertainty was conceptualised and quantified, (b) how different uncertainty sources were quantified and decomposed and (c) major gaps and future research needs. We found that input uncertainty was the most frequently considered uncertainty type, followed by parameter uncertainty, with structural uncertainty the least examined. In most uncertainty research, crop yield was the most common model output examined. In addition, most of these studies were conducted in rice, wheat and maize. In most research, uncertainty was quantified using sensitivity analysis techniques. The extraction of metrics from the articles and the comparison of contributions from various sources of uncertainty was highly challenging as there were differences in how the results were presented. Our findings reveal that standard and uniform methodology/ approach for uncertainty estimation is still lacking. The key gaps identified in this chapter were subsequently addressed in Chapters 3, 4 and 5. Following the findings and key research gaps highlighted from the systematic review, Chapter 3 of this thesis investigates the contribution of model structural uncertainty in simulated agronomic (yield, irrigation requirement), environmental (water drainage, nitrogen leaching) and economic [partial gross margin (PGM)] model outputs for potato (Solanum tuberosum L.). Since soil variability plays a major role in variance of model outputs, Chapter 4 further aimed to examine the effect of soil types on model structural uncertainty. Additionally, this thesis also explored the contribution of uncertainty vs. variability in modelling outputs in Chapter 5 as often these terms are confused in modelling studies which make it difficult to provide accurate advice to stakeholders.
Crop model structure is one of the major sources of uncertainty, however, its quantification is difficult due to limitations in controlling confounding effects. Therefore, Chapter 3 quantified the contribution of structural uncertainty to model outputs produced by the Agricultural Production Systems sIMulator (APSIM). Eight model structures differing in choice of soil water model, crop model and irrigation model were developed within a single APSIM version and tested under three contrasting environments across 121 years (1900-2020). We quantified: (i) the model structural uncertainty using analysis of variance and deviation analysis; and (ii) the variability of outputs due to model structure and climate using the coefficient of variation. Most structural uncertainty resulted from first order effects driven by the model components (crop:12.2-98.9%, irrigation:0-78.4%, soil:1-33.7%) rather than second order interactions between components. Furthermore, uncertainty from the choice of sub-model used (for example the irrigation model) was not necessarily related to the structural complexity of these components. The effects of structural uncertainty on predictions were strongly impacted by site (driven mainly by soil type), highlighting the need for any uncertainty assessment to cover the entire range of potential conditions for model applications.
Soil type plays a major role in nutrient dynamics and soil water which impact crop growth and yield. The influence of soil characteristics on crop growth are usually evaluated through field experimentation (in the short term) and through crop-soil modelling (in the long-term). However, there has been limited research which has looked at the effect of model structural uncertainty of model outputs in different soil types. To analyse the impact of soil inputs on model structural uncertainty, we used the developed eight model structures (from Chapter 3) across three soil types (Ferralsols, Alisols and Chernozems). By decomposing the mean proportion of variance and simulated values of the model outputs (yield, irrigation, drainage, nitrogen leaching and PGM) we identified the influence of soil type on the magnitude of model structural uncertainty (Chapter 4). For all soil types, crop model was the most significant source of structural uncertainty, contributing >60% to variability for most modelled variables, except irrigation demand which was dominated by the choice of irrigation model applied. Relative to first order interactions, there were minimal (<12%) contributions to uncertainty from the second order interactions (i.e., inter-model components). We found that a higher mean proportion of variance does not necessarily imply high contribution in actual values. For example, the choice of crop model used contributed more than 90% in yield variance for all three soil types. However, the range of standard deviation in simulated yield was only 0.2 – 1.0 t ha^-1 , which was small compared to the overall average crop yields (14.6 t ha-¹ ). We identified the need to include the mean proportion of variance of model uncertainty, along with the magnitude of the contribution in measured units (e.g., t ha-¹ ). Both types of uncertainty were required for crop model uncertainty assessments to provide useful information for agronomic decision making or policy formulation. The findings highlighted the sensitivity of agricultural models to the impacts of moisture availability, suggesting that it is important to give more attention to structural uncertainty when modelling dry/wet conditions depending on the model outputs analysed.
Uncertainty and variability are different from one another. However, researchers have generally used them as synonyms in past uncertainty studies. Hence, there is a strong need to decompose and identify the contribution of uncertainty and variability factors in model outputs which would help better identify main drivers of variance in model outputs and consequently increase confidence in modelling simulations. These improvements will enable more accurate advice based on crop model predictions. Thus, the objective of Chapter 5 was to investigate and quantify the contribution of model structural uncertainty, temporal variability and spatial variability to the variance in simulated model outputs. Eight model structures were developed for three planting dates, three soil types and three sites for a period of 121 years. Then, mean proportion of variance was calculated using ANOVA and actual values were calculated in terms of standard deviation. This chapter of the thesis also investigated how the relative contribution of these factors varied over time. The results of our study demonstrate how the impact and extent of uncertainty and variability differ across different outcomes examined. Further, similar observations were made for different sites. The findings of the study indicate the importance of adopting site-specific approaches while using crop model outputs for decision-making processes.
The findings from this thesis have broad implications for different stakeholders, including scientists, policymakers, industry professionals and farmers. This thesis is mostly helpful to scientists/consultants to better understand how they best use models, what to consider when in project design and what guidance they can provide to their stakeholders. The findings can help stakeholders understand the limitations of crop models under diverse environmental conditions, along with the inherent uncertainties of modelling, which can lead to more informed decisions and better risk management. Additionally, the conclusions drawn from this thesis can increase the transparency of modelling, by identifying the sources of uncertainty and variability and making them explicit which may lead to increased trust in crop models and greater acceptance of their results.