Data Pre-processing for Agricultural Simulations

Jan JarolímekJan PavlíkJana KholovaSwarna Ronanki

DOI 10.7160/aol.2019.110105
No 1/2019, March
pp. 49-53

Jarolímek, J., Pavlík, J., Kholova, J. and Ronanki, S. (2019) “Data Pre-processing for Agricultural Simulations", AGRIS on-line Papers in Economics and Informatics, Vol. 11, No. 1, pp. 49-53. ISSN 1804-1930. DOI 10.7160/aol.2019.110105.

Abstract

The process of agricultural simulation using APSIM requires input meteorological data to be prepared in a specific format and the simulation setting file to be ready before the simulation processing starts. Because of possible time savings when conducting large number of simulations at once, it is preferable to create all the input and settings files for all the simulations beforehand and process the simulations in batches as large as possible. This article specifically deals with the data acquisition, transformation and preparation process. It also outlines initial testing and computing time estimations and discusses scheduling, parallel processing and other possible simulation optimization methods.

Keywords

APSIM, big data, data processing, yield optimization, software automation, parallel processing.

References

  1. Aurbacher, J., Parker, P. S., Sanchez, G. A. C., Steinbach, J., Reinmuth, E., Ingwersen, J. and Dabbert, S. (2013) “Influence of climate change on short term management of field crops - A modelling approach”, Agricultural Systems, Vol. 119, pp. 44-57. ISSN 0308-521X. DOI 10.1016/j.agsy.2013.04.005.
  2. Fujimoto, R. M. (2016) “Research Challenges in Parallel and Distributed Simulation”, ACM Transactions On Modeling And Computer Simulation, Vol. 26, No. 4. 10493301. DOI 10.1145/2866577.
  3. Hewitson, B. C. and Crane, R. G. (1996) “Climate downscaling: Techniques and application” Climate Research, Vol. 7, pp. 85-95. E-ISSN 1616-1572, ISSN 0936-577X. DOI 10.3354/cr007085.
  4. Holzworth, D., Huth, N. I., de Voil, P. G., Zurcher, E. J., Herrmann, N. I., McLean, G., Keating, B. A. (2014) “APSIM - Evolution towards a New Generation of Agricultural Systems Simulation.” Environmental Modelling & Software, Vol. 62, pp. 327-350. ISSN 1364-8152. DOI 10.1016/j.envsoft.2014.07.009.
  5. Holzworth, D., Huth, N. I., Fainges, J., Brown, H., Zurcher, E., Cichota, R., Verrall, S., Herrmann, N. I., Zheng, B. and Snow. V. (2018) “APSIM Next Generation: Overcoming Challenges in Modernising a Farming Systems Model”, Environmental Modelling & Software, Vol. 103, pp. 43-51. ISSN 1364-8152. DOI 10.1016/j.envsoft.2018.02.002.
  6. Kambadur, M., Tang, K., Lopez, J. and Kim, M. A. (2013) “Parallel scaling properties from a basic block view”, ACM SIGMETRICS Performance Evaluation Review, Vol. 41, pp. 365-366. ISSN 0163-5999. DOI 10.1145/2494232.2465748.
  7. Kim, K. S., Yoo, B. H., Shelia, V., Porter, C. H. and Hoogenboom, G. (2018) “START: A data preparation tool for crop simulation models using web-based soil databases”, Computers and Electronics in Agriculture, vol. 154, pp. 256-264. ISSN 0168-1699. DOI 10.1016/j.compag.2018.08.023.
  8. Kirby, A. C., Yang, Z., Mavriplis, D. J., Duque, E. P. N. and Whitlock, B. J. (2018) “Visualization and data analytics challenges of large-scale high-fidelity numerical simulations of wind energy applications”, AIAA Aerospace Sciences Meeting. AIAA SciTech Forum, Kissimmee, Florida. DOI 10.2514/6.2018-1171.
  9. Mass, C. F., Ovens, D., Westrick, K. and Colle, B. A. (2002) “Does increasing horizontal resolution produce more skillful forecasts? The results of two years of real-time numerical weather prediction over the Pacific northwest”, Bulletin of the American Meteorological Society, Vol. 83, No. 3. DOI 10.1175/1520-0477(2002)083<0407:DIHRPM>2.3.CO;2.
  10. Reinmuth, E. and Dabbert, S. (2017) “Toward more efficient model development for farming systems research - An integrative review”, Computers And Electronics In Agriculture, Vol. 138, pp. 29-38. ISSN 0168-1699. DOI 10.1016/j.compag.2017.04.007.
  11. Skoogh, A., Michaloski, J. and Bengtsson, N. (2010) “Towards continuously updated simulation models: Combining automated raw data collection and automated data processing”, Proceedings - Winter Simulation Conference, pp. 1678-1689. ISSN 08917736. DOI 10.1109/WSC.2010.5678901.
  12. Stoces, M., Masner, J., Kanska, E. and Jarolimek J. (2018) "Processing of Big Data in Internet of Things and Precision Agriculture", Agrarian Perspectives XXVII.: Food Safety - Food Security, Proceedings of the 27th International Scientific Conference, pp. 353-358. ISBN 978-80-213-2890-7. ISSN 1213-7979.
  13. Szufel, P., Czupryna, M. and Kaminski, B. (2017) "Optimal execution of large scale simulations in the cloud. The case of route-To-pa sim online preference simulation“, Proceedings - Winter Simulation Conference, pp. 3702-3703. DOI 10.1109/WSC.2016.7822408.
  14. Zhao, G., Bryan, B. A., King, D., Luo, Z., Wang, E., Bende-Michl, U., Song, X. and Yu, Q. (2013) “Large-scale, high-resolution agricultural systems modeling using a hybrid approach combining grid computing and parallel processing”, Environmental Modelling & Software, Vol. 41, pp. 231-238. ISSN 1364-8152. DOI 10.1016/j.envsoft.2012.08.007.

Full paper

  Full paper (.pdf, 589.53 KB).