Publications¶

Zhang, W., Xu, M., Evans, K., Norman, M., Morales-Hernandez, M., Mahajan, S., et al. (2021). Progress towards accelerating the unified model on hybrid multi-core systems. In Proceedings of the Platform for Advanced Scientific Computing Conference (pp. 1–9). New York, NY, USA: Association for Computing Machinery. (1) (2)

@inproceedings{zhangProgressAcceleratingUnified2021,
author = "Zhang, Wei and Xu, Min and Evans, Katherine and Norman, Matthew and Morales-Hernandez, Mario and Mahajan, Salil and Hill, Adrian and Manners, James and Shipway, Ben and Christopher, Maynard",
address = "New York, NY, USA",
series = "PASC '21",
title = "Progress towards accelerating the unified model on hybrid multi-core systems",
isbn = "978-1-4503-8563-3",
urldate = "2021-12-28",
booktitle = "Proceedings of the Platform for Advanced Scientific Computing Conference",
publisher = "Association for Computing Machinery",
month = "July",
year = "2021",
pages = "1--9" }
The cloud microphysics scheme, CASIM, and the radiation scheme, SOCRATES, are two computationally intensive parts within the Met Office's Unified Model (UM). This study enables CASIM and SOCRATES to use accelerated multi-core systems for optimal computational performance of the UM. Using profiling to guide our efforts, we refactored the code for optimal threading and kernel arrangement and implemented OpenACC directives manually or through the CLAW source-to-source translator. Initial porting results achieved 10.02x and 9.25x speedup in CASIM and SOCRATES respectively on 1 GPU compared with 1 CPU core. A granular performance analysis of the strategy and bottlenecks are discussed. These improvements will enable UM to run on heterogeneous computers and a path forward for further improvements is provided.

Tang, R., Mao, J., Jin, M., Chen, A., Yu, Y., Shi, X., et al. (2021). Interannual variability and climatic sensitivity of global wildfire activity. Advances in Climate Change Research. (1) (2)

@article{tangInterannualVariabilityClimatic2021,
author = "Tang, Rongyun and Mao, Jiafu and Jin, Mingzhou and Chen, Anping and Yu, Yan and Shi, Xiaoying and Zhang, Yulong and Hoffman, Forrest M. and Xu, Min and Wang, Yaoping",
title = "Interannual variability and climatic sensitivity of global wildfire activity",
issn = "1674-9278",
language = "en",
urldate = "2021-09-24",
journal = "Advances in Climate Change Research",
month = "July",
year = "2021" }
Understanding historical wildfire variations and their environmental driving mechanisms is key to predicting and mitigating wildfires. However, current knowledge of climatic responses and regional contributions to the interannual variability (IAV) of global burned area remains limited. Using recent satellite-derived wildfire products and simulations from version v1.0 of the land component of the U.S. Department of Energy's Energy Exascale Earth System Model (E3SM land model [ELM] v1) driven by three different climate forcings, we investigated the burned area IAV and its climatic sensitivity globally and across nine biomes from 1997 to 2018. We found that 1) the ELM simulations generally agreed with the satellite observations in terms of the burned area IAV magnitudes, regional contributions, and covariations with climate factors, confirming the robustness of the ELM to the usage of different climate forcing sources; 2) tropical savannas, tropical forests, and semi-arid grasslands near deserts were primary contributors to the global burned area IAV, collectively accounting for 71.7\%–99.7\% of the global wildfire IAV estimated by both the satellite observations and ELM simulations; 3) precipitation was a major fire suppressing factor and dominated the global and regional burned area IAVs, and temperature and shortwave solar radiation were mostly positively related with burned area IAVs; and 4) noticeable local discrepancies between the ELM and remote-sensing results occurred in semi-arid grasslands, croplands, boreal forests, and wetlands, likely caused by uncertainties in the current ELM fire scheme and the imperfectly derived satellite observations. Our findings revealed the spatiotemporal diversity of wildfire variations, regional contributions and climatic responses, and provided new metrics for wildfire modeling, facilitating the wildfire prediction and management.

Sun, L., Liang, X.-Z., Ling, T., Xu, M., & Lee, X. (2020). Improving a Multilevel Turbulence Closure Model for a Shallow Lake in Comparison With Other 1-D Models. Journal of Advances in Modeling Earth Systems, 12(7), e2019MS001971. (1) (2)

@article{sunImprovingMultilevelTurbulence2020,
author = "Sun, Lei and Liang, Xin-Zhong and Ling, Tiejun and Xu, Min and Lee, Xuhui",
title = "Improving a Multilevel Turbulence Closure Model for a Shallow Lake in Comparison With Other 1-D Models",
volume = "12",
copyright = "©2020. The Authors.",
issn = "1942-2466",
language = "en",
number = "7",
urldate = "2020-12-07",
journal = "Journal of Advances in Modeling Earth Systems",
year = "2020",
pages = "e2019MS001971" }
Lakes differ from lands in water availability, heat capacity, albedo, and roughness, which affect local surface-atmospheric interactions. This study modified a multilevel upper ocean model (UOM) for lake applications and evaluated its performance in Lake Taihu (China) with comprehensive measurements against three popular one-dimensional (1-D) lake models. These models were based on different concepts, including the self-similarity (FLake), the wind-driven eddy diffusion (LISSS), the k-ε turbulence closure (SIMSTRAT), and a simplified turbulence closure (UOM). The surface flux scheme in these models was unified to exclude the discrepancies in representing air-lake exchanges. All models in their default formulations presented obvious cold water temperature biases and largely underestimated the lake surface temperature (LST) diurnal range. For each model, these deficiencies were significantly reduced by incorporating new physics schemes or calibrated tunable parameters based on systematic sensitivity tests. The primary modifications for UOM included (1) a new scheme of decreased surface roughness lengths to better characterize the shallow lake, (2) a solar radiation penetration scheme with increased light extinction coefficient and surface absorption fraction to account for the high water turbidity, and (3) turbulent Prandtl number increased by a factor of 20 to reduce the turbulent vertical mixing. All other models were improved in these three aspects (roughness, extinction, and mixing) within their original formulations. Given these improvements, UOM showed superior performance to other models in capturing LST diurnal cycle and daily to seasonal variations, as well as summer-autumn vertical stratification changes. The new UOM is well suited for application in shallow lakes.

Durden, D. J., Metzger, S., Chu, H., Collier, N., Davis, K. J., Desai, A. R., et al. (2020). Automated Integration of Continental-Scale Observations in Near-Real Time for Simulation and Analysis of Biosphere–Atmosphere Interactions. In J. Nichols, B. Verastegui, A. ‘Barney’. Maccabe, O. Hernandez, S. Parete-Koon, & T. Ahearn (Eds.), Driving Scientific and Engineering Discoveries Through the Convergence of HPC, Big Data and AI (Vol. 1315, pp. 204–225). Cham: Springer International Publishing. (1)

@incollection{durdenAutomatedIntegrationContinentalScale2020,
author = "Durden, David J. and Metzger, Stefan and Chu, Housen and Collier, Nathan and Davis, Kenneth J. and Desai, Ankur R. and Kumar, Jitendra and Wieder, William R. and Xu, Min and Hoffman, Forrest M.",
editor = "Nichols, Jeffrey and Verastegui, Becky and Maccabe, Arthur ‘Barney’ and Hernandez, Oscar and Parete-Koon, Suzanne and Ahearn, Theresa",
address = "Cham",
title = "Automated Integration of Continental-Scale Observations in Near-Real Time for Simulation and Analysis of Biosphere–Atmosphere Interactions",
volume = "1315",
isbn = "978-3-030-63392-9 978-3-030-63393-6",
language = "en",
urldate = "2021-01-06",
booktitle = "Driving Scientific and Engineering Discoveries Through the Convergence of HPC, Big Data and AI",
publisher = "Springer International Publishing",
year = "2020",
pages = "204--225" }

Yang, X., Ricciuto, D. M., Thornton, P. E., Shi, X., Xu, M., Hoffman, F., & Norby, R. J. (2019). The Effects of Phosphorus Cycle Dynamics on Carbon Sources and Sinks in the Amazon Region: A Modeling Study Using ELM v1. Journal of Geophysical Research: Biogeosciences, 124(12), 3686–3698. (1)

@article{yangEffectsPhosphorusCycle2019,
author = "Yang, Xiaojuan and Ricciuto, Daniel M. and Thornton, Peter E. and Shi, Xiaoying and Xu, Min and Hoffman, Forrest and Norby, Richard J.",
title = "The Effects of Phosphorus Cycle Dynamics on Carbon Sources and Sinks in the Amazon Region: A Modeling Study Using ELM v1",
volume = "124",
issn = "2169-8953, 2169-8961",
shorttitle = "The Effects of Phosphorus Cycle Dynamics on Carbon Sources and Sinks in the Amazon Region",
language = "en",
number = "12",
urldate = "2020-02-16",
journal = "Journal of Geophysical Research: Biogeosciences",
month = "December",
year = "2019",
pages = "3686--3698" }

Yan, B., Mao, J., Shi, X., Hoffman, F. M., Notaro, M., Zhou, T., et al. (2019). Predictability of tropical vegetation greenness using sea surface temperatures. Environmental Research Communications, 1(3), 031003. (1) (2)

@article{yanPredictabilityTropicalVegetation2019,
author = "Yan, Binyan and Mao, Jiafu and Shi, Xiaoying and Hoffman, Forrest M. and Notaro, Michael and Zhou, Tianjun and Mcdowell, Nate and Dickinson, Robert E. and Xu, Min and Gu, Lianhong and Ricciuto, Daniel M.",
title = "Predictability of tropical vegetation greenness using sea surface temperatures",
volume = "1",
issn = "2515-7620",
language = "en",
number = "3",
urldate = "2019-09-06",
journal = "Environmental Research Communications",
month = "April",
year = "2019",
pages = "031003" }
Much research has examined the sensitivity of tropical terrestrial ecosystems to various environmental drivers. The predictability of tropical vegetation greenness based on sea surface temperatures (SSTs), however, has not been well explored. This study employed fine spatial resolution remotely-sensed Enhanced Vegetation Index (EVI) and SST indices from tropical ocean basins to investigate the predictability of tropical vegetation greenness in response to SSTs and established empirical models with optimal parameters for hindcast predictions. Three evaluation metrics were used to assess the model performance, i.e., correlations between historical observed and predicted values, percentage of correctly predicted signs of EVI anomalies, and percentage of correct signs for extreme EVI anomalies. Our findings reveal that the pan-tropical EVI was tightly connected to the SSTs over tropical ocean basins. The strongest impacts of SSTs on EVI were identified mainly over the arid or semi-arid tropical regions. The spatially-averaged correlation between historical observed and predicted EVI time series was 0.30 with its maximum value reaching up to 0.84. Vegetated areas across South America (25.76\%), Africa (33.13\%), and Southeast Asia (39.94\%) were diagnosed to be associated with significant SST-EVI correlations (p {\textless} 0.01). In general, statistical models correctly predicted the sign of EVI anomalies, with their predictability increasing from ∼60\% to nearly 100\% when EVI was abnormal (anomalies exceeding one standard deviation). These results provide a basis for the prediction of changes in greenness of tropical terrestrial ecosystems at seasonal to intra-seasonal scales. Moreover, the statistics-based observational relationships have the potential to facilitate the benchmarking of Earth System Models regarding their ability to capture the responses of tropical vegetation growth to long-term signals of oceanic forcings.

Xu, M., Mahajan, S., Hoffman, F. M., & Shi, X. (2019). Evaluating Carbon Extremes in a Coupled Climate-Carbon Cycle Simulation. In 2019 International Conference on Data Mining Workshops (ICDMW) (pp. 303–310). Beijing, China: IEEE. (1)

@inproceedings{xuEvaluatingCarbonExtremes2019,
author = "Xu, Min and Mahajan, Salil and Hoffman, Forrest M. and Shi, Xiaoying",
address = "Beijing, China",
title = "Evaluating Carbon Extremes in a Coupled Climate-Carbon Cycle Simulation",
isbn = "978-1-72814-896-0",
urldate = "2020-02-16",
booktitle = "2019 International Conference on Data Mining Workshops (ICDMW)",
publisher = "IEEE",
month = "November",
year = "2019",
pages = "303--310" }

Mahajan, S., Evans, K. J., Kennedy, J. H., Xu, M., & Norman, M. R. (2019). A Multivariate Approach to Ensure Statistical Reproducibility of Climate Model Simulations. In Proceedings of the Platform for Advanced Scientific Computing Conference (pp. 14:1–14:10). New York, NY, USA: ACM. (1) (2)

@inproceedings{mahajanMultivariateApproachEnsure2019,
author = "Mahajan, Salil and Evans, Katherine J. and Kennedy, Joseph H. and Xu, Min and Norman, Matthew R.",
address = "New York, NY, USA",
series = "PASC '19",
title = "A Multivariate Approach to Ensure Statistical Reproducibility of Climate Model Simulations",
isbn = "978-1-4503-6770-7",
urldate = "2019-09-06",
booktitle = "Proceedings of the Platform for Advanced Scientific Computing Conference",
publisher = "ACM",
year = "2019",
pages = "14:1--14:10" }
Effective utilization of novel hybrid architectures of pre-exascale and exascale machines requires transformations to global climate modeling systems that may not reproduce the original model solution bit-for-bit. Round-off level differences grow rapidly in these non-linear and chaotic systems. This makes it difficult to isolate bugs/errors from innocuous growth expected from round-off level differences. Here, we apply two modern multivariate two sample equality of distribution tests to evaluate statistical reproducibility of global climate model simulations using standard monthly output of short ({\textasciitilde} 1-year) simulation ensembles of a control model and a modified model of US Department of Energy's Energy Exascale Earth System Model (E3SM). Both the tests are able to identify changes induced by modifications to some model tuning parameters. We also conduct formal power analyses of the tests by applying them on designed suites of short simulation ensembles each with an increasingly different climate from the control ensemble. The results are compared against those from another such test. These power analyses provide a framework to quantify the degree of differences that can be detected confidently by the tests for a given ensemble size (sample size). This will allow model developers using the tests to make an informed decision when accepting/rejecting an unintentional non-bit-for-bit change to the model solution.

Mahajan, S., Evans, K. J., Kennedy, J. H., Xu, M., Norman, M. R., & Branstetter, M. L. (2019). Ongoing solution reproducibility of earth system models as they progress toward exascale computing. The International Journal of High Performance Computing Applications, 33(5), 784–790. (1) (2)

@article{mahajanOngoingSolutionReproducibility2019,
author = "Mahajan, Salil and Evans, Katherine J and Kennedy, Joseph H and Xu, Min and Norman, Mathew R and Branstetter, Marcia L",
title = "Ongoing solution reproducibility of earth system models as they progress toward exascale computing",
volume = "33",
issn = "1094-3420",
language = "en",
number = "5",
urldate = "2019-09-06",
journal = "The International Journal of High Performance Computing Applications",
month = "September",
year = "2019",
pages = "784--790" }
We present a methodology for solution reproducibility for the Energy Exascale Earth System Model during its ongoing software infrastructure development to prepare for exascale computers. The nonlinear chaotic nature of climate system simulations precludes traditional model verification approaches since machine precision differences—resulting from code refactoring, changes in software environment, and so on—grow exponentially to a different weather state. Here, we leverage the nature of climate as a statistical description of the atmosphere in order to establish model reproducibility. We evaluate the degree to which two-sample equality of distribution tests can confidently detect the change in climate from minor tuning parameter changes on model output variables in order to establish the level of difference that indicates a new climate. To apply this (baselined test), we target a section of the model’s development cycle wherein no intentional science changes have been applied to its source code. We compare an ensemble of short simulations that were conducted using a verified model configuration against a new ensemble with the same configuration but with the latest software infrastructure (Common Infrastructure for Modeling the Earth, CIME5.0), compiler versions, and software libraries. We also compare these against ensemble simulations conducted using the original version of the software infrastructure (CIME4.0) of the earlier model configuration, but with the latest compilers and software libraries, to test the impact of new compilers and libraries in isolation from additional software infrastructure. The two-sample equality of distribution tests indicates that these ensembles indeed represent the same climate.

Liang, X.-Z., Sun, C., Zheng, X., Dai, Y., Xu, M., Choi, H. I., et al. (2018). CWRF performance at downscaling China climate characteristics. Climate Dynamics, 1–26. (1) (2)

@article{liangCWRFPerformanceDownscaling2018,
author = "Liang, Xin-Zhong and Sun, Chao and Zheng, Xiaohui and Dai, Yongjiu and Xu, Min and Choi, Hyun I. and Ling, Tiejun and Qiao, Fengxue and Kong, Xianghui and Bi, Xunqiang and Song, Lianchun and Wang, Fang",
title = "CWRF performance at downscaling China climate characteristics",
issn = "0930-7575, 1432-0894",
language = "en",
urldate = "2018-06-09",
journal = "Climate Dynamics",
month = "May",
year = "2018",
pages = "1--26" }
The performance of the regional Climate-Weather Research and Forecasting model (CWRF) for downscaling China climate characteristics is evaluated using a 1980–2015 simulation at 30 km grid spacing driven by the ECMWF Interim reanalysis (ERI). It is shown that CWRF outperforms the popular Regional Climate Modeling system (RegCM4.6) in key features including monsoon rain bands, diurnal temperature ranges, surface winds, interannual precipitation and temperature anomalies, humidity couplings, and 95th percentile daily precipitation. Even compared with ERI, which assimilates surface observations, CWRF better represents the geographic distributions of seasonal mean climate and extreme precipitation. These results indicate that CWRF may significantly enhance China climate modeling capabilities.

Levine, P. A., Randerson, J. T., Chen, Y., Pritchard, M. S., Xu, M., & Hoffman, F. M. (2018). Soil Moisture Variability Intensifies and Prolongs Eastern Amazon Temperature and Carbon Cycle Response to El Niño–Southern Oscillation. Journal of Climate, 32(4), 1273–1292. (1) (2)

@article{levineSoilMoistureVariability2018,
author = "Levine, Paul A. and Randerson, James T. and Chen, Yang and Pritchard, Michael S. and Xu, Min and Hoffman, Forrest M.",
title = "Soil Moisture Variability Intensifies and Prolongs Eastern Amazon Temperature and Carbon Cycle Response to El Niño–Southern Oscillation",
volume = "32",
issn = "0894-8755",
number = "4",
urldate = "2019-01-31",
journal = "Journal of Climate",
month = "December",
year = "2018",
pages = "1273--1292" }
El Niño–Southern Oscillation (ENSO) is an important driver of climate and carbon cycle variability in the Amazon. Sea surface temperature (SST) anomalies in the equatorial Pacific drive teleconnections with temperature directly through changes in atmospheric circulation. These circulation changes also impact precipitation and, consequently, soil moisture, enabling additional indirect effects on temperature through land–atmosphere coupling. To separate the direct influence of ENSO SST anomalies from the indirect effects of soil moisture, a mechanism-denial experiment was performed to decouple their variability in the Energy Exascale Earth System Model (E3SM) forced with observed SSTs from 1982 to 2016. Soil moisture variability was found to amplify and extend the effects of SST forcing on eastern Amazon temperature and carbon fluxes in E3SM. During the wet season, the direct, circulation-driven effect of ENSO SST anomalies dominated temperature and carbon cycle variability throughout the Amazon. During the following dry season, after ENSO SST anomalies had dissipated, soil moisture variability became the dominant driver in the east, explaining 67\%–82\% of the temperature difference between El Niño and La Niña years, and 85\%–91\% of the difference in carbon fluxes. These results highlight the need to consider the interdependence between temperature and hydrology when attributing the relative contributions of these factors to interannual variability in the terrestrial carbon cycle. Specifically, when offline models are forced with observations or reanalysis, the contribution of temperature may be overestimated when its own variability is modulated by hydrology via land–atmosphere coupling.

Xu, M., & Hoffman, F. (2015). Evaluations of CMIP5 simulations over cropland. In (Vol. 9610, pp. 961003-961003-15). (1) (2)

@inproceedings{xuEvaluationsCMIP5Simulations2015,
author = "Xu, Min and Hoffman, Forrest",
title = "Evaluations of CMIP5 simulations over cropland",
volume = "9610",
urldate = "2015-09-08",
year = "2015",
pages = "961003--961003--15" }
Cropland is the major source of carbon lost to the atmosphere and contribute directly to emissions of greenhouse gases. There is, however, large potential for cropland to reduce its carbon ux to the atmosphere and sequester soil carbon through soil and crop managements. The managements include no-tillage, perennial and/or deep root crops, irrigation, and organic fertilization etc. But these estimations over cropland remain largest uncertain among all other terrestrial biomes. In most models in CMIP5, the cropland is generally treated similarly as grassland without accounting for realistic crop phenology and physiology processes and crop and soil manage- ments. In this study, we will evaluate how well cropland is represented in CMIP5 simulations and how to improve the representations and reduce the uncertainties over cropland. We will compare the modeled biogeochemical variables against multiple observational data including various remote sensing products and in-situ data.

Ling, T., Xu, M., Liang, X.-Z., Wang, J. X. L., & Noh, Y. (2015). A multilevel ocean mixed layer model resolving the diurnal cycle: Development and validation. Journal of Advances in Modeling Earth Systems, n/a–n/a. (1) (2)

@article{lingMultilevelOceanMixed2015,
author = "Ling, Tiejun and Xu, Min and Liang, Xin-Zhong and Wang, Julian X.L. and Noh, Yign",
title = "A multilevel ocean mixed layer model resolving the diurnal cycle: Development and validation",
issn = "1942-2466",
shorttitle = "A multilevel ocean mixed layer model resolving the diurnal cycle",
language = "en",
urldate = "2015-09-01",
journal = "Journal of Advances in Modeling Earth Systems",
month = "August",
year = "2015",
pages = "n/a--n/a" }
The representation of transient air-sea interactions is critical to the prediction of the sea surface temperature diurnal cycle and daily variability. This study develops a multi-level upper ocean model to more realistically resolve these interactions. The model is based on the one-dimensional turbulence kinetic energy closure developed by Noh et al. [2011], and incorporates new numerical techniques and improved schemes for model physics. The primary improvements include: (1) a surface momentum flux penetration scheme to better depict velocity shear in the diurnal mixed layer; (2) a solar penetration scheme to improve the penetration of visible and near-infrared bands of solar radiation into the mixed layer ocean; (3) a scheme to resolve the cool-skin and warm-layer effects on sea skin temperature; (4) a vertical grid stretch scheme to achieve higher near-surface resolution with fewer vertical levels; (5) a trapezoidal time integration scheme for flexible time steps; (6) a relaxation term of the previous daily mean difference between observed and modeled the sea surface temperature. According to the numerical experiments based on the TOGA-COARE IMET mooring buoy data and the validation by observations from the National Data Buoy Center, NOAA, the results indicate that the new upper ocean mixed layer model improves the simulation of the diurnal cycle of SST and sea skin temperature, especially in amplitude. This article is protected by copyright. All rights reserved.

Sreepathi, S., Xu, M., Collier, N., Kumar, J., Mao, J., & Hoffman, F. M. (n.d.). Land Model Testbed: Accelerating Development, Benchmarking and Analysis of Land Surface Models. (1) (2)

@article{sreepathiLandModelTestbed,
author = "Sreepathi, Sarat and Xu, Min and Collier, Nathan and Kumar, Jitendra and Mao, Jiafu and Hoffman, Forrest M",
title = "Land Model Testbed: Accelerating Development, Benchmarking and Analysis of Land Surface Models",
language = "en" }
A Land Model Testbed (LMT), designed to provide a computational framework for systematically assessing model ﬁdelity and supporting rapid development of complex multiscale models, offers a general-purpose workﬂow for conducting large ensemble simulations of multiple land surface models, postprocessing large volumes of model output, and evaluating model results. It leverages existing tools for launching model simulations and the International Land Model Benchmarking (ILAMB) package for assessing model ﬁdelity through comparison with best-available observational datasets. Increased complexity and proliferation of uncertain parameters in process representations in land surface models has driven the need for frequent and intensive testing and evaluating of models to quantify uncertainties and optimize parameters such that results are consistent with observations. The LMT described here meets these needs by providing tools to run thousands of ensemble simulations simultaneously and post-process their output ﬁles, by automating execution of an enhanced version of ILAMB with site-speciﬁc benchmarks and multivariate functional relationships, and by offering ensemble diagnostics and a customizable dashboard for displaying model performance metrics and associated graphics. We envision the LMT capabilities will serve as a foundational computational resource for a proposed user facility focused on terrestrial multiscale model–data integration.