ai2cm/pace: v0.1.0 GMD release

This is the initial release of the full Pace ecosystem at time of submission to Geoscientific Model Development (GMD). This is alpha research software not meant for production use. What's Changed Add submodules for existing repositories by @mcgibbon in https://github.com/ai2cm/pace/pull/1 does this do a thing? by @twicki in https://github.com/ai2cm/pace/pull/3 Docker image for top level directory by @elynnwu in https://github.com/ai2cm/pace/pull/4 Add physics savepoint test by @elynnwu in https://github.com/ai2cm/pace/pull/5 Add plotting tools by @elynnwu in https://github.com/ai2cm/pace/pull/6 Update sumodules to latest by @elynnwu in https://github.com/ai2cm/pace/pull/7 Add dependencies graph for project to README by @mcgibbon in https://github.com/ai2cm/pace/pull/8 Add script to check if package or its dependencies have changed for tests by @mcgibbon in https://github.com/ai2cm/pace/pull/9 Feature/plotting tools by @FlorianDeconinck in https://github.com/ai2cm/pace/pull/13 Merge submodules as subtrees by @mcgibbon in https://github.com/ai2cm/pace/pull/15 change cache plan location by @twicki in https://github.com/ai2cm/pace/pull/18 Allow objects with storage attribute as call arguments to FrozenStencil by @mcgibbon in https://github.com/ai2cm/pace/pull/19 Update cache location by @elynnwu in https://github.com/ai2cm/pace/pull/20 Add jenkins at top level by @elynnwu in https://github.com/ai2cm/pace/pull/24 fix grid tests for 54 ranks by @rheacangeo in https://github.com/ai2cm/pace/pull/17 Jenkins physics tests by @elynnwu in https://github.com/ai2cm/pace/pull/21 override failing tests by @rheacangeo in https://github.com/ai2cm/pace/pull/26 fv3core uses the MetricTerms by @rheacangeo in https://github.com/ai2cm/pace/pull/16 Merge buildenv and daint_venv as files in pace by @mcgibbon in https://github.com/ai2cm/pace/pull/30 Re-enable CircleCI for linting and for fv3gfs-util by @mcgibbon in https://github.com/ai2cm/pace/pull/22 Jenkins env name clarification by @elynnwu in https://github.com/ai2cm/pace/pull/28 Make jenkins green by @rheacangeo in https://github.com/ai2cm/pace/pull/32 Enable tests for fv3core infrastructure code on CircleCI by @mcgibbon in https://github.com/ai2cm/pace/pull/23 baroclinic initialization by @rheacangeo in https://github.com/ai2cm/pace/pull/31 Clean up old files in physics by @elynnwu in https://github.com/ai2cm/pace/pull/34 Add file caching infrastructure and test for fv3gfs-util using it by @mcgibbon in https://github.com/ai2cm/pace/pull/27 Point daint_venv GT4PY_VERSION.txt reference to pace git repo by @elynnwu in https://github.com/ai2cm/pace/pull/37 Feature/add shared by @elynnwu in https://github.com/ai2cm/pace/pull/25 Restore buildenv as a submodule instead of hard files by @mcgibbon in https://github.com/ai2cm/pace/pull/38 Make jenkins green... again by @rheacangeo in https://github.com/ai2cm/pace/pull/35 Enable flake8 tests on fv3gfs-util by @mcgibbon in https://github.com/ai2cm/pace/pull/42 run_on_daint fixed for more layouts by @rheacangeo in https://github.com/ai2cm/pace/pull/41 Rename fv3gfs-util to pace-util by @mcgibbon in https://github.com/ai2cm/pace/pull/43 Add required backend arg to make_storage methods by @mcgibbon in https://github.com/ai2cm/pace/pull/40 Physics use GridData by @elynnwu in https://github.com/ai2cm/pace/pull/46 fix the baroclinic test case option by @rheacangeo in https://github.com/ai2cm/pace/pull/45 Enable linting in physics by @elynnwu in https://github.com/ai2cm/pace/pull/48 Fix/enable physics linting by @elynnwu in https://github.com/ai2cm/pace/pull/49 Re-enable KCacheDetection for FVSubgridZ.init stencil by @eddie-c-davis in https://github.com/ai2cm/pace/pull/52 Add unit tests of DynamicalCore.call by @mcgibbon in https://github.com/ai2cm/pace/pull/47 Test all packages when non-subproject files change by @mcgibbon in https://github.com/ai2cm/pace/pull/55 stop re-downloading of data on gsutil fail by @rheacangeo in https://github.com/ai2cm/pace/pull/58 update_dwind_phys 54 ranks fix by @rheacangeo in https://github.com/ai2cm/pace/pull/57 Feature/move gt4py utils to pace dsl by @elynnwu in https://github.com/ai2cm/pace/pull/50 physics and dycore share constants by @rheacangeo in https://github.com/ai2cm/pace/pull/51 Gt4py is used as a submodule by @mcgibbon in https://github.com/ai2cm/pace/pull/56 Revert "Gt4py is used as a submodule" by @mcgibbon in https://github.com/ai2cm/pace/pull/59 Remove env.sh and references to it, use buildenv submodule instead by @mcgibbon in https://github.com/ai2cm/pace/pull/53 fix a typo in the overrides file by @rheacangeo in https://github.com/ai2cm/pace/pull/63 parallel and sequential tests cache compiled stencil to the same folder by @rheacangeo in https://github.com/ai2cm/pace/pull/65 Add print statements when fetching gt_cache by @elynnwu in https://github.com/ai2cm/pace/pull/62 init baroclinic gpu by @rheacangeo in https://github.com/ai2cm/pace/pull/67 Consolidate make_storage_from_shape by @elynnwu in https://github.com/ai2cm/pace/pull/68 update buildenv directory in run_on_daint.sh by @mcgibbon in https://github.com/ai2cm/pace/pull/69 Allow relative path in install virtualenv by @elynnwu in https://github.com/ai2cm/pace/pull/70 ensure that artifacts are stored if in the main branch by @twicki in https://github.com/ai2cm/pace/pull/71 create template again by @twicki in https://github.com/ai2cm/pace/pull/72 Consolidate daint logic into buildenv (env vars) and daint_venv (python packages) by @mcgibbon in https://github.com/ai2cm/pace/pull/54 remove G2G arg from run_command, consolidate gpu logic in buildenv by @mcgibbon in https://github.com/ai2cm/pace/pull/76 Remove global access in fv3core and physics tests by @elynnwu in https://github.com/ai2cm/pace/pull/60 Add timesteps arg to run_standalone.sh by @mcgibbon in https://github.com/ai2cm/pace/pull/77 Bump buildenv from fd940f2 to 88465c1 by @dependabot in https://github.com/ai2cm/pace/pull/78 Remove references to s1053, use installdir instead by @mcgibbon in https://github.com/ai2cm/pace/pull/80 add pace driver, update dycore and physics state by @rheacangeo in https://github.com/ai2cm/pace/pull/79 Bump buildenv from fc27296 to e72a0c7 by @dependabot in https://github.com/ai2cm/pace/pull/85 driver final review adjustments by @rheacangeo in https://github.com/ai2cm/pace/pull/86 Update PR checklist and readme by @elynnwu in https://github.com/ai2cm/pace/pull/87 pace-util release v0.7.0 by @mcgibbon in https://github.com/ai2cm/pace/pull/88 fix pace-util make release by adding dependent targets by @mcgibbon in https://github.com/ai2cm/pace/pull/89 Bump external/fv3gfs-wrapper from 9d62bbf to 5e9dc78 by @dependabot in https://github.com/ai2cm/pace/pull/74 Bump external/fv3gfs-fortran from 9d76b0c to 5db2c24 by @dependabot in https://github.com/ai2cm/pace/pull/73 fix script argument bug in buildenv by @mcgibbon in https://github.com/ai2cm/pace/pull/90 stabilize fv3core performance by @rheacangeo in https://github.com/ai2cm/pace/pull/91 DycoreState uses Quantity attributes by @mcgibbon in https://github.com/ai2cm/pace/pull/83 Cleanup pace stencils testing by @elynnwu in https://github.com/ai2cm/pace/pull/84 refactor DycoreState initialization routines to use only GridSizer when possible by @mcgibbon in https://github.com/ai2cm/pace/pull/92 Pace util mostly does not depend on dsl by @elynnwu in https://github.com/ai2cm/pace/pull/93 Gt4py is used as a submodule by @mcgibbon in https://github.com/ai2cm/pace/pull/64 Remove unused run_cached_fv_dynamics.sh by @mcgibbon in https://github.com/ai2cm/pace/pull/95 driver can run from serialized init data again and is tested by @rheacangeo in https://github.com/ai2cm/pace/pull/96 Renaming DummyComm to LocalComm, Adding Tags by @oelbert in https://github.com/ai2cm/pace/pull/97 Add Sphinx documentation by @mcgibbon in https://github.com/ai2cm/pace/pull/94 Namelist class in pace.util by @elynnwu in https://github.com/ai2cm/pace/pull/98 Update top level documentation by @elynnwu in https://github.com/ai2cm/pace/pull/100 Docker runs gtc cpu backend by @elynnwu in https://github.com/ai2cm/pace/pull/101 Driver demo for AMS by @mcgibbon in https://github.com/ai2cm/pace/pull/99 remove fv3gfs-wrapper, fv3gfs-fortran, and wrapper docker infrastructure by @mcgibbon in https://github.com/ai2cm/pace/pull/102 Remove unused with current namelist sum_te in remapping by @FlorianDeconinck in https://github.com/ai2cm/pace/pull/104 Add edge_interior_ratio argument to TilePartitioner by @yniederm in https://github.com/ai2cm/pace/pull/44 Feature/driver init from serialbox by @elynnwu in https://github.com/ai2cm/pace/pull/103 Passing down q_min through in tracer remapping by @FlorianDeconinck in https://github.com/ai2cm/pace/pull/105 Add Yannick to contributors by @mcgibbon in https://github.com/ai2cm/pace/pull/108 Dockerfile for plotting maps in pace by @elynnwu in https://github.com/ai2cm/pace/pull/106 Report stencil argument intents, update fv3core docstrings by @mcgibbon in https://github.com/ai2cm/pace/pull/107 fix daint compilation by @rheacangeo in https://github.com/ai2cm/pace/pull/114 Adding In/Out to Remapping by @oelbert in https://github.com/ai2cm/pace/pull/110 Update requirements to Python 3.8 by @elynnwu in https://github.com/ai2cm/pace/pull/113 Remove pytest dependency from non-testing files by @jdahm in https://github.com/ai2cm/pace/pull/111 Feature/physics cache plan by @elynnwu in https://github.com/ai2cm/pace/pull/82 convert u_contra_dxc and v_contra_dyc to temporaries, remove ptc input form D_SW by @mcgibbon in https://github.com/ai2cm/pace/pull/115 testing new buildenv by @rheacangeo in https://github.com/ai2cm/pace/pull/121 switch to https submodule by @elynnwu in https://github.com/ai2cm/pace/pull/127 Daint compilation by @rheacangeo in https://github.com/ai2cm/pace/pull/128 Driver outputs performance timing by @elynnwu in https://github.com/ai2cm/pace/pull/109 Clarify tmp_wk and tmp_vort usages in d_sw.py by @mcgibbon in https://github.com/ai2cm/pace/pull/118 fix mirror_grid for even layouts by @rheacangeo in https://github.com/ai2cm/pace/pull/132 Driver diagnostics write out lat/lon info by @elynnwu in https://github.com/ai2cm/pace/pull/125 merge main_uc and main_vc stencils in fxadv by @mcgibbon in https://github.com/ai2cm/pace/pull/129 single node compilation by @rheacangeo in https://github.com/ai2cm/pace/pull/133 Fix translate init tests by @elynnwu in https://github.com/ai2cm/pace/pull/137 Clean up concrete externals for fv3core, even on failure by @mcgibbon in https://github.com/ai2cm/pace/pull/140 Transfer dace changes for fvtp2d/d_sw, remove CopiedCorners by @mcgibbon in https://github.com/ai2cm/pace/pull/139 Update grid thresholds for higher resolution by @oelbert in https://github.com/ai2cm/pace/pull/144 add Driver regression test, change data organization by @rheacangeo in https://github.com/ai2cm/pace/pull/138 Pin click version when running black in pre-commit by @mcgibbon in https://github.com/ai2cm/pace/pull/148 Use https URL for buildenv by @jdahm in https://github.com/ai2cm/pace/pull/150 Add compare_to_numpy=True to StencilConfig by @mcgibbon in https://github.com/ai2cm/pace/pull/152 Add CachingCommReader and CachingCommWriter by @mcgibbon in https://github.com/ai2cm/pace/pull/153 Feature/jenkins baroclinic init plan by @elynnwu in https://github.com/ai2cm/pace/pull/135 Format source when using numpy backend by @jdahm in https://github.com/ai2cm/pace/pull/149 Update to latest gt4py by @jdahm in https://github.com/ai2cm/pace/pull/143 Remove numpy -> gtc:numpy hack by @jdahm in https://github.com/ai2cm/pace/pull/160 Fix/jenkins cache setup by @elynnwu in https://github.com/ai2cm/pace/pull/161 Add pace.driver.run config for CachingCommReader, CachingCommWriter by @mcgibbon in https://github.com/ai2cm/pace/pull/157 Delete deprecated files by @mcgibbon in https://github.com/ai2cm/pace/pull/163 Add gt4py cache settings to env.daint.sh by @mcgibbon in https://github.com/ai2cm/pace/pull/164 Split pace.driver.run module into multiple files by @mcgibbon in https://github.com/ai2cm/pace/pull/165 Add a configuration option to driver to disable step_physics call by @mcgibbon in https://github.com/ai2cm/pace/pull/169 util/v0.8.0 + zarr monitor: allow transposed diagnostics to be written by @brianhenn in https://github.com/ai2cm/pace/pull/170 merge two stencils in d_sw, clarify kinetic energy variable by @mcgibbon in https://github.com/ai2cm/pace/pull/146 disable driver diagnostics if no path given by @mcgibbon in https://github.com/ai2cm/pace/pull/172 Fix/driver translate test on Jenkins by @elynnwu in https://github.com/ai2cm/pace/pull/173 Remove SerialboxConfig initialization configuration from pace.driver by @mcgibbon in https://github.com/ai2cm/pace/pull/168 Consolidate driver registration code for init and comm configuration by @mcgibbon in https://github.com/ai2cm/pace/pull/167 Add Jupyter notebook example running driver with write-read by @mcgibbon in https://github.com/ai2cm/pace/pull/174 pace-util: zarr monitor quantity attribute handling by @brianhenn in https://github.com/ai2cm/pace/pull/175 pace-util release v0.9.0 by @brianhenn in https://github.com/ai2cm/pace/pull/176 consolidate imports of mpi4py into pace.util, remove fv3core null_comm by @mcgibbon in https://github.com/ai2cm/pace/pull/179 Fix Overrides by @rheacangeo in https://github.com/ai2cm/pace/pull/178 Refactor optional dependencies, make zarr optional in pace.driver by @mcgibbon in https://github.com/ai2cm/pace/pull/180 Add restart capability to pace.driver by @elynnwu in https://github.com/ai2cm/pace/pull/185 Fix/remove RESTART instead of make clean by @elynnwu in https://github.com/ai2cm/pace/pull/187 Fix jenkins driver test by @elynnwu in https://github.com/ai2cm/pace/pull/189 Move to data version 8.0.5 by @rheacangeo in https://github.com/ai2cm/pace/pull/155 Add NullCheckpointer called for C_SW and D_SW by @mcgibbon in https://github.com/ai2cm/pace/pull/190 Update gt4py by @jdahm in https://github.com/ai2cm/pace/pull/183 Use netcdf data for savepoint regressions in fv3core by @mcgibbon in https://github.com/ai2cm/pace/pull/193 Revert serialbox init by @elynnwu in https://github.com/ai2cm/pace/pull/195 hotfix grid for caching by @twicki in https://github.com/ai2cm/pace/pull/197 Add comment about fortran restart format by @mcgibbon in https://github.com/ai2cm/pace/pull/202 Remove temporary backend string replace calls by @jdahm in https://github.com/ai2cm/pace/pull/207 Fix save_netcdf for sequential savepoints by @mcgibbon in https://github.com/ai2cm/pace/pull/201 Refactor dycore translate tests to use Namelist class by @mcgibbon in https://github.com/ai2cm/pace/pull/205 Add SnapshotCheckpointer by @mcgibbon in https://github.com/ai2cm/pace/pull/200 Zero heat_source temporary between dycore calls by @mcgibbon in https://github.com/ai2cm/pace/pull/196 Fix/restart gpu backend by @elynnwu in https://github.com/ai2cm/pace/pull/212 Use netCDF data loading in physics tests by @mcgibbon in https://github.com/ai2cm/pace/pull/209 add diagnostics frequency by @twicki in https://github.com/ai2cm/pace/pull/194 Add plotting script by @elynnwu in https://github.com/ai2cm/pace/pull/199 DaCe orchestration by @FlorianDeconinck in https://github.com/ai2cm/pace/pull/124 Write intermediate restart files when specified by @elynnwu in https://github.com/ai2cm/pace/pull/215 Set gt_cache directory at module import time and remove future stencil by @jdahm in https://github.com/ai2cm/pace/pull/188 Update gt4py by @jdahm in https://github.com/ai2cm/pace/pull/204 Updated readme for pace by @ajdas1 in https://github.com/ai2cm/pace/pull/221 Add support for halo updates in TileCommunicator, use in dycore by @mcgibbon in https://github.com/ai2cm/pace/pull/216 Revert "Add support for halo updates in TileCommunicator, use in dycore" by @mcgibbon in https://github.com/ai2cm/pace/pull/226 Documentation/pace doc update by @ajdas1 in https://github.com/ai2cm/pace/pull/230 Collect build and execution timings by @jdahm in https://github.com/ai2cm/pace/pull/220 Fix cache building by @jdahm in https://github.com/ai2cm/pace/pull/229 Match timings by @jdahm in https://github.com/ai2cm/pace/pull/224 Restore dynamical core statefulness test by @mcgibbon in https://github.com/ai2cm/pace/pull/234 Add test of physics statefulness to CircleCI by @mcgibbon in https://github.com/ai2cm/pace/pull/235 Automatically calculate translate thresholds using perturbations by @mcgibbon in https://github.com/ai2cm/pace/pull/222 Add support for halo updates in TileCommunicator, use in dycore by @mcgibbon in https://github.com/ai2cm/pace/pull/227 Fix caching plan by @elynnwu in https://github.com/ai2cm/pace/pull/239 Add top-level test directory for "main" tests by @mcgibbon in https://github.com/ai2cm/pace/pull/236 Fixed an unallocated temporary causing NaNs in qvapor by @oelbert in https://github.com/ai2cm/pace/pull/242 Diagnostics include column integrated tracer by @elynnwu in https://github.com/ai2cm/pace/pull/219 fix race condition by @twicki in https://github.com/ai2cm/pace/pull/244 Ajda pointed out a deprecated link by @twicki in https://github.com/ai2cm/pace/pull/247 Reorchestrate all modules by @FlorianDeconinck in https://github.com/ai2cm/pace/pull/228 Bad timestep calculation by @FlorianDeconinck in https://github.com/ai2cm/pace/pull/249 Efficient building of caches by @twicki in https://github.com/ai2cm/pace/pull/248 Use much simpler dockerfile for make test targets by @mcgibbon in https://github.com/ai2cm/pace/pull/241 Add "pair_debug" configuration option for driver by @mcgibbon in https://github.com/ai2cm/pace/pull/252 Run make savepoint_tests_mpi on CircleCI by @mcgibbon in https://github.com/ai2cm/pace/pull/218 Driver: increment time and delete unused function by @elynnwu in https://github.com/ai2cm/pace/pull/258 fix fv3core daint_venv path in performance plan by @mcgibbon in https://github.com/ai2cm/pace/pull/259 Ingest code for GT4py by @twicki in https://github.com/ai2cm/pace/pull/254 Fix/potential race condition by @elynnwu in https://github.com/ai2cm/pace/pull/260 Fix normalize vector by @ajdas1 in https://github.com/ai2cm/pace/pull/263 Enable fetching basic test data from public FTP server. by @ofuhrer in https://github.com/ai2cm/pace/pull/264 Revise top-level README.md documentation by @ofuhrer in https://github.com/ai2cm/pace/pull/265 Add LICENSE.md by @ofuhrer in https://github.com/ai2cm/pace/pull/267 Feature/refactor grid data by @ajdas1 in https://github.com/ai2cm/pace/pull/271 Notebook stencil by @ajdas1 in https://github.com/ai2cm/pace/pull/246 Ensure kslice_from_inputs makes a slice of ints by @jdahm in https://github.com/ai2cm/pace/pull/273 Run init tests on savepoint_tests_mpi by @jdahm in https://github.com/ai2cm/pace/pull/275 Update README.md by @ofuhrer in https://github.com/ai2cm/pace/pull/277 Fix fetching test data from FTP server by @ofuhrer in https://github.com/ai2cm/pace/pull/282 clean up checking by @twicki in https://github.com/ai2cm/pace/pull/284 Fix the model inconsistencies between 6 and 54 rank runs by @twicki in https://github.com/ai2cm/pace/pull/283 do not build physics if disabled by @twicki in https://github.com/ai2cm/pace/pull/290 Clean up test decomposition by @elynnwu in https://github.com/ai2cm/pace/pull/292 Add ValidationCheckpointer and ThresholdCalibrationCheckpointer by @mcgibbon in https://github.com/ai2cm/pace/pull/287 Stable orchestration by @FlorianDeconinck in https://github.com/ai2cm/pace/pull/250 Update gt4py on August 19, 2022 by @jdahm in https://github.com/ai2cm/pace/pull/285 CI visualize validation against fortran by @elynnwu in https://github.com/ai2cm/pace/pull/280 Speed up initialization by @twicki in https://github.com/ai2cm/pace/pull/294 Update gt4py by @jdahm in https://github.com/ai2cm/pace/pull/296 Replace dace_ with dace: in backend name by @jdahm in https://github.com/ai2cm/pace/pull/298 Consolidate makefile logic for data download into top level by @mcgibbon in https://github.com/ai2cm/pace/pull/293 DaCe VRAM pooling by @FlorianDeconinck in https://github.com/ai2cm/pace/pull/295 read version from top level get data Makefile by @elynnwu in https://github.com/ai2cm/pace/pull/305 Rename packages and package directories to be consistent by @mcgibbon in https://github.com/ai2cm/pace/pull/304 Feature/example notebooks in circle ci by @ajdas1 in https://github.com/ai2cm/pace/pull/303 Add 54rank savepoint tests to CircleCI by @mcgibbon in https://github.com/ai2cm/pace/pull/255 Make DycoreState a statically-defined type by @mcgibbon in https://github.com/ai2cm/pace/pull/308 Fix caching of gt4py caches in 54rank test by @mcgibbon in https://github.com/ai2cm/pace/pull/310 Extend CircleCI to include sequential tests and cpu backend by @mcgibbon in https://github.com/ai2cm/pace/pull/311 Fix run_standalone execution by @mcgibbon in https://github.com/ai2cm/pace/pull/312 Update the fv3core readme to include bare-metal installation instructions by @oelbert in https://github.com/ai2cm/pace/pull/266 Introduce kernel_theoritical_timing from a SDFG by @FlorianDeconinck in https://github.com/ai2cm/pace/pull/306 Microphysics savepoint test passes on standard case by @elynnwu in https://github.com/ai2cm/pace/pull/313 Feature/transform grid stretch by @ajdas1 in https://github.com/ai2cm/pace/pull/316 Add test that init is decomposition invariant by @mcgibbon in https://github.com/ai2cm/pace/pull/317 Update gt4py on 2022.09.08 by @jdahm in https://github.com/ai2cm/pace/pull/315 Hack in -Wno-set-but-unused-variable warning for fv3core by @jdahm in https://github.com/ai2cm/pace/pull/321 Consolidate reqs into top level requirements_dev.txt by @mcgibbon in https://github.com/ai2cm/pace/pull/307 DaCe Orchestration: fix parsing, de-inline grid data scalars & m