xbitinfo: Retrieve information content and compress accordingly#

https://github.com/observingClouds/xbitinfo/actions/workflows/ci.yaml/badge.svg https://results.pre-commit.ci/badge/github/observingClouds/xbitinfo/main.svg https://img.shields.io/pypi/v/xbitinfo.svg https://img.shields.io/readthedocs/bitinfo/latest.svg?style=flat https://mybinder.org/badge_logo.svg

This is an xarray-wrapper around BitInformation.jl to retrieve and apply bitrounding from within python. The package intends to present an easy pipeline to compress (climate) datasets based on the real information content.

How the science works#

Paper#

Klöwer, M., Razinger, M., Dominguez, J. J., Düben, P. D., & Palmer, T. N. (2021). Compressing atmospheric data into its real information content. Nature Computational Science, 1(11), 713–724. doi: 10/gnm4jj

Video#

https://img.youtube.com/vi/kcbOdwfskmY/0.jpg

Julia Repository#

BitInformation.jl

How to install#

You can install the latest release of xbitinfo soon using pip:

pip install xbitinfo

You can also install the bleeding edge (pre-release versions) by running:

pip install git+https://github.com/observingClouds/xbitinfo

pip install git+https://github.com/observingClouds/xbitinfo.git#egg=xbitinfo[complete]

How to use#

import xarray as xr
import xbitinfo as xb

ds = xr.tutorial.load_dataset(inpath)
bitinfo = xb.get_bitinformation(ds, dim="lon")  # calling bitinformation.jl.bitinformation
keepbits = xb.get_keepbits(bitinfo, inflevel=0.99)  # get number of mantissa bits to keep for 99% real information
ds_bitrounded = xb.xr_bitround(ds, keepbits)  # bitrounding keeping only keepbits mantissa bits
ds_bitrounded.to_compressed_netcdf(outpath)  # save to netcdf with compression

Credits#

Getting Started

Help & Reference