Skip to contents

Changelog

Source: NEWS.md

greta 0.6.0

Changes

  • as.unknowns() now handles plain numeric vectors (and dim<- being set to NULL), fixing an mcmc() error “no applicable method for ‘as.unknowns’” that surfaced when re-building vignettes under R-devel (#582).
  • log.greta_array() function warns if user uses the base arg, as it was unused, (#597).
  • outer() (and %o%) now works with greta arrays when FUN = "*", instead of silently returning a base array of NAs; base R’s "*" fast path used as.vector(), which dropped the greta operation (#582).
  • Reshaping a greta array with dim<- now keeps the ? placeholder display for unknown values instead of showing NA (#582).
  • Add warmup information to MCMC print method (#652, resolved by #755).
  • Add more options to level of detail in greta_sitrep() with “verbosity” argument. There are three levels, “minimal” (default), “detailed”, and “quiet”. (#612, resolved by #679).
  • Use .batch_size instead of batch_size internally, to avoid rare name clash errors (#634).
  • Resolve issues with Tensorflow version in DESCRIPTION (no longer can specify == 2.16.0, must be >= 2.16.0).
  • When the number of cores requested exceeds the number of cores detected, then the number of cores detected will be used.
  • Ensure n_cores arg defaults to 2 cores, and chains defaults to 2 chains.
  • Cap TensorFlow’s internal CPU threadpool inside vignette builds (via TF_NUM_INTRAOP_THREADS and TF_NUM_INTEROP_THREADS) so CRAN’s CPU/elapsed timing on vignette rebuild stays under the two-core limit (#796).
  • greta now also caps TensorFlow’s CPU thread pools at 2 when running under R CMD check (detected via the _R_CHECK_LIMIT_CORES_ environment variable), so checks on CRAN machines respect the two-core limit (#796).

Installation and dependencies

  • greta now resolves its Python environment more flexibly instead of always forcing reticulate’s managed (uv) environment: it respects a user-set RETICULATE_PYTHON, a stored greta preference, or an existing greta-env-tf2 conda environment, and otherwise uses the managed (uv) environment (#801).
  • The managed (uv) backend now works without an internet connection, by enabling uv’s offline mode (via environment variable UV_OFFLINE) once reticulate’s uv cache is installed. You can set UV_OFFLINE=0 (or =1) yourself to force online or offline resolution, respectively (#814).
  • Fixed library(greta) failing with an error when greta’s stored Python preference file exists but is empty (#809).
  • greta_deps_spec() now only checks that the requested TensorFlow version is one greta supports (TensorFlow 2.16 and later are not supported, as they ship Keras 3); compatible TensorFlow Probability and Python versions are left to uv or conda to resolve rather than being validated against a fixed compatibility table (#675).
  • greta_list_py_modules() shows the Python packages installed in a specific TF2 environment (#801, #809).
  • greta_remove() consolidates greta’s Python removal helpers behind a single what argument to remove the greta-env-tf2 conda environment, miniconda, reticulate’s uv cache, the stored Python preference, the stored dependency versions, or all of them. This means destroy_greta_deps(), greta_remove_all_deps(), remove_greta_env(), remove_miniconda(), and remove_reticulate_uv_cache(), are now superceded by greta_remove() (#814).
  • greta_remove() no longer leaves a stale, deleted environment active for the rest of the session: it now invalidates greta’s cached Python backend and nudges you to restart R if you try to use greta again without restarting, instead of silently failing or falsely reporting the removed environment as still available.
  • reinstall_greta_env(), and reinstall_miniconda() are now deprecated in favour of reinstall_greta_deps() (#814).
  • greta_set_deps() persistently chooses which TensorFlow, TensorFlow Probability, and Python versions greta uses; the managed (uv) environment installs them on next load and install_greta_deps() uses them as its default; clear with greta_remove("deps") (#817).
  • greta_set_python() and greta_reset_python() let you choose, persistently, which Python environment greta uses - the managed (uv) environment (backend = "uv", the default), a conda environment (backend = "conda"), or a specific Python (backend = "path"); each reports the stored preference, warns if RETICULATE_PYTHON takes precedence, and shows what greta will resolve to on its next load (#801, #809, #817).
  • greta_set_python("path", path = ...) accepts either a Python binary or an environment directory (a virtualenv or conda prefix), looking for bin/python (or Scripts/python.exe on Windows) inside it, which eases offline and pre-installed setups (#814).
  • greta_sitrep() now reports the resolved Python backend and whether greta can start offline: for the managed (uv) environment, the UV_OFFLINE setting and whether reticulate’s uv cache is populated; a missing conda environment is reported neutrally as “not used” on the managed (uv) backend rather than as a failure (#801, #817).
  • greta_sitrep() now requires Python 3.9 or later (previously 3.8), matching the Python versions greta supports (#809).
  • install_greta_deps() now records the location of the greta-env-tf2 conda environment at install time, so greta auto-detects it in any conda installation, not just reticulate’s miniconda (#809).
  • install_greta_deps() is no longer required for most users, as greta now installs TensorFlow and TensorFlow Probability automatically via uv on first use; it remains for installing a conda environment (for example, for offline use), which you can then select with greta_set_python("conda") (#801).
  • remove_greta_env(), remove_miniconda(), and remove_reticulate_uv_cache() ask for confirmation before removing, gain an ask argument (default interactive()) so they work non-interactively, and invisibly return whether anything was removed (#809).
  • remove_reticulate_uv_cache() removes reticulate’s uv cache; note this cache is shared by all reticulate packages and is not greta-specific, and a system-wide uv cache is left untouched (#801, #809).

greta 0.5.0

CRAN release: 2024-11-12

This version of greta uses Tensorflow 2.0.0, which comes with it a host of new very exciting features!

Optimizers

The latest interface to optimizers in tensorflow are now used, these changes are described.

  • gradient_descent gains momentum and nesterov arguments, as described here in TF docs
  • adagrad gains epsilon argument
  • removes momentum optimizer, as this has been folded into gradient_descent arguments
  • Adds amsgrad argument to adam optimizer, as described in TF docs
  • Adds adamax optimiser, see TF docs
  • Adds l2_shrinkage_regularization_strength and beta arguments to ftrl optimiser.
  • adds nadam optimiser - see docs.
  • In rms_prop optimiser, changes decay parameter to rho, and adds centered parameter - see docs

The following optimisers are removed, as they are no longer supported by Tensorflow:

Installation revamp

This release provides a few improvements to installation in greta. It should now provide more information about installation progress, and be more robust. The intention is, it should just work, and if it doesn’t, it should fail gracefully with some useful advice on problem solving.

New Print methods

  • New print method for greta_mcmc_list. This means MCMC output will be shorter and more informative (#644).
  • greta arrays now have a print method that stops them from printing too many rows into the console. Similar to MCMC print method, you can control the print output with the n argument: print(object, n = <elements to print>) (#644).

Minor

Internals

  • Internally we are replacing most of the error handling code as separate check_* functions.

  • Implemented cli::cli_abort/warn/inform() in place of cli::format_error/warning/message() + stop/warning/message(msg, call. = FALSE) pattern.

  • Uses legacy optimizer internally (Use tf$keras$optimizers$legacy$METHOD over tf$keras$optimizers$METHOD). No user impact expected.

  • Update photo of Grete Hermann (#598).

  • Use %||% internally to replace the pattern: if (is.null(x)) x <- thing with x <- x %||% thing (#630).

  • Add more explaining variables - replace if (thing & thing & what == this) with if (explanation_of_thing).

  • Refactored repeated uses of vapply into functions (#377, #658).

  • Add internal data files .deps_tf and .deps_tfp to track dependencies of TF and TFP. Related to #666.

  • Posterior density checks (#720):

    • Don’t run Geweke on CI as it takes 30 minutes to run.
    • Add thinning to Geweke tests.
    • Fix broken geweke tests from TF1–>TF2 change.
    • Increase the number of effective samples for check_samples for lkj distribution
    • Add more checks to posterior to run on CI/on each test of greta

Bug fixes

  • Fix bug where matrix multiply had dimension error before coercing to greta array. (#464)
  • Fixes for Wishart and LKJ Correlation distributions (#729 #733 #734):
    • Add bijection density to choleskied distributions.
    • Note about some issues with LKJ and our normalisation constant for the density.
    • Removed our custom forward_log_det_jacobian() function from tf_correlation_cholesky_bijector() (used in lkj_correlation()). Previously, it did not work with unknown dimensions, but it now works with them.
    • Ensure wishart uses sigma_chol in scale_tril
    • Wishart uses tf$matmul(chol_draws, chol_draws, adjoint_b = TRUE) instead of tf_chol2symm(chol_draws).
    • Test log prob function returns valid numeric numbers.
    • Addresses issue with log prob returning NaNs–replace FillTriangular with FillScaleTriL and apply Chaining to first transpose input.

greta 0.4.5

CRAN release: 2024-03-11

Bug Fixes

  • Remove trailing comma bug in glue #618

greta 0.4.4

CRAN release: 2024-02-02

Bug fixes

  • Some small documentation bugs were fixed, namely the sentinel “_PACKAGE” documentation, and various small changes to correctly export S3 methods.

greta 0.4.3

CRAN release: 2022-09-08

Features

Fixes

  • Issue where future and parallely packages error when a CPU with only one core is provided (#513, #516).
  • Removes any use of multiprocess as it is deprecated in the future package (#394)

greta 0.4.2

CRAN release: 2022-03-22

Fixes

  • workaround for M1 issues (#507)

greta 0.4.1 (2022-03-14)

CRAN release: 2022-03-15

Fixes:

  • Python is now initialised when a greta_array is created (#468).

  • head and tail S3 methods for greta_array are now consistent with head and tail methods for R versions 3 and 4 (#384).

  • greta_mcmc_list objects (returned by mcmc()) are now no longer modified by operations (like coda::gelman.diag()).

  • joint distributions of uniform variables now have the correct constraints when sampling (#377).

  • array-scalar dispatch with 3D arrays is now less buggy (#298).

  • greta now provides R versions of all of R’s primitive functions (I think), to prevent them from silently not executing (#317).

  • Uses Sys.unsetenv("RETICULATE_PYTHON") in .onload on package startup, to prevent an issue introduced with the “ghost orchid” version of RStudio where they do not find the current version of RStudio. See #444 for more details.

  • Internal change to code to ensure future continues to support parallelisation of chains. See #447 for more details.

  • greta now depends on future version 1.22.1, tensorflow (the R package) 2.7.0, and parallelly 1.29.0. This should see no changes on the user side.

API changes:

Features:

  • New install_greta_deps() - provides installation of python dependencies (#417). This saves exact versions of Python (3.7), and the python modules NumPy (1.16.4), Tensorflow (1.14.0), and Tensorflow Probability (0.7.0) into a conda environment, “greta-env”. When initialising Python, greta now searches for this conda environment first, which presents a great advantage as it isolates these exact versions of these modules from other Python installations. It is not required to use the conda environment, “greta-env”. Overall this means that users can run the function install_greta_deps(), follow the prompts, and have all the python modules they need installed, without contaminating other software that use different python modules.

  • calculate() now enables simulation of greta array values from their priors, optionally conditioned on fixed values or posterior samples. This enables prior and posterior predictive checking of models, and simulation of data.

  • A simulate() method for greta models is now also provided, to simulate the values of all greta arrays in a model from their priors.

  • variable() now accepts arrays for upper and lower, enabling users to define variables with different constraints.

  • There are three new variable constructor functions: cholesky_variable(), simplex_variable(), and ordered_variable(), for variables with these constraints but no probability distribution.

  • New chol2symm() is the inverse of chol().

  • mcmc(), stashed_samples(), and calculate() now return objects of class greta_mcmc_list which inherit from coda’s mcmc.list class, but enable custom greta methods for manipulating mcmc outputs, including a window() function.

  • mcmc() and calculate() now have a trace_batch_size argument enabling users to trade-off computation speed versus memory requirements when calculating posterior samples for target greta arrays (#236).

  • Many message, warning, and error prompts have been replaced internally with the {cli} R package for nicer printing. This is a minor change that should result in a more pleasant user experience (#423 #425).

  • Internally, where sensible, greta now uses the glue package to create messages/ouputs (#378).

  • New FAQ page and updated installation instructions for installing Python dependencies (#424)

  • New greta_sitrep() function to generate a situation report of the software that is available for use, and also initialising python so greta is ready to use. (#441)

greta 0.3.1

CRAN release: 2019-08-09

This release is predominantly a patch to make greta work with recent versions of TensorFlow and TensorFlow Probability, which were not backward compatible with the versions on which greta previously depended. From this release forward, greta will depend on specific (rather than minimum) versions of these two pieces of software to avoid it breaking if more changes are made to the APIS of these packages.

  • greta now (only) works with TensorFlow 1.14.0 and TensorFlow Probability 0.7.0 (#289, #290)

  • behaviour of the pb_update argument to mcmc() has been changed slightly to avoid a bad interaction with thinning (#284)

  • various edits to the documentation to fix spelling mistakes and typos

greta 0.3.0

CRAN release: 2018-10-30

This is a very large update which adds a number of features and major speed improvements. We now depend on the TensorFlow Probability Python package, and use functionality in that package wherever possible. Sampling a simple model now takes ~10s, rather than ~2m (>10x speedup).

Fixes:

operation bugs

  • dim<-() now always rearranges elements in column-major order (R-style, not Python-style)

performance bugs

  • removed excessive checking of TF installation by operation greta arrays (was slowing down greta array creation for complex models)
  • sped up detection of sub-DAGs in model creation (was slowing down model definition for complex models)
  • reduced passing between R, Python, and TensorFlow during sampling (was slowing down sampling)

New Functionality:

inference methods

  • 18 new optimisers have been added
  • initial values can now be passed for some or all parameters
  • 2 new MCMC samplers have been added: random-walk Metropolis-Hastings (thanks to @michaelquinn32) and slice sampling
  • improved tuning of MCMC during warmup (thanks to @martiningram)
  • integration with the future package for execution of MCMC chains on remote machines. Note: it is not advised to use future for parallel execution of chains on the same machine, that is now automatically handled by greta.
  • the one_by_one argument to MCMC can handle serious numerical errors (such as failed matrix inversions) as ‘bad’ samples
  • new extra_samples() function to continue sampling from a model.
  • calculate() works on the output of MCMC, to enable post-hoc posterior prediction

distributions

  • multivariate distributions now accept matrices of parameter values
  • added mixture() and joint() distribution constructors

operations

  • added functions: abind(), aperm(), apply(), chol2inv(), cov2cor(), eigen(), identity(), kronecker(), rdist(), and tapply() (thanks to @jdyen)
  • we now automatically skip operations if possible, e.g. computing binomial and poisson densities with log-, logit- or probit-transformed parameters where they exist, or skipping cholesky decomposition of a matrix if it was created from its cholesky factor. This increases numerical stability as well as speed.

misc

  • ability to change the colour of the model plot (thanks to @dirmeier)
  • ability to reshape greta arrays using greta_array()

API changes:

inference methods

  • mcmc now runs 4 chains (simultaneously on all available cores), 1000 warmup steps, and 1000 samples by default
  • optimisation and mcmc methods are now passed to opt() and mcmc() as objects, with defined tuning parameters. The control argument to these functions is now defunct.
  • columns names for parameters now give the array indices for each scalar rather than a number (i.e. x[2, 3], rather than x.6)

distributions

  • multivariate distributions now define each realisation as a row, and parameters must therefore have the same orientation

misc

  • plot.greta_model() now returns a DiagrammeR::grViz object (thanks to @flyaflya). This is less modifiable, but renders the plot more much consistently across different environments and notebook types. The DiagrammeR dgr_graph object use to create the grViz object is included as an attribute of this object, named "dgr_graph".

documentation

testing

  • added tests of the validity of posterior samples drawn by MCMC (for known distributions and with Geweke tests)

greta 0.2.5

Minor patch to handle an API change in the progress package. No changes in functionality.

greta 0.2.4

Fixes:

  • improved error checking/messages in model(), %*%
  • switched docs and examples to always use <- for assignment
  • fixed the n_cores argument to model()

New functionality:

  • added a calculate() function to compute the values of greta arrays conditional on provided values for others
  • added imultilogit() transform
  • added a chains argument to model()
  • improved HMC self-tuning, including a diagonal euclidean metric

greta 0.2.3

CRAN release: 2018-01-23

Fixes:

  • fixed breaking change in extraDistr API (caused test errors on CRAN builds)
  • added dontrun statements to pass CRAN checks on winbuilder
  • fixed breaking change in tensorflow API (1-based indexing)

New functionality:

greta 0.2.2

New functionality:

greta 0.2.1

New functionality:

  • export internal functions via .internals object to enable extension packages

API changes:

  • removed the deprecated define_model(), an alias for model()
  • removed the dynamics module, to be replaced by the gretaDynamics package