Skip to contents

Grid workflows

Source: vignettes/grid.qmd

Scope

The package supports two current grid workflows:

  1. a deterministic maximum-likelihood (MLE) grid, used to fit and validate one optimum for each structural cell; and
  2. a posterior grid, used to run a resumable MCMC within every structural cell and then combine accepted cells with balanced draw allocation.

The examples below vary steepness (h), the reproductive-output power parameter (psi), and the natural-mortality anchors (m0 and m10). A real assessment must define its grid and acceptance thresholds before fitting.

Important

Completed cells are not automatically accepted cells. Require optimization, estimability, MCMC, provenance, payload, and biological-state checks to pass before a grid is used for inference or projection.

Assessment MLE grid

The previous assessment grid crossed four steepness values, three reproductive-output power values, three M0 values, and three M10 values. The ordering below reproduces its 108 cells: M0 varies fastest, followed by M10, steepness, and psi.

library(sbt)
library(dplyr)

fixture <- new.env(parent = emptyenv())
load(system.file("extdata", "opt.rda", package = "sbt"), envir = fixture)

grid <- expand.grid(
  m0 = c(0.40, 0.45, 0.50),
  m10 = c(0.065, 0.085, 0.105),
  h = c(0.55, 0.63, 0.72, 0.80),
  psi = c(1.50, 1.75, 2.00)
)
grid <- data.frame(
  Cell = seq_len(nrow(grid)),
  grid[c("h", "psi", "m0", "m10")]
)

grid_parameters <- make_mle_grid_parameters(
  grid = grid,
  parameters = fixture$parameters
)
knitr::kable(
  data.frame(
    Coordinate = c("h", "psi", "m0", "m10"),
    Values = c(
      "0.55, 0.63, 0.72, 0.80",
      "1.50, 1.75, 2.00",
      "0.40, 0.45, 0.50",
      "0.065, 0.085, 0.105"
    )
  ),
  caption = "Structural values in the 108-cell assessment grid."
)
Structural values in the 108-cell assessment grid.
Coordinate Values
h 0.55, 0.63, 0.72, 0.80
psi 1.50, 1.75, 2.00
m0 0.40, 0.45, 0.50
m10 0.065, 0.085, 0.105

run_grid() estimates all active parameters separately within each cell. The full 108-cell run should be made in the assessment workflow and retained as portable fitted states, so the fitting code below is shown but is not rerun while this article is built. The map fixes the four structural coordinates, while the named starting vector supplies the same optimised active parameters to every cell.

grid_fits <- run_grid(
  data = fixture$data,
  grid_parameters = grid_parameters,
  bounds = fixture$bounds,
  map = fixture$map,
  start = fixture$opt$par,
  n_passes = 1L,
  verbose = FALSE
)

mle_grid <- summarise_mle_grid(
  grid_fits = grid_fits,
  grid = grid,
  data = fixture$data,
  check_estimability_cells = FALSE
)

mle_grid$summary |>
  select(
    Cell, h, psi, fixed_m0, fixed_m10, objective, convergence,
    max_gradient, state_passes
  ) |>
  knitr::kable(digits = 5)

The saved fitted states must round-trip through a fresh objective. This checks the objective, gradient, reported fixed coordinates, biological states, biomass path, and payload checksums.

validate_mle_grid_state_records(
  records = mle_grid$state_records,
  saved_summary = mle_grid$summary,
  saved_biomass = mle_grid$biomass,
  grid = grid,
  data = fixture$data,
  parameters = fixture$parameters,
  map = fixture$map
)

sample_grid() can sample deterministic cells using relative likelihood and explicit structural-prior weights. Passing NULL gives equal prior weight to each unique h and psi level.

mle_draws <- sample_grid(
  grid = mle_grid$summary,
  n_samples = 24L,
  prior_h = NULL,
  prior_psi = NULL,
  seed = 42L
)

mle_draws$grid_freq |>
  select(Cell, h, psi, nll, prob, Freq) |>
  knitr::kable(digits = 5)

This deterministic likelihood/prior sampling is not MCMC and is distinct from the balanced posterior-draw allocation below.

Production posterior grid

Construction and restart policy

run_grid_mcmc() optimizes, checks, samples, diagnoses, and saves each cell as a portable grid<N>.sbt.rds file. The code is deliberately not run while the website is built.

fit <- sbt_fit_read("accepted-base-model.sbt.rds", strict = TRUE)

production_grid <- grid

grid_run <- run_grid_mcmc(
  data = fit$data,
  parameters = fit$parameters,
  grid = production_grid,
  bounds = fit$bounds,
  map = fit$map,
  random = fit$random,
  grid_dir = "grid_mcmc",
  n_passes = 3L,
  check = TRUE,
  stop_on_failure = TRUE,
  mcmc_args = list(
    num_samples = 1000L,
    num_warmup = 1000L,
    chains = 4L,
    cores = 4L,
    control = list(adapt_delta = 0.999)
  ),
  resume = TRUE,
  overwrite = FALSE,
  run_metadata = list(assessment = "ESC", configuration = "base")
)

With resume = TRUE, a completed cell is skipped only after its inputs, bounds, sampler globals, run settings, biological-state diagnostics, retained posterior payload, and signatures have been revalidated. An incompatible or damaged file stops the run. Set overwrite = TRUE only when intentionally replacing those exact cell files. Use cells = c(...) to rerun a known subset without renumbering it.

Independent workers may safely write disjoint cell sets, but their aggregate CSV/RDS sidecars are snapshots. Reconcile the canonical cell files after all workers finish rather than treating the last-written sidecar as authoritative.

Acceptance and diagnostics

The returned summary includes MLE convergence and estimability, R-hat, bulk and tail effective sample size, divergences, maximum-treedepth hits, biological state checks, source signatures, and file paths. Apply the thresholds agreed for the assessment to every cell. For example:

accepted <- grid_run$summary |>
  filter(
    convergence == 0L,
    estimable,
    max_rhat <= 1.01,
    min_bulk_ess >= 400,
    min_tail_ess >= 400,
    divergences == 0L,
    max_treedepth_hits == 0L,
    state_passes
  )

stopifnot(nrow(accepted) == nrow(production_grid))

A scientifically accepted grid should also have reviewed trace plots, marginal distributions, objective and gradient behaviour, cell weights, biological trajectories, and sensitivity to the grid definition. A cache, zero exit code, or complete file list is not that review.

Balanced posterior combination

Only canonical .sbt.rds cells are accepted by grid_mcmc_to_tmbfit(). The converter revalidates model, grid, run, payload, and biological-state provenance, then allocates draws as evenly as possible across cells.

combined <- grid_mcmc_to_tmbfit(
  grid_files = grid_run$files,
  n_draws = 2000L,
  seed = 44L
)

combined$draw_plan
posterior <- combined$fit

combined$draw_metadata, combined$draw_plan, and combined$source_signatures should be retained with downstream results. The converter deliberately does not waive cell-level diagnostic thresholds; the acceptance gate remains the responsibility of the production workflow.

Legacy compatibility

get_grid(), rerun_grid(), and check_grid() support historical direct-anchor, in-memory grids. The list-object routes in sample_grid() and grid_to_tmbfit() are also retained for compatibility. New work should use the current data-frame/state-record routes described above, or the canonical posterior grid workflow.