Comparison with the ADMB model

Executable V1 implementation check

Introduction

This vignette compares outputs from the Automatic Differentiation Model Builder (ADMB) operating model (OM) to the RTMB OM within the sbt package. This is done using a fixed parameter run and then by optimising. In the fixed parameter run the parameter values from a single cell of the ADMB model are used as fixed value inputs to sbt and outputs from sbt are generated then compared to the ADMB outputs.

Note

This is a historical V1 implementation-comparison workflow. Its direct MakeADFun() and nlminb() calls intentionally reproduce the historical ADMB calculation; they are not the current user-facing fit workflow. New analyses should use sbt_fit() followed by sbt_optimise() as shown in vignette("sbt").

Important

The model inputs are loaded at render time from the checksum-verified 2023 release in the private sbtdata package. The published page contains comparison summaries and figures, but does not publish the underlying assessment data.

Fixed parameter run

Comparison table

Table 2 includes transformed parameters (i.e., in natural space), derived quantities, priors, penalties, likelihoods, and objective function values for the ADMB model, the RTMB model, and the difference and percent difference between the two models.

The bundled ADMB reference is grid cell h3m1M3O2C1a1p2 (scenario 3132112). It uses h = 0.72 and psi = 1.75.

Table 1: Selected legacy ADMB reference-grid cell.
Grid cell Scenario h psi M0 M4 M10 M30
h3m1M3O2C1a1p2 3132112 0.72 1.75 0.4 0.147638 0.105 0.412891
Table 2: Comparison of ADMB and RTMB model outputs including transformed parameters, derived quantities, priors, penalties, likelihoods, and objective function values.
ADMB RTMB Difference Percent difference
Parameters
B0 6.09441e+06 6.094412e+06 -2.0272708 0e+00
psi 1.75000e+00 1.750000e+00 0.0000000 0e+00
sigmaR 6.00000e-01 6.000000e-01 0.0000000 0e+00
h 7.20000e-01 7.200000e-01 0.0000000 0e+00
q HSP 1.00000e+00 1.000000e+00 0.0000000 0e+00
m0 4.00000e-01 4.000000e-01 0.0000000 0e+00
m4 1.47637e-01 1.476375e-01 -0.0000005 4e-04
m10 1.05000e-01 1.050000e-01 0.0000000 0e+00
m30 4.12891e-01 4.128912e-01 -0.0000002 0e+00
Derived quantities
R0 6.80738e+06 6.807383e+06 -2.7070421 0e+00
alpha 7.54049e+06 7.540485e+06 4.5398918 1e-04
beta 6.56321e+05 6.563213e+05 -0.2952445 0e+00
tau_ac2 6.36503e-01 6.365034e-01 -0.0000004 1e-04
Priors & penalties
kludge 0.00000e+00 0.000000e+00 0.0000000 0e+00
sel.change 6.95676e+01 6.956764e+01 -0.0000361 1e-04
sel.smooth 3.33450e+01 3.334503e+01 -0.0000295 1e-04
rec -3.12152e+01 -3.121520e+01 0.0000042 0e+00
m0 0.00000e+00 0.000000e+00 0.0000000 0e+00
m10 0.00000e+00 0.000000e+00 0.0000000 0e+00
steep 0.00000e+00 0.000000e+00 0.0000000 0e+00
omega 0.00000e+00 0.000000e+00 0.0000000 0e+00
expl 0.00000e+00 0.000000e+00 0.0000000 0e+00
sel.init 3.00000e-07 3.000000e-07 0.0000000 0e+00
hstar 7.62801e+00 7.628005e+00 0.0000047 1e-04
Likelihoods & objective function
LL1 1.94666e+02 1.946657e+02 0.0003179 2e-04
LL2 3.09130e+01 3.091308e+01 -0.0000847 3e-04
LL3 3.46979e+01 3.469793e+01 -0.0000314 1e-04
LL4 3.75607e+01 3.756078e+01 -0.0000777 2e-04
Indo 9.15768e+01 9.157678e+01 0.0000238 0e+00
Aus 4.49229e+01 4.492290e+01 0.0000025 0e+00
CPUE -6.56728e+01 -6.567290e+01 0.0000953 1e-04
Tags 1.79633e+02 1.796326e+02 0.0004414 2e-04
Aerial 7.30625e+00 7.306254e+00 -0.0000039 1e-04
Troll 0.00000e+00 0.000000e+00 0.0000000 0e+00
POP 1.75788e+03 1.757879e+03 0.0006990 0e+00
HSP 2.19816e+03 2.198162e+03 -0.0016910 1e-04
GT 1.45928e+03 1.459276e+03 0.0040753 3e-04
ObjF 6.05025e+03 6.050245e+03 0.0045493 1e-04

Comparison figures

The following figures compare outputs from the ADMB and RTMB models: average weight at age (Figure 1), natural mortality at age (Figure 2), initial numbers at age (Figure 3), recruitment deviates (Figure 4), recruitment (Figure 5), total numbers (Figure 6), total reproductive output (Figure 7), CPUE (Figure 8), phi at age (Figure 9), selectivity at age (Figure 10), and age-frequency observations and predictions (Figure 12, Figure 13).

Figure 1: Average weight (tonnes per fish) at age.
Figure 2: Natural mortality (M) at age.
Figure 3: Initial numbers at age in the population.
Figure 4: Recruitment deviates each year.
Figure 5: Recruitment each year.
Figure 6: Total number of individuals each year.
Figure 7: Total reproductive output.
Figure 8: Catch per unit effort (CPUE).
Figure 9: Phi at age for a subset of years.
Figure 10: Selectivity at age for a subset of years.

The LF observations in the ADMB lab report are stored after applying the same minimum-bin aggregation used by the LF likelihood. For example, fishery 4 has lf_minbin = 11, so bins 1–11 are combined into bin 11 before the likelihood is evaluated. The processed RTMB input keeps the pre-aggregation proportions in data$lf_obs, so plotting those raw values made the 1970 fishery-4 panel show a small visual mismatch even though the likelihoods agreed. Figure 11 applies the same aggregation to the RTMB series before plotting; remaining differences are numerical precision only.

Figure 11: LF observations for a subset of years.
Figure 12: AF observations for a subset of years.
Figure 13: AF predictions for a subset of years.

Optimising

The selected ADMB result represents one fixed structural grid cell. The V1 RTMB optimisation applies the same conditioning: h, psi, M0, and M10 are fixed at the grid-cell values, the observation-scale and externally conditioned parameters listed below remain fixed, and the remaining population, mortality, catchability, recruitment-deviation, and selectivity parameters are estimated. A parameter is fixed when all of its map entries are NA; parameters absent from the map are estimated.

Table 3: Parameters fixed and estimated in the V1 RTMB optimisation of the selected ADMB grid cell.
Parameter Treatment Natural-scale value or role
par_log_psi Fixed 1.75
par_log_m0 Fixed 0.4
par_log_m10 Fixed 0.105
par_log_h Fixed 0.72
par_log_sigma_r Fixed 0.6
par_log_cpue_sigma Fixed 0.2
par_log_cpue_omega Fixed 1
par_log_troll_tau Fixed 0.001
par_log_aerial_tau Fixed 0.22
par_log_aerial_sel Fixed 1, 1
par_log_hsp_q Fixed 1
par_log_tag_H_factor Fixed 1
par_log_B0 Estimated Unfished biomass
par_log_m4 Estimated Mortality at age 4
par_log_m30 Estimated Mortality at age 30
par_log_cpue_q Estimated CPUE catchability
par_logit_hstar_i Estimated 17 hstar effects
par_rdev_y Estimated 92 annual deviations
par_sels_init_i Estimated 83 initial selectivity effects
par_sels_change_i Estimated 1132 selectivity-change effects

This is the same scientific setup as the previous ADMB model within the selected grid cell: it uses the same assessment inputs, switches, likelihoods, selectivity parameterisation, fixed grid values, and parameter bounds. The comparison changes the estimation engine from ADMB to RTMB with nlminb; it does not introduce a current-assessment configuration. Recovering the same optimum therefore checks the implementation rather than comparing two different model specifications.

The sbt model was optimised using nlminb to see if RTMB produces the same result. The nlminb function finds the same optimum as ADMB (Figure 14).

Figure 14: Total reproductive output.
Table 4: Comparison of ADMB and RTMB model outputs including transformed parameters, derived quantities, priors, penalties, likelihoods, and objective function values.
ADMB RTMB Difference Percent difference
Parameters
B0 6.09441e+06 6.092792e+06 1618.1534139 0.0266
psi 1.75000e+00 1.750000e+00 0.0000000 0.0000
sigmaR 6.00000e-01 6.000000e-01 0.0000000 0.0000
h 7.20000e-01 7.200000e-01 0.0000000 0.0000
q HSP 1.00000e+00 1.000000e+00 0.0000000 0.0000
m0 4.00000e-01 4.000000e-01 0.0000000 0.0000
m4 1.47637e-01 1.476585e-01 -0.0000215 0.0146
m10 1.05000e-01 1.050000e-01 0.0000000 0.0000
m30 4.12891e-01 4.128094e-01 0.0000816 0.0198
Derived quantities
R0 6.80738e+06 6.806354e+06 1026.0617939 0.0151
alpha 7.54049e+06 7.539346e+06 1144.0992179 0.0152
beta 6.56321e+05 6.561468e+05 174.1857523 0.0265
tau_ac2 6.36503e-01 6.367887e-01 -0.0002857 0.0449
Priors & penalties
kludge 0.00000e+00 0.000000e+00 0.0000000 0.0000
sel.change 6.95676e+01 6.954895e+01 0.0186479 0.0268
sel.smooth 3.33450e+01 3.331723e+01 0.0277664 0.0833
rec -3.12152e+01 -3.119649e+01 -0.0187106 0.0599
m0 0.00000e+00 0.000000e+00 0.0000000 0.0000
m10 0.00000e+00 0.000000e+00 0.0000000 0.0000
steep 0.00000e+00 0.000000e+00 0.0000000 0.0000
omega 0.00000e+00 0.000000e+00 0.0000000 0.0000
expl 0.00000e+00 0.000000e+00 0.0000000 0.0000
sel.init 3.00000e-07 0.000000e+00 0.0000003 0.0000
hstar 7.62801e+00 7.628098e+00 -0.0000884 0.0012
Likelihoods & objective function
LL1 1.94666e+02 1.946724e+02 -0.0063520 0.0033
LL2 3.09130e+01 3.091280e+01 0.0001994 0.0006
LL3 3.46979e+01 3.468581e+01 0.0120856 0.0348
LL4 3.75607e+01 3.757091e+01 -0.0102128 0.0272
Indo 9.15768e+01 9.157496e+01 0.0018360 0.0020
Aus 4.49229e+01 4.492930e+01 -0.0064032 0.0143
CPUE -6.56728e+01 -6.566823e+01 -0.0045748 0.0070
Tags 1.79633e+02 1.796299e+02 0.0030741 0.0017
Aerial 7.30625e+00 7.308100e+00 -0.0018504 0.0253
Troll 0.00000e+00 0.000000e+00 0.0000000 0.0000
POP 1.75788e+03 1.757881e+03 -0.0014903 0.0001
HSP 2.19816e+03 2.198161e+03 -0.0011683 0.0001
GT 1.45928e+03 1.459276e+03 0.0043636 0.0003
ObjF 6.05025e+03 6.050232e+03 0.0179622 0.0003

Previous grid evidence

The single-cell comparison above is extended here to all 108 cells in the 2023 assessment grid. Each cell fixes the same four structural quantities used by ADMB—four h values, three psi values, three M0 values, and three M10 values—and estimates the remaining parameters by maximum likelihood with the V1 RTMB model. The comparison uses the archived ADMB maximum-likelihood reports, the checksum-verified 2023 grid coordinates from sbtdata, cell-specific legacy mortality bounds, and at least two successive nlminb passes per cell. Cells above a projected-gradient tolerance of 1e-5 receive a bound-respecting L-BFGS-B step followed by a centred, uniformly rescaled nlminb pass. Removing the irrelevant objective constant for this final pass avoids scale-driven relative-stopping behaviour without changing the likelihood or its MLE.

MLE coverage and agreement

The RTMB run completed all 108 cells. All 108 returned nlminb convergence code zero. The largest raw absolute gradient was 4.327e-04, and the largest bound-aware projected gradient was 4.327e-04. All 108 cells were below 1e-3, and 105 were below 1e-4. Across the grid, the largest objective-function difference was 3.361e-02 objective units. The largest percentage difference among B0, terminal TRO, M4, and M30 was 3.326e-02%.

Table 5: Coverage, optimisation diagnostics, ADMB--SBT agreement, and the effect on likelihood-weighted cell probabilities across the complete previous assessment grid. Percentage differences are absolute.
Diagnostic Value
Grid cells fitted 108
Cells with convergence code zero 108
Cells with projected gradient below 1e-3 108
Cells with projected gradient below 1e-4 105
Cells receiving scale-sensitive refinement 42
Cells with an active legacy bound 0
Largest raw absolute gradient 4.326582e-04
Largest projected absolute gradient 4.326582e-04
Largest absolute objective difference 3.360642e-02
Largest objective percent difference 5.558059e-04
Largest B0 percent difference 3.280411e-02
Largest terminal TRO percent difference 3.325947e-02
Largest M4 percent difference 1.785904e-02
Largest M30 percent difference 2.607755e-02
Total variation in cell probabilities 1.547397e-03
Figure 15: Maximum-likelihood objective values for all 108 previous-assessment grid cells. The dashed line is exact one-to-one agreement; colour identifies fixed M10.

Figure 15 compares the maximised objective directly rather than centring each model on its best cell. This makes both a constant offset and cell-specific differences visible.

Figure 16: Absolute percentage difference between ADMB and SBT total reproductive output across all 108 MLE grid cells. The ribbon contains 90% of cells; the solid line is the median, and the dashed line is the maximum.

The same all-cell check for natural mortality has a maximum absolute percentage difference of 2.608e-02% over all 3,348 cell-age combinations.

Effect on grid weights

For comparability with the previous assessment, h receives equal mass across its four levels, psi = 1.5, 1.75, 2.0 receives mass 0.25, 0.50, 0.25, and the nine mortality cells within each h–psi stratum are weighted by their relative maximised likelihood. The total variation distance between the ADMB and SBT cell-probability vectors is 1.547e-03.

Figure 17: Marginal structural-grid probabilities calculated from the ADMB and SBT maximum-likelihood objectives.
Table 6: The ten grid cells with the largest percentage discrepancy among B0, terminal TRO, M4, and M30.
Cell h psi M0 M10 Objective difference B0 difference (%) Terminal TRO difference (%) M4 difference (%) M30 difference (%)
28 0.8 1.50 0.40 0.065 -0.03361 0.03007 0.03326 0.01761 0.02461
102 0.8 2.00 0.50 0.065 -0.02772 0.03280 0.02729 0.01539 0.01830
29 0.8 1.50 0.45 0.065 -0.03168 0.03098 0.03278 0.01738 0.02435
66 0.8 1.75 0.50 0.065 -0.03163 0.03239 0.02890 0.01605 0.01978
101 0.8 2.00 0.45 0.065 -0.02777 0.03219 0.02903 0.01619 0.02106
30 0.8 1.50 0.50 0.065 -0.02743 0.03185 0.03203 0.01771 0.02392
69 0.8 1.75 0.50 0.085 -0.01828 0.03202 0.02743 0.01581 0.02122
105 0.8 2.00 0.50 0.085 -0.01659 0.03197 0.02572 0.01532 0.01925
33 0.8 1.50 0.50 0.085 -0.02232 0.03176 0.02979 0.01664 0.02336
65 0.8 1.75 0.45 0.065 -0.02743 0.03171 0.03072 0.01648 0.02243

These results extend the implementation check across the entire previous grid; they do not make that historical grid a current assessment product. See the ESC31 grid page for the current deterministic and posterior grid APIs, restart rules, validation gates, balanced draw allocation, and provenance requirements. Historical grid artifacts must not be combined with current fitted states or presented as a validated current posterior.