Package {weightflow}

Title:	Declarative Recipes for Staged Survey Weighting with Recipe-Aware Replicate Variances
Version:	1.1.0
Description:	Builds survey analysis weights by declaring the whole weighting process as an ordered recipe of explicit adjustments and estimating it in a single call. The recipe follows the stages of a real household survey: redistribution of unresolved eligibility, within-household selection, nonresponse adjustment by weighting classes, by response-propensity models (logistic regression or machine-learning learners such as trees, random forests and gradient boosting, with optional cross-fitting) or by two-phase calibration for nonresponse, calibration to known population totals following Deville and Sarndal (1992) <doi:10.2307/2290268>, optional model-assisted (model) calibration to auxiliary predictions following Wu and Sitter (2001) <doi:10.1198/016214501750333054>, and range-restricted trimming that bounds the weights while preserving the calibration totals. Variances come from a recipe-aware bootstrap and jackknife that resample or delete primary sampling units and re-apply the entire cascade on each replicate, following the rescaling bootstrap of Rao and Wu (1988) <doi:10.1080/01621459.1988.10478591>, so the replicate weights carry the variability of every adjustment instead of treating the adjustments as known. A self-contained HTML report documents each step with its diagnostics, and the weights bridge to the 'survey' and 'srvyr' packages for design-based inference.
License:	MIT + file LICENSE
Encoding:	UTF-8
Language:	en-US
Depends:	R (≥ 4.1.0)
Imports:	stats, utils, graphics, parallel
Suggests:	MASS, rpart, ranger, testthat (≥ 3.0.0), survey, srvyr, dplyr, tidyr, ggplot2, haven, archive, knitr, rmarkdown, spelling, xgboost
Config/roxygen2/version:	8.0.0
URL:	https://github.com/jpferreira33/weightflow, https://jpferreira33.github.io/weightflow/
BugReports:	https://github.com/jpferreira33/weightflow/issues
Config/testthat/edition:	3
LazyData:	true
VignetteBuilder:	knitr
NeedsCompilation:	no
Packaged:	2026-08-19 10:15:32 UTC; jp
Author:	Juan Pablo Ferreira [aut, cre], Andrés Gutiérrez [aut]
Maintainer:	Juan Pablo Ferreira <juanpablo.ferreira@fcea.edu.uy>
Repository:	CRAN
Date/Publication:	2026-08-19 11:20:02 UTC

weightflow: declarative survey weighting

Description

Builds analysis weights from design base weights by declaring the weighting process as an ordered recipe of explicit adjustments – unknown eligibility, within-cluster selection (e.g. within household), nonresponse, calibration, trimming, rounding, rescaling, assertions – and then estimating that recipe in one call. The package also produces replicate weights and design-based standard errors that carry the variability of the whole cascade, so a weighting project no longer has to end at the weights.

Details

Start with weighting_spec(), add ⁠step_*()⁠ adjustments, estimate the cascade with prep(), and extract the weights with collect_weights(). Inspect with summary(), plot() and report_weighting().

Author(s)

Maintainer: Juan Pablo Ferreira juanpablo.ferreira@fcea.edu.uy (ORCID)

Authors:

• Juan Pablo Ferreira juanpablo.ferreira@fcea.edu.uy (ORCID)

• Andrés Gutiérrez (ORCID)

See Also

Useful links:

• https://github.com/jpferreira33/weightflow

• https://jpferreira33.github.io/weightflow/

• Report bugs at https://github.com/jpferreira33/weightflow/issues

Export weightflow weights to a survey design

Description

as_svydesign() builds a linearization (ultimate-cluster) survey.design from a prepped recipe, treating the final weights as fixed constants. as_svrepdesign() builds a replicate-weights svyrep.design from a bootstrap_weights() or jackknife_weights() object. Both are the bridge to the survey package, and therefore to svytotal(), svymean(), svyratio(), svyby(), svyglm() and domain estimation generally.

Usage

as_svydesign(object, ids, strata = NULL, weight_name = ".weight", ...)

as_svrepdesign(object, ...)

Arguments

Details

Only as_svrepdesign() propagates the variability of the weighting adjustments (nonresponse, calibration, ...), because each replicate re-runs the whole recipe. as_svydesign() is design-based linearization on the fixed final weights: its standard errors reflect the sampling design but treat the adjustments as known without error, so they are usually smaller. Use as_svrepdesign() (with bootstrap_weights() / jackknife_weights()) when the adjustment variability should be included.

Value

A survey.design / svyrep.design object.

Bootstrap estimate, standard error and confidence interval

Description

Applies a statistic to the point weights and to every bootstrap replicate, and returns the estimate with its bootstrap standard error and a normal confidence interval. boot_total() and boot_mean() are the two shortcuts you will use most: a weighted total and a weighted mean of one column.

Usage

bootstrap_estimate(boot, statistic, level = 0.95)

boot_total(boot, variable)

boot_mean(boot, variable)

Arguments

Details

The bootstrap variance takes the replicate estimates \hat\theta^{*}_b around the point estimate \hat\theta (the mse = TRUE convention of survey), over the R valid replicates (a failed replicate is dropped, not counted),

\widehat V_{\mathrm{boot}}(\hat\theta) = \frac{1}{R}\sum_{b=1}^{R}\big(\hat\theta^{*}_b - \hat\theta\big)^2.

Value

A data frame with estimate, se, ci_lower, ci_upper.

Recipe-aware bootstrap replicate weights

Description

Builds bootstrap replicate weights by resampling primary sampling units (PSUs) with replacement within strata and re-running the entire weighting recipe on each replicate – every estimated stage (nonresponse, calibration, model calibration, trimming), not just one. Reach for this when those stages are estimated from the sample and you want their uncertainty inside the standard error, instead of conditioning on them as if they were known.

Usage

bootstrap_weights(
  object,
  replicates = 200L,
  strata = NULL,
  psu = NULL,
  m = NULL,
  lonely_psu = c("certainty", "collapse"),
  seed = NULL,
  cores = 1L,
  progress = TRUE
)

Arguments

Details

The multiplier is the Rao-Wu rescaling bootstrap: within a stratum with n PSUs, m PSUs are drawn with replacement (default m = n - 1) and unit i in PSU k gets \lambda = 1 - \sqrt{m/(n-1)} + \sqrt{m/(n-1)}\,(n/m)\,t_k, with t_k the number of times its PSU was drawn.

Value

An object of class weightflow_boot with the replicates matrix (units x replicates), the point weights, and the design metadata.

Examples

spec <- weighting_spec(sample_survey, base_weights = pw) |>
  step_calibrate(method = "raking",
                 margins = list(region = c(table(population$region))))
boot <- bootstrap_weights(spec, replicates = 50, strata = "region",
                          psu = "psu", seed = 1)
boot_total(boot, "responded")

Recover the fitted response propensities of a nonresponse step

Description

A step_nonresponse(method = "propensity") step fits a response-propensity model and adjusts the weights by 1/\hat p. prep() keeps the full per-unit propensity vector \hat p (out-of-fold when cross-fitting is used) on the step, but it is not returned by collect_weights(). This accessor extracts it aligned to the sample, so you can inspect its distribution and confirm the nonresponse model is well fitted before trusting the adjusted weights. It works the same way whether the adjustment was made at the unit level or, through cluster, at the household level (there the household propensity is broadcast to its members).

Usage

collect_propensities(object, step = NULL)

Arguments

Value

The sample data.frame with columns appended: .propensity (the fitted response propensity \hat p, NA for units outside the model, i.e. ineligible / already dropped), .responded (the response indicator the model used), .weight_in (the weight reaching the step, see below), .factor (the multiplier the step actually applied to the unit), .status (a factor that labels each unit as "eligible respondent", "eligible nonrespondent" or "not in propensity model"), and, when the step uses propensity classes (num_classes), .class (the assigned class). Units not in the propensity model carry NA in the per-unit columns. .weight_in is the weight reaching the nonresponse step – it already carries any earlier adjustment (unknown-eligibility redistribution, within-cluster selection), not the raw base weight. At the unit level it is also the weight the propensity model is fitted with (unless weight_model = FALSE); with cluster, the model is fitted at the household level with the household weight, which equals .weight_in only when weights are uniform within the household. .factor equals 1/\hat p only when num_classes = NULL; with propensity classes it is the class-level adjustment, so 1/.propensity does not reconstruct the applied factor – use .factor. The stage-by-stage weights are available through weight_factors() and domain_summary().

See Also

step_nonresponse(), collect_weights()

Examples

fit <- weighting_spec(sample_survey, base_weights = pw) |>
  step_nonresponse(respondent = responded, method = "propensity",
                   formula = ~ sex + region, engine = "logit") |>
  prep()
p <- collect_propensities(fit)
summary(p$.propensity)

Collect replicate weights into a data frame ready for srvyr

Description

Returns the data with the point weight and every replicate weight as ordinary columns, plus the replication design as attributes. This is the form srvyr::as_survey_rep() and survey::svrepdesign() expect, and the form to write out when the analysis continues in another session, another script or another language.

Usage

collect_replicate_weights(
  object,
  weight_name = ".weight",
  prefix = "rep_",
  drop_zero = TRUE
)

Arguments

Value

A data frame: the original columns, weight_name, and one column per replicate. The number of replicates is in attribute "R", and the replication design in attributes "type", "scale" and "rscales".

Examples

spec <- weighting_spec(sample_survey, base_weights = pw) |>
  step_calibrate(method = "raking",
                 margins = list(region = c(table(population$region))))
boot <- bootstrap_weights(spec, replicates = 30, strata = "region",
                          psu = "psu", seed = 1, progress = FALSE)
df <- collect_replicate_weights(boot)   # or a weightflow_jack object

if (requireNamespace("srvyr", quietly = TRUE) &&
    requireNamespace("dplyr", quietly = TRUE)) {
  srvyr::as_survey_rep(df, weights = .weight,
                       repweights = dplyr::starts_with("rep_"),
                       type = attr(df, "type"), combined.weights = TRUE,
                       scale = attr(df, "scale"), rscales = attr(df, "rscales"),
                       mse = TRUE)
}

Per-unit detail of one step of the cascade

Description

A generic companion to collect_weights() that returns, for a single step, the weight it received and the multiplier it applied to every unit, plus any quantities that step computed internally (for a propensity step, the fitted propensity and its class). .weight_in and .factor are read from the stage-by-stage weights that prep() already stores, so .weight_in * .factor equals the weight leaving the step by construction, for any step. Native columns (those a step exposes on its own) are NA for units the step did not touch.

Usage

collect_step_detail(object, step = NULL)

Arguments

Value

The sample data.frame with .weight_in (the weight reaching the step, carrying every earlier adjustment) and .factor (the multiplier the step applied to each unit, NA where the incoming weight is zero) appended, plus any native columns of the chosen step (for a propensity step: .propensity, .responded, and .class when propensity classes are used), which are NA outside the units the step covers. Here .factor is defined for every unit with a nonzero incoming weight, so an active unit the step did not touch reports .factor = 1; this differs from collect_propensities(), where .factor is NA outside the propensity model (see its .status).

See Also

collect_weights(), collect_propensities(), weight_factors()

Examples

fit <- weighting_spec(sample_survey, base_weights = pw) |>
  step_nonresponse(respondent = responded, method = "propensity",
                   formula = ~ sex + region, engine = "logit") |>
  prep()
d <- collect_step_detail(fit, step = 1)
head(d[!is.na(d$.factor), c(".weight_in", ".factor", ".propensity")])

Extract the data with the computed weights

Description

Returns the sample as a data.frame with the final analysis weight attached as a column, ready to hand to an estimation routine. By default the units that left the cascade (weight 0) are dropped, so what comes back is the responding, in-scope sample and its weights.

Usage

collect_weights(
  object,
  drop_zero = TRUE,
  keep_intermediate = FALSE,
  weight_name = ".weight"
)

Arguments

Value

data.frame.

Examples

fitted <- weighting_spec(sample_survey, base_weights = pw) |>
  step_nonresponse(respondent = responded, method = "weighting_class", by = "region") |>
  prep()
head(collect_weights(fitted))

Kish design effect from unequal weighting

Description

Computes Kish's design effect due to unequal weighting, deff = 1 + CV^2(w) = m \sum w^2 / (\sum w)^2, and the effective sample size n_\mathrm{eff} = m / deff it implies. It is the standard one-number summary of what a weighting cascade cost in precision, and it is what the summary() and plot() methods of a prepped recipe report step by step.

Usage

design_effect(w)

Arguments

Details

Zero weights are dropped (they are the "dropped-unit" marker); negative weights – a valid but unusual output of unbounded linear/GREG calibration – are kept active, so the count n matches collect_weights(). Be aware, however, that the Kish formula assumes non-negative weights: a negative weight shrinks \sum w and enlarges \sum w^2 at once, so with negatives present deff is inflated and no longer interpretable as an effective-sample summary. prep() raises an alert when a calibration produces negative weights; prefer bounds to keep the factor positive if you need the design effect to be meaningful.

Value

list with deff, n_eff, cv and n.

Examples

design_effect(sample_survey$pw)

Per-domain weight summary at every stage of the cascade

Description

For quality control by study domain (for example a department / DAM), this summarises how the weights move within each domain at every stage of the recipe: the base weights, then the weights after each step. It reads the stage-by-stage weights that prep() already stores, so it adds no computation to the cascade and never changes a weight.

Usage

domain_summary(object, by)

Arguments

Value

A data.frame with one row per stage x domain and the columns stage (an ordered factor: base weights, then ⁠1. <step>⁠, ⁠2. <step>⁠, ...), domain (an ordered factor), n_active (active units in the domain at that stage), sum_w (sum of the active weights), mean_w, deff (the Kish design effect within the domain) and n_eff. Reading down a domain shows how its weight total and dispersion evolve step by step.

See Also

design_effect(), weight_factors(), summary.prepped_weighting_spec()

Examples

fit <- weighting_spec(sample_survey, base_weights = pw) |>
  step_nonresponse(respondent = responded, method = "weighting_class", by = "region") |>
  step_calibrate(method = "raking", margins = list(region = c(table(population$region)))) |>
  prep()
domain_summary(fit, by = "region")

Jackknife estimate, standard error and confidence interval

Description

Applies a statistic to the point weights and to every delete-a-PSU replicate, and returns the estimate with its stratified jackknife (JKn) standard error and a normal confidence interval. jack_total() and jack_mean() are the shortcuts for a weighted total and a weighted mean of one column.

Usage

jackknife_estimate(jack, statistic, level = 0.95)

jack_total(jack, variable)

jack_mean(jack, variable)

Arguments

Details

The stratified (JKn) variance sums each stratum's delete-a-PSU spread,

\widehat V_{JK} = \sum_h \frac{n_h - 1}{n_h}\sum_{i \in h}\big(\hat\theta_{(hi)} - \hat\theta_h\big)^2,

with \hat\theta_{(hi)} the estimate with PSU i of stratum h deleted and \hat\theta_h their within-stratum mean; the unstratified JK1 uses a single stratum. No finite population correction is applied.

Value

A data frame with estimate, se, ci_lower, ci_upper.

Note

jack_total() / jack_mean() center the replicate deviations on the per-stratum mean of the deleted-PSU estimates (the standard JKn). The survey design built by as_svrepdesign() instead uses mse = TRUE, which centers on the point estimate. Both are legitimate, so the standard errors from jack_total() and from svytotal() on the same object can differ slightly.

Examples

spec <- weighting_spec(sample_one, base_weights = pw) |>
  step_calibrate(method = "raking",
                 margins = list(region = c(table(population$region))))
jk <- jackknife_weights(spec, strata = "region", psu = "psu", progress = FALSE)
jackknife_estimate(jk, function(w, d) sum(w * d$employed, na.rm = TRUE))

Recipe-aware delete-a-PSU jackknife replicate weights

Description

Builds jackknife replicate weights by deleting one primary sampling unit (PSU) at a time and re-running the entire weighting recipe on each replicate. This is the deterministic sibling of bootstrap_weights(): same recipe-aware variance, no random number generation, and a replicate count fixed by the design rather than chosen by the analyst.

Usage

jackknife_weights(
  object,
  strata = NULL,
  psu = NULL,
  lonely_psu = c("certainty", "collapse"),
  cores = 1L,
  progress = TRUE
)

Arguments

Details

For a stratum h with n_h PSUs, the replicate that deletes PSU i zeros the base weight of that PSU and inflates the remaining PSUs of the stratum by n_h/(n_h-1); other strata are unchanged. There is one replicate per PSU. Strata with a single PSU contribute no variance and are skipped. This is the stratified jackknife (JKn); with strata = NULL it is the unstratified jackknife (JK1), and with psu = NULL each unit is its own PSU (delete-one-unit jackknife).

Value

An object of class weightflow_jack with the replicates matrix (units x replicates), the point weights, the per-replicate stratum and stratum size (used by jackknife_estimate()), and the design metadata.

Examples

spec <- weighting_spec(sample_one, base_weights = pw) |>
  step_calibrate(method = "raking",
                 margins = list(region = c(table(population$region))))
jk <- jackknife_weights(spec, strata = "region", psu = "psu", progress = FALSE)
jack_total(jk, "employed")

Diagnostic plots for the weights

Description

Draws the weighting cascade: one histogram of the adjustment factor per step, plus a four-panel summary of the final weights, the cumulative factor, base against final weight, and the design effect by stage. Base graphics only, no dependencies. Use it after prep() to see how the weights moved, where summary() tells you by how much.

Usage

## S3 method for class 'prepped_weighting_spec'
plot(x, type = c("all", "factors", "summary"), ...)

Arguments

Value

Invisibly, the prepped object x. Called for its side effect of drawing the diagnostic plots described above.

Examples

fitted <- weighting_spec(sample_survey, base_weights = pw) |>
  step_nonresponse(respondent = responded, method = "weighting_class", by = "region") |>
  prep()
plot(fitted)

Synthetic target population (sampling frame)

Description

The simulated frame every weightflow example draws from: 4,495 persons nested in 1,882 households, in 120 primary sampling units, in 4 regions used as strata. Because the whole population is observed, it supplies the known population totals a calibration step needs, and the true values against which a weighted estimate can be checked.

Usage

population

Format

A data frame with one row per person:

person_id

individual identifier

household_id

household identifier (cluster)

psu

primary sampling unit (segment) within the stratum

region

stratum: North, South, East or West

sex

F or M

age

age in years (18-95)

income

annual income

employed

employment indicator (0/1)

Estimate the weighting cascade

Description

Runs an inert weighting_spec() recipe. Starting from the design base weights, prep() applies each step in the order it was piped, multiplying the current weight by that step's adjustment factor, and returns an object holding the weight at every stage, the per-step diagnostics and the quality alerts. This is the only function that computes weights.

Usage

prep(spec, min_cell_n = 30, max_factor = 2.5, warn = FALSE)

Arguments

Value

a "prepped_weighting_spec" object.

Examples

rec <- weighting_spec(sample_survey, base_weights = pw) |>
  step_nonresponse(respondent = responded, method = "weighting_class", by = "region")
prep(rec)

Print a bootstrap replicate-weight object

Description

Compact one-screen summary of a weightflow_boot object: how many replicates were requested, how many units are in the data, how many of them are still active (final weight above zero), and which columns defined the resampling design.

Usage

## S3 method for class 'weightflow_boot'
print(x, ...)

Arguments

Value

(invisibly) the object.

Print a jackknife replicate-weight object

Description

Compact one-screen summary of a weightflow_jack object: how many delete-a-PSU replicates were built, how many units are in the data, how many of them are still active (final weight above zero), and which columns defined the deletion design.

Usage

## S3 method for class 'weightflow_jack'
print(x, ...)

Arguments

Value

(invisibly) the object.

Self-contained HTML quality report for a weighting recipe

Description

Writes a single, self-contained HTML file documenting how a prepped recipe turned the design base weights into the final weights: the cascade, the parameters requested at each step, the per-stage weight summary, the step-specific diagnostics (calibration, nonresponse, trimming), the fieldwork outcome rates and an audit trail. It is the deliverable to attach to a methodological report or a weighting annex: everything is inline, so the file opens offline and can be archived or emailed as one artifact.

Usage

report_weighting(
  object,
  file = NULL,
  open = TRUE,
  plots = TRUE,
  narrative = TRUE,
  lang = c("en", "es"),
  metadata = NULL,
  replicates = NULL,
  domains = NULL,
  y_vars = NULL
)

Arguments

Value

(invisibly) the path to the HTML file.

Examples

fitted <- weighting_spec(sample_survey, base_weights = pw) |>
  step_nonresponse(respondent = responded, method = "weighting_class", by = "region") |>
  prep()

# writes a self-contained HTML report to a temporary file (open = FALSE so
# nothing is launched); use open = TRUE to view it in the browser.
path <- report_weighting(fitted, open = FALSE)

Synthetic address sample with one selected person per household

Description

A multistage sample of 417 addresses (stratum, then PSU, then household, then one person inside the reached household) carrying every complication a household survey meets: unresolved eligibility, out-of-scope addresses, household nonresponse, unequal within-household selection and person nonresponse. It is the dataset that exercises the complete weighting cascade.

Usage

sample_one

Format

A data frame with one row per sampled household (the selected person, or a single placeholder row for non-roster cases):

person_id, household_id, psu

identifiers

region

stratum

sex, age

selected person's attributes (NA on non-roster rows)

pw

design base weight (product of the stage selection probabilities)

status

"eligible", "ineligible" or "unknown"

disposition

full field disposition as a single factor (a recode of the indicator columns): "eligible respondent", "eligible nonrespondent", "household nonresponse", "ineligible" or "unknown eligibility"

unknown_elig

1 if eligibility is unknown (no roster)

ineligible

1 if the address is out of scope (no roster)

hh_responded

1 reached, 0 household nonresponse, NA for non-eligible

responded

1 if the selected person responded (NA on non-roster rows)

n_elig

number of eligible persons in the household (NA on non-roster rows)

p_within

within-household selection probability of the selected person

income, employed

survey outcomes; NA unless the person responded

Synthetic person sample with a take-all household roster

Description

A stratified two-stage sample of 467 persons drawn from population: PSUs within region, then households within PSU, then every adult of the selected household (take-all roster). It carries unequal design base weights, an unknown-eligibility flag and a person-level response indicator, and it is the dataset the short examples in this package use.

Usage

sample_survey

Format

A data frame with one row per sampled person:

person_id, household_id, psu

identifiers

region, sex, age

frame auxiliaries, known for all units

pw

design base weight (inverse sampling fraction)

unknown_elig

1 if eligibility is unknown

responded

1 if the person responded

income, employed

survey outcomes; NA for nonrespondents

Assert quality conditions on the weights

Description

A checkpoint that leaves the weights untouched and instead verifies that they meet quality thresholds at this point of the cascade, raising an error or a warning when they do not. Use it to stop a production pipeline before bad weights are published, in the spirit of a validation step inside a recipe.

Usage

step_assert(
  spec,
  max_deff = NULL,
  max_weight_ratio = NULL,
  min_n_eff = NULL,
  on_fail = c("error", "warning")
)

Arguments

Value

The input weighting_spec with this checkpoint appended to its recipe. The check is recorded only; it is evaluated when prep() is called and does not modify the weights.

Examples

weighting_spec(sample_survey, base_weights = pw) |>
  step_assert(max_deff = 5, on_fail = "warning") |> prep()

Calibration to population totals

Description

Adjusts the weights so that the weighted sample reproduces known population totals of auxiliary variables, while moving the incoming weights as little as possible. This is the calibration estimator of Deville and Sarndal (1992), with raking, post-stratification and linear (GREG) calibration, optional bounds on the adjustment factor, ridge relaxation of the targets, domain partitions and one-weight-per-cluster (integrative) calibration.

Usage

step_calibrate(
  spec,
  margins = NULL,
  method = c("raking", "poststratify", "linear"),
  formula = NULL,
  totals = NULL,
  count = NULL,
  by = NULL,
  cluster = NULL,
  equal_within_cluster = FALSE,
  calfun = c("linear", "logit", "raking"),
  bounds = NULL,
  maxit = 50L,
  tol = 1e-06,
  penalty = NULL
)

Arguments

Details

The calibrated weight is w_i = d_i g_i, close to the incoming weights d_i and reproducing the totals \sum_{i \in s} w_i \mathbf{x}_i = \mathbf{X}, with the factor g_i minimizing a distance to d_i: linear g_i = 1 + \mathbf{x}_i'\boldsymbol\lambda or exponential ("raking") g_i = \exp(\mathbf{x}_i'\boldsymbol\lambda). The penalty (ridge) relaxes the exact constraint to steady extreme weights when the auxiliaries are many or collinear, minimizing

\sum_i \frac{(w_i - d_i)^2}{d_i q_i} + \frac{1}{s}\sum_j c_j (\hat X_j - X_j)^2,

where c_j is the cost of missing constraint j: as c_j \to \infty the constraint is met exactly, as c_j \to 0 the weights return to d_i.

Value

The input weighting_spec with this step appended to its recipe. The step is recorded only; it is evaluated when prep() is called.

Examples

# Raking to population margins
weighting_spec(sample_survey, base_weights = pw) |>
  step_nonresponse(respondent = responded, method = "weighting_class", by = "region") |>
  step_calibrate(method = "raking",
                 margins = list(sex    = c(table(population$sex)),
                                region = c(table(population$region)))) |>
  prep()

# ridge (penalized) calibration: relaxes the targets to control extreme
# weights; a smaller penalty relaxes more. Uses only base R.
pop_tot <- c("(Intercept)" = nrow(population),
             regionSouth = sum(population$region == "South"),
             regionEast  = sum(population$region == "East"),
             regionWest  = sum(population$region == "West"),
             sexM        = sum(population$sex == "M"))
weighting_spec(sample_survey, base_weights = pw) |>
  step_nonresponse(respondent = responded, method = "weighting_class", by = "region") |>
  step_calibrate(method = "linear", formula = ~ region + sex,
                 totals = pop_tot, penalty = 1) |>
  prep()

# --- Tidy `totals` format (recommended) ---------------------------------
# Post-stratification: give the population counts as a data frame with one or
# more category columns plus a counts column named by `count`.
ps_totals <- as.data.frame(table(region = population$region, sex = population$sex))
weighting_spec(sample_survey, base_weights = pw) |>
  step_calibrate(method = "poststratify", totals = ps_totals, count = "Freq") |>
  prep()

# Raking: a list of data frames, one per margin.
m_region <- as.data.frame(table(region = population$region))
m_sex    <- as.data.frame(table(sex = population$sex))
weighting_spec(sample_survey, base_weights = pw) |>
  step_calibrate(method = "raking",
                 totals = list(m_region, m_sex), count = "Freq") |>
  prep()

# Linear/GREG with mixed auxiliaries: data frames for categoricals (all
# categories) and a single number for a continuous total. weightflow builds
# the model.matrix totals internally, so you never drop a reference category.
resp <- subset(sample_survey, responded == 1)
weighting_spec(resp, base_weights = pw) |>
  step_calibrate(method = "linear", formula = ~ region + sex + income,
                 totals = list(region = m_region, sex = m_sex,
                               income = sum(population$income)),
                 count = "Freq") |>
  prep()

# Domain (partitioned) calibration: `by` calibrates independently within each
# domain, each to its own totals. The domain is an extra column in the tidy
# totals, not a term in the formula/margins. Here we rake two margins (sex and
# age group) within each region, which a single global cross could not express.
pop  <- transform(population,
  age_grp = cut(age, c(0, 30, 45, 60, Inf), labels = c("18-30","31-45","46-60","60+")))
samp <- transform(sample_survey,
  age_grp = cut(age, c(0, 30, 45, 60, Inf), labels = c("18-30","31-45","46-60","60+")))
sex_by_region <- as.data.frame(table(region = pop$region, sex     = pop$sex))
age_by_region <- as.data.frame(table(region = pop$region, age_grp = pop$age_grp))
weighting_spec(samp, base_weights = pw) |>
  step_calibrate(method = "raking",
                 totals = list(sex_by_region, age_by_region),
                 count = "Freq", by = "region") |>
  prep()

Drop ineligible (out-of-scope) units

Description

Sets the weight of the units known to be outside the target population to zero, so they leave the cascade and take no part in any later step or in collect_weights(). Their weight is discarded, not redistributed: the weight total is meant to fall by exactly the mass they carried. Use it once eligibility has been resolved, immediately after step_unknown_eligibility().

Usage

step_drop_ineligible(spec, ineligible)

Arguments

Details

Apply it AFTER step_unknown_eligibility: ineligibles must be present and NOT flagged as unknown during that step, so they take part in the known-eligibility group and receive their share of the redistributed unknown weight. Their weight is then correctly discarded here (it represents the ineligible share of the unknown units, which are out of scope).

Value

The input weighting_spec with this step appended to its recipe. The step is recorded only; it is evaluated when prep() is called.

Examples

df <- transform(sample_survey,
                ineligible = as.integer(region == "West" & age > 90))
weighting_spec(df, base_weights = pw) |>
  step_drop_ineligible(ineligible = ineligible) |>
  prep()

Model-assisted calibration (Wu and Sitter 2001)

Description

Fits a working model for each study variable, predicts it over the whole population, and calibrates the weights so that the sample total of every prediction matches its population total, on top of the usual auxiliary totals – which may come from the population frame itself or from an external source (a census table, an administrative register). Reach for it when you hold, or can supply, those control totals and the outcome is well predicted by the auxiliaries: the predictions act as extra, highly relevant controls and buy precision that calibrating on x alone cannot.

Usage

step_model_calibration(
  spec,
  x_formula,
  models,
  population,
  x_totals = NULL,
  count = "Freq",
  cluster = NULL,
  equal_within_cluster = FALSE,
  crossfit = NULL,
  crossfit_seed = NULL
)

Arguments

Details

Requires COMPLETE auxiliary information: a data.frame population with the x_formula columns and the model predictors for the whole population (or a reference frame/census).

The predictions \hat y_i enter as extra constraints, \sum_{i \in s} w_i \hat y_i = \sum_{i \in U} \hat y_i, solved together with the benchmark auxiliary totals \mathbf{X}. When the working model is linear this reduces to GREG; a nonlinear learner adds efficiency through the prediction constraint while the totals \mathbf{X} preserve design consistency even if the model is misspecified.

Value

The input weighting_spec with this step appended to its recipe. The step is recorded only; it is evaluated when prep() is called.

References

Wu, C. and Sitter, R. R. (2001). A model-calibration approach to using complete auxiliary information from survey data. Journal of the American Statistical Association, 96(453), 185-193. doi:10.1198/016214501750333054.

Examples

weighting_spec(sample_survey, base_weights = pw) |>
  step_nonresponse(respondent = responded, method = "weighting_class", by = "region") |>
  step_model_calibration(
    x_formula  = ~ sex + region,
    models     = list(income = y_model(income ~ age + sex, engine = "glm")),
    population = population) |>
  prep()

# with cross-fitting (out-of-fold predictions, avoids overfitting)
weighting_spec(sample_survey, base_weights = pw) |>
  step_nonresponse(respondent = responded, method = "weighting_class", by = "region") |>
  step_model_calibration(
    x_formula  = ~ sex + region,
    models     = list(income = y_model(income ~ age + sex, engine = "glm")),
    population = population, crossfit = 5, crossfit_seed = 1) |>
  prep()

# consistency totals from an external source (tidy format): a data frame per
# factor and a single number per continuous total. `population` is still used
# for the model predictions. Adjust for nonresponse first, since the outcome
# is only observed for respondents.
m_region <- as.data.frame(table(region = population$region))
weighting_spec(sample_survey, base_weights = pw) |>
  step_nonresponse(respondent = responded, method = "weighting_class", by = "region") |>
  step_model_calibration(
    x_formula  = ~ region + age,
    models     = list(income = y_model(income ~ age + sex, engine = "glm")),
    population = population,
    x_totals   = list(region = m_region, age = sum(population$age)),
    count      = "Freq") |>
  prep()

# equal weights within a household (integrative, Lemaitre-Dufour): one weight
# per cluster, so person and household estimates stay coherent. The final
# weights are constant within each cluster among its active members.
fit_hh <- weighting_spec(sample_survey, base_weights = pw) |>
  step_nonresponse(respondent = responded, method = "weighting_class", by = "region") |>
  step_model_calibration(
    x_formula  = ~ sex + region,
    models     = list(income = y_model(income ~ age + sex, engine = "glm")),
    population  = population,
    cluster = "household_id", equal_within_cluster = TRUE) |>
  prep()
w <- fit_hh$final_weight
max(tapply(w[w > 0], sample_survey$household_id[w > 0],
           function(x) diff(range(x))))    # 0: one weight per household

Nonresponse adjustment

Description

Inflates the weights of the eligible respondents so they also represent the eligible nonrespondents, under the assumption that response is ignorable given the information used. Three estimators are available – weighting classes, a response-propensity model (four engines, with optional cross-fitting), and calibration of the respondents to auxiliary totals – at the unit level or, through cluster, at a coarser level (e.g. the household).

Usage

step_nonresponse(
  spec,
  respondent,
  method = c("weighting_class", "propensity", "calibration"),
  by = NULL,
  formula = NULL,
  engine = c("logit", "tree", "forest", "boost"),
  weight_model = TRUE,
  num_classes = 5L,
  cluster = NULL,
  crossfit = NULL,
  crossfit_seed = NULL,
  totals = NULL,
  count = NULL,
  calfun = c("linear", "logit", "raking"),
  bounds = NULL,
  penalty = NULL,
  equal_within_cluster = FALSE,
  maxit = 50L,
  tol = 1e-06
)

Arguments

Details

The three estimators are the same operation with a different inflation factor. Weighting classes inflate the responding weight to the cell total, f_c = \sum_{i \in c} w_i / \sum_{i \in c} r_i w_i (with r_i = 1 if i responds), applied as w_i \leftarrow f_c w_i. A response-propensity model instead adjusts unit by unit, w_i \leftarrow w_i / \hat\phi_i, with \hat\phi_i the estimated response propensity. Calibration of the respondents solves for a factor v_i that makes the respondents reproduce a reference total (the two-phase / Sarndal-Lundstrom approach).

Value

The input weighting_spec with this step appended to its recipe. The step is recorded only; it is evaluated when prep() is called.

Examples

weighting_spec(sample_survey, base_weights = pw) |>
  step_nonresponse(respondent = responded, method = "weighting_class",
                   by = "region")

# household-level nonresponse (whole household responds or not)
weighting_spec(sample_survey, base_weights = pw) |>
  step_nonresponse(respondent = responded, method = "weighting_class",
                   by = "region", cluster = "household_id") |>
  prep()
# propensity with cross-fitting (out-of-sample, avoids overfitting)
weighting_spec(sample_survey, base_weights = pw) |>
  step_nonresponse(respondent = responded, method = "propensity",
                   formula = ~ region + sex, engine = "logit",
                   num_classes = 5, crossfit = 5, crossfit_seed = 1) |>
  prep()

# gradient boosting engine (requires the 'xgboost' package)

if (requireNamespace("xgboost", quietly = TRUE)) {
  weighting_spec(sample_survey, base_weights = pw) |>
    step_nonresponse(respondent = responded, method = "propensity",
                     formula = ~ region + sex + age, engine = "boost",
                     num_classes = 5, crossfit = 5) |>
    prep()
}


# nonresponse by calibration (two-phase): calibrate the respondents to the
# R+NR design-weighted totals of the auxiliaries at that stage, so their
# estimates reproduce the pre-nonresponse ones (Sarndal & Lundstrom 2005)
weighting_spec(sample_survey, base_weights = pw) |>
  step_nonresponse(respondent = responded, method = "calibration",
                   formula = ~ region + sex) |>
  prep()

Rescale the weights to a fixed sum

Description

Multiplies the active weights by a single constant so that they add up to a chosen total: either the number of active units (mean weight 1) or an arbitrary number. Use it as a presentation step, when the analysis wants normalized weights rather than population-scale ones.

Usage

step_rescale(spec, to = c("n", "total"), total = NULL, by = NULL)

Arguments

Value

The input weighting_spec with this step appended to its recipe. The step is recorded only; it is evaluated when prep() is called.

Examples

weighting_spec(sample_survey, base_weights = pw) |>
  step_rescale(to = "n") |> prep()

Round the final weights

Description

Rounds the weights to a given number of decimals, either unit by unit ("nearest") or with the largest-remainder method ("preserve_total"), which keeps the weighted total exactly. Typically the last step of a recipe, after calibration, when the weights have to be delivered as integers or with a fixed number of decimals.

Usage

step_round(spec, digits = 0L, method = c("nearest", "preserve_total"))

Arguments

Value

The input weighting_spec with this step appended to its recipe. The step is recorded only; it is evaluated when prep() is called.

Examples

weighting_spec(sample_survey, base_weights = pw) |>
  step_round(digits = 0) |> prep()

Within-cluster selection adjustment

Description

Undoes one stage of subsampling inside a cluster: when only some of the eligible units of a household (or dwelling, or area segment) were selected, the selected ones must represent the whole cluster, so their weight is multiplied by the inverse of the within-cluster selection probability. Apply it after the cluster-level eligibility and nonresponse steps and before the person-level nonresponse step.

Usage

step_select_within(spec, prob = NULL, n_eligible = NULL, n_selected = NULL)

Arguments

Details

Despite the name, the cluster need not be a household and the unit need not be a person: the step is the generic within-cluster subsampling adjustment. In a multi-stage design it can appear more than once – e.g. dwellings selected within sampled area segments, then persons selected within dwellings – each occurrence undoing one stage of subsampling.

Value

The input weighting_spec with this step appended to its recipe. The step is recorded only; it is evaluated when prep() is called.

Examples

# simple random selection of one eligible person per household
df <- transform(sample_survey,
                n_elig = ave(person_id, household_id, FUN = length))
weighting_spec(df, base_weights = pw) |>
  step_select_within(n_eligible = n_elig)

# simple random selection of two eligible persons per household
weighting_spec(df, base_weights = pw) |>
  step_select_within(n_eligible = n_elig, n_selected = 2)

Trim extreme weights against a ratio

Description

Caps the current weights at a multiple of a reference (each unit's base weight, the group median, or an absolute value) and, by default, redistributes the removed mass among the untrimmed units so the weighted total survives the trim. With by, the reference and the cap are computed separately within each subgroup. An optional step that can appear anywhere in the recipe, more than once.

Usage

step_trim(
  spec,
  max_ratio,
  min_ratio = NULL,
  reference = c("base", "median", "value"),
  redistribute = TRUE,
  by = NULL,
  maxit = 50L
)

Arguments

Details

There is no standard threshold: max_ratio is an analyst decision, a bias-variance trade-off. Use Kish's design effect (see summary) to judge whether trimming is worth it.

Value

The input weighting_spec with this step appended to its recipe. The step is recorded only; it is evaluated when prep() is called.

Examples

weighting_spec(sample_survey, base_weights = pw) |>
  step_trim(max_ratio = 3, reference = "base")

Trimmed calibration (range-restricted, totals-preserving)

Description

Pulls already-calibrated weights into an absolute interval ⁠[lower, upper]⁠ without breaking the calibration: instead of capping and redistributing, it re-solves a bounded calibration whose targets are the totals the incoming weights already reproduce, optionally with its own band per subgroup through by. It is the only one of the three trimming steps that leaves the calibration totals intact.

Usage

step_trim_calibrated(
  spec,
  formula,
  lower = NULL,
  upper = NULL,
  calfun = c("linear", "raking"),
  by = NULL,
  cluster = NULL,
  equal_within_cluster = FALSE,
  maxit = 100L,
  tol = 1e-07
)

Arguments

Details

The absolute-weight bound is imposed as a per-unit factor bound ⁠w_new / w in [lower/w, upper/w]⁠ on top of the incoming weights, using a bounded (range-restricted) calibration with the truncated Deville-Sarndal distances: the range-restricted Euclidean distance (calfun = "linear", the default) or the multiplicative one (calfun = "raking"). Weights inside the range that are not needed to restore the totals stay put; the out-of-range ones saturate at their bound and the rest move as little as possible. If the range is too tight to preserve every total, the totals that cannot be met are relaxed and a warning is raised.

This step is meant to run after a step_calibrate(): it acts on the positive incoming weights and leaves dropped units (weight 0) alone.

Value

The input weighting_spec with this step appended to its recipe. The step is recorded only; it is evaluated when prep() is called.

Examples

# calibrate, then trim the calibrated weights into [5.5, 13.5] without breaking
# the region/sex totals (the calibrated weights of sample_survey live in ~[5.4, 14])
weighting_spec(sample_survey, base_weights = pw) |>
  step_calibrate(method = "raking",
                 margins = list(region = c(table(population$region)),
                                sex    = c(table(population$sex)))) |>
  step_trim_calibrated(~ region + sex, lower = 5.5, upper = 13.5) |>
  prep()

Automatic weight trimming to an absolute band

Description

Caps the weights into an absolute interval ⁠[lower, upper]⁠ and hands the removed mass back to the units that were not capped, so the weighted total is preserved. This is the step to use when you have not calibrated yet (or will calibrate afterwards) and you want the cutoff chosen from the data rather than argued for: with upper = NULL it picks one by the Tukey far-out fence or by Potter's MSE rule.

Usage

step_trim_weights(
  spec,
  lower = 1,
  upper = NULL,
  method = c("tukey", "potter"),
  redistribute = c("proportional", "uniform"),
  strict = TRUE,
  maxit = 50L
)

Arguments

Value

The input weighting_spec with this step appended to its recipe. The step is recorded only; it is evaluated when prep() is called.

Examples

weighting_spec(sample_survey, base_weights = pw) |>
  step_nonresponse(respondent = responded, method = "weighting_class", by = "region") |>
  step_trim_weights(lower = 1, strict = TRUE) |> prep()

# Potter MSE-optimal cutoff chosen from the data
weighting_spec(sample_survey, base_weights = pw) |>
  step_nonresponse(respondent = responded, method = "weighting_class", by = "region") |>
  step_trim_weights(method = "potter") |> prep()

Unknown-eligibility adjustment

Description

Redistributes the weight of the cases whose eligibility was never resolved onto the resolved cases of the same adjustment cell, so the resolved units stand in for the unresolved share of the frame. Reach for it as the first step of the cascade, while the known-ineligible units are still in the data.

Usage

step_unknown_eligibility(spec, unknown, by = NULL, cluster = NULL)

Arguments

Details

Within each cell c the resolved cases are scaled up to also carry the unresolved ones,

w_i^{\mathrm{out}} = w_i \, \frac{\sum_{j \in c} w_j}{\sum_{j \in c,\, \mathrm{resolved}} w_j},

with every weight on the right the weight entering the step, so the ratio is one number per cell, the cell total is conserved exactly, and the result does not depend on the order in which units are updated.

Value

The input weighting_spec with this step appended to its recipe. The step is recorded only; it is evaluated when prep() is called.

Examples

weighting_spec(sample_survey, base_weights = pw) |>
  step_unknown_eligibility(unknown = unknown_elig, by = "region")

# household-level redistribution (unknown units without roster)
weighting_spec(sample_survey, base_weights = pw) |>
  step_unknown_eligibility(unknown = unknown_elig, by = "region",
                           cluster = "household_id")

Detailed per-step diagnostics

Description

Prints the full audit of an estimated recipe: the stage-by-stage evolution of the weights, then one block per step with that step's own diagnostics table and the design effect before and after it, and finally the R-indicator when the recipe adjusted for nonresponse. Where print() answers "what is in this recipe?", summary() answers "what did each step do, and what did it cost?".

Usage

## S3 method for class 'prepped_weighting_spec'
summary(object, ...)

Arguments

Value

(invisibly) the prepped object.

Examples

fitted <- weighting_spec(sample_survey, base_weights = pw) |>
  step_nonresponse(respondent = responded, method = "weighting_class", by = "region") |>
  prep()
summary(fitted)

Per-unit adjustment factors table

Description

Unrolls the cascade into a data.frame: the weight of every unit at every stage, plus the factor each step applied to it. This is the tidy form of prep()'s ⁠$history⁠, and the starting point for any diagnostic that plot() does not already draw.

Usage

weight_factors(object)

Arguments

Value

data.frame with one weight column per stage and one factor per step.

Examples

fitted <- weighting_spec(sample_survey, base_weights = pw) |>
  step_nonresponse(respondent = responded, method = "weighting_class", by = "region") |>
  prep()
head(weight_factors(fitted))

Conventions shared by every weightflow step

Description

Five things behave the same way in all twelve ⁠step_*()⁠ functions and are easier to learn once than twelve times: what the active set is and how a unit leaves it, why the order of the cascade is not arbitrary, the three different scales on which weight bounds are expressed, which arguments take a bare column name and which take a string, and how to read the diagnostics that prep() stores.

Details

The active set. A unit is active when its weight is finite and non-zero (is.finite(w) & w != 0). A weight of exactly 0 is the "dropped" marker: step_drop_ineligible() sets out-of-scope units to zero, an empty adjustment cell collapses to zero, and a unit that leaves the active set takes no part in any later step and is not returned by collect_weights(). A negative weight, by contrast, is a valid (if unusual) output of unbounded linear/GREG calibration and stays active: it is counted by collect_weights(), the stage funnel and design_effect(), so the reported totals match the weights actually returned. (One caveat: the Kish design effect assumes non-negative weights, so with negatives present its value is inflated – see design_effect().)

Why the order is not arbitrary. Each step multiplies the current weight, i.e. the weight leaving the previous step, so the cascade is read top to bottom. The methodological order of a household survey is: resolve unknown eligibility (step_unknown_eligibility()) while the ineligible units are still present, then drop the ineligible (step_drop_ineligible()), undo any within-cluster subsampling (step_select_within()), adjust for nonresponse (step_nonresponse()), calibrate to population totals (step_calibrate()), and finally trim, round or rescale for delivery. Putting calibration before a trim, or a trim before the nonresponse adjustment, changes the estimator, which is why the steps are explicit rather than inferred.

Three scales for weight bounds. Bounds are expressed on three different scales, and mixing them up is the most common mistake for users coming from survey:

• step_trim() max_ratio / min_ratio are a ratio to a reference (each unit's base weight, the group median, or an absolute value): max_ratio = 3 caps a weight at three times its reference.

• step_trim_weights() and step_trim_calibrated() lower / upper are absolute weights: upper = 400 caps the weight itself at 400.

• step_calibrate() bounds = c(L, U) bound the calibration g-factor w_new / d (the multiplicative adjustment), not the weight.

Bare column name vs string. Arguments that name a single variable to be evaluated on the data take an unquoted (bare) column name or condition: base_weights in weighting_spec(), and respondent, unknown, ineligible, prob, n_eligible, n_selected in the steps. Arguments that name grouping or design structure take character strings: by, cluster, and strata / psu in the variance functions. So it is step_nonresponse(respondent = responded, by = "region") – responded bare, "region" quoted.

Reading the diagnostics. prep() returns an object that carries the weight at every stage (⁠$history⁠, a named list of vectors), one entry per step (⁠$steps⁠, each with its own ⁠$diagnostics⁠ table and ⁠$alerts⁠), and the recipe-level ⁠$alerts⁠. Rather than read those directly, use summary() for the stage-by-stage audit, plot() for the visual cascade, weight_factors() for the per-unit factor table, design_effect() for the Kish design effect, and report_weighting() for the full self-contained HTML report.

Start a weighting specification

Description

Opens a weighting recipe on a sample and its design base weights. The object it returns is inert: it holds the data, the name of the base-weight column and an empty list of steps, and computes nothing. Every ⁠step_*()⁠ function takes such an object and returns it with one more step appended; prep() estimates the result.

Usage

weighting_spec(data, base_weights)

Arguments

Value

an object of class "weighting_spec".

Examples

rec <- weighting_spec(sample_survey, base_weights = pw)
rec

Specify a working model for a study variable y

Description

Declares the working model that step_model_calibration() fits for one study variable: a formula, a learner (a linear/glm model or a machine-learning method such as a regression tree, a random forest or gradient boosting) and, for glm, a family. It builds no model and touches no data – it records the specification that the calibration step will fit on the sample and predict over the population.

Usage

y_model(formula, engine = c("glm", "tree", "forest", "boost"), family = NULL)

Arguments

Value

a model specification list.

Examples

y_model(income ~ age + sex, engine = "glm")