Simulation Vignette for FETWFE: From Coefficients to True Treatment Effects

Gregory Faletto

2025-05-14

# Load necessary libraries
library(dplyr)
#> 
#> Attaching package: 'dplyr'
#> The following objects are masked from 'package:stats':
#> 
#>     filter, lag
#> The following objects are masked from 'package:base':
#> 
#>     intersect, setdiff, setequal, union
library(fetwfe)

Introduction

This vignette demonstrates how to conduct simulation studies with the {fetwfe} package. In particular, we will:

The workflow here follows the simulation‐study design outlined in the paper, so you may wish to skim its setup section for additional context.

Simulation Workflow Using Piping

Below is a complete simulation pipeline, step by step.

Step 1: Generate Simulation Coefficients

The genCoefs() function returns an object of class "FETWFE_coefs", containing both the coefficient vector and its simulation parameters. In this example we set:

# Generate the coefficient object for simulation
sim_coefs <- genCoefs(
  R         = 3, 
  T         = 6, 
  d         = 2, 
  density   = 0.1, 
  eff_size  = 2, 
  seed      = 101
)

(Again, for more details on the meaning of these parameters, see the simulation study section of the paper.)

Step 2: Simulate Panel Data

Next, we simulate a panel data set using the generated coefficient object with the simulateData() function. With simulateData(), we generate:

Here we choose:

# Simulate panel data based on the coefficients
sim_data <- simulateData(
  sim_coefs,
  N = 30,
  sig_eps_sq = 5,
  sig_eps_c_sq = 5,
  distribution = "gaussian"
  )

The dataframe is stored in sim_data$pdata, so we can take a quick look at the results:

head(sim_data$pdata)
#>   time   unit treatment          y      cov1     cov2
#> 1    1 unit01         0 -0.8394412 0.4061679 2.478849
#> 2    2 unit01         0 -4.3685319 0.4061679 2.478849
#> 3    3 unit01         0 -6.7668056 0.4061679 2.478849
#> 4    4 unit01         0 -5.9939733 0.4061679 2.478849
#> 5    5 unit01         0  3.0107793 0.4061679 2.478849
#> 6    6 unit01         0  1.0709035 0.4061679 2.478849

Step 3: Run the FETWFE Estimator on Simulated Data

We then run the estimator on the simulated data using fetwfeWithSimulatedData(). (We could get the same results by manually unpacking sim_data and passing the arguments appropriately to fewtfe(). fetwfeWithSimulatedData() is just a wrapper function that takes care of this for us.)

result <- fetwfeWithSimulatedData(sim_data)

We can now extract the results from result in the same way that we can with the standard fetwfe() function.

# Overall ATT estimate
cat("Estimated Overall ATT:", result$att_hat, "\n")
#> Estimated Overall ATT: -1.061722

# 95% confidence interval
ci_lower <- result$att_hat - qnorm(0.975) * result$att_se
ci_upper <- result$att_hat + qnorm(0.975) * result$att_se
cat("95% CI for ATT: [", ci_lower, ", ", ci_upper, "]\n")
#> 95% CI for ATT: [ -1.820098 ,  -0.3033451 ]

# Cohort‐specific ATTs
print(result$catt_df)
#>   Cohort Estimated TE        SE ConfIntLow ConfIntHigh
#> 1      2     0.000000 0.0000000   0.000000   0.0000000
#> 2      3    -1.459867 0.4660013  -2.373213  -0.5465217
#> 3      4    -1.459867 0.4660013  -2.373213  -0.5465217

Step 4: Extract True Treatment Effects

To evaluate the estimated ATT, we can compute the true treatment effects using the original coefficient object. The getTes() function extracts both the overall average treatment effect and the cohort-specific effects.

# Extract the true treatment effects
true_tes <- getTes(sim_coefs)

# Print the true overall treatment effect
cat("True Overall ATT:", true_tes$att_true, "\n")
#> True Overall ATT: -0.6666667

# Print the cohort-specific treatment effects
print(true_tes$actual_cohort_tes)
#> [1]  0  0 -2

We can use this to calculate metrics to evaluate our estimated treatment effect, like squared error:

squared_error <- (result$att_hat - true_tes$att_true)^2

cat("Squared error of ATT estimate:", squared_error, "\n")
#> Squared error of ATT estimate: 0.1560685

Combining the Workflow in One Pipeline

You can also chain the simulation functions together with the pipe operator. The following code generates the coefficients, simulates the data, and runs the estimator all in one pipeline:

coefs <- genCoefs(R = 3, T = 6, d = 2, density = 0.1, eff_size = 2, seed = 2025)

result_piped <- coefs |>
  simulateData(N = 30, sig_eps_sq = 5, sig_eps_c_sq = 5) |>
  fetwfeWithSimulatedData()

cat("Estimated Overall ATT from piped workflow:", result_piped$att_hat, "\n")
#> Estimated Overall ATT from piped workflow: 0

true_tes_piped <- coefs |> getTes()

# Print the true overall treatment effect
cat("True Overall ATT:", true_tes_piped$att_true, "\n")
#> True Overall ATT: 0

# Print the squared estimation error
squared_error_piped = (result_piped$att_hat - true_tes_piped$att_true)^2

cat("Squared estimation error:", squared_error_piped, "\n")
#> Squared estimation error: 0

Conclusion

In this vignette, we walked through how to use the simulation functions in the {fetwfe} package to simulate data and run simulations similar to the ones in the simulation studies section of the FETWFE paper.

This pipeline streamlines simulation experiments so you can rapidly evaluate FETWFE’s performance under varying scenarios. For more details, consult the package documentation or reach out to the author.