Sensitivity and identification quantification by a relative latent model complexity perturbation in Bayesian meta-analysis

In recent years, Bayesian meta-analysis expressed by a normal-normal hierarchical model (NNHM) has been widely used for combining evidence from multiple studies. Data provided for the NNHM are frequently based on a small number of studies and on uncertain within-study standard deviation values. Despite the widespread use of Bayesian NNHM, it has always been unclear to what extent the posterior inference is impacted by the heterogeneity prior (sensitivity S) and by the uncertainty in the within-study standard deviation values (identification I). Thus to answer this question, we developed a unified method to simultaneously quantify both sensitivity and identification (S-I) for all model parameters in a Bayesian NNHM, based on derivatives of the Bhattacharyya coefficient with respect to relative latent model complexity (RLMC) perturbations. Three case studies exemplify the applicability of the method proposed: historical data for a conventional therapy, data from which one large study is first included and then excluded, and two subgroup meta-analyses specified by their randomization status. We analyzed 6 scenarios, crossing three RLMC targets with two heterogeneity priors (half-normal, half-Cauchy). The results show that S-I explicitly reveals which parameters are affected by the heterogeneity prior and by the uncertainty in the within-study standard deviation values. In addition, we compare the impact of both heterogeneity priors and quantify how S-I values are affected by omitting one large study and by the randomization status. Finally, the range of applicability of S-I is extended to Bayesian NtHM. A dedicated R package facilitates automatic S-I quantification in applied Bayesian meta-analyses.