When Effects Cannot be Estimated: Redefining Estimands to Understand the Effects of Naloxone Access Laws
Violations of the positivity assumption (also called the common support condition) challenge health policy research and can result in significant bias, large variance, and invalid inference. We define positivity in the single- and multiple-timepoint (i.e., longitudinal) health policy evaluation setting, and discuss real-world threats to positivity. We show empirical evidence of the practical positivity violations that can result when attempting to estimate the effects of health policies (in this case, Naloxone Access Laws). In such scenarios, an alternative is to estimate the effect of a shift in law enactment (e.g., the effect if enactment had been delayed by some number of years). Such an effect corresponds to what is called a modified treatment policy, and dramatically weakens the required positivity assumption, thereby offering a means to estimate policy effects even in scenarios with serious positivity problems. We apply the approach to define and estimate the longitudinal effects of Naloxone Access Laws on opioid overdose rates.
Sociodemographic and geographic disparities in excess fatal drug overdoses during the COVID-19 pandemic in California: A population-based study
Background/UNASSIGNED:The coronavirus disease 2019 (COVID-19) pandemic is co-occurring with a drug addiction and overdose crisis. Methods/UNASSIGNED:We fit overdispersed Poisson models, accounting for seasonality and secular trends, to estimate the excess fatal drug overdoses (i.e., deaths greater than expected), using data on all deaths in California from 2016 to 2020. Findings/UNASSIGNED:Between January 5, 2020 and December 26, 2020, there were 8605 fatal drug overdoses-a 44% increase over the same period one year prior. We estimated 2084 (95% CI: 1925 to 2243) fatal drug overdoses were excess deaths, representing 5Â·28 (4Â·88 to 5Â·68) excess fatal drug overdoses per 100,000 population. Excess fatal drug overdoses were driven by opioids (4Â·48 [95% CI: 4Â·18 to 4Â·77] per 100,000), especially synthetic opioids (2Â·85 [95% CI: 2Â·56 to 3Â·13] per 100,000). The non-Hispanic Black and Other non-Hispanic populations were disproportionately affected with 10Â·1 (95% CI: 7Â·6 to 12Â·5) and 13Â·26 (95% CI: 11Â·0 to 15Â·5) excess fatal drug overdoses per 100,000 population, respectively, compared to 5Â·99 (95% CI: 5.2 to 6.8) per 100,000 population in the non-Hispanic white population. There was a steep, nonlinear educational gradient with the highest rate among those with only a high school degree. There was a strong spatial patterning with the highest levels of excess mortality in the southernmost region and consistently lower levels at progressively more northern latitudes (7Â·73 vs 1Â·96 per 100,000). Interpretation/UNASSIGNED:Fatal drug overdoses disproportionately increased in 2020 among structurally marginalized populations and showed a strong geographic gradient. Local, tailored public health interventions are urgently needed to reduce growing inequities in overdose deaths. Funding/UNASSIGNED:US National Institutes of Health and Department of Veterans Affairs.
Heterogeneous treatment effects in social policy studies: An assessment of contemporary articles in the health and social sciences
PURPOSE/OBJECTIVE:Social policies are important determinants of population health but may have varying effects on subgroups of people. Evaluating heterogeneous treatment effects (HTEs) of social policies is critical to determine how social policies will affect health inequities. Methods for evaluating HTEs are not standardized. Little is known about how often and by what methods HTEs are assessed in social policy and health research. METHODS:A sample of 55 articles from 2019 on the health effects of social policies were evaluated for frequency of reporting HTEs; for what subgroupings HTEs were reported; frequency of a priori specification of intent to assess HTEs; and methods used for assessing HTEs. RESULTS:A total of 24 (44%) studies described some form of HTE assessment, including by age, gender, education, race/ethnicity, and/or geography. Among studies assessing HTEs, 63% specified HTE assessment a priori, and most (71%) used descriptive methods such as stratification; 21% used statistical tests (e.g., interaction terms in a regression); and no studies used data-driven algorithms. CONCLUSIONS:Although understanding HTEs could enhance policy and practice-based efforts to reduce inequities, it is not routine research practice. Increased evaluation of HTEs across relevant subgroups is needed.
Systematic Review Reveals Lack of Causal Methodology Applied to Pooled Longitudinal Observational Infectious Disease Studies
OBJECTIVES/OBJECTIVE:Among ID studies seeking to make causal inferences and pooling individual-level longitudinal data from multiple infectious disease cohorts, we sought to assess what methods are being used, how those methods are being reported, and whether these factors have changed over time. STUDY DESIGN AND SETTING/METHODS:Systematic review of longitudinal observational infectious disease studies pooling individual-level patient data from 2+ studies published in English in 2009, 2014, or 2019. This systematic review protocol is registered with PROSPERO (CRD42020204104). RESULTS:Our search yielded 1,462 unique articles. Of these, 16 were included in the final review. Our analysis showed a lack of causal inference methods and of clear reporting on methods and the required assumptions. CONCLUSION/CONCLUSIONS:There are many approaches to causal inference which may help facilitate accurate inference in the presence of unmeasured and time-varying confounding. In observational ID studies leveraging pooled, longitudinal IPD, the absence of these causal inference methods and gaps in the reporting of key methodological considerations suggests there is ample opportunity to enhance the rigor and reporting of research in this field. Interdisciplinary collaborations between substantive and methodological experts would strengthen future work.
Occupation and Educational Attainment Characteristics Associated With COVID-19 Mortality by Race and Ethnicity in California
Importance:Racial and ethnic inequities in COVID-19 mortality may be driven by occupation and education, but limited evidence has assessed these mechanisms. Objective:To estimate whether occupational characteristics or educational attainment explained the associations between race and ethnicity and COVID-19 mortality. Design, Setting, and Participants:This population-based retrospective cohort study of Californians aged 18 to 65 years linked COVID-19 deaths to population estimates within strata defined by race and ethnicity, gender, age, nativity in the US, region of residence, education, and occupation. Analysis was conducted from September 2020 to February 2022. Exposures:Education and occupational characteristics associated with COVID-19 exposure (essential sector, telework option, wages). Main Outcomes and Measures:All confirmed COVID-19 deaths in California through February 12, 2021. The study estimated what COVID-19 mortality would have been if each racial and ethnic group had (1) the COVID-19 mortality risk associated with the education and occupation distribution of White people and (2) the COVID-19 mortality risk associated with the lowest-risk educational and occupational positions. Results:Of 25â€¯235â€¯092 participants (mean [SD] age, 40  years; 12â€¯730â€¯395 [50%] men), 14â€¯783 died of COVID-19, 8â€¯125â€¯565 (32%) had a Bachelor's degree or higher, 13â€¯345â€¯829 (53%) worked in essential sectors, 11â€¯783â€¯017 (47%) could not telework, and 12â€¯812â€¯095 (51%) had annual wages under $51â€¯700. COVID-19 mortality ranged from 15 deaths per 100â€¯000 for White women and Asian women to 139 deaths per 100â€¯000 for Latinx men. Accounting for differences in age, nativity, and region of residence, if all races and ethnicities had the COVID-19 mortality associated with the occupational characteristics of White people (sector, telework, wages), COVID-19 mortality would be reduced by 10% (95% CI, 6% to 14%) for Latinx men, but increased by 5% (95% CI, -8% to 17%) for Black men. If all working-age Californians had the COVID-19 mortality associated with the lowest-risk educational and occupational position (Bachelor's degree, nonessential, telework, and highest wage quintile), there would have been 43% fewer COVID-19 deaths among working-age adults (8441 fewer deaths; 95% CI, 32%-54%), with the largest absolute risk reductions for Latinx men (3755 deaths averted; 95% CI, 3304-4255 deaths) and Latinx women (2329 deaths averted; 95% CI, 2038-2621 deaths). Conclusions and Relevance:In this population-based cohort study of working-age California adults, occupational disadvantage was associated with excess COVID-19 mortality for Latinx men. For all racial and ethnic groups, excess risk associated with low-education, essential, on-site, and low-wage jobs accounted for a substantial fraction of COVID-19 mortality.
A descriptive analysis of 2020 California Occupational Safety and Health Administration covid-19-related complaints
COVID-19 mortality has disproportionately affected specific occupations and industries. The Occupational Safety and Health Administration (OSHA) protects the health and safety of workers by setting and enforcing standards for working conditions. Workers may file OSHA complaints about unsafe conditions. Complaints may indicate poor workplace safety during the pandemic. We evaluated COVID-19-related complaints filed with California (Cal)/OSHA between January 1, 2020 and December 14, 2020 across seven industries. To assess whether workers in occupations with high COVID-19-related mortality were also most likely to file Cal/OSHA complaints, we compared industry-specific per-capita COVID-19 confirmed deaths from the California Department of Public Health with COVID-19-related complaints. Although 7820 COVID-19-related complaints were deemed valid by Cal/OSHA, only 627 onsite inspections occurred, and 32 citations were issued. Agricultural workers had the highest per-capita COVID-19 death rates (402 per 100,000 workers) but were least represented among workplace complaints (44 per 100,000 workers). Health Care workers had the highest complaint rates (81 per 100,000 workers) but the second lowest COVID-19 death rate (81 per 100,000 workers). Industries with the highest inspection rates also had high COVID-19 mortality. Our findings suggest complaints are not proportional to COVID-19 risk. Instead, higher complaint rates may reflect worker groups with greater empowerment, resources, or capacity to advocate for better protections. This capacity to advocate for safe workplaces may account for relatively low mortality rates in potentially high-risk occupations. Future research should examine factors determining worker complaints and complaint systems to promote participation of those with the greatest need of protection.
What to Do When Everything Happens at Once: Analytic Approaches to Estimate the Health Effects of Co-Occurring Social Policies
Social policies have great potential to improve population health and reduce health disparities. Thus, increasing empirical research seeks to quantify the health effects of social policies by exploiting variation in the timing of policy changes across places. Multiple social policies are often adopted simultaneously or in close succession in the same locations, creating co-occurrence which must be handled analytically for valid inferences. Although this is a substantial methodological challenge for studies aiming to isolate social policy effects, limited prior work has systematically considered analytic solutions within a causal framework or assessed whether these solutions are being adopted. We designated seven analytic solutions to policy co-occurrence, including efforts to disentangle individual policy effects and efforts to estimate the combined effects of co-occurring policies. We leveraged an existing systematic review of social policies and health to evaluate how often policy co-occurrence is identified as a threat to validity and how often each analytic solution is applied in practice. Of the 55 studies, only 17 (31%) reported checking for any co-occurring policies, although 36 (67%) used at least one approach that helps address policy co-occurrence. The most common approaches were: adjusting for measures of co-occurring policies; defining the outcome on subpopulations likely to be affected by the policy of interest (but not other co-occurring policies); and selecting a less-correlated measure of policy exposure. As health research increasingly focuses on policy changes, we must systematically assess policy co-occurrence and apply analytic solutions to strengthen future studies on the health effects of social policies.
The Revolution Will Be Hard to Evaluate: How Co-Occurring Policy Changes Affect Research on the Health Effects of Social Policies
Extensive empirical health research leverages variation in the timing and location of policy changes as quasi-experiments. Multiple social policies may be adopted simultaneously in the same locations, creating co-occurrence which must be addressed analytically for valid inferences. The pervasiveness and consequences of co-occurring policies have received limited attention. We analyzed a systematic sample of 13 social policy databases covering diverse domains including poverty, paid family leave, and tobacco. We quantified policy co-occurrence in each database as the fraction of variation in each policy measure across different jurisdictions and times that could be explained by co-variation with other policies (R2). We used simulations to estimate the ratio of the variance of effect estimates under the observed policy co-occurrence to variance if policies were independent. Policy co-occurrence ranged from very high for state-level cannabis policies to low for country-level sexual minority rights policies. For 65% of policies, greater than 90% of the place-time variation was explained by other policies. Policy co-occurrence increased the variance of effect estimates by a median of 57-fold. Co-occurring policies are common and pose a major methodological challenge to rigorously evaluating health effects of individual social policies. When uncontrolled, co-occurring policies confound one another, and when controlled, resulting positivity violations may substantially inflate the variance of estimated effects. Tools to enhance validity and precision for evaluating co-occurring policies are needed.
Conducting density-sampled case-control studies using survey data with complex sampling designs: A simulation study
PURPOSE/OBJECTIVE:Population-based surveys are possible sources from which to draw representative control data for case-control studies. However, these surveys involve complex sampling that could lead to biased estimates of measures of association if not properly accounted for in analyses. Approaches to incorporating complex-sampled controls in density-sampled case-control designs have not been examined. METHODS:We used a simulation study to evaluate the performance of different approaches to estimating incidence density ratios (IDR) from case-control studies with controls drawn from complex survey data using risk-set sampling. In simulated population data, we applied four survey sampling approaches, with varying survey sizes, and assessed the performance of four analysis methods for incorporating survey-based controls. RESULTS:Estimates of the IDR were unbiased for methods that conducted risk-set sampling with probability of selection proportional to survey weights. Estimates of the IDR were biased when sampling weights were not incorporated, or only included in regression modeling. The unbiased analysis methods performed comparably and produced estimates with variance comparable to biased methods. Variance increased and confidence interval coverage decreased as survey size decreased. CONCLUSIONS:Unbiased estimates are obtainable in risk-set sampled case-control studies using controls drawn from complex survey data when weights are properly incorporated.
California's Mental Health Services Act and Mortality Due to Suicide, Homicide, and Acute Effects of Alcohol: A Synthetic Control Application
California's Mental Health Services Act (MHSA) substantially expanded funding of county mental health services through a state tax, and led to broad prevention efforts and intensive services for individuals experiencing serious mental disorders. We estimated the associations between MHSA and mortality due to suicide, homicide, and acute effects of alcohol. Using annual cause-specific mortality data for each US state and the District of Columbia from 1976-2015, we used a generalization of the quasi-experimental synthetic control method to predict California's mortality rate for each outcome in the absence of MHSA using a weighted combination of comparison states. We calculated the association between MHSA and each outcome as the absolute difference and percentage difference between California's observed and predicted average annual rates over the postintervention years (2007-2015). MHSA was associated with modest decreases in average annual rates of homicide (-0.81/100,000 persons, corresponding to a 13% reduction) and mortality from acute alcohol effects (-0.35/100,000 persons, corresponding to a 12% reduction). Placebo test inference suggested that the associations were unlikely to be due to chance. MHSA was not associated with suicide. Protective associations with mortality due to homicide and acute alcohol effects provide evidence for modest health benefits of MHSA at the population level.