Policy Evaluation

EPICER: EVALUATING THE POLICY IMPACT OF CHILDHOOD EDUCATION REFORMS

Our research aims to understand the impacts of large-scale Early Childhood Education and Care (ECEC) reforms occurring in high-income countries between 1995-2008 and determine which interventions have been most effective for increasing coverage of services, and ultimately, adolescent scholastic achievement, health, and well-being.

Investigators: Arijit Nandi (PI), Emis Akbari, Frank Elgar, Sam Harper, Seungmi Yang, Jere Behrman, Linda White

Funding: Social Sciences and Humanities Research Council of Canada.

evaluating quebec childcare policy

The changing composition of families and the work force has led to a number of policy innovations in Canada to alter the balance of parental care and advance early childhood education. The resulting policy instruments have taken a distinct nature in Quebec, which has introduced both a more generous and accessible parental leave program and universal publicly funded childcare program. As part of a policy agenda to combat social inequality and poverty, there is further potential for such policies to mitigate social inequalities in health and development outcomes in children. This project takes an interdisciplinary approach to investigate the distribution of health and developmental outcomes of these programs, and to analyze the degree to which they have contributed to greater health equity in Quebec.

Investigators: Thomas Charters, Arijit Nandi, Seungmi Yang, Daniel Weinstock

Funding: Canadian Institutes of Health Research

Paid Family and Medical Leave: A data-driven approach to learning what works and can have transformative impact in the United States

Parental and medical leave policies allow employees to take time off work for pregnancy and birth, for personal illness, or to care for sick children, parents and spouses. By 2013, all OECD countries other than the United States offered some form of national paid leave policy. Over the past two decades, there have been hundreds of changes to legislation governing paid leave from work. Although recent trends are towards more generous benefits and government-mandated leave, there is still substantial variation in allowances and benefits, both cross-nationally and sub-nationally. The wide variation in the design and generosity of leave policies may result in heterogeneous effects, both across economic, social, and health outcomes and across population groups.

The Paid Family and Medical Leave research project is examining paid leave policies in the United States and across the OECD. So far, the team has examined the health effects of paid parental leave in California and New Jersey on infants, and the impact of paid leave on employment in European OECD countries.

Investigators: Arijit Nandi (PI), Erika Braithwaite, Michelle Dimitris, Sam Harper, Deepa Jahagirdar, Jay Kaufman, Erin Strumpf

Funding: Anonymous gift to the University of California at Los Angeles (by sub-award)

PROSPERED: Policy-Relevant Observational Studies for Population HEALTH Equity and Responsible Development

The overarching objective of this project is to produce rigorous evidence concerning the impact of public policies on the health-related targets set forth by the United Nations Sustainable Development Goals (SDGs), which include reducing maternal mortality, ending preventable child deaths, eradicating major communicable diseases, and mitigating premature mortality due to non-communicable diseases (NCDs), among others. Focusing on public policies intended to influence healthcare services, public health programs, social welfare, economic opportunity, gender equality, and the environment, we are using quasi-experimental techniques to learn about the consequences of specific policy reforms in terms of their health effects, influence on gender and social inequalities, and mechanisms of action. The fundamental premise of the approach is that evaluations of existing policy reforms can help generate timely and actionable evidence for improving population health and mitigating health inequalities. This project includes partners in academia, government and civil society in Canada, India, Peru, Tanzania, the United States and the United Kingdom, and includes a training program to mentor the next generation of researchers.

Investigators:

Madhav Badami, Tim Brewer, Cesar Carcamo, Marnie Davidson, John Frank, Patricia Garcia, Eveline Geubbels, Dave Gordon, Sam Harper, Jody Heymann, Jay Kaufman, Roger Mathisen, Arnab Mukherji, Devaki Nambiar, Arijit Nandi, Shailen Nandy, Gilles Paradis, Helen Scott, Erin Strumpf, Hema Swaminathan, Frank Welsh

Funding: Canadian Institutes of Health Research

Seat Belt Laws and Inequality in Motor Vehicle Accident Mortality

Public health policies that nominally apply to all may not impact the health of all social groups equally. We estimate the differential effects of US state seat belt laws across education groups, showing that they increase rates of seat belt use most among groups with less education. We also document trends in inequality in motor vehicle accident mortality over time. Future work will examine how seat belt laws effect inequalities in deaths from motor vehicle accidents.

Investigators: Sam Harper, Erin Strumpf

Funding: Canadian Institutes of Health Research.

The impacts of being formally enrolled with a GP on continuity and integration of care: Evidence from a comparison of Quebec and British columbia

Some Canadian provinces have included patient enrolment with a General Practitioner (GP) as a foundation of their efforts to make primary health care more accessible, more continuous, and of higher quality. The idea behind patient enrolment – also called rostering or registration – is that all patients have a family doctor who takes responsibility for their care. Other provinces are currently considering such programs, while still others think they are not the best way to ensure accessible, timely health care for Canadians. Unfortunately, little is known about the actual benefits and costs of patient enrolment policies in Canada.

This research project brings together patients, providers, decision makers and researchers to provide a better understanding of how patient enrolment affects patient experiences, continuity and quality of care, and policy-relevant outcomes that are important for the sustainability of our health care systems.  Because Quebec and British Columbia have already implemented patient enrolment programs, and because those programs are designed quite differently, we focus on those two provinces. We will talk to patients and family doctors to hear about their experiences and their priorities. We will also use health care data and statistical methods designed to evaluate policy interventions to understand what patient enrolment means for quality and continuity of care.  We will try to understand if patient enrolment works better for some patients than for others, paying special attention to Canadians who need and use the health care system most. Our results will facilitate future cross-provincial studies of health care policy innovations and will support decision makers to build improved, patient-oriented, integrated health care and social services systems.

Investigators: E. Strumpf (NPA); PKUs A.Groulx, H. Davidson; PAs L. Goldsmith, C. Loignon, K. McGrail; Patient Partner A. Bouchard; Co-As M. Dumas Pilon, C. Hudon, F. Légaré, M-J. Ouimet, R. Lavergne

Funding: Canadian Institutes of Health Research, Michael Smith Foundation, Quebec Ministry of Health, Réseau-1 QC, McGill University, CR-HCLM

THE Impact Of Reproductive Health Restrictions On Women’s Health Outcomes

Abortion rates have been steadily decreasing in both Canada and the US, but the cause remains poorly understood. In the US, where reproductive health policies vary considerably from state to state, abortion opponents credit a recent upsurge in state-level abortion restrictions (including mandatory waiting periods, mandatory counseling, and provider licensing requirements), while advocates argue that the decline is due to empowering initiatives, such as improved access to contraception and sex education programs. Existing research, particularly on restrictions, is largely descriptive; this prevents us from understanding the causal effects of, and potentially complex interplay between, restrictive and empowering policies on abortion rates. Disentangling the effects of these factors is challenging but important, as these policies likely have a marked effect on population health. This project will bridge this knowledge gap by deconstructing the recent decline in US abortion rates and teasing apart the causal impact of empowering and restrictive initiatives. The substantial state-to-state variation in the timing of policy implementation in the US will permit the use of quasi-experimental study study designs to minimize confounding. Abortion is a common and safe medical procedure, but abortion policy decisions are currently being made with low quality evidence on effectiveness or long-term health implications. This work will ultimately provide a better understanding of why abortion rates are declining, which is vital in informing future policy decisions.

Investigators: Sam Harper, Erin Strumpf

Funding: Fonds de recherche du Québec – Santé (FRQS)

Program Evaluation

Ontario’s HPV Vaccination Program (completed)

As HPV vaccination campaigns and programs began to be considered around the world after 2006, concerns were raised about their real-world impacts and unintended consequences, including on the sexual behaviour of adolescents. We estimated the effects of Ontario’s vaccination program for grade 8 girls, and the effects of vaccine receipt, on both clinical and behavioural health outcomes at the population level. Our results provide information to aid decision making by policymakers, health professionals, parents, and young adults.

Investigators: Jay Kaufman, Linda Levesque, Leah Smith, Erin Strumpf

Funding: Canadian Institutes of Health Research

Randomized evaluation of a TB control program in Peru

Until recently, the focus of TB programs worldwide has simply been on treating TB cases who present to clinics, however this strategy of awaiting cases has done little to accelerate the goals of one day eliminating TB. Cases who are coughing and do not start treatment can continue to spread the disease, and quickly finding and engaging those with TB disease in the community is considered a top public health priority. Peru has one of the most well-established TB programs, yet continues to see new cases of TB every year and has one of the highest TB rates in the Americas. The local Direccion de Salud in San Juan de Lurigancho, the district with the highest TB incidence in Lima, Peru, has decided to proactively introduce a program for evaluating families of TB cases in order to more rapidly detect secondary cases within households. We took advantage of the roll out of this policy innovation in order to evaluate the cost-effectiveness of planned active case finding for all household contacts of smear-positive TB cases in 34 district healthcare centres using a stepped-wedge randomized trial.  Data collection ended in August 2016 and data are currently being cleaned and analyzed.  

Shah L, Rojas Peña M, Mori O, Zamudio C, Kaufman JS, Otero L, Gotuzzo E, Seas C, Brewer TF. A pragmatic stepped-wedge cluster randomized trial to evaluate the effectiveness and cost-effectiveness of active case finding for household contacts within a routine tuberculosis program, San Juan de Lurigancho, Lima, Peru. Int J Infect Dis. 2020 Sep 18;100:95-103. PMID: 32950737

Investigators:

Tim Brewer, Jay Kaufman, Lena Shah

Funding: Canadian Institutes of Health Research

TAMANI: Tabora Maternal Newborn Health Project

We are monitoring and evaluating a health systems strengthening program being implemented by CARE Canada, in partnership with the Society of Obstetricians and Gynaecologists of Canada (SOGC) and the Canadian Society for International Health (CSIH), in the Tabora region of Tanzania. The aim of this program is to reduce maternal and newborn mortality and morbidity by improving the availability of quality maternal and newborn health. Activities include: training and mentoring of Regional and District Council Health Management Teams (R/CHMTs) in data management and usage, supply chain management, budgeting and leadership; development of an emergency transportation system; refurbishing health centre and hospital maternity wards and procuring Basic Emergency Obstetric and Newborn CARE (BEmONC) and Comprehensive Emergency Obstetric and Newborn CARE (CEmONC) equipment; training and mentoring of health care workers (HCWs) on BEmONC, CEmONC and family planning;  and training and support of Community Health Workers (CHWs) to deliver quality maternal and newborn health education and promote utilization of health care services. We will be assessing the program’s impacts by integrating monitoring and evaluation with the delivery of interventions using a randomized phase-in or “stepped wedge” design.

Investigators:

Marnie Davidson, Sam Harper, Arijit Nandi

Funding: Global Affairs Canada

Team-Based Primary Care: Quebec’s Family Medicine Groups

Improved primary health care is often cited as key to better quality of care, increased efficiency in the health care system, and better health outcomes. While reforms and innovations are frequent, rigorous evaluations are not. In this set of projects, we examine the effects of interdisciplinary, team-based primary care practice on access to primary care, quality of care, hospital readmissions, health care utilization and health care costs. We use longitudinal administrative health databases and quasi-experimental methods to address biases driven by selective participation by patients and providers in these new models of care.

Investigators: Mehdi Ammi, Roxane Borges da Silva, Eric Latimer, Marie-Jo Ouimet, Sylvie Provost, Erin Strumpf, Pierre Tousignant

Funding: Canadian Institutes of Health Research, Fonds de recherche du Québec – Santé (FRQS), Ministère de la Santé et des Services sociaux (MSSS)

The Uttam Unnati Study: Affordable Daycare to Empower Indian Women

In partnership with the Institute of Financial Management and Research—Leveraging Evidence for Access and Development (IFMR-LEAD) and the non-government voluntary organization Seva Mandir, we are conducting a cluster-randomized impact evaluation for estimating the effect of a community-based day-care program on empowerment, economic opportunity, and health among mothers living in rural Rajasthan, India. The responsibility of providing childcare is imposed primarily on women and contributes to gender inequalities experienced by women over the life-course. The provision of affordable and reliable day-care services is a potentially important policy lever for empowering Indian women. Access to day-care might reduce barriers to labor force entry and generate economic opportunities for women. However, empirical evidence concerning the effects of day-care programs in low-and-middle-income countries is scarce. This interdisciplinary research initiative will address an important research gap and has the potential to inform policies for improving the day-care system in India in ways that promote inclusive economic growth.

Investigators: Parul Agarwal, Sam Harper, Shannon Maloney, Arijit Nandi, Priyanka Singh

Funding source(s): International Development Research Centre, Spencer Foundation

Evaluation of Clinical Interventions

Risks and Benefits of Obstetric Intervention in Childbirth

Interventions in childbirth through Cesarean delivery or medical/surgical induction of labour has increased dramatically in recent decades. Although these are valuable tools that can save mothers’ and newborns’ lives, they are also costly procedures with risks of medical complications. Reliable evidence to inform guidelines on optimal use of obstetric interventions is lacking for a number of clinical indications. This study aims to establish the impact of obstetric intervention in childbirth on maternal and newborn health and healthcare costs through the use of quasi-experimental study designs. We take advantage of natural experiments such as changes to hospital policies and clinical guidelines to establish the risks, benefits, and costs of obstetric interventions while avoiding the biases inherent in conventional observational studies.

Investigators: Sam Harper, Jennifer Hutcheon, Erin Strumpf

The Impacts of Cancer Screening AND ORGANIZED SCREENING PROGRAMS

While clinical trials have provided evidence regarding the effectiveness of certain cancer screening tests, less is known about the impacts of population-based screening recommendations or programs on clinical practice patterns and population health outcomes. We have examined the impacts of age-specific cancer screening guidelines for breast and colorectal cancer on screening rates. We then used these policy-driven changes in screening rates at particular ages to estimate the effect of screening on detection and stage-at-detection. We are currently investigating whether organized screening programs impact cancer outcomes (e.g., stage-at-detection, health services utilization) and equity in such outcomes across socioeconomic groups.

Investigators: Geetanjali Datta, Srikanth Kadiyala, Erin Strumpf

Funding: Canadian Institutes of Health Research