Gregory and colleagues examined the effectiveness of RJP policies and implementation in minimizing the suspension gap between White students and students of color. Researchers have utilized multi-level modeling of existing school discipline records to examine differences in suspensions and expulsions between schools implementing RJP programs and those utilizing punitive measures. Others have taken a population-level approach, assessing knowledge and awareness of RJPs through secondary data available via the California Healthy Kids Survey . Darling-Hammond and colleagues utilized CHKS data to examine student experiences with exposure to RJP programs, not necessarily impacts at the school-level. Data from these types of studies provide insight into perceptions and potential effects on student behaviors and knowledge of RJPs. Researchers have also utilized interrupted time series methods using administrative data to examine trends in suspension rates prior to and after implementation of RJP program components. Focusing on a large, urban school district, Hashim and colleagues analyzed aggregate suspension data to assess potential associations with RJP program implementation. In their considerations, trimming cannabis the authors noted RJP evaluations have focused on larger school districts in urban settings, thus highlighting the need for more studies on the effectiveness of RJPs in smaller, rural school systems.
Qualitative data collection and mixed method study designs have also been employed to measure impacts of RJP programs and to gather data on stakeholder experiences. These methods have mainly been implemented with the aim of filling the gap in the literature on stakeholder perceptions and experiences with RJP programs at the school-level. Gathering stakeholder feedback, including student perceptions and experiences, has allowed researchers to further examine recommendations for improving the structure, development, and implementation of RJP programs. Researchers have highlighted the importance of qualitative interviews in documenting staff and student perceptions of RJP approaches and the way they are implemented in real-world settings. Qualitative interviews have also been found to be useful in measuring staff and student perceptions of cultural shifts that often arise with the implementation of RJPs. Furthermore, researchers have also been able to determine which staff are generally more involved in specific RJP components via staff interviews which in turn inform recommendations for ideal staff roles to lead RJP development and implementation. For instance, Sedillo-Hamman and colleagues highlighted the important role that school counselors and social workers might play in ensuring implementation fidelity and access to RJP resources among students. Limitations with these RJP program evaluation methods include the use of aggregate discipline data to assess impacts on suspension and expulsion rates. It is difficult to examine individual-level impacts on behavioral outcomes, youth development, and program perceptions; although, qualitative data have been found to adequately inform these dimensions.
Additionally, research that isolates single RJP components for evaluationfails to consider other crucial aspects of an RJP program such as relationship restoration, apology letters, counseling/social-emotional support, and other resources that may be provided to students. A limited number of previous studies have assessed implementation fidelity of RJP components in school settings. This leaves a gap in the literature on the importance of assessing process measures to examine how well programs are being implemented as intended. Understanding an educational agency’s initial scope of work and any goals outlined in RJP program proposals are crucial for a comprehensive evaluation.Current RJP program evaluation research on changes in participant behavioral outcomes, disciplinary incident rates, and school climate/culture is limited. Although a significant shift to RJPs has been observed in school systems throughout the United States, current RJP evaluative research is not meeting the rate at which these programs are being implemented. Nonetheless, most of the current studies have reported positive impacts of RJP programs on student behavior, academic achievement, school connectedness, and school climate. More specifically, researchers have noted significant improvements in student behavioral outcomes and attitudes, academic achievement, and school connectedness. Students at schools that implement RJP programs have also noted improved student-teacher relationships. The most significant impacts have been seen at the individual versus school levels. RJP programs have also been found to be associated with significant improvements in behavioral outcomes and academic achievement for Black and Latino students.
However, current studies did not find an impact on the discipline gap in suspensions or expulsions between White students and students of color. Nonetheless, students who were exposed to RJP programs and their components had lower odds of being re-suspended after going through RJP interventions compared to those at non-RJP schools. Within the context of qualitative RJP evaluations, there are limited findings assessing student, staff, and educator perceptions of RJP programs. One qualitative study found that students had fewer positive comments on RJP learning outcomes in comparison to the communication skills and social-emotional learning components. Furthermore, students have also highlighted their preference for “meaningful consequences” seen within RJP programs as opposed to traditional punitive measures. The tailored consequences and reintegration of the offender were two main components that were positively viewed by students. Nonetheless, more research on student experiences and perceptions of RJPs is needed to better understand individual perspectives and internal behavioral impacts. Collection of this data could help to improve current RJP program development, structure, and implementation strategies. Staff and educators on the other hand tend to recognize the importance and positive impacts of RJP programs in school settings. When asked about RJP programs within the context of discipline, most educators and school staff understand the potentially positive impacts these programs can have on the suspension gap. Additionally, educators have also voiced their concerns with current punitive and zero-tolerance policies in schools, implying a preference for RJP approaches. Despite preference and support for RJPs, there are challenges with the actual implementation of said programs. For instance, staff buy-in and competing priorities were highlighted as barriers to successful RJP program implementation. Public school systems are often understaffed and under-resourced, leading to high staff burnout and poor support for any nonacademic programming that may further cut into crucial class time. Therefore, researchers highlight the importance of staff trainings and having several staff engaged and involved in the development and implementation of RJP programs, fostering a culture shift from “within” the staff community. Nonetheless, it is crucial that staff are given the tools and training they need to implement RJP programs efficiently and effectively. For instance, trainings in which different RJP components are modeled may increase self-efficacy among staff to implement these activities on their own. Overall, current RJP program evaluation research has found this approach to be a positive alternative to punitive, zero-tolerance disciplinary approaches. Both experimental and observational studies reported that RJPs may improve discipline rates and decrease the gap between White students and students of color. Qualitative findings have also shown that staff and students generally have positive perceptions of RJP programs; however, more research is needed to assess the individual-level impacts of such approaches on program enrollees.Located in the western Sierra Nevada foothills, Nevada County consists of two large towns, Nevada City and Grass Valley, surrounded by smaller rural communities hosting a total population of approximately 100,000 residents. Its location in Northeast California places the county in a region known for prevalent cannabis cultivation. When voters passed “Proposition 64: The Adult Use of Marijuana Act” in 2016, it provided a legal avenue for the integration of the marijuana industry in communities throughout Nevada County. However, despite a state-led effort to regulate cultivation and distribution, gardening rack there still exists a low-cost, black market driving higher use rates among Nevada youth. Closer examination of overall use shows that Nevada youth report higher substance use rates when compared to state averages. Data from the 2019-2021 California Healthy Kids Survey dashboard shows that among 7th, 9th, and 11th graders in Nevada County, 5%, 25% and 32%, respectively, reported AOD use in the past 30 days.34 In comparison, 7%, 15%, and 23% of 7th, 9th, and 11th graders, respectively, reported past 30-day AOD use at the state level.
Looking specifically at marijuana, 3% of 7th graders, 15% of 9th graders and 20% of 11th graders reported using marijuana at least once in the last 30 days. At the state level, 4% of 7th graders, 10% of 9th graders, and 16% of 11th graders reported past 30-day marijuana use. Current vape or ecigarette use was higher among Nevada County high schoolers with 3% of 7th graders, 18% of 9th graders, and 18% of 11th graders reporting using a vape in comparison to 4% of 7th graders, 9% of 9th graders, and 11% of 11th graders at the state level. To address the disproportionate use rates among the local student population, the Nevada County Superintendent of Schools partnered with the county’s public health department and Tobacco Use Prevention Education office to develop and launch the Restorative Accountable Youth Solutions program. NCSOS coordinators were granted funds through the Bureau of State Community Corrections Prop 64 Public Health & Safety Grant program, which aims to address community-level impacts of the passage of Proposition 64. Training and outreach were conducted from spring through summer of 2021 followed by a formal launch in August of 2021. The RAYS program has thus far been established at four public school sites in Nevada County – one middle school, two comprehensive high schools, and one continuation high school. The program’s target population includes 6-12th graders currently enrolled at one of these four target sites who have committed a suspendable offense.The RAYS program is rooted in RJPs with the main aim being to provide an alternative to suspension, non-punitive option for students who commit a suspendable infraction at one of the four target schools. The three main goals outlined in the NCSOS Local Evaluation Plan are as follows: to reduce suspension rates at the four target sites, reduce marijuana and other substance use among youth, and increase student access to drug treatment services and counseling . A more comprehensive description of RAYS program goals and objectives is provided in Table 11 . While RAYS program components are tailored to address drug-related incidents, services are also provided for students who have committed non-drug-related offenses . Critical components of the program include a 1-hour peer-led restorative circle, a 2-to-3-hour alcohol and drug safety skills class , community engagement activities, harm reparation, individual or group counseling sessions, and youth advocate engagement. Offending students who are referred for disciplinary action first meet with a site administrator. The administrator reviews the student’s potential options, at which point the student may voluntarily elect either traditional suspension or to enroll in the RAYS program. Should the student select RAYS, they are referred to the program’s Restorative Practices Coordinator, at which point they formally enroll in the program and schedule their restorative circle. During the restorative circle, students collaborate with a team of their peers to develop a Restorative Plan – a contract outlining the various activities that the offending student must complete in order to successfully exit the program. The activities outlined in a student’s Restorative Plan are dependent on the offense and what the peer team believes would be most beneficial for the offending student. Figure 2 provides a visualization of the RAYS enrollment process and critical program components. Core components of the RAYS program and respective definitions are listed below. The current study is an intermediate evaluation of Nevada County’s RAYS program with the aim of assessing the overall impact on school disciplinary incidents, student AOD use behaviors, perceptions of substances and use, self-responsibility, and awareness of school substance use and mental health resources. This project is intermediate in that it seeks to inform larger evaluation efforts outlined in a contract agreement between Professor Shu-Hong Zhu’s research team at the University of California, San Diego and the NCSOS RAYS team. Furthermore, this evaluation may inform limited research of RJP program evaluation with a small, rural school district in a region with a prevalent cannabis cultivation industry. Findings from this project may help to fill the gap in the literature on restorative program evaluation for drug-related incidents in school-based settings. This may inform efforts for developing, implementing, and evaluating public school-based restorative programs with a substance use focus. Data from this project may also provide the grantee with crucial intermediate findings to inform the continuous development and optimization of RAYS program components and implementation strategies.