PROTOCOL: Criminal justice interventions for preventing terrorism and radicalisation: An evidence and gap map

Abstract This is the protocol for a Campbell systematic review. The objectives are as follows: to identify the existing evidence that considers the effectiveness of criminal justice interventions in preventing terrorism and radicalisation and to identify existing gaps in the evidence where new primary research could be undertaken and where future synthesis could be conducted.

1 | BACKGROUND 1.1 | The problem, condition or issue Violent radicalisation and terrorism continue to represent an ongoing threat to global security, with 44 countries experiencing at least one death from terrorism in the past year (Institute for Economics and Peace IEP, 2022). While definitions vary (Borum, 2015;Desmarais et al., 2017;Horgan, 2008;Kruglanski et al., 2019;Sarma, 2017) violent radicalisation is broadly defined as the process by 'which a person adopts extremist views and moves towards committing a violent act' (Hardy, 2018;p. 76; see also Irwin, 2015;Jensen et al., 2018). Once radicalised, individuals and groups may go on to carry out terrorist attacks against innocent civilians, key infrastructure, symbolic targets or specific groups (Wilner & Dubouloz, 2010) or travel to conflict zones to engage in violence with extremist groups (Lindekilde et al., 2016). Preventing violent radicalisation has thus become a key focus of counterterrorism agendas globally (Dudenhoefer, 2018;Elshimi, 2017;Heath-Kelly, 2013).
The harms associated with violent radicalisation and terrorism are far-reaching. Between 2009 and 2019, violent radicalisation and terrorism were listed as the cause of 21,000 deaths on average per year, with the majority occurring in the Middle East, Africa or South Asia (Ritchie et al., 2019). The estimated economic impact of terrorism is similarly staggering. According to Bardwell and Iqbal (2020); the cost of terrorism to the world economy between 2000 and 2018 was 855 billion USD. Beyond the human casualties and economic consequences, violent radicalisation and terrorism are linked to a broad range of psychological, social and political impacts.
Studies find experiences of terrorism can have negative mental health implications particularly among first responders and victims who face high levels of trauma and depression in the aftermath (Razik et al., 2013;Salguero et al., 2011). Violent radicalisation and terrorism can also antagonise public attitudes towards immigration and groups perceived as a terrorist threat (Williamson & Murphy, 2020).
Right-wing political parties frequently manipulate fear of terrorism and violent radicalisation to garner support for hard-line and discriminatory counterterrorism policies (Haner et al., 2019(Haner et al., , 2021 to educate others on how to identify and report radicalised individuals (Carter, 2014;Davis et al., 2016). Some police agencies also work with communities to build resilience to protect against the influence of extremist messages and recruiters (Cherney & Hartley, 2017;Mazerolle et al., 2020;Schanzer et al., 2016). Such initiatives focus on bolstering community engagement, trust and connectedness while reducing feelings of social isolation (Mazerolle et al., 2020).
The courts and correctional agencies also play a critical role in preventing terrorism and radicalisation. Court officials may direct radicalised individuals or individuals who have engaged in violent extremism and/or terrorist activity to various support services or mandate participation in rehabilitation programs designed to counter violent extremism . Prisons are widely believed to provide a prime opportunity for radicalisation to flourish, threatening the safety and security of both inmates and the community more broadly (van der Heide & Kearney, 2020). Training initiatives and awareness programs with corrective services staff may assist in the detection and prevention of both radicalisation and violent extremism (Vejvodová & Kolář, 2020). In an attempt to prevent the spread of radicalised ideologies behind bars, corrective services agencies often segregate radicalised individuals from the general prison population (Jones & Narag, 2018). Indeed, some jurisdictions operate specialised terrorist units to house extremist offenders (van der Heide & Kearney, 2020), with the aim to prevent radicalised inmates from influencing other prisoners and allow staff interacting with radicalised inmates to be upskilled in best practice approaches. Corrective services agencies may also design and deliver deradicalisation programs to radicalised inmates or inmates convicted of terror-related offences (Cherney & Belton, 2021). Upon release, community corrections staff may provide radicalised offenders with the support necessary for successful reintegration back into the community (Cherney, 2021).
Recognising the complexities surrounding radicalisation and terrorism, many criminal justice agencies are increasingly taking an intersectoral and multi-agency approach. Such approaches acknowledge the difficulties a single agency or organisation may face in trying to combat the problem in isolation . These initiatives include promoting information sharing between agencies (Knight, 2009), community outreach activities (Lamb, 2013;Schanzer et al., 2016), intelligence gathering (Lewandowski, 2012) and training programs (Davis et al., 2016;Kerry, 2007). For example, the World Organisation for Resource Development Education (WORDE) program which was led by a non-profit group of Muslim scholars and community leaders, partnered with several agencies including law enforcement. The program comprised three key components: (1) community education via town hall meetings; (2) the development of a referral network for highrisk individuals; and (3) participation in community activities (Williams et al., 2016). By partnering with other agencies and community groups, the intervention aimed to tackle the problem of radicalisation in a multifaceted manner (Beutel & Weinberger, 2016;Mazerolle et al., 2021).
Criminal justice interventions to prevent terrorism and violent radicalisation thus cover a broad range of policies and programs (Hardy, 2020). Prevention efforts can be divided into primary prevention (interventions targeted at a population), secondary prevention (interventions targeted at high-risk individuals) and tertiary prevention (interventions targeted at already radicalised individuals) (Harris-Hogan et al., 2015).
Criminal justice interventions which respond to terrorism/radicalisation are thus delivered across a variety of settings with participants ranging from community members to convicted terrorists (Gielen, 2019). As such, there is no singular theoretical framework underpinning criminal justice interventions for preventing terrorism and radicalisation. Prevention efforts by criminal justice agencies vary considerably in their underlying mechanisms and intended outcomes (Hardy, 2020). For instance, primary intervention programs may focus on building community capacity/ cohesion and feelings of inclusion (Mazerolle et al., 2020), secondary prevention programs may aim to reduce vulnerability or risk of radicalisation (Jones & Narag, 2018) and tertiary prevention programs may seek to deradicalise convicted extremists (Cherney & Belton, 2021).

| Why it is important to develop the EGM
Built using data collected via systematic search methods, EGMs provide a visual display of the available research evidence related to a particular topic area . EGMs are typically presented as a matrix comprising intervention categories (rows) and outcomes (columns). Interactive EGMs may offer additional filters which allow users to view the information based on certain classifications (e.g., study location, document type, study quality) . By locating and visually presenting all available evidence, EGMs improve stakeholder access to impact evaluations. While EGMs are a relatively new method of evidence synthesis, they are increasingly used to map evidence related to crime and justice issues (see e.g., Eggins et al., 2021;Pundir et al., 2020).
There are currently no existing EGMs that focus on criminal justice responses to radicalisation, violent extremism, and/or terrorism. Several systematic reviews have synthesised the effectiveness literature. However, these reviews have either had a broad scope including criminal justice agencies amongst other agencies (Lum et al., 2006) or a more targeted scope focusing on just a particular branch of the criminal justice system (see e.g., Mazerolle et al., 2021 for multi-agency responses to radicalisation with police as a partner).
Other reviews have focused on a particular type of program. For example, Mazerolle et al. (2020) reviewed police programs that seek to increase community connectedness for reducing violent extremism behaviour, attitudes and beliefs. Our systematic search will include all types of criminal justice interventions that prevent or respond to radicalisation, violent extremism and/or terrorism. It is expected that this EGM will help promote the translation of evidence into practice by providing practitioners and policymakers with easy access to evaluation evidence. This EGM will be used to demonstrate gaps in the evaluation research, allowing key stakeholders to identify future funding priorities for research and development. Specifically, if the EGM indicates a lack of research related to a particular intervention/outcome, this may provide grounds to justify future primary studies. Additionally, if the EGM illustrates several studies in one domain, this may lead to further syntheses such as meta-analysis.

| OBJECTIVES
This EGM aims to provide a comprehensive and systematic display of the existing evidence on criminal justice interventions which aim to prevent and reduce terrorism and radicalisation. It will map the extant evaluation evidence across the policing, courts, and corrections arms of the criminal justice system including multi-agency responses. The key objectives of this EGM are as follows: 1. To identify the existing evidence that considers the effectiveness of criminal justice interventions in preventing terrorism and radicalisation.
2. To identify existing gaps in the evidence where new primary research could be undertaken and where future synthesis could be conducted.

| Evidence and gap map: Definition and purpose
EGMs provide an interactive and graphical representation of the available evidence concerning a particular topic area (Snilstveit et al., 2016). EGMs are usually presented as a matrix with interventions displayed in rows and outcomes presented in columns . Like systematic reviews, EGMs utilise rigorous systematic search methods . However, EGMs are usually broader in scope than systematic reviews. EGMs are often viewed as 'public goods' as they improve acess to high-quality evidence for policymakers, practitioners and the wider public (Snilstveit et al., 2016). Our EGM will help to highlight both the density and paucity of existing research by including both primary studies and systematic reviews related to criminal justice interventions for preventing and responding to terrorism and radicalisation.

| Framework development and scope
The framework used in the EGM will be both theoretically and empirically informed. The initial framework to design inclusion and exclusion criteria and search strategies is based on an a priori schema, described below, and will be informed by the practical experiences and academic knowledge of the Advisory Group. Building on this framework, we will conduct a thematic cluster analysis of the included studies to develop an empirical classification schema for the final EGM categories and domains.

| Stakeholder engagement
Initial stakeholder consultation took place with the Department of Homeland Security in July 2021 to determine the outcome categories of interest and EGM scope. We will further consult a Virtual Advisory Group comprised of key experts in the field and government stakeholders. We have invited stakeholders from various government agencies across the United States, United Kingdom, Australia, New Zealand, Canada and Sweden to join our Advisory Group. We have additionally invited researchers from Spain, Israel, the United States and Australia. When finalised, our Advisory Group will provide a diverse range of perspectives regarding criminal justice responses to terrorism/radicalisation. Due to varied time zones, we will continue to contact Advisory Group members via email. As part of their role, members of the Advisory Group have been asked to provide feedback on search locations, recommend potentially eligible studies and help to locate hard to reach documents. Advisory Group members will additionally be offered the opportunity to review and provide feedback on the final draft report. A list of all represented agencies on the Advisory Group will be provided in the final report.  (Glass, 1997) • Long interrupted time series designs with or without a control (≥25 preintervention and postintervention observations (Glass, 1997).
In line with previous reviews (Mazerolle et al., 2020;Mazerolle et al., 2021) which recognise the significant shift in counterterrorism policies post 9/11 (Gaibulloev & Sandler, 2019), only studies that are published from or report on impact evaluations conducted between January 2002 and December 2021 will be included in the EGM. We will not include qualitative research as we are exclusively interested in mapping high quality impact evaluations. Should our searches identify any ongoing studies that meet our selection criteria but for which findings have not yet been published, we will include these as ongoing studies. These will be assessed for possible inclusion when the review is updated by the authors.

| Types of intervention/problem
Our EGM will capture criminal justice interventions that aim to prevent or respond to radicalisation, violent extremism, and/or terrorism.
A criminal justice intervention is defined as a strategy, technique, approach, activity, campaign, training, directive, funding, or organisational change that involves at least one criminal justice agency.
Criminal justice agencies include police, courts and corrections (both custodial and community based). Interventions that involve partnerships between at least one criminal justice agency and at least one external agency/organisation will also be included. Mazerolle et al., 2018). Examples of criminal justice interventions may include: • Criminal justice system initiation, development, or leadership of the intervention.
• Criminal justice system staff or populations as recipients of the intervention.
• Criminal justice system practices as the focus or target of the intervention.
• The criminal justice system delivers or implements the intervention.
Radicalisation is defined as the process of an individual or group adopting extreme political, social, or religious ideals (Hardy, 2018;Jensen et al., 2018). Analogous concepts are disengagement and/or deradicalisation, which are often encompassed within conceptualisations of violent extremism. Disengagement refers to reducing or ceasing physical involvement in violent or radical activities, while deradicalisation is defined as the psychological shift in attitudes or beliefs (Windisch et al., 2016).
Violent extremism is defined as 'advocating, engaging in, preparing, or otherwise supporting ideologically motivated or justified violence to further social, economic, and political objectives' (US Agency for International Development, 2016).
Terrorism is defined as 'the unlawful use of force or violence against persons or property to intimidate or coerce a government, the civilian population or any segment thereof, in furtherance of political or social objectives' (Shanahan, 2016, p. 108).
Eligible interventions may include but are not limited to: • Police training programs to improve officer recognition and responses to radicalisation and terrorism (Davis et al., 2021).
• Police working with communities to build resilience to protect against the influence of extremist messages and recruiters (Cherney & Hartley, 2017;Mazerolle et al., 2020;Schanzer et al., 2016).
• Court officials mandating participation in rehabilitation programs designed to counter violent extremism .
• Corrections agencies installing terrorist wings within prisons to separate radicalised prisoners or prisoners convicted of terrorist acts from the general prison population (van der Heide & Kearney, 2020).
• Training initiatives and awareness programs with corrective services staff focused on the detection and prevention of both radicalisation and violent extremism (Vejvodová & Kolář, 2020).
• Re-entry programs focused on building pro-social networks and access to support services to assist with prisoner release (Cherney, 2021).
• Multi-agency initiatives focused on delivering training to various stakeholders on how to identify and report radicalised individuals (Carter, 2014;Davis et al., 2016;Kerry, 2007).
Eligible comparison conditions/groups for inclusion in the EGM will include no treatment, placebo, 'business as usual,' waitlist control, or an alternative treatment. If a study reports on multiple interventions whereby only a subset meets the eligibility criteria, only the eligible interventions will be included in the map.

| Types of population
The following populations will be included in the EGM: • Criminal justice practitioners 3. Other: a. Should a pattern emerge in the outcomes initially categorised as 'other', we will group these together under an appropriate label (e.g., psychosocial, perceptions of police).
Unintended or adverse outcomes will be included.

Types of location/situation
We will place no limits on the geographical region reported in the study. No restrictions will be placed on the language a document is SYDES ET AL. | 5 of 16 written in. Similar to other reviews (e.g., Mazerolle et al., 2020;Davis et al., 2021), titles/abstracts written in a language other than English will be translated using Google Translate to determine if they are potentially eligible for the EGM. We will likewise translate portions of full-text documents in a language other than English using Google Translate to ascertain eligibility. If we cannot determine eligibility through this approach, we will contact study authors to determine eligibility.

Types of settings
No restrictions will be placed on the settings used in eligible studies. To capture studies for this review, we will use terms related to terrorism and radicalisation to search the GPD corpus of full-text documents that have been screened as reporting on a quantitative impact evaluation of a policing intervention. Specifically, we will use the following terms to search the title and abstract fields of the corpus of documents published between 1 January 2002 and 31

|
December 2021: extremis* OR 'far#left*' OR 'far#right*' OR 'foreign#fight*' OR 'freedom#fight*' OR guerrilla OR 'homeland#security' OR 'ideological violence*' OR 'ideologically#motivat*' OR indoctrinat* OR 'left#wing*' OR 'lone#wol*' OR militant* OR 'national#security' OR 'political violence*' OR 'politically#motivat*' OR radicali* OR rebel* OR 'religious violence*' OR 'religiously#motivat*' OR 'right#wing*' OR 'single#issue*' OR supremacis* OR terror* OR vigilante* OR vigilantism OR deradicali* OR 'de-radicali*' OR 'counter-terror*' OR counterterror* OR 'counter-extremis*' OR counterextremis* OR separatis* OR militia* OR jihad* 3.5.2 | Other database searches To capture interventions in the courts, corrections and multi-agency spheres not encapsulated within the GPD, we will conduct searches of key academic sources in criminology and criminal justice (see Table 1). We will search these databases on three sets of terms across title, abstract, keyword, and/or subject/indexing fields. These aim to capture literature on (1) terrorism within the (2) criminal justice system that uses a term indicative of a (3) evaluation of an intervention. Terms within each set will be combined with Boolean OR operators, and the three sets will then be combined with Boolean AND operators (see Supporting Information: Appendix 3 for an example search syntax). This search approach has been informed by Eggins et al. (2021) systematic review of criminal justice system responses to child sexual abuse material offending.
We will search the following across the title, abstract, keyword, and/or subject/indexing fields (adapting the search to the database if fields are unavailable): OR 'right#wing*' OR 'single#issue*' OR supremacis* OR terror* OR vigilante* OR vigilantism OR deradicali* OR 'de-radicali*' OR 'counter-terror*' OR counterterror* OR 'counter-extremis*' OR counterextremis* OR separatis* OR militia* OR jihad* AND (2) accused OR acquit* OR adjourn* OR adjudicat* OR *admiss* OR affida* OR appeal* OR appellate OR apprehend* OR arbitrat* OR arraign* OR *arrest* OR attorney* OR authorit* OR bail* OR barrister* OR breach* OR 'case#manage*' OR caution* OR charge* OR clerk* OR confinement* OR convict* OR coroner* OR correction* OR court* OR crime* OR criminal* OR 'cross#examin*' OR custod* OR defendant* OR defense OR defence OR detain* OR detention* OR deter* OR divert* OR diversion* OR enforc* OR execut* OR felon* OR forensic* OR gaol* OR guilt* OR 'high#security' OR 'halfway#house' OR *imprison* OR incarcerat* OR indict* OR infract* OR infring* OR injunct* OR inquest* OR innocen* OR inmate* OR juris* OR jail* OR judge* OR judic* OR juror* OR juries OR jury OR justice OR law* OR legal* OR legislat* OR litigat* OR 'low#security' OR magistrate* OR mandat* OR mitigat* OR marshal* OR misdem* OR 'medium#security*' OR offend* OR offence* OR officer* OR official* OR ordinance OR parole* OR pardon* OR penal* OR plea* OR precedent* OR prevent* OR prison* OR probat* OR prohibit* OR prosecut* OR punish*OR recividis* OR rehab* OR reintegrat* OR remand* OR reoffend* OR 're-offend*' OR ruling* OR sanction* OR sentenc* OR solicitor* OR statut* OR subpoena* OR supervis* OR surveil* OR suspect* OR testif*

| Searching other resources
In addition to the database searches, we will use the following search strategies: • Searching government and non-government websites related to terrorism and radicalisation research (see Table 1) • Searching government and non-government websites related to courts and corrections for research related to terrorism and radicalisation interventions (see Table 1) • Searching journals related to terrorism and radicalisation (see Table 1) 3.6 | Analysis and presentation 3.6.1 | Report structure The EGM will comprise the following sections: (1) executive summary; (2) background; (3) objectives (4) methods (5) results and (6) discussion. The executive summary will provide an overview of the EGM findings, detailing the number of studies captured in the review across the criminal justice system (police, courts, corrections and multi-agency). The background section will detail the problem of radicalisation, violent extremism and terrorism, outlining why and how the criminal justice system provides a prime opportunity for intervention to reduce and prevent its occurrence.
The methods will provide information on the systematic search, including key search terms and search locations. The methods will also describe the screening and data extraction processes and software programs utilised. The results will provide an overview of the number of eligible studies captured by the systematic search, detailed in a PRISMA figure. Our results will also include our interactive EGM, providing a graphical representation of the available evidence. The discussion will reflect on both the paucity and density of research across different intervention and outcome domains. We will recommend areas for future research and synthesis, and implications for policy and practice.
Planned Tables and Figures: • Figure: PRISMA • Table: Search locations • Table: EGM-primary studies (non-interactive) • Table: EGM-systematic reviews (non-interactive) • Table: online interactive EGM-with filters 3.6.2 | Filters for presentation The results of our online interactive EGM will be presented as a matrix of rows (comprising two layers: (1) intervention agency; (2) intervention type) and columns (outcomes). We will additionally employ the following filters: • In maps that include systematic reviews, we will visually differentiate between systematic reviews that have included studies and those that do not by the use of colour coding .

| Dependency
The unit of analysis is the primary study, not the publication or document that reports on the study. We are conscious of the risk of using documents as the unit of analysis, as a visual tool such as an EGM can appear to indicate a preponderance of evidence on an intervention where there are multiple publications drawn from the same research study.
The two most common ways in which dependency can manifest in an EGM are as many: one dependency or as one: many dependency. Firstly, there may be more than one document reporting on the same study (many: one dependency). In this situation, all documents that report on the one study will be linked, and the study is what will appear on the map. Secondly, one document may report on more than one study (one: many dependency). If a document reports on more than one study, we will link that document to both 'parent' studies.
Systematic reviews provide a unique challenge in terms of the 2. We will include systematic reviews in addition to their component studies, but systematic reviews will not be visible at the same time as primary studies. We will do this using a strict toggle filter for Study Type, with mutually exclusive options of research synthesis or research studies. This will allow the user to choose between an EGM of primary studies, and an EGM of systematic reviews, to not visually inflate the weight of evidence.
A further form of dependency is where one study compares two eligible treatments head-to-head. In these cases, the study itself may be eligible to appear in two different columns of the map. In addition to linking any related documents, we will flag studies of this design so that it is clear where there is the potential for double-counting.
Similarly, if a systematic review is eligible to appear in more than one cell on the map, we will flag the review to ensure that the reader is aware of the potential for double-counting.

| Screening and study selection
Our screening approach will differ slightly for the different sources, but all documents will be assessed on the same final eligibility criteria.
Documents from the GPD have already been screened on their fulltext as reporting on a quantitative impact evaluation of a policing intervention. Therefore, we will assess the full-text of these documents for their eligibility to the review. Documents from other sources have not been verified as quantitative impact evaluations, so we plan to screen these on their title and abstract to remove documents not related to criminal justice responses to terrorism/ radicalisation, and then screen eligible full-text documents on the same criteria as for the GPD data.
For all stages described below, we will develop standardised screening companions and inter-rater reliability tests to ensure consistency in decision-making across the research team. For title and abstract screening, each team member will screen the same set of 30 records before commencing screening. Similarly, at the full-text screening stages, before commencing screening, team members will screen the same set of 30 documents. All team member decisions will be compiled and compared to determine any divergence in decisions.
Feedback on screening decisions will be provided with further training when necessary. We will double screen 5% of each screener's excluded documents to identify any potential falsenegative decisions. In instances where a screener's decisions are considered unreliable, their exclusion screenings will be reassigned.
Where disagreements about screening decisions occur, these will be mediated by a third team member.
All documents will be screened in DistillerSR-a web based, reference management software that incorporates artificial intelligence (AI) to expedite the review process (Evidence Partners, 2022).
DistillerSR has an inbuilt AI function which cross-checks exclusion decisions. This helps to ensure consistency across reviewers and reduce human error. DistillerSR also draws on machine learning to reorder references based on their likelihood of inclusion, learning from the decisions made by human users (Evidence Partners, 2022).
Using this function, we will screen documents up until the point that DistillerSR indicates 100% of all potentially eligible records are included. At this point, we will screen a random sample of 30 additional records. We will repeat this process until no eligible records are identified. We will use the AI function for Title and Abstract screening only.
Following Campbell Collaboration guidance ), we will screen systematic reviews on their PICOS rather than on their included studies. In this way, we will retain 'empty reviews', and allow the EGM user to see where synthesis has been attempted but was unsuccessful.

GPD sourced documents
First, we will import all documents reporting on a quantitative impact evaluation of a policing intervention captured by the systematic search into DistillerSR. Quantitative impact evaluations indexed in the GPD generally have a pre-existing full-text document attached, and document retrieval is therefore not required. We will then screen the full-text of the records according to the following exclusion criteria: 1. Document is not unique 2. Document is not about a criminal justice intervention for preventing/responding to terrorism/radicalisation.

Document does not include an impact evaluation of a criminal
justice intervention that aims to prevent or respond to terrorism/ radicalisation.
While all efforts are made to remove duplicate records within the GPD, exclusion criterion 1 will be used to remove any that were missed during the initial data cleaning. Exclusion criterion 2 will be used to remove records unrelated to terrorism, radicalisation or extremism. Exclusion criterion 3 will be used to produce a corpus of studies that report on quantitative impact evaluations of an eligible intervention, using an eligible research design. Before screening, we will make all efforts to remove duplicates and ineligible document types (such as book reviews, blog posts etc).
However, criteria 1 and 2 will allow us to remove any that were missed during data cleaning. Exclusion criterion 3 will be used to remove titles and abstracts unrelated to terrorism, radicalisation or extremism.
Records that successfully progress through the title and abstract screening stage will move to a literature retrieval stage. For records without documents, we will endeavour to locate the full-text document via existing university resources. In instances where we cannot locate the record, we will order the document through the university library or by directly contacting study authors. Documents for which we cannot locate the full-text and cannot unequivocally exclude on title and abstract will be listed in 'Studies awaiting classification'.
Once the literature is retrieved, all potentially eligible records will then progress to full-text eligibility screening. The full-text of the records according to the following exclusion criteria: 1. Document is not unique 2. Ineligible document type 3. Document does not report on quantitative bivariate or multivariate data which may be indicative of an evaluation 4. Document is not about a criminal justice intervention for preventing/responding to terrorism/radicalisation.

Document does not include an impact evaluation of a criminal
justice intervention that aims to prevent or respond to terrorism/ radicalisation.
The addition of criterion 3 allows us to refine the corpus of studies to those which do not contain bivariate or multivariate data which may be indicative of an impact evaluation. A similar approach is taken for the GPD (Higginson et al., 2015). Criteria 4 and 5 will be used to remove records unrelated to terrorism, radicalisation or extremism and produce a corpus of studies that report on quantitative impact evaluations of an eligible intervention, using an eligible research design.

| Data extraction and management
Once deemed eligible for the review, documents will be coded in EPPI-reviewer. We will use a standardised coding companion (see All coders will code the same four eligible studies to ensure consistency across team members. These will be compiled and compared to determine any divergence in coding, and feedback will be provided before independent coding. To check for inter-rater reliability, 5% of each person's coding will be double-coded by a second person. Where disagreements arise, a third review author will be consulted. Study authors will be contacted directly if documents are missing any pertinent information.

| Tools for assessing risk of bias/study quality of included studies
Each eligible systematic review in the EGM will be assessed for Risk of Bias using the AMSTAR 2 critical appraisal tool (Shea et al., 2017).
Based on ratings across 16 items, the tool generates a rating of overall confidence in the systematic review's results: High (no more than one non-critical weakness, but no critical flaws); Moderate (more than one non-critical weakness, but no critical flaws); Low (one critical flaw, with or without non-critical weaknesses); or Critically Low (more than one critical flaw, with or without non-critical weaknesses). Two coders will independently code each systematic review using the AMSTAR 2 tool. Any discrepancies between the two coders will be resolved by discussion.
Due to the anticipated size of the EGM, we do not plan to assess confidence in the included primary studies, but note that our methodological thresholds exclude study designs with the highest inherent risk of bias.

| Methods for mapping
We will use EPPI-reviewer to create the EGM (Eppi-Centre, 2019).