Instruments to measure fear of COVID-19: a diagnostic systematic review

Background The COVID-19 pandemic has become a source of fear across the world. Measuring the level or significance of fear in different populations may help identify populations and areas in need of public health and education campaigns. We were interested in diagnostic tests developed to assess or diagnose COVID-19-related fear or phobia. Methods We performed a systematic review of studies that examined instruments diagnosing or assessing fear or phobia of COVID-19 (PROSPERO registration: CRD42020197100). We utilized the Norwegian Institute of Public Health’s Live map of covid-19 evidence, a database of pre-screened and pre-categorized studies. The Live map of covid-19 evidence identified references published since 1 December 2019 in MEDLINE, Embase, and the Centers for Disease Control and Prevention. Following biweekly searches, two researchers independently categorized all studies according to topic (seven main topics, 52 subordinate topics), population (41 available groups), study design, and publication type. For this review, we assessed for eligibility all studies that had been categorized to the topic “Experiences and perceptions, consequences; social, political, economic aspects” as of 25 September 2020, in addition to hand-searching included studies’ reference lists. We meta-analyzed correlation coefficients of fear scores to the most common reference tests (self-reports of anxiety, depression, and stress), and reported additional concurrent validity to other reference tests such as specific phobias. We assessed study quality using the QUADAS-2 for the minority of studies that presented diagnostic accuracy statistics. Results We found 18 studies that validated fear instruments. Fifteen validated the Fear of COVID-19 scale (FCV-19S). We found no studies that proposed a diagnosis of fear of COVID-19 or a threshold of significant/clinical versus non-significant/subclinical fear. Study quality was low, with the most common potential biases related to sampling strategy and un-blinded data analysis. The FSV-19S total score correlated strongly with severe phobia (r = 0.703, 95%CI 0.634–0.761) in one study, and moderately with anxiety in a meta-analysis. Conclusions The accuracy of the FSV-19S needs to be measured further using fear-related reference instruments, and future studies need to provide cut-off scores and normative values. Further evaluation of the remaining three instruments is required. Supplementary Information The online version contains supplementary material available at 10.1186/s12874-021-01262-5.


Background
Fear is an emotional response to a threat as perceived by an individual, and considered a functional, adaptive, and transient response to stimuli, briefly resulting in physiological changes [1,2]. Fear can become pathological when physiological changes are chronic instead of transient, when fear reactions are triggered in the absence of actual danger, and/or when an individual is unable to learn safety signals, that is, to inhibit fear reactions by understanding safety cues (see [3,4]).
Infectious diseases are a particularly salient source of fear because they are transmissible, imminent, and invisible [2], and the COVID-19 pandemic has become a source of fear across the world. Schimmenti and colleagues [5] have suggested four dialectical elements of COVID-19 fear: fear of and for one's body, as one is both a potential vector and victim; fear of and for others, also related to the tension of prescribed social distancing from important interpersonal relationship; fear of ignorance of the virus as well as knowledge, as information is required for protection but can also be overwhelming and anxiety-inducing; and fear of both personal action and inaction, related to the behavioral consequences of fear. More than four out of five respondents reported one of the respective fears in two large surveys that included 1421 Japanese workers [6] and 669 dental practitioners worldwide [7]. In a recent systematic review of the mental health impact of COVID-19, healthcare workers reported that fear tied to their professional responsibilities extended into their personal lives [8]. Fear of getting sick with COVID-19 at work because they were unable to protect themselves extended to fear of infecting family members at home. This review found that fear correlated with greater exposure risk, supporting the general understanding of fear to be an appropriate response to an external threat.
Several surveys have also demonstrated that those more directly impacted by the pandemic or those at risk of being personally impacted are more afraid, as seen in people currently laid off or furloughed [9], and in people at high risk due to comorbidities [6,10]. Fear may have both positive consequences, such as greater adherence to infection prevention and control strategies, and negative consequences, such as avoiding health care services and settings. Harper et al. [11] recently reported a moderate positive correlation of fear to transmissionreducing behavior change such as hand-washing and social distancing (r = 0.31) in an international survey of 324 respondents. Karacin et al. [12] found that 14% of a Turkish center's oncology patients cited COVID-19 fear as the reason they delayed their chemotherapy (by an average of 47 days), despite no COVID-19 cases among patients or staff during the data collection period. Patients' and relatives' fear of infection was the overwhelming reason hypothesized by staff at 227 of China's 280 stroke hospitalsonly half of which treated COVID-19 patientsto explain nationwide reductions in hospital admissions for thrombolysis/thrombectomy [13].
Individuals' fear can amass into critical social and public health problems. Taylor and colleague's early questionnaires reported that fear of COVID-19 was highly correlated with stigmatization of both healthcare workers and foreigners [14,15]. These findings are an unfortunate reminder of Ebola outbreak research that pointed to fear-driven behavior with serious economic and social consequences, including stigmatization and discrimination of survivors [16][17][18]. Bali et al. [19] introduced the term "fearonomics" to describe these effects during Ebola. However, the threshold at which individuals' fear transforms into population-level problems is unclear: do a certain amount of people need to experience pathological or phobic fear, or a larger amount of people experiencing subclinical fear? Does the position of people in a society experiencing feare.g. a minority of policymakers versus a majority of votersinfluence when fears become problematic on a population level?
Measuring the level or significance of fear in different populations may help identify populations and areas in need of public health and education campaigns. Several validated instruments are available to serve as gold standard reference tests to diagnosis phobic and nonphobic fears related to illness, such as the Illness Attitudes Scales [20], the Fear Survey Schedule III [21], and the Perceived Vulnerability to Disease [22]. These gold standard reference tests and others have been used in the development of fear scales of Ebola [23] and swine flu [24].
Many studies measuring fear of COVID-19 have used unvalidated instruments, although the Fear of COVID-19 Scale [25] is increasingly used. It is imperative that fear of COVID-19 be measured appropriately. This systematic review examines scales used to assess or diagnosis fear of COVID-19.

Method
We conducted a systematic review according to our protocol registered in PROSPERO (CRD42020197100), and following Cochrane's guidelines for diagnostic accuracy tests.

Inclusion criteria
We included all studies that identified themselves as diagnostic accuracy or validation studies, and that measured the accuracy or validated the psychometric properties of any instrument measuring fear of COVID-19. Widening the selection of study designs beyond diagnostic accuracy studies was based on Umemneku Chikere and colleagues' recent systematic review of diagnostic accuracy tests conducted in the absence of a gold standard reference test, which found the use of validation studies to comprise one of four methodological alternatives [26]. We had no exclusion criteria related to population, intervention, comparator, outcome, or language; however, Chinese-language studies were not identified in our search strategy.

Literature search and article selection
We identified relevant studies by searching the Norwegian Institute of Public Health's (NIPH's) publicly available Live map of covid-19 evidence (https://www.fhi.no/ en/qk/systematic-reviews-hta/map/) on 25 September 2020. This map is one of several evidence maps and living evidence databases that attempt to enable faster identification of relevant publications with primary data. Not only has the speed with which covid-19 publications proceed through the peer-review and publication processes increased exponentially, even when compared to previous pandemics [27], the majority of publications on COVID-19 do not in fact present data [28]. Researchers and evidence synthesizers face significant challenges in identifying relevant publications, beginning simply by filtering out those lacking empirical data. Numerous initiatives have begun that use human effort, machine learning, or a combination, to identify relevant studies, categorize them according to intervention or population group, and in some cases extract and analyze data before a systematic review has been ordered, so that researchers will be positioned to rapidly produce high-quality evidence syntheses on a variety of topics as soon as policymakers request them. Some of the larger initiatives include the COVID-NMA project, focused on continuously updating analyses of randomized trials [29]; Epistemonikos' LOVE database, which categorizes studies according to "type of question", suitable for systematic reviews [30]; and the EPPI Centre's living map, containing 11 heterogenous and mutually exclusive categories such as vaccine development, genetics, and mental health impacts [31].
NIPH's Live map of covid-19 evidence is one of the most granular evidence maps, and all studies are categorized according to topic (seven main topics, 52 subordinate topics), population (41 available groups), study design, and publication type, by two researchers independently. The map's protocol describes the methodology in detail [32]. The search strategy of the Live map of covid-19 evidence has developed dynamically since March 2020, and is updated online [32]. The Live map of covid-19 evidence first searched for references on 12 March 2020, and identified references published since 1 December 2019 in MEDLINE, Embase, Centers for Disease Control and Prevention, and the Center for Evidence-Based Medicine. EPPI Centre at University College London has conducted the majority of searches (MEDLINE and Embase) and screening from May 2020 and onwards [31].
For this systematic review, we identified references categorized to the topic "Experiences and perceptions, consequences; social, political, economic aspects". One researcher screened all identified references specifically for the inclusion criteria for this systematic review, and read in full-text those containing the words fear, phobi*, diagnos*, accuracy, psychometric*, validation, or scale, in title/abstract. We also hand-searched included studies' reference lists. On the search day, the map's website stated that all relevant studies identified in database searches on or before 4 May 2020 were categorized, along with systematic reviews, health technology assessments, and randomized and non-randomized controlled trials up until 24 August 2020.

Data extraction and methodological quality assessment
We developed a data extraction form to collect data on country, target group and participants (age, gender, COVID-19 characteristics and socioeconomic characteristics as reported by study author), instrument description, methods of accuracy/validity assessment (such as reference tests), and outcomes. One researcher extracted data, and a second researcher checked extraction for accuracy.
We planned on using the QUADAS-2 [33] for diagnostic accuracy studies to assess methodological quality. Only three studies [34][35][36] tested a fear instrument against a dichotomous reference test and could therefore be assessed as diagnostic accuracy studies; the remainder measured convergent validity to continuous scores of other patient-reported outcomes. We assessed only these three studies using the QUADAS-2.

Data presentation and analyses
While we intended to conduct a summary ROC analysis to assess sensitivity and specificity of fear instruments, only one of the included studies produced an area under the curve score. In lieu of a summary ROC analysis, we reported each study's validation assessments using classical test theory or Rasch analysis, namely measures of internal consistency reliability, factor structure, and goodness-of-fit tests (Additional file 1). We metaanalyzed correlation coefficients between total fear instrument scores and other reference tests' total scores. We reported the correlation of fear to other constructs reported by at least one study, and calculated 95% confidence intervals. We conducted all analyses in STATA v.16 [37] using the metan command.

Description of studies
Eighteen studies were included, with a large geographic spread. Three studies took place in Turkey, two studies each in China, Peru, and Israel, and one each in Iran, Bangladesh, Italy, Saudi Arabia, Vietnam, Unites States, and Greece; one study sampled participants from both Belarus and Russia. All studies but one were online questionnaires that recruited via social media or used other convenience sampling methods. Most studies targeted the general population; four sampled among university students/graduates, one recruited participants working at public health departments, with no further description of sampling methods [45], and another    [41].
No studies were conducted specifically among healthcare personnel. Sample sizes ranged from 228 to 8550 participants (Table 1).

Instruments
The eighteen included studies validated four instruments. No study validated more than one instrument.

Fear of COVID-19 Scale (FCV-19S)
Fifteen studies validated the Fear of COVID-19 Scale (FCV-19S), including the original development study. This seven-item instrument was developed by Ahorsu et al. [25]. The authors first identified thirty general instruments on fear through a literature review, pooled and de-duplicated relevant items, solicited expert input to further reduce the amount of items, piloted the resulting ten-item instrument among 46 non-clinical participants for feasibility, administered it among a larger sample (N = 717), and finally, removed three items following classical test theory analysis. Respondents indicate their agreement with each question, as for example "I am afraid of losing my life because of coronavirus", on a 1-5 Likert-type scale from "strongly disagree" to "strongly agree". All answers are summed to produce a total score, from 7 to 35, with higher scores indicating more fear. While the original authors and eleven subsequent studies concluded the FCV-19S was unidimensional, Bitan et al. [40] found a two-factor fit of emotional fear and symptomatic expressions of fear. Huarcaya-Victorial et al. [44] found a bifactor model, in which all seven items load onto one general factor and onto one of two subordinate factors of emotional fear and symptomatic expressions, as in Bitan et al. Zolotov et al. reported weak fit indices for a one-factor and a two-factor model, but did not report the items comprising the two factors. With the exception of Zolotov et al., all studies utilizing classical test theory or Rasch analysis reported a stable factor structure, and adequate fit indices (see Additional file 1).
Most studies that used reference tests used validated and patient-reported screening measures of symptom severity. No studies used fear diagnoses as reference tests. Nguyen et al. [35] reported the area under the curve and 95% confidence intervals to predict clinically significant anxiety on the Generalized Anxiety Disorder Screener (scores ≥8). The area under the curve was 0.63 (0.60- Thirteen studies tested convergent/concurrent validity to reference tests with continuous scores as a validation method, with all reporting significant bivariate correlations between the FCV-19 total score (and each subscore, in the case of Bitan et al. [40]; not reported) and the various psychological constructs tested (Fig. 2).
Across nine studies, greater anxiety correlated with greater fear on the FSV-19S (r = 0.55, CI 0.46-0.64), as displayed in Fig. 2. Zolotov et al. [36] measured anxiety dichotomously, and found that the total FSV-10S score distinguished between self-reports of feeling more anxious as a result of COVID-19 in the past month, versus not. Figure 3 shows that greater depression also correlated with greater fear (r = 0.4, CI 0.34-0.46). The total score distinguished between participants who reported feeling more depressed as a result of the pandemic in the past month and those who did not report feeling more depressed, reported by Zolotov et al. [36]. Stress was measured in three studies (Fig. 4), and more stress correlated with more fear (r = 0.39, CI 0.28-0.50).
The Hospital Anxiety and Depression Scale total score, a measure of general distress, was correlated with the FSV-19S (r = 0.60, CI 0.51-0.69) (Fig. 5).
Each meta-analysis contained a very large amount of unexplained heterogeneity, a strong suggestion that the studies' populations, contexts, or methods were not similar enough to combine. It is therefore useful for readers to visually examine individual studies' results in each forest plot.

Fear perception and magnitude of the issue (MED-COVID-19) scale
The MED-COVID-19 was developed by a group of experts, piloted, and administered to approximately 400 public employees in Peru [45]. Only one validation study was identified for this instrument, which measured both the extent of fear and fear sources, such as media, Fig. 2 Correlation between FSV-19S and anxiety. Eight studies measured the correlation of fear scores to anxiety healthcare providers, and family/friends. Three factors were uncovered within twelve items through Rasch analysis. The first factor contained items relating to the perception that media sources were exaggerating COVID-19, the second factor related to magnitude of fear from media, and the third factor to both perception and magnitude of fear arising from healthcare providers, family, and friends. The validation study did not assess the MED-COVID-19 against a reference test, assess concurrent validity, or report subscale scores or total scores.

Scale of COVID-19 Related Psychological Distress in the healthy public (CORPD)
The Scale of COVID-19 [42] related psychological distress in the healthy public (CORPD), consists of two factors: fear/anxiety and suspicion. Both dimensions had

COVID-19 Phobia Scale (C19P-S)
The COVID-19 Phobia Scale [39] is based on a pool of 102 DSM-5 diagnostic criteria for phobia, adapted to COVID-19. Key criteria were excessive or unreasonable persistent fear, immediate anxiety provoked by exposure which may manifest as panic, recognition by the individual that the fear is excessive or unreasonable, avoidance of exposure, and anxiety that significantly interferes with normal routines if exposure cannot be avoided. A panel of physicians and psychometricians rated each item according to relevance; 70 items were identified and surveyed among 1250 participants. Exploratory factor analysis suggested retaining 20 items, and confirmatory factor analysis was conducted after a separate sample of 2143 participants answered these 20 items. The total score was a sum of four subscales: psychological, psychosomatic, economic, and social. The only reference test administered was COVID-19 infection, and the total C19P-S score as well as each factor score distinguished between infected and non-infected participants.  All but seven studies used anxiety as a reference test. Only two [25,51] used disease fear or phobia-specific reference tests, and it is noteworthy that the highest Table 2 Methodological quality assessment with QUADAS-2 correlation coefficient was between a self-reported measure of specific phobia (r = 0.703 CI 0.634-0.761) and fear (FSV-19S total score). The only study that reported a standard diagnostic accuracy metric used FSV-19S to discriminate among clinically significant vs nonsignificant self-reported anxiety scores, which was possible in only 63% of cases: a poor level of accuracy [53]. Accuracy may have been higher had a fear, not anxiety, reference test been used. It is likely that sampling procedures and data collection modesmainly convenience samples recruited through online social medianecessitated administration of simpler and non-diagnostic reference tests. The high amounts of unexplained heterogeneity in meta-analyses may be partially accounted for by the variety of reference tests. Heterogeneity may also be explained by an unstable factor structure. While twelve studies reported the FSV-19 to be unidimensional, one study found a two-factor fit and another a bi-factor fit; further research needs to explore dimensionality. As Schimmenti and colleagues [54] and Mertens and colleagues [55] have noted, fear may be more than pathological. Socioeconomic and interpersonal aspects of fear may be distinct factors or mediators of physiological manifestations of fear, and it is crucial that fear instruments are able to stably measure these dimensions.

Methodological quality assessment
The authors of included studies were able to quickly administer and collect data by choosing online convenience sampling, but therefore introduced potential sampling bias that reduced methodological quality in the three studies assessed with the QUADAS-2. The evidence so far supports the continued exploration of the FSV-19S based on a high correlation to severe phobia, and a moderate correlation to anxiety. To increase trustworthiness in these results, future studies should confirm this among non-selected samples, and should utilize fear-specific reference tests. The available evidence on the additional three instruments is not robust enough for recommending their use. The C19P-S was recently validated among an American sample in a forthcoming article (personal communication with I. Arpaci); depending on the findings, it may be a more appropriate instrument that uses the DSM-5-specific phobia criteria to measure COVID-19 fear.
While fear is understood to be adaptive and transient, and an appropriate response to a threat, Ebola research has emphasized the negative effects of disease-related fear on the community and international level [56]. Individual fear of infection and fear resulting from witnessing disease progression and death can turn into "a cyclical pattern of fear" on the community level (p.211), in which normal community interactions are disrupted because individuals loose trust in health services and reproduce stigma against infected individuals and survivors. In the current pandemic, discrimination and hate crimes against Asians have already been reported in North America [57] and Europe [58,59], a worrying reminder of the stigma and discrimination that Africans reported globally during the 2013/2014 Ebola outbreak [56]. Fear of COVID-19 is increasingly being reported in systematic reviews related to mental health as a distinct outcome [60] or an identified risk factor for mental health problems [61]. With methodologically strong instruments, fear of COVID-19 can be measured and populations with more fear identified to receive public education and public health campaigns.

Strengths and weaknesses
This review is the first that we know of to systematically search for and assess diagnostic accuracy or other validation studies of fear of COVID-19 instruments. Our quality assessment of studies should help other researchers in the evidence synthesis process, if they choose to use methodological quality in their inclusion criteria. An additional methodological strength is our utilization of the publicly available Live map of covid-19 evidence, one of the first reviews to do so (others include Muller at al. [8], as well as a diagnostic accuracy review that used the map as one of multiple sources [62]). By using this map, we quickly identified 394 studies that had already been categorized to our topic of interest, without having to search in academic databases and screen again. Although this timely approach has increased the availability of overarching research, it includes the risk of missing relevant studies missing if they were not grouped to the relevant categories. We did not use any of the methodological shortcuts recently reported in a survey of rapid diagnostic review study authors [63].

Conclusion
At least four instruments have been developed to measure fear of COVID-19, ten months after the pandemic began. Most studies are assessing the validity of instruments against other patient-reported mental health problems, rather than measuring diagnostic accuracy against fear-related reference tests. The Fear of COVID-19 scale (FCV-19S) instrument has already been translated to 13 languages, and our included studies reported convergent validity to phobia, disease-related fear, and anxiety. Future studies should assess diagnostic accuracy against validated fear-specific reference tests.