1. Introduction

Outbreaks of diseases have always been a part of human existence. However, the consequences of the COVID-19 pandemic are largely unprecedented because of the global impact. The risks associated with novel infectious diseases such as Ebola or HIV have typically been downplayed – or even considered negligible - by individuals from Western countries [1, 2]. This is likely because pandemics are often circumscribed to specific, non-Western regions of the world (e.g., Africa, Asia), or associated with marginal groups of society (e.g., gay men). In either case, the perceived risks and associated levels of threat are insignificant for large portions of the global population (often, western industrialised countries). This is not true for COVID-19 – in fact, 6 months into the pandemic (September, 2020), the top 10 most affected countries varied widely in terms of cultural, economic, and linguistic composition, and included several Western and non-Western countries – including India, the United States, Brazil, Argentina, Colombia, Spain, France, Peru and Russia [3]. In December 2019, a novel and unknown strain of coronavirus - COVID-19 - began to spread across the globe and was declared a pandemic on the 11th of March 2020 by the World Health Organisation (WHO, 2020). By July 2021, WHO had received reports of over 190,000,000 confirmed cases of COVID-19, including over 4,000,000 people who have died from the virus [3]. The pandemic poses an unparalleled threat for individuals, groups, and societies, with consequences that are largely unprecedented [4].

On a global economic level, traditional goods and service supply chains have been significantly disrupted, impacting distribution around the world [5]. In addition, an increasing number of businesses worldwide have shut-down their operations, revised their strategies or business models, and/or enforced layoffs amongst staff [6]. COVID-19 has also sparked fears of long-term economic hardship, with recessions predicted across many countries with further potential for economic depression [7]. A down-turn of business has led to widespread increases in unemployment, leading to difficulties in paying rent, foreclosures on mortgages and even challenges in affording more fundamental necessities like food [8]. The spread of COVID-19 has also resulted in significant social change with governments worldwide both encouraging and enforcing individuals to isolate and distance themselves from family and friends. For example, in the year 2020, there were periods of time in which at least one third of the world’s population were under some form of isolation regulations [9]. These containment measures have effectively severed traditional social ties leading to numerous negative psychological outcomes including loss of motivation, meaning, and decreased self-worth [10]. The uncertainty of the situation, followed by actions taken to reduce the physical risks, has had serious consequences on people’s mental health. Indeed, results from studies around the world have unanimously shown that people’s mental health has been negatively impacted [11, 12]. Despite advances in public health responses to the spread of the virus, and the rapid development of a vaccine, the pandemic continues to spread in many parts of the world. Additionally, newer and more transmissible strains of the virus continue to emerge - it remains unclear how effective vaccines will prove to be in the long run, and thus COVID-19 does and will continue to impact the lives of people around the world. In this paper, we present the development and initial evidence for the COVID-19 Multifaceted Threat Scale – a new measure for use in understanding COVID-19 specific threats as experienced by people in different national and cultural contexts.

Lessons Learnt? Threats previously observed during the spread of Infectious Diseases

With the heightened mass panic surrounding the COVID-19 pandemic [13], researchers and community members alike are attempting to find ways to reduce and prevent the spread of the virus itself, but also to respond to the impacts associated with the threats arising from the pandemic. By reviewing prior knowledge on disease outbreaks and natural disasters, [4] expanded the definition of threat to include all “external or internal stressors that are appraised as a potential danger to physical or psychological goals relevant to the personal or social self, and to group cohesion and survival” (p2). In is worth noting the overview of the most prevalent stressors triggered by past infectious diseases provided by [14]. They reported that threat perceptions can be clustered into four broad dimensions: A first category comprises personal wellbeing concerns, including fears of getting infected, passing an infection, or suffering of psychological distress [15, 16]. A second category describes social concerns in a broader sense, such as interrupting activities, disrupting social networks, and experienced stigmatization [17, 18]. A third category includes material and economical concerns, for instance disrupted supply chains, and financial issues. Finally, a fourth category of institutional concerns relates to threats emerging from distrust with health authorities and media [19, 20].

Given the wide range of impacts of COVID-19 threats, it is necessary to accurately measure COVID-specific threats among the general population. Some scholarly attempts to measure levels of threat associated with the pandemic have been made, but these assessments tend to a focus on physical and mental health, and most of them lack ecological validity and descriptions of the scale’s development (e.g., the 6-item Perceived Coronavirus Threat Questionnaire [21]; the Questionnaire on Perception of Threat from COVID-19 [Pérez-Fuentes et al., 2020 ]; the 7-item Fear of COVID-19 Scale [23]).

These measures are brief, and have been well-received by researchers working on understanding the psychology of COVID-19. However, there is a need to systematize current knowledge and establish an ecologically valid understanding of the threats associated with the spread of the COVID-19 pandemic. Notably, existing measures of COVID-19 threat were exclusively self-focused. The COVID-19 outbreak on the other hand has escalated to a collective threat that concerns communities and societies, and even the broadest collective of humankind (for other collective threats, see [24, 25]). Along the same lines, some sparse attempts have been made to incorporate collective aspects into measures assessing threat appraisals (e.g., the 36-item COVID Stress Scale [26], the 10-item Integrated COVID-19 Threat scale [27]; the 20-item COVID-19 Phobia Scale Scale [28]). The current study was designed to fill this gap.

Research Overview

To best understand the variety of threats that are induced by the pandemic, and to develop a measure that accurately assesses the full range of threat dimensions, we employed a rigorous scale validation approach, innovatively mixing qualitative bottom-up approaches to item generation with quantitative approaches based in classical test theory to test the scale’s factor structure and provide evidence for its validity and reliability across countries that are culturally disparate (and differentially impacted by COVID-19). Across three studies, in this paper we present the initial evidence for such a measure. First, we present a cross-cultural, inductive qualitative approach to understanding the multi-dimensionality of COVID-19 threat across a diverse range of 11 countries (Study 1). A thematic analysis of this data identified 10 dimensions of threat. We then developed a pool of 100 items that could plausibly assess these dimensions. Data from participants in the United States were subjected to an exploratory factor analysis to reduce the number of items (Study 2). A reduced pool of 30 items was then selected based on the psychometric properties of the longer scale and cross-cultural validation was finally achieved in the United States and India using multi-group confirmatory factor analysis (Study 3). We also present evidence for several forms of validity and estimates for the internal consistency of our measure.

2. Study 1: Qualitative approach to item construction search design

In order to acquire an ecologically valid understanding of the threats associated with the spread of the COVID-19 pandemic, we administered a free association task across eleven different national samples (i.e., Australia, Germany, Italy, Poland, USA, UK, India, Malaysia, Peru, Colombia, South Korea). The free association task has previously been used to gain insights into how lay publics engage with unfamiliar issues, such as climate change, earthquakes, and emerging communicative diseases [2]. This method has the advantage of allowing participants to freely generate responses about COVID-19-relevant threats, without being constrained by pre-imposed taxonomies decided upon by the research team. Given the exploratory nature of this study, no predictions were made for this Study.

2.1 Method

Participants: Our online survey was responded to by 194 participants (59.6% female, 39.7% male, 0.7% unknown; M_age = 38.3 years, SD = 14.3) in April 2020, one months after the pandemic was declared. Participants were volunteers recruited on social media. The sample size from each country ranged from 12 (Germany) to 23 (India). Most participants were working full-time (42.3%), part-time (17.5%), or were students (16.1%). A series of χ2 tests of independence revealed that gender, age, and employment status were balanced across the eight different countries (ps > .091). A summary of the participant demographic information is presented in Table 1.

Table 1: Demographic distribution of participants across countries.

Analytical Strategy: In line with previous research using free association methods, participants were presented with a grid containing ten empty boxes and requested to write down the main concerns, worries, or fears they had when thinking about the COVID-19 pandemic. A total of 1,728 COVID-19 related threats (M = 8.92) were collected and synthesized guided by the premise of thematic analysis, which is a method for identifying and clustering prevalent patterns of meaning in a dataset [29]. After a first stage of familiarization with the data, threats were clustered based on inductive codes, grounded in the content of the data, and then refined in light of the existing literature. The data were coded by two people, with near-perfect inter-coder accuracy between the two independent coders (κ = .847[.01], p < .001; [30]. Conflicting codes were discussed and resolved through deliberation and mutual agreement.

Rigour: Credibility, dependability, and transferability of data need to be established in qualitative research [31]. Credibility (i.e., the confidence that the findings will be truthful) was established in the current study by using a well-used method to elicit free association responses. Further, and in line with recommendations by [29] we did not set any a priori data saturation criterion (for a discussion on saturation, see [32]). Dependability (i.e., confidence that the findings will be consistent and reproduceable) was achieved in the current study in several ways: (a) online administration ensured consistency across all countries, (b) back-translation was used for materials in all languages, (c) a large corpus of responses was included in analysis (with a low non-classification rate [96.6% of responses were coded into the themes]), and (d) throughout analysis there was constant comparison and refinement of the coding frame was used, which helped ensure consistency of categorization. Transferability (i.e., confidence that the methods and findings could be generalised to other contexts, or settings with different participants) was achieved by providing thorough explanations of how data were collected and analysed, and the inclusion of online supplementary materials/data increased the transferability of findings.

2.2 Results:

Among the total 1,728 threats collected, 66 (3.8%) did not fit in the coding frame. The remaining items were clustered into 10 dimensions. The observed percentages for each category as a function of the participant’s country of residency are presented in Table 2. The raw data, including full participant demographic data, and the coding details, along with supplementary analyses (differences in threat perceptions as a function of country and other socio-demographic variables) are available online at

https://osf.io/dx2eg/.

Table 2: Percentages of Reported Personal and Collective Threats Across 11 Countries (n_total = 137)

Note: These categories have been post hoc re-arranged to reflect the order in which the factors emerged in the exploratory factor analysis EFA in Study 2 (i.e., in decreasing order of factor loading strength). Online supplementary Table S1 contains the information about the data that were not clustered into these dimensions

Health: Unsurprisingly, the largest cluster of threats pertained to the health of participants or those close to them as a major threat, accounting for 21.1% of the total corpus. This category was reflected in responses such as the 19-year-old Malaysian male student (#M4) who expressed fears of “catching the virus” or of “friends catching it”. Given the pandemic consists of a novel coronavirus, resulting in severe or even deadly respiratory infections, this is unsurprising.

Existential threats: Also important were existential threats (6.3%), including feelings such as “losing meaning” (Australian, 32 years, male, public servant, #A1), “isolation” (Indian, 63 years, female, unemployed, IN11), “loss of control” (German, 34 years, male, analyst, #G5), or fears of “dying alone” (Polish, 29 years, male, teacher, #P12). A large body of psychological research has outlined the negative impact of such deeply rooted psychological anxieties on people´s psychological health [33].

Relational: Additionally, threats emerged pertaining to relationships (3.8%), concerned restrictions such as “not being able to check in on friends and family” (Malaysian, 28 years, female, Manager, #M1) and “not seeing friends for a long time” (German, 35 years, male, Psychotherapist, #G9).

Lifestyle: Threats emerging from lifestyle interruptions (4.5%), included relatively mundane concerns such as “not being able to go to the cinema” (UK, 40 years, female, researcher, #UK18) and “uncertainty about vacations” (Italy, 42 years, female, veterinarian, #It10).

Disrupted supply chain: Material threats concerned actual and potential disrupted supply chains (6.0%), reflected in responses about “food shortages” (US, 50-year-old, female, educator, #US9), or due to being “unable to buy essentials, medicine, and groceries” (Indian, 35, male, banker, #In5).

Economic concerns: Participants named economical concerns (14.2%), reflected in responses of 34-year-old US female retailer (#US5) who feared “I may lose my job”, or the 67-year-old German female business owner (#G3) who expressed concerns about “no income and running costs”. Balancing these two categories reflects a situation in which the sword of Damocles hangs for governments and citizens alike. While movement and business restrictions are a key instrument to decrease epidemic growth and ensure the highest levels of health protection, the same measures also cause wide-ranging economic costs. Growing discussions about the economical side effects of prevention measures indicate that the topic carries the potential for public divide, which on the long run may undermine social cohesion. Buffering negative economic consequences thus seems to reflect an important factor to maintain acceptance for health-focused restrictions.

Social fabric: A category emerged concerning threats to the stability of the social order (5.5%). Here participants expressed fears of non-solidarity and anti-social behaviors emerging from the pandemic, such as “looting” (Malaysian, 44 years, male, #M11) and “increase of populism and nationalism” (Polish, 22 years, male, student, #P21).

Vulnerable groups: Participants were particularly concerned about vulnerable groups (5.2%). For instance, participants expressed concerns about “older people, and people with pre-existing medical conditions” (Indian, 25 years, male, mechanic, #In21), or marginalized groups such as “…people who are refugees or homeless” (German, 22 years, student, #G11).

Healthcare systems: There were concerns for the sustainability of healthcare systems during the pandemic (4.9%), for example concerns about an “overloaded healthcare system” (Polish, 35 years, male, translator, #P4).

Politics: Responses reflected threats stemming from dissatisfaction of how authorities govern the pandemic resulting in blatant mistrust of authorities (3.9%). One 21-year-old female student (#UK14) expressed fears that “the government [is] acting too slowly”. Participants in the Polish sample also expressed blatant mistrust with the government, for instance expressing concerns that political leaders are “tightening up authoritarian power” (Polish, 37 years, female, educator, #P24). Finally, participants in the US sample feared that the pandemic may “effect the 2020 [presidential] election” (US, 18 years, student, #US10).

2.3 Discussion

This study was a first step in understanding the threat being experienced by individuals across the globe, which then informed the development of items for use in our new measure of multi-dimensional COVID-19 threat. The goal of Study 1 was to summarize key threats people are facing during the COVID-19 pandemic. Due to the unprecedented nature of current events, we used an inductive approach to understanding COVID-19-relevant threats based on participants' responses to a free association task. The obtained findings supported observations from the previous disease spreads regarding important personal threat perceptions. The range and intensity of the current lockdown measures, however, also revealed self-focused threats that might sometimes be overlooked, such as existential fears stemming from feelings of meaninglessness and isolation. Perhaps most critically, the present research emphasizes the need to expand the perspective of threat perceptions from a rather exclusive focus on the individual and proximal environment to broader collective concerns. Our observations indicate that threat perceptions go beyond the individual self-focus to further include group survival. Taken together, this highlights the need for a multidimensional measure of threat.

Item construction and predictions remaining studies

Based on the findings of Study 1, we developed an item pool of threat-relevant statements that participants could endorse. Based on the 10 substantive categories that emerged from the qualitative analysis, we predicted 10 factors to emerge from a factor analysis of data responding to these items. The following two studies are quantitative, and thus we formulated the following hypotheses to test the psychometric properties of our new measure:

• • H1: Factor structure hypotheses (Studies 2 & 3) – In Study 2, an exploratory factor analysis is expected to yield 10 interrelated, yet distinct factors from analysis of the 100 items of the item pool. In Study 3, the reduced item pool emerging from this analysis will then be subjected to confirmatory factor analysis, for which we predict values of CFI > .90 and RMSEA < .08 [34], and SRMR < .060 [35].
• • H2: Reliability hypotheses (Studies 2, & 3)– internal consistency estimates are predicted to be above .70 [36].
• • H3: Cross-cultural invariance hypotheses (Study 3) – scale measurement is predicted to be stable between two countries that are disparate culturally (and in terms of the impact and response to the spread of the COVID-19 Pandemic. As such, measurement and structural invariance between the United States and India should be met [37].

3. Study 2: Exploratory factor analysis

Study 2 uses exploratory factor analytic (EFA) for dual purposes. First to reduce the number of items in the item pool, and second to evidence the 10 underlying structures of COVID-19 threat that exist within the new measure. The items (presented in the appendix) were endorsed on a scale ranging from 1 (not at all concerned) to 7 (extremely concerned) and were presented in a randomized order.

3.1 Method

Participants and Procedure: Participants were recruited through Amazon’s Mechanical Turk in July 2020 (approximately 4 months after the pandemic had been declared) and were reimbursed US$1 in exchange for their participation. In total, 344 participants commenced the survey, however, 22 were excluded for not giving consent (n = 9), failing attention checks (n = 3), or not responding to the dependent variables (n = 10). The final sample comprised 322 MTurk workers (age range: 18-76 years, M = 38.12, SD = 14.22; Gender: male = 152, female = 164, gender diverse = 6). All participants lived in the United States (the majority were also born there (86%). Half the sample worked full-time (50%), others worked in another capacity (18.9%) or were unemployed (10.9%), retired (7.5%), or were students (7.8%, 5% did not disclose employment status). Notably, 17.4% of the sample had already lost their job from factors related to the pandemic, and another 15.2% believed they were likely to lose their jobs in the near future. Almost half the sample were working from home (48.8%) because of the pandemic. The participants read an online consent form, and those who agreed to participate were redirected to the website hosting the survey. Demographic questions were administered before they were asked their level of agreement to the 100 items in the pool about COVID-19 specific threats. Items were endorsed on a Likert-type scale from 1 (strongly disagree) to 7 (strongly agree). These items were randomized to prevent order effects. Participants were thanked for their time and debriefed.

3.2 Results and Discussion

Initially, we conducted a principal axis factoring analysis using an equimax rotation. This analysis revealed 14 factors with varying strength of factor loadings. The first 10 factors resembled the 10 themes emerging from Study 1, and as such the 5 items with the highest factor loadings from those 10 factors were retained and entered into a subsequent EFA, which we report here as the primary analysis for this Study (the initial EFA, including the original 100 items, is presented in online supplementary Table S2 on the open science framework). On these 50 items, we conducted a principal axis factoring analysis using an equimax rotation. This analysis produced a scree plot and Eigenvalues (ranging from 1.05 to 15.67; in combination these factors accounted for 74.65% of the variance), each of which revealed the predicted 10 factors. A parallel analysis based on 1000 permutations of parallel data from the raw data set suggested that all 10 eigenvalues were statistically significant, (i.e., exceed their relevant 95th percentile benchmark criterion eigenvalues [both observed and benchmark criterion eigenvalues are available in online supplements]). The pattern matrix loadings are presented in Table 3. After calculating factor scores, we screened the data and found there were no excessive cases, but that four of the factors were positively skewed (health, social fabric, healthcare systems, and political). The skew of these variables was corrected by applying logarithmic transformations prior to analyses. The item level descriptive data and zero-order correlations are presented in Table 4.

The data presented in Study 2 provide preliminary evidence for the factor structure and the internal reliability of the COVID-19 Multifaceted Threat Scale. Specifically, supporting the factor structure hypotheses, the EFA revealed the 10 predicted underlying structures of the scale. A visual inspection of the 30 items reveals that they all factored onto the appropriate hypothesized factors. This study also provides initial support for the reliability hypothesis, by providing evidence for the internal consistency of the scales (as > .787).

Table 3: Pattern Matrix Loadings (EFA; Study 1) and Standardised Item Loadings (CFA; Study 2) for items in the COVID-19 Multifaceted Threat Scale.

Notes: all responses are in completing the question stem “I am concerned because…” EFA = exploratory factor analysis

Table 4: Descriptive Statistics, Internal Reliability Coefficients, and Bivariate Correlations for the Subscales of the COVID-19 Multifaceted Threat Scale.

Note: EFA = exploratory factor analysis; CFA = confirmatory factor analysis. **p < .001, * p < .05.

4. Study 3: Confirmatory factor analysis and preliminary validity evidence

Study 3 aims to replicate the factor structure and reliability estimates of the COVID-19 Multifaceted Threat Scale, and to provide initial evidence for the validity of the new measure validity hypotheses. To do that we employ multi-group CFA and SEM models and assess measurement and structural invariance between the United States and India. After removing measurement error from the model structure, covariations and regression slopes between the 10 expected threat dimensions and two measures of personal and social wellbeing are assessed (see [38]). Personal and social wellbeing measures are used to test cross-cultural discriminant validity of the scale, while providing first insights about complex relations between COVID-19 threats and wellbeing.

4.1 Method

Participants

The total sample were 954 participants recruited from both India and USA (age range: 18-76 years, M = 33.73, SD = 11.89; Gender: male = 460, female = 469, gender diverse = 16).

For the Indian sample, a total of 423 participants (age range: 18-76 years, M = 35.22, SD = 11.67; Gender: male = 256, female = 211, gender diverse = 6), were recruited through online – 164 were recruited through Amazon’s MTurk and paid USD$1 to complete the study, and 309 participants were volunteers recruited using snowballing sampling on social media. In this sample, 56.2% worked full time, 10.8% worked part-time, 3% were contractual workers, 5.5 % were unemployed, 4.4 % were retired, 16.5 % were students, and the remaining 3.6% of people worked in another capacity. A large portion (79.8%) of the sample were unsure if they would keep their jobs and 5.7 % of the sample had already lost their jobs due to the pandemic. More than half the sample (65.1%) were working from home.

For the USA sample, a total of 471 participants (age range: 18-73 years, M = 32.23, SD = 11.92; Gender: male = 213, female = 249, gender diverse = 20), were recruited via the online platform Prolific, wherein participants were paid at the rate of 5 GBP per hour. In this sample, 41.5% worked full time, 17.2% worked part-time, 1.9% were contractual workers, 15% were unemployed, 3% were retired, 17.6 % were students, and the remaining 3.8% of people worked in another capacity. Again, a large portion (80.1%) of the sample were unsure if they would keep their jobs and 8% of the sample lost their jobs due to the pandemic. More than half the sample (60.4%) were working from home.

The samples were relatively similar on most demographic characteristics. For instance, the age range did not differ between the two countries, F (942,1) = 0.142, p = .71. However, it is worth noting that there were differences in gender composition, χ² (1, N = 929) = 7.06, p =.008 (more females from United States, more males from India), employment status χ²(5, N = 910) = 40.94, p < .001 (more part-time/unemployed participants in United States, more full-time employed participants in India), and COVID-19 based job loss χ²(5, N = 581) = 37.29, p < .001 (more job loss and anticipated job n United States than India).

Measures

COVID-19 Multifaceted Threat Scale: The EFA of the scale that was conducted in Study 2 led to 30-item version of the scale which was administered in Study 3. Participants were asked to read 30 statements concerning the COVID-19 pandemic (see Appendix) and then asked to indicate the extent to which they felt worried or concerned on a you feel worried or concerned on a Likert scale ranging from 1 (not at all concerned) to 7 (extremely concerned). Higher the score indicates more threat perception. The overall reliability of the subscales ranges from α = .71 to .93.[1]

Personal Coping Scale: The 28-item personal coping scale [39] with was administered wherein participants were asked to indicate on a 7-point scale ranging from 1 (not at all) to 7 (extremely) the extent to which they were able to cope with the described activities concerning the COVID-19 situation. Sample items include “I'm taking action to try to make the situation better”, “I'm looking for something good in what is happening” and “I’m getting emotional support from others”. The overall reliability of the scale is α = .88.

Collective Coping Scale: The 8-item collective coping scale [40] was administered wherein participants were asked to indicate on a 7-point scale ranging from 1 (not at all) to 7 (extremely) the extent to which their close social network (e.g., family, friends, neighbours) were able to cope with the described activities concerning the COVID-19 situation. Sample items include “We tell each other how we feel.”, “We try not to bother each other with negative thoughts and feelings” and “We try to “make peace” with the situation”. The overall reliability of the scale is α = .75.

Procedure

Participants first agreed to an informed consent form, a data protection declaration and an information that the participation of the study is voluntary. On signing the online consent form participants were directed to the survey starting with answering some basic demographic questions. This was then followed by the COVID-19 multidimensional threat scale then the personal and collective coping scales (see Measures). All materials were presented in English. Finally, participants were thanked for their time and debriefed.

4.2 Results

Before the main analyses, we first evaluated the assumptions of multivariate normality via the MVN R package [41]. Because indicators followed a non-normal multivariate distribution, MLM estimation with Satorra–Bentler scaled chi-squared statistics was selected [42]. Missing values were negligible for any observed indicator (all < 1%), so that listwise deletion was used [43]. Multi-group Confirmatory Factor Analyses were then performed using the R package “Lavaan” [44]. Reference indicators were derived via Exploratory factor analysis using ML and Oblimin rotation. Cut-off criteria of fit measures CFI > 0.90 and RMSEA < 0.08 [34], and SRMR < 0.060 [35]. Differences between models and increasing levels of invariance were assessed using Chi-squared statistics (Δ χ²), changes in Bayesian information criterion (ΔBIC) and comparative fit index (ΔCFI), as suggested by [45].

Latent structure of COVID-19 threat measures

The unconstrained model was estimated first, where only configural invariance was screened and all parameters were left free to vary between US and India. The configural measurement model composed by the 10 expected threat dimensions, each composed by three indicators, provided good fit, χ² (720) = 1120.07, p < .001; CFI = .97; RMSA = .03, 90% CI [.03; .04], p > .99; SRMR = .04, suggesting similar latent configurations both in United States (χ² = 623.75) and India (χ² = 496.30). Weak metric measurement invariance was then estimated, constraining all factor loading to equality across countries. As compared to the unconstrained model, the fully constrained model showed significantly worse χ², negligeable worse CFI, but better BIC, Δ χ² (20) = 69.78, p < .001; ΔBIC = –61; ΔCFI = −.003. To improve the model fit we scrutinized differences in factor loadings (λ – lambda) between the two countries. By releasing two factor loadings out of 30, partial metric invariance was met, Δ χ² (18) = 23.70, p = .17; ΔBIC = –96; ΔCFI = .000. Next, strong scalar measurement invariance was estimated, constraining all intercepts of observed indicators (τ - tau) to be equal across countries. The model fit worsened significantly, Δ χ² (30) = 533.87, p < .001; ΔBIC = 53; ΔCFI = −.01. By releasing 17 observed intercepts out of 30, partial scalar invariance was met, Δ χ² (13) = 19.22, p = .12; ΔBIC = –8; ΔCFI = .000. Finally, strict residual measurement invariance was estimated, constraining all residual variances of observed indicators (θ – theta) to be equal across countries. The model fit worsened significantly, Δ χ² (30) = 275.44, p < .001; ΔBIC = 123; ΔCFI = −.06. By releasing 24 observed residual variances out of 30, partial residual measurement invariance was met, Δ χ² (6) = 7.58, p = .27; ΔBIC = –19; ΔCFI = –.001.

As soon as partial residual measurement invariance was met, we proceeded with structural invariance. All residual variances of latent factors (θ) were first constrained to be equal across countries. The model fit worsened significantly, Δ χ² (10) = 236.96, p < .001; ΔBIC = 82; ΔCFI = −.007. Only by releasing nine out of 10 latent residual variances partial invariance was met, Δ χ² (1) = 2.53, p = .11; ΔBIC = − 04; ΔCFI = .000. Then, all covariances between latent factors (φ – phi) were constrained to be equal across countries. The model fit worsened significantly, Δ χ² (45) = 179.49, p < .001; ΔBIC = 96; ΔCFI = −.008. By releasing 20 out of 45 covariances, partial invariance was met, Δ χ² (25) = 30.08, p = .22; ΔBIC = − 137; ΔCFI = .000. Confirming the multidimensionality and cross-cultural validity of our COVID-19 threat scale, the final model with partial measurement and structural invariance showed excellent fit to the data, χ² (783) = 1204.95, p < .001; CFI = .97; RMSA = .03, 90% CI [.03; .04], p > .99; SRMR = .05, both in US (χ² = 657.71) and India (χ² = 547.23, see online supplementary Table S3 to retrieve measurement and structural components of the COVID-19 threat measure).

Latent structure of personal and social wellbeing measures

The social wellbeing measure was expected to load on four dimensions, each composed by three indicators, then converging onto a second higher-order latent factor. Conversely, the personal wellbeing measure was expected to load on a single latent factor composed of five indicators. The corresponding configural measurement model provided good fit, χ² (228) = 432.90, p < .001; CFI = .98; RMSA = .04, 90% CI [.04; .05], p = .84; SRMR = .05, suggesting similar latent configurations both in US (χ² = 220.36) and India (χ2 = 212.54). Moreover, this hierarchical model provided way better fit than the non-hierarchical model, whereby social wellbeing was modelled as unidimensional, Δ χ² (8) = 1263.2, p < .001; ΔBIC = 1824; ΔCFI = –.275.

Weak metric measurement invariance was then estimated, constraining all factor loading to be equal across countries. As compared to the unconstrained model, the fully constrained model showed significantly worse χ², but slightly better BIC and CFI, χ² (15) = 37.94, p < .001; ΔBIC = –55; ΔCFI = .003. To improve the model fit we scrutinized differences in factor loadings (λ) between the two countries. By releasing two factor loadings out of 18 and leaving the four underlying dimensions of social wellbeing free to vary between US and India, partial metric invariance was met, Δ χ² (11) = 13.73, p = .25; ΔBIC = –60; ΔCFI = .003. Next, strong scalar measurement invariance was estimated, constraining all intercepts of observed indicators (τ) to be equal across countries. The model fit was slightly worse, Δ χ² (11) = 84.74, p < .001; ΔBIC = − 3; ΔCFI = −.004, and could not be improved via partial invariance, meaning that mean levels of the underlying wellbeing indicators vary significantly between countries. Finally, strict residual measurement invariance was estimated, constraining all residual variances of observed indicators (θ) to be equal across countries. The model fit worsened significantly, Δ χ² (17) = 77.34, p < .001; ΔBIC = 111; ΔCFI = −.015. By releasing 11 observed residual variances out of 17, partial residual measurement invariance was met, Δ χ² (6) = 9.54, p = .15; ΔBIC = –22; ΔCFI = .002.

As soon as partial residual measurement invariance was met, we proceeded with structural invariance. All residual variances of latent factors (θ) were first constrained to be equal across countries. The model fit worsened significantly, Δ χ² (6) = 188.43, p < .001; ΔBIC = 54; ΔCFI = −.009. Only by releasing four out of six latent residual variances partial invariance was met, Δ χ² (2) = 3.35, p = .19; ΔBIC = − 11; ΔCFI = .000. Then, the covariance between the two latent factors (φ – phi) was constrained to be equal across countries. The model fit worsened significantly, Δ χ² (1) = 17.34, p < .001; ΔBIC = 9; ΔCFI = −.002, revealing stronger relations between personal and social wellbeing in US, cov = .94(.08), p < .001, than in India, cov = .62(.07), p < .001. Confirming the multi-dimensionality and cross-cultural validity of social and personal wellbeing, however, the final model with partial measurement and structural invariance showed excellent fit to the data, χ² (247) = 460.41, p < .001; CFI = .97; RMSA = .03, 90% CI [.04; .05], p > .99; SRMR = .05, both in United States (χ² = 231,62) and India (χ² = 228,79, see supplementary materials to retrieve measurement and structural components of the personal and social wellbeing measures).

Latent covariations between COVID-19 threat and wellbeing measures

In a final step, we merged the two measurement models together to estimate latent covariances between COVID-19 threat and wellbeing measures. The model fit was good, χ² (2010) = 2967.97, p < .001; CFI = .96; RMSA = .03, 90% CI [.03; .04], p > .99; SRMR = .06, thereby revealing that COVID-19 threat and wellbeing were two separate although correlated constructs. By constraining all covariances between COVID-19 threat and wellbeing measures to equality in US and India, the χ² statistic worsened significantly, the BIC improved, and the CFI remained virtually the same. To locate specific differences between COVID-19 threat and wellbeing measures in the United States and India, we proceeded by constraining covariances one by one. As shown in Table 5, only lifestyle threat differently related to personal wellbeing in the United States and in India, Δ χ² (1) = 16.15, p < .001; ΔBIC = 7; ΔCFI = −.001. All the other covariances did not differ significantly and were therefore constrained to equality, Δ χ² (1) = 28.25, p = .08; ΔBIC = − 96; ΔCFI = .000. Interestingly, almost all COVID-19 threats were negatively related to personal wellbeing. Conversely, relational threat and threat for vulnerable groups were positively related to social wellbeing (see Table 5).

Table 5: Covariances between COVID-19 threat and wellbeing measures in US and India (Study 3).

Note: All covariances were constrained to equality between US and India but lifestyle threat. † < .10, * < .05, ** <.01, *** <.001. Significant findings presented in boldface.

To assess the unique contribution of each COVID-19 threat on personal and social wellbeing while controlling for the other threat dimensions, we calculated regression estimates from a SEM model. In line with covariances retrieved from the previous CFA, political threat, b = − 0.18(.06), p = .001, financial b = − 0.04(.02), p = .05, and existential, b = − 0.23(.03), p < .001, threats were related to decreases in personal wellbeing. Conversely, political, b = − 0.18(.06), p = .001, and existential, b = − 0.21 (.04), p = .001, threats were related to decreases in social wellbeing, while threat for vulnerable groups, b = 0.22(.06), p = .006, and relational threat, b = 0.32(.05), p < .001, were related to increases in social wellbeing.

4.3 Discussion

The data presented in Study 3 provide additional evidence for the factor structure and the internal reliability of the COVID-19 Multifaceted Threat Scale. Specifically, supporting the factor structure hypotheses, the multi-group CFA revealed the 10 predicted underlying structures of the scale both in the United States and India. By testing measurement invariance in two countries heavily affected by the pandemic and yet extremely different in terms of cultures and social arrangements, we provided first evidence of the cross-cultural validity of our multidimensional scale. Furthermore, we found similar results in the United States and India concerning covariations between the COVID-19 Multifaceted Threat Scale and personal and social wellbeing measures. While COVID-19 threats were generally associated with decreased personal wellbeing, some threat dimensions were associated with increased social wellbeing, such as threats for vulnerable groups and relational threats. This finding suggests that emphatic concerns for others’ welfare and disrupted social bonds, if not accompanied by distrust towards politics and existential anxieties, may trigger heightened sense of belonging to a community and making a contribution to society.

5. General discussion

With little knowledge about how to contain the virus or manage its spread, governments around the world were forced into implementing sudden and wide-ranging restrictions to counteract the rapid growth of infection rates. In many cases, whole countries were set into temporal lockdown, completely disrupting the status quo and leaving individuals, families, and communities in turmoil as they indefinitely adjusted to this new way of life. Given the rapid evolution of the pandemic and the succession of regulations adopted by national governments to reduce contagion as a result, it is likely that threat perceptions among populations will remain for some time. Although at this point, it is still hard to make predictions for the future, one may expect that while some threats may dissolve rather quickly along with the ease of restrictions, other threats (e.g., economic disruptions) will take substantially more time to disappear. We developed the COVID-19 Multifaceted Threat Scale to provide researchers with a tool that permits them to capture a wide range of potential threat experiences with different outcomes for individuals and societies.

A major benefit of having such an exhaustive measurement tool is that researchers will be able to study the dynamic developments of threat experiences in multiple contexts and over time. While in many places of the world, in-place restrictions begin to ease, the looming danger of new and more transmittable virus mutations remains. While writing these lines, some governments decided to re-introduce harsher lockdown measures, which will likely result in a re-appearance of certain threats. Gaining a differentiated understanding about the functionality of coping mechanisms and people´s resilience towards Covid-19 related threats can provide an image about the psychological costs associated with different strategies to combat virus spreading and eventually inform governmental decision-making processes. Interestingly, our data also suggests some positive relationships between threats related to social relationships and concerns for vulnerable groups and social well-being. It is possible, that despite all the negative consequences, in some people the pandemic has led to an increased awareness for important personal social bonds as well as for structural inequalities (on posttraumatic growth see [46]). Exploiting this potential may strengthen social bonds that can help navigating communities and societies through current and future crises [47, 48]. In sum, we believe that the proposed scale adds a valuable contribution to efforts attempting to understand the ongoing personal and social challenges of the Covid-19 pandemic.

Limitations and future directions

This study has several limitations that warrant consideration. It is important to note that the inductive approach we adopted is not suitable for tackling the hierarchical structure of specific threat dimensions, nor to draw any generalization from the data to a given population. For future research, we suggest focusing on the development of a psychometric assessment of these COVID-19 threat facets. In addition, in Study 1 we have presented data from a range of countries, but we concede that we were opportunistic in our selection process. That is, we selected countries that the authors could easily recruit from and were fluent in (allowing translation of the materials and the participant responses). We do not claim to have covered the full range of cultures or countries and note that there are some marked absences (Central America, Africa) in our corpus, and indeed, the scale has only been fully validated in samples from the United States and India. Moreover, we also aimed to incorporate a wide range of potentially experienced threats, novel threats have emerged since and were consequently not incorporated into our measure. Perhaps most notably in this regard is the spread of vaccine hesitancy and vaccine-related conspiracy beliefs. However, we would expect that such outcomes can be informed by some of the items that are included into our scale, namely political threats.

Conclusion

As the COVID-19 pandemic has indeed disrupted life as we know it, a new theoretical approach is needed to thoroughly understand the accompanying stressors, worries, fears, and concerns (i.e., threats) experienced by people. In this paper, we took a mixed-method approach, first exploring perceptions of threat associated with the pandemic and then validating a psychometric scale based on this preliminary inductive approach. Across three studies the COVID-19 Multifaceted Threat Scale was presented, an item pool was generated from the analysis of qualitative data, the factor structure was explored and verified, evidence supporting the validity and reliability hypotheses was presented, and initial evidence for the application of the 10 subscales was put forward. Although there are a range of options available for measuring COVID-19 based threat, this is to our knowledge the first to measure the full-range of possible threats experienced during the virus outbreak, and to have used an inductive, bottom-up approach to item generation, thus increasing both the internal validity and the external generalisability of the measure.

The COVID-19 multifaceted threat scale

There are several reasons why you might be concerned or worried about the spread of the COVID-19 pandemic. Please read each of the following statements carefully, and then indicate to what extent you feel worried or concerned with regard to the pandemic situation on a scale ranging from 1 (not at all concerned) to 7 (extremely concerned).

I am concerned because….

1. … I might catch COVID-19
2. … I am very paranoid about germs these days
3. … I am following all the advice to avoid getting sick
4. … I have a sense of uselessness since the pandemic started
5. … my life has less meaning these days
6. ...I feel trapped with no way to escape
7. … I have had to become less social
8. … I miss my friends
9. … the lack of social contact is noticeable
10. … I don't know when my next vacation will be
11. … I do not know when I will be able to travel again
12. … I can no longer visit places outside the area where I live (e.g., seaside, mountains, countryside)
13. … there's a shortage of sanitizers and hygiene products
14. .. there are shortages of essential goods (e.g., toilet paper, water)
15. … there could be food shortages in supermarkets
16. … my financial situation is less stable
17. … I might run out of money
18. … I might not be able to pay my bills
19. … there are too many irresponsible people who do not respect social distancing
20. … people don't seem to care that they might be infected and passive on COVID-19
21. … people simply don't respect governments orders that are designed to contain COVID-19
22. … the virus will impact humanitarian work in regions of conflict (e.g. Syria, Yemen)
23. … COVID-19 is spreading through refugee camps
24. … homeless people are not able to protect themselves
25. … local intensive care facilities cannot handle the impact of the virus
26. … the hospitals are struggling to cope with the demands they are under
27. … medical staff are unable to keep up with what is needed of them
28. … the government's response to the coronavirus is being used for political gains
29. … there is so much misinformation being spread for political purposes
30. … the Government is unable to deal with the COVID-19 pandemic

Note: randomize the presentation order of the items.

Scoring:

• Health dimension – items 1 – 3.
• Existential dimension – items 4 – 6.
• Relational dimension – items 7 – 9
• Lifestyle dimension – items 10 – 12
• Supplies dimension – items 13 – 15
• Financial dimension – items 16 – 18
• Social fabric dimension – items 19 – 21
• Vulnerable groups dimension – items 22 – 24
• Healthcare system dimension – items 25 – 27
• Politics dimension – items 29 - 30

We recommend scoring the scale so that there are 10 subscales. To do so, average the participants responses to the items in each subscale.

Note: there is no need to reverse score any items.

Declarations:

Data availability: Original data and coding information can be retrieved from: https://osf.io/5wtgv/

Funding: Adrian Lueders was funded by the Agence Nationale de la Recherche (ANR-18-0RAR-0003-02). Emanuele Politi was funded by the Fonds National Suisse (P2LAP1_187709). Joel Anderson was supported by a faculty grant from Australian Catholic University and the Australian Research Council (DE230101636).

Conflict of Interest: The authors declare no conflict of interest.

Ethics: The research has been approved by the ethical committee of the Australian Catholic University (2020-113E; 2020-114E). Informed consent was obtained from all research participants.

Credit author statement

Joel Anderson: Conceptualization, Methodology, Software, Formal Analysis, Investigation, Writing - Original Draft

Adrian Lueders: Conceptualization, Methodology, Software, Formal Analysis, Investigation, Writing – Review and Editing

Sindhuja Sankaran: Conceptualization, Methodology, Software, Investigation, Writing – Review and Editing

Eva Green: Supervision, Funding Acquisition, Writing – Review and Editing

Emanuele Politi: Conceptualization, Methodology, Software, Formal Analysis, Investigation, Writing – Review and Editing, Project Administration.

References

1. Eicher V, and Bangerter A. Social representations of infectious diseases. In Handbook of social representations (2015) : 385-296.
2. Mondragon N I, Gil de Montes L, and Valencia J. Ebola in the public sphere: a comparison between mass media and social networks. Science Communication 39 (2017): 101-124
3. World Health Organisation. WHO Coronavirus (COVID-19) (2021).
4. Politi E, Lueders A, Sankaran S, et al. The impact of COVID-19 on the majority population, ethno-racial minorities, and immigrants: A systematic review on threat appraisals from an inter-group perspective. European Psychologist 26 (2022): 298-309.
5. Garnett P, Doherty B, and Heron T. Vulnerability of the United Kingdom’s food supply chains exposed by COVID-19. Nature Food 1 (2020): 315-318.
6. Fernandes N. Economic effects of coronavirus outbreak (COVID-19) on the world economy (2020).
7. Witteveen D, and Velthorst E. Economic hardship and mental health complaints during COVID-19. Proceedings of the National Academy of Sciences 117 (2020): 27277-27284.
8. Tabri N, Hollingshead S, and Wohl M. Framing COVID-19 as an existential threat predicts anxious arousal and prejudice towards Chinese people (2020).
9. Jetten J. Together apart: The psychology of COVID-19. Sage (2020).
10. Williams S N, Armitage C J, Tampe T, et al. Public perceptions and experiences of social distancing and social isolation during the COVID-19 pandemic: A UK-based focus group study. BMJ open 10 (2020): e039334.
11. Gruber J, Prinstein M J, Clark L A, et al. Mental health and clinical psychological science in the time of COVID-19: Challenges, opportunities, and a call to action. American Psychologist (2020).
12. Xiong J, Lipsitz O, Nasri F, et al. Impact of COVID-19 pandemic on mental health in the general population: A systematic review. Journal of Affective Disorders (2020).
13. Depoux A, Martin S, Karafillakis E, et al. The pandemic of social media panic travels faster than the COVID-19 outbreak. In: Oxford University Press (2020).
14. Brooks S K, Webster R K, Smith L E, et al. The psychological impact of quarantine and how to reduce it: rapid review of the evidence. The Lancet 395 (2020): 912-920.
15. Desclaux A, Badji D, Ndione A G, et al. Accepted monitoring or endured quarantine? Ebola contacts' perceptions in Senegal. Social Science & Medicine 178 (2017): 38-45.
16. Jeong S J, Italiano C, Chaiwarith R, et al. Late presentation into care of HIV disease and its associated factors in Asia: results of TAHOD. AIDS research and human retroviruses 3 2(2016): 255-261.
17. Lee S, Chan L Y, Chau A M, et al. The experience of SARS-related stigma at Amoy Gardens. Social Science & Medicine 61 (2005): 2038-2046.
18. Wilken J A, Pordell P, Goode B, et al. Knowledge, attitudes, and practices among members of households actively monitored or quarantined to prevent transmission of Ebola Virus Disease—Margibi County, Liberia: February-March 2015. Prehospital and disaster medicine 32 (2017): 673-678.
19. Braunack-Mayer A, Tooher R, Collins J E, et al. Understanding the school community’s response to school closures during the H1N1 2009 influenza pandemic. BMC public health 13 (2013): 1-15.
20. Gilles I, Bangerter A, Clémence A, et al. Trust in medical organizations predicts pandemic (H1N1) 2009 vaccination behavior and perceived efficacy of protection measures in the Swiss public. European journal of epidemiology 26 (2011): 203-210.
21. Conway I, Gideon L, Woodard S R, et al. Social psychological measurements of COVID-19: Coronavirus perceived threat, government response, impacts, and experiences questionnaires (2020).
22. Pérez-Fuentes M D C, Molero Jurado M d M, Oropesa Ruiz N F, et al. Questionnaire on Perception of Threat from COVID-19. Journal of Clinical Medicine 9 (2020): 1196.
23. Ahorsu D K, Lin C-Y, Imani V, et al. The fear of COVID-19 scale: development and initial validation. International journal of mental health and addiction (2020): 1-9.
24. Ernst-Vintila A, Delouvée S, and Rouquette M-L. La crise financière de 2008: menace collective ou défi individuel? Une analyse de la pensée sociale mobilisée en situation de crise. Les cahiers internationaux de psychologie sociale (2010): 515-542.
25. Krauth-Gruber S, Bonnot V, and Drozda-Senkowska E. Menaces et peurs collectives: Apeurés, restons nous des citoyens éclairés. Les peurs collectives (2013): 151-168.
26. Taylor S, Landry C A, Paluszek M M, et al. Development and initial validation of the COVID Stress Scales. Journal of Anxiety Disorders 72 (2020): 102232.
27. Kachanoff F J, Bigman Y E, Kapsaskis K, et al. Measuring realistic and symbolic threats of COVID-19 and their unique impacts on well-being and adherence to public health behaviors. Social Psychological and Personality Science 12 (2021): 603-616.
28. Arpaci I, Karatas K, and Baloglu M. The development and initial tests for the psychometric properties of the COVID-19 Phobia Scale (C19P-S). Personality and individual differences 164 (2020): 110108.
29. Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative research in psychology, 3(2), 77-101. https://doi.org/10.1191/1478088706qp063oa
30. Landis, J. R., & Koch, G. G. (1977). An application of hierarchical kappa-type statistics in the assessment of majority agreement among multiple observers. Biometrics, 363-374. https://doi.org/10.2307/2529786
31. Graneheim U H, and Lundman B. Qualitative content analysis in nursing research: concepts, procedures and measures to achieve trustworthiness. Nurse Education Today 24 (2004): 105-112.
32. Francis J J, Johnston M, Robertson C, et al. What is an adequate sample size? Operationalising data saturation for theory-based interview studies. Psychology and health 25 (2010): 1229-1245.
33. Greenberg J, Koole S L, and Pyszczynski T A. Handbook of experimental existential psychology. Guilford Press (2004).
34. Kline R B. Principles and Practice of Structural Equation Modeling. Guilford (1999).
35. Hu L T, and Bentler P M. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal 6 (1999): 1-55.
36. Cronbach L. Essentials of psychological testing. Happer and Row publishers (1990).
37. Vandenberg R J. Toward a further understanding of and improvement in measurement invariance methods and procedures. Organizational research methods 5 (2002): 139-158.
38. Sibley C G, Greaves L M, Satherley N, et al. Effects of the COVID-19 pandemic and nationwide lockdown on trust, attitudes toward government, and well-being. American Psychologist 75 (2020): 618-630.
39. Carver C S. You want to measure coping but your protocol’too long: Consider the brief cope. International journal of behavioral medicine 4 (1997): 92-100.
40. Wlodarczyk A, Basabe N, Páez D, et al. Positive effects of communal coping in the aftermath of a collective trauma: The case of the 2010 Chilean earthquake. European Journal of Education and Psychology 9 (2016): 9-19.
41. Korkmaz S, Goksuluk D, and Zararsiz G. MVN: an R package for assessing multivariate normality. R J 6 (2014): 151.
42. Chou C P, Bentler P M, and Satorra A. Scaled test statistics and robust standard errors for non-normal data in covariance structure analysis: a Monte Carlo study. British Journal of Mathematical and Statistical Psychology 44 (1991): 347-357.
43. McKnight P E, McKnight K M, Sidani S, et al. Missing data: A gentle introduction. Guilford Press (2007).
44. Rosseel Y. Lavaan: An R package for structural equation modeling and more. Version 0.5–12 (BETA). Journal of statistical software 48 (2012): 1-36.
45. Vandenberg R J, and Lance C E. A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational research methods 3 (2000): 4-70.
46. Tedeschi R G, and Calhoun L G. " Posttraumatic growth: conceptual foundations and empirical evidence". Psychological Inquiry 15 (2004): 1-18.
47. Muldoon O T, Acharya K, Jay S, et al. Community identity and collective efficacy: A social cure for traumatic stress in post-earthquake Nepal. European Journal of Social Psychology 47 (2017): 904-915.
48. Wlodarczyk A, Basabe N, Páez D, et al. Communal coping and posttraumatic growth in a context of natural disasters in Spain, Chile, and Colombia. Cross-Cultural Research 50 (2016): 325-355.
49. [1] Cronbach’s alpha coefficients for each facet and for each national group are available in the supplementary materials on the Open Science Framework.