1. Background

Health systems in the low and middle-income countries (LMICs) is facing a rapidly increasing burden from noncommunicable diseases (NCDs). NCDs contributes for 72% of deaths out of total deaths globally [1]. In 2016, more than 3/4^th of NCD deaths happened in low and middle-income countries and 46% of deaths are in below the age of 70 years [2]. In India, in 1990, the 53.6% deaths were due to NCDs whereas in 2016, these have increased to 61.8% [3]. The major NCDs are CVDs, chronic respiratory diseases, cancer, diabetes and other endocrine disorders contributing to mortality in India. Increased attributable risk to NCDs are contributed by unhealthy diet, tobacco use, alcohol use, low physical activity, increased blood pressure, raised fasting plasma glucose, raised total cholesterol and high body mass index [3]. To assess the exact burden and magnitude of NCDs, high-quality population based NCD surveillance data is required to assess the; (1) Burden, patterns, and outcomes so as to (2) inform prevention, detection and management activities for NCDs; and (3) evaluate interventions trends so that optimal approaches should be adopted. So as to tackle inequalities in coverage of NCD surveillance systems globally, with limited information currently available [4]. So, the tools for surveillance of NCDs in developing countries or LMICs as in India are disease registries which work as a powerful method to record the data [5].

The registries generate high quality data of diseases surveillance for estimating the disease incidence, mortality, trends in population for different geographical regions and with different cultures [6]. The data from registries set a tone for undertaking prevention and control measures which includes working for certain health policies and for further research in areas of epidemiology and health system [7]. Though there are various tools to for separate registries such as Cancer, MACE, Stroke, Diabetes etc. However, all these diseases have not been integrated in a single tool with inclusion of major risk factors for NCDs [8]. In this perspective in the present study, a practical and systematized Integrated NCD registry tool based on the international and national literature available has been developed and validated.

2. Methodology

The stages for the development and validation of an instrument encompass four distinct phases: planning, construction, quantitative analysis, and validation [9-11]. Figure 1 outlines the development of the tool. The instrument planning and construction phases consisted of an extensive literature review of the existing integrated and individual registry forms available online and by gathering information from the regional functional registries and expert opinion.

Figure 1: Steps of Validation of the Integrated Registry tool.

2.1 Instrument development

Based on the literature review, the first version of the instrument was elaborated, consisting of 125 items. Items were divided into four domains: “General data of registered patient,” “Patient sociodemographic details,” “Patient history (including behavioural risk factors for NCDs as Tobacco use, alcohol use, diet, dietary salt, physical activity, anthropometric measurements, family history, obstetric history),” and “Medical condition details (Young diabetes, Stroke, Cancer and Acute coronary artery events).”

In the second phase, the face and the content of the instrument was validated by a panel of specialist judges/experts. The Specialists were selected on the basis of the criteria of area of specialization in the field of specific NCDs or those who are already running the individual NCD registries or have field experience of running a population-based registry or have been working in their respective areas for a period of more than one year, or who have scientific publications in the relevant field. Those who did not respond to the invitation to participate within the established deadline, were excluded as an expert. Total of 10 experts were approached and 6 responded. The qualitative judgement of the experts was also taken as an open-ended question.

2.2 Judgement

This step entails confirmation by a specific number of experts, indicating that instrument items and the entire instrument have content validity. For this purpose, an expert panel is contacted. This includes Content validity and Face validity.

2.3 Content validation

Content Validity Index: The researchers asked the panel of experts to give their view points on the items generated for the construct of integrated NCD registry tool. The CVI was calculated for all individual items (I-CVI) and the overall scale (S-CVI) [12]. For CVI, the panel of experts was asked to rate each scale item in terms of its relevance to the underlying construct. A 4-point scale was used to avoid a neutral point. The four points used along the item for rating continuum (1 = not relevant, 2 = somewhat relevant, 3 = quite relevant, 4 = highly relevant). For each item, I-CVI was computed as the number of experts giving a rating of 3 or 4, divided by the total number of experts. For example, an item rated 3 or 4 by four out of five experts has I-CVI of 0.80. It is advised that I-CVI should be 1.00 in case of five or fewer judges and in case of six or more judges; I-CVI should not be less than 0.78. The S-CVI was computed for ensuring content validity of the overall scale. It can be conceptualized in two ways—S-CVI (universal Agreement) and S-CVI (Average). S-CVI (Universal agreement) reflects the proportion of items on an instrument that achieved a rating of 3 or 4 by all the experts in the panel. S-CVI (Average) is the liberal interpretation of Scale validity Index, and it is computed by using average I-CVI. S-CVI (Average) emphasizes on average item quality rather than on average performance of the experts. It is recommended that a minimum S-CVI should be 0.8 for reflecting content validity [12-14]. Six experts participated in the first phase of content validation, the acceptable CVI value for each item ranged from 1.00 to 0.78, and the Kappa Modified value from 1.00 to 0.65 [13].

Kappa Statistic coefficient: CVI is extensively used by researchers for determining the content validity. However, it does not consider the inflated values that may occur because of possibility of chance agreement. Therefore, computation of Kappa coefficient ensures better understanding of content validity as it removes any random chance agreement. Kappa statistic is a consensus index of interrater agreement that supplements CVI to ensure that the agreement among experts is beyond chance. Computation of Kappa Statistic requires the calculation of probability of chance agreement, that is, Pc = [N! / A! (N – A)!] × 0.5N. In this formula, N =number of experts in the panel, A=number of experts in the panel who agree that the item is relevant. Kappa statistic is then calculated as K= (I-CVI – Pc) / (1 – Pc). Evaluation criteria for Kappa is that values above 0.74, between 0.6 and 0.74, and the ones between 0.4 and 0.59 are considered to be excellent, good, and fair, respectively [9,13].

Content Validity Ratio (CVR): CVR according to the Lawshe test is computed to specify whether an item is necessary for operating a construct in a set of items or not. For this, the expert panel was asked to give a score of 1 to 3 to each item ranging from essential, useful but not essential, and not necessary. The formula for computation of CVR= (Ne – N / 2) / (N / 2) in which Ne is the number of panelists indicating “essential” and N is the total number of panelists. The numeric value of CVR ranges from -1 to 1 [15]. High scores of CVR indicate the agreement of members on the necessity of an item in the instrument [16]. A positive CVR indicates that atleast half of the panelists agree on the necessity of the item for the construct [15,16].

2.4 Face validity

Face validity answers this question whether an instrument apparently has validity for subjects, patients and/or other participants. Face validity means if the designed instrument is apparently related to the construct underlying study. The face validity of the tool was assessed using the calculation of item impact score. The experts were required to evaluate the items with respect to 10 domains i.e. Matches Understanding Level, Technical content ok, Has logical sequence of questions, Continuity of items is fine, Language is understandable, Terminology, Given options are simple to understand, Is not loaded with unnecessary information, Permits answering properly and Is useful with practical value using the 4 point Likert scale i.e. strongly disagree, disagree, agree and strongly agree ranging from 1 to 4. All the questionnaires were collected and analysed, the impact score was computed for each item using the formula: Impact Score = Frequency (proportion of raters who scored 3&4) * Importance (mean score for the importance on the basis of domains). If the item impact of an item is equal to or greater than 1.5 (which corresponds to a mean frequency of 50%), it is maintained in the instrument; otherwise it is eliminated [9,17].

3. Results

3.1 Instrument development: The first version of the tool was developed and with a total of 138 variables divided into four domains: “General data of registered patient (1 with 4 subparts),” “Patient sociodemographic details (1 with 15 subparts),” “Patient history (including behavioural risk factors for NCDs as Tobacco use (17 subparts), alcohol use (7 subparts), diet (4 subparts), dietary salt (8 subparts), physical activity (7 subparts), anthropometric measurements, family history, obstetric history) (8 items),” and “Medical condition details (Young diabetes (36), Stroke (21), Cancer (10), Acute coronary artery events (20), Aplastic anaemia (13) and follow up (33).”

This tool was generated based on the review of the existing literature on i.e. tools which existing NCD registries are using although the individual tools are being used by them i.e. Philippines [18] and Barbados [19], MACE (ICMR) [20], YDR (ICMR) [21], Stroke (ICMR and SITS) [22,23] and Cancer (PBCR Chandigarh and ICMR) [24,25]. After this initial pool of questions/ variables was generated, experts were identified and invited for the validation of the tool for face and content validation. The final tool is given in Supplementary material file 1.

3.2 Quantification of the validity

Content validity

The I-CVI for all the items ranged from 0.5 to 1. The S-CVI (Average) for Basic details, Module 1, Module 2, Module 3 and Module 4 was 0.87, 0.92, 0.91, 0.87 and 0.99 respectively. The overall S-CVI for the 126 items scale was 0.91 which represents high content validity of all the items. Items with 0.67 I-CVI represents the requirement for revision. Kappa statistic ranged from 0.81 to 1 for most of the items. Nine items with negative kappa coefficient calls a disagreement among raters for their inclusion in the tool. CVR for the variables showed the percentage of panelists rating an item as “essential/mandatory” None of the variable has CVR as negative. CVR ranged from 0 to 1 for other items on the scale indicating that half or a greater number of panelists rated these items to be essential for the integrated registry tool. The details of CVI, CVR and Kappa statistic are given in Supplementary file 2.

Face Validity

The expert scoring on the 10 domains identified for face validity are presented in Figure 2. The median score for the rating on Likert scale for 10 domains was assessed which is above 3 for each domain and is presented in Figure 3.

At this stage, all the experts stated that all the questionnaire items were simple, clear, and related to the objectives. Additionally, the impact scores of all the items were above 1.5 which is acceptable. The item impact score for the face validity is Table 1.

Table 1: Impact Score of the questionnaire for the 10 domains of face validity.

Figure 2: Expert Scoring of the tool for the 10 domains of face validity ((1 = not relevant, 2 = somewhat relevant, 3 = quite relevant, 4 = highly relevant).

Figure 3: Median score of the expert validation on the 10 domains of face validity.

4. Discussion

The tool for the Integrated NCD Registry, is a multidimensional tool for the major NCDs i.e. Young diabetes, Cancer, Stroke and Acute cardiac events. In the study, the items of the tool were constructed on the basis of comments and suggestions of the specialist and public health experts to produce the final version of the tool. Though, the reporting format or the registry tools are different for different registries as per the funding agency requirements. There was no validated integrated tool developed so far globally for the registry purpose. Despite of the fact that these NCD shares common risk factors, no integrated tool for data collection on NCD data is available. Globally, many tools are available as per the survey requirements such as for tobacco GATS11, GYTS12, STEP-wise surveillance tool for risk factors of NCDs but no such tool is available for reporting of mortality and morbidity related data. The tool for the Integrated NCD Registry, is a multidimensional tool for the major NCDs i.e., young diabetes, Cancer, Stroke and Acute cardiac events.

Content validity is related to the robustness of score interpretations of an instrument and indicates the degree to which these scores measure what they claim to measure [26]. In this context, the phase of content validation in the present study enabled a reduction in the size of the instrument with the exclusion of 10 items including the language editing. Among the 10 items that were excluded, 6 belonged to the General Characteristics Module and 2 belonged to the Young diabetes module and 2 to Cancer module. The items in the General Characteristics module that did not reach satisfactory levels of content validity were considered to have no value. In this phase, important changes were also performed in the item wordings for better understanding based on the qualitative comments received from experts which provided the robustness to the tool. The results of the content validation process were determined by the CVI and Modified Kappa Coefficient. The items that presented CVI and Modified Kappa values above 0.70 were considered good and excellent [13]. Thus, considering the final version of the instrument, it was observed that for the 125 items, 92% of the items were validated in the content validity.

The final version of the tool contains of Integrated NCD Registry for four NCDs includes the major 2 parts i.e. Part 1 includes general characteristic i.e. General data on reporting facility details, patient information, patient history including the behavioural risk factors for NCDs, Part 2 includes 4 disease modules i.e. Young diabetes, Stroke, Cancer and Acute Cardiac events and follow up. In the he final version, it was observed that out of 125 items, 92% of the items were validated. Some of the strong points of the study is doing a complete content and face validation with the inclusion of content validity index, content validity ratio, kappa statistic and impact score in terms of quantitative analysis of validation. The Present paper demonstrates quantities indices for content validity of a new instrument and outlines them designing of Integrated Registry tool.

5. Conclusion

Validity of a study tool is subjective and subjective procedure. In the first stage, tool design is developed and in the second stage, judgment by experts on tool items is performed along with the content study by experts for accordance between theoretical and operational definitions as per the objectives of tool development [9,12^]. Content validity mandates the operationalization of the construct which is dependent on items from the specific domain of content relevant with intended objectives of tool.²⁶. In the judgment stage, 6 experts from different domains were asked to rate the items on the basis of their necessity and relevance. The quantification indicated high content validity for the items based on content validity of CVI (I-CVI & S-CVI), Kappa coefficient, and CVR. Calculation of content validity for the construct helped in reducing the inappropriateness [27]. Therefore, a Registry tool was constructed and validated professionals to collect the NCD patient’s data in a timely, effective and quality manner. So, the integrated NCD Registry tool developed and validated in the current study will benefit in term of reporting of mortality and morbidity data with risk factors for NCDs in a uniform way. The data collected by this way can further be comparable across the regions and help in further trend analysis of various NCDs.

References

1. Noncommunicable Diseases Progress Monitor. Geneva 27, Switzerland (2015).
2. GBD 2016 Causes of Death Collaborators. Global, regional, and national age-sex speci?c mortality for 264 causes of death, 1980-2016: a systematic analysis for the Global Burden of Disease Study 2016. Lancet 390 (2017): 1151e-1210.
3. Global Health Estimates 2015. Deaths by Cause, Age, Sex, by Country and by Region, 2000-2015. Geneva: World Health Organization (2016).
4. Kroll M, Phalkey RK, Kraas F. Challenges to the surveillance of non-communicable diseases – a review of selected approaches. BMC Public Health 15 (2015): 1243.
5. Thakur JS. NCD surveillance. In: Public Health Approaches to Non-Communicable Diseases. New Delhi: Wolters Kluwer 1 (2015): 193-209.
6. Thakur JS. Role of registries in NCDs: A public health perspective. In: Public Health Approaches to Non-Communicable Diseases. New Delhi: Wolters Kluwer 1 (2015): 221-237.
7. Brooke EM, World Health Organization. The current and future use of registers in health information systems / Eileen M. Brooke. World Health Organization (1974).
8. Paika R, Thakur JS, Khurana D, et al. Development and cost estimates of an integrated noncommunicable disease registry in North India: A study protocol. Int J Non-Commun Dis 4 (2019): 49-52.
9. Zamanzadeh V, Rassouli M, Abbaszadeh A, et al. Details of content validity and objectifying it in instrument development. Nursing Practice Today 1 (2014): 163-171.
10. Benson J, Clark F. A guide for instrument development and validation. Am J Occup Ther 36 (1982): 789-800.
11. Grant JS, Davis LL. Selection and use of content experts for instrument development. Res Nurs Health 20 (1997): 269-7 4 .
12. Lynn M. Determination and quantification of content validity. Nursing Research 35 (1986): 382-386.
13. Polit D, Beck C. The content validity index: Are you sure you know what’s being reported? Critique and recommendations. Research in Nursing & Health 29 (2006): 489-497.
14. Rubio D, Berg Weger M, Tebb S, et al. Objectifying content validity: Conducting a content validity study in social work research. Social Work Research 27 (2003): 94-104.
15. Lawshe C. A quantitative approach to content validity. Personnel Psychology 28 (1975): 563-575.
16. Ayre C, Scally AJ. Critical Values for Lawshe’s Content Validity Ratio: Revisiting the Original Methods of Calculation. Measurement and Evaluation in Counseling and Development 47 (2014): 79-86.
17. Mousazadeh S, Rakhshan M, Mohammadi F. Investigation of Content and Face Validity and Reliability of Sociocultural Attitude towards Appearance Questionnaire-3 (SATAQ-3) among Female Adolescents. Iran J Psychiatry 12 (2017): 15-20.
18. http://chronic.doh.gov.ph/forms/ICNCDRS%20Manual%20of%20Operations.pdf.
19. The Barbados National Registry for Chronic Non Communicable Diseases. Available from: http://www.bnr.org.bb/cms/.
20. Management of acute coronary event registry. Indian council of medical research.
21. Registry of People with Diabetes with Young Age at Onset (YDR). Indian Council of Medical Research 2006-2011. http://www.icmr.nic.in/final/diabetes/Consolidated%20REport.pdf
22. National Stroke registry programme. Indian council of medical research.
23. SITS International. Safe Implementation of Treatments of Stroke. https://www.sitsinternational.org/
24. Population Based Cancer Registries at Chandigarh and SAS Nagar, Sangrur, Mansa Districts Punjab State. India: (2016). http://pbhealth.gov.in/Punjab%20PBCR%20summary%2023%20Feb%202016.pdf
25. Data Quality and Indices of reliability, Chapter 9. Three-year report of Population based cancer registries 2009-2011. Accessed from www.ncrpindia.org
26. Polit DF, Beck CT, Owen SV. Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Res Nurs Health 30 (2007): 459-67.
27. Almanasreh E, Moles R, Chen TF. Evaluation of methods used for estimating content validity. Res Social Adm Pharm 15 (2019): 214-221.