- Research article
- Open Access
- Open Peer Review
Adaptation and psychometric properties of the Norwegian version of the heart continuity of care questionnaire (HCCQ)
BMC Medical Research Methodology volume 19, Article number: 62 (2019)
Continuity of cardiac care after hospital discharge is a priority, especially as healthcare systems become increasingly complex and fragmented. There are few available instruments to measure continuity of cardiac care, especially from the patient perspective. The aim of this study was (1) to translate and adapt the Heart Continuity of Care Questionnaire (HCCQ) to conditions in Norway, and (2) to determine its psychometric properties in self-report format administered to patients after percutaneous coronary intervention (PCI).
The HCCQ was first translated into Norwegian from the original English version, following a widely used cross-cultural adaptation process. Data were collected before hospital discharge and in a follow-up after 2 months. To assess psychometric properties, a confirmatory factor analysis (CFA) was performed and three aspects of construct validity were evaluated: structural validity, hypotheses testing and cross-cultural validation. Internal consistency of the HCCQ subscales was calculated using Cronbach’s alpha, while intra-class correlation (ICC) was used to assess test-retest reliability. Additionally, socio-demographic and patient-reported data were collected to correlate with HCCQ scores.
Of those included at baseline, 436 (76%) completed the questionnaires after 2 months. CFA suggested that the fit of the HCCQ data to a 3-factor model was modest (RMSEA = 0.11, CFI = 0.90, TLI = 0.90). However, convergent validity was satisfactory, based on existing research. Internal consistency was good, as indicated by its Cronbach’s alphas: total continuity of care (0.95); informational (0.93), relational (0.87), and management (0.89) continuity. The ICC for the total HCCQ score was 0.80 (95% CI [0.71, 0.87] p < 0.001). As indicated by negative care experiences (rated as 1 or 2 on the five-point scale), patients seemed to have limited knowledge about medical treatment, lifestyle modification and follow-up after PCI. Participation in cardiac rehabilitation and longer consultations with the general practitioner after hospital discharge were positively correlated with better continuity of care.
Implementation of the HCCQ will likely support healthcare providers and researchers in identifying problem areas of continuity of cardiac care and in evaluating interventions aimed at improving continuity of care.
Continuity of care among different healthcare providers and from one healthcare setting to another is a priority for patients, healthcare providers and policymakers. It is becoming a major concern as healthcare systems become increasingly complex and fragmented [1,2,3,4].
Percutaneous coronary intervention (PCI) is the most widely performed procedure to treat patients with coronary heart disease [5, 6]. The treatment does not end with the PCI procedure, as continuity is a critical component of secondary prevention and favorable outcomes . In addition, as healthcare policy is increasingly focusing on minimizing length of hospital stay, a trend has emerged in which patients are discharged earlier from hospital, with the recovery process being followed up in the primary care . Thus, in a field where interventional cardiology technology continues to develop rapidly, continuity of care is a major concern for PCI patients.
Several definitions have been proposed to describe the concept of continuity of care [3, 7,8,9]. In a literature review, Haggerty et al.  identified three types of continuity: informational, relational, and management continuity. Informational continuity refers to the use of information from previous events to provide adequate care to the patient. Relational continuity is described as the ongoing relationship between a patient and one or more healthcare providers. Management continuity is viewed as the provision of complementary healthcare services with shared management. Even though continuity of care is important and a priority within healthcare, there are few instruments available to assess patients’ experiences with multiple dimensions of continuity of care [3, 8, 11, 12]. Such a measure has the potential to guide quality improvement initiatives related to continuity of care [3, 7, 9].
Four promising instruments have been employed to measure patients’ experiences with continuity of care; one of these is the Heart Continuity of Care Questionnaire (HCCQ) . The HCCQ was the first to measure multiple dimensions of continuity of care specifically among cardiac patients. It also corresponds well to the three aspects of continuity of care that Haggerty et al. identified . HCCQ has been reported to be a comprehensive, valid and reliable instrument for measuring continuity of care from the patient perspective in patients with congestive heart failure, atrial fibrillation or patients hospitalized for acute coronary syndrome [12, 14, 15]. Despite this, further attention on validity and reliability is warranted  to determine whether the HCCQ can be administered in a self-report format .
The use of a single instrument in cardiac care can help to crystallize conceptualizations of continuity of care, and as healthcare research has increasingly become international in scope it can also facilitate comparisons among findings in different countries . The cross-cultural adaptation of an instrument requires a specific methodology to reach equivalence between the original source and target languages [18, 19]. The aim of this study was (1) to translate and adapt the HCCQ to conditions in Norway, and (2) to determine its psychometric properties in self-report format administered to patients after PCI.
This methodological sub-study was part of a larger prospective multicenter, register-based study, the CONCARDPCI. The study population for this paper included patients from one university hospital. Procedures were consistent with ethical guidelines of the World Medical Association, Helsinki declaration . Patients gave written informed consent. Confidentiality and the right to withdraw from the study were assured. The methodological sub-study was approved by the Norwegian Regional Committee for Ethics in Medical Research (REK 2015/57).
The study included 436 patients of 571 included at baseline from June 2017 to March 2018.
Inclusion criteria were patients undergoing PCI, hospitalized 2 months earlier, ≥ 18 years of age, living at home at the time of inclusion, and having answered at least one baseline HCCQ question. Exclusion criteria were unable to speak Norwegian or unable to fill out the questionnaire due to reduced capacities. Institutionalized patients or patients who might likely die in less than a year were excluded. Additionally, patients undergoing PCI without stent implantation and patients undergoing PCI related to transcatheter aortic valve implantation or MitraClip® were not included. Finally, patients who were later readmitted to hospital were also excluded.
Retrospective baseline self-reports of demographic information and certain clinical data were obtained after PCI, but before discharge from hospital. Two months after discharge, questionnaires were mailed by post to all patients included in the study. The two-month time interval was chosen to allow for sufficient follow-up care and so that patients could give an adequate evaluation of early post-discharge continuity of care. A pre-stamped envelope was included and non-responders were reminded once if they did not response within a certain period of time. Pilot testing was done with a version of the HCCQ with 55 PCI patients before using the final adapted instrument in the main cohort study. A random sub-group of 95 patients was approached for a HCCQ test-retest after 2 weeks .
Cross-cultural adaptation of HCCQ
A cross-cultural adaptation was completed to reach equivalence between the original source and the target Norwegian version [18, 21]. The translation process was conducted systematically in six steps as described by De Vet et al.  and Beaton et al. .
In any research where different cultures are involved, systematic measurement bias may occur that affects the results of the study. Polit and Yang et al.  introduced five levels of cross-cultural equivalence; a) conceptual equivalence concerns the extent to which the construct of interest exists in another culture and whether the construct has similar meaning, b) content equivalence concerns the cultural relevance of individual items for the focal construct within the culture under consideration, c) semantic equivalence is the extent to which the meaning of an item is the same in the target culture after translation as in the original version, d) technical equivalence concerns the equivalence of assumptions about the methods of instrument administration, and e) measurement equivalence concerns the comparability of various measurement properties in the original and translated version of a scale.
A critical review of the equivalence was discussed with an expert group including a physician in community medicine, a professor of cardiac nursing, two cardiac nurse specialists, and a professional translator. These reviewers had experience with translation procedures and were knowledgeable about continuity of care and issues in cardiology. In addition, a patient representative identified in collaboration with the Norwegian Heart and Lung Foundation provided input to the planning of the study and was also involved in the translation process. When asking the patient representative to comment on the instrument in general, the experts employed a cognitive pretesting framework .
Figure 1 provides a summary of the overall translation procedure.
Step 1. Two forward translations of the English HCCQ were made. The experts selected two bilingual translators who had the target language (Norwegian) as their mother tongue. The translators worked independently, and both wrote a report that identified challenging phrases and described their rationale for final translation choices. Examples of difficult words or phrases to translate into Norwegian were “satisfied with the level of care,” “overall treatment plan,” “dietary needs,” and “open line of communication.” The two translations (TL1 and TL2) were compared and discrepancies were identified.
Step 2. The experts synthesized TL1 and TL2 into one consensus version (TL3) and described how they resolved discrepancies. Furthermore, the experts asked two healthcare providers familiar with the cardiac patient population to evaluate TL3 from a clinical perspective and to evaluate its face validity. Their recommendations were used to shape the final HCCQ. The experts discussed any differences in the translation and selected the best and most accurate version of the translated instrument.
Step 3. To further ensure the accuracy of the HCCQ, two persons who had a good understanding of English who also spoke Norwegian fluently independently translated TL3 back into English (TL4 and TL5). This back-translation process provided critical feedback on the vocabulary used in TL3 and achieved different aspects of cross-cultural validity.
Step 4. The experts and a professional translator synthesized TL4 and TL5, and agreed on a modified Norwegian version of the HCCQ (TL6). The experts also discussed semantic and technical equivalence with the original developer of the instrument from Canada: for example, timing of administration, scoring of the results, and meaning of certain words and items in the original version. The main challenge when translating the English HCCQ was to re-structure sentences from English to Norwegian to produce readily comprehensible questions. In order to increase the understanding of the items a few words were changed or small changes were made to the items of the questionnaire.
Step 5. A pre-final version (TL6) was sent to 100 patients, of which 55 answered. The responses were explored concerning how each person interpreted the items in the instrument and in particular identifying the proportion of missing items. After patients had completed the instrument, they were asked to answer six questions identifying words and phrases that might be difficult to understand and commented on the overall impression of the instrument.
Step 6. The experts evaluated the adapted HCCQ (TL6) and the patients’ experiences and answers to the six questions. In the pre-final version, 3–7% had missing data on at least one question. Five patients thought some questions were repeated and that the instrument was too extensive. None of the patients reported any difficulties understanding the response categories. Two patients found it challenging to answer items on whether healthcare providers communicated effectively after discharge. Three patients did not know which hospital to evaluate for the questions, since they had visited several. Therefore, the experts decided to separate the HCCQ into two sections: one section addressing care before discharge, and another section addressing care after discharge. Additionally, they decided to elaborate on the introduction to the HCCQ to clarify which situation(s) should be evaluated. After the translation and adaptation process, the instrument was evaluated for psychometric properties and measurement equivalence.
The heart continuity of care questionnaire (HCCQ)
The original English HCCQ is a 33-item self-report questionnaire that assesses patients’ experiences with continuity of cardiac care. It measures this continuity along three dimensions: informational (17 items), relational (10 items), and management (6 items) [12, 15]. From the perspective of the patient, the self-report instrument covers major topics in cardiac care: heart condition explained, communication among healthcare providers, preparation for discharge, post-hospital care, post-hospital review of treatment, consistent information, information on medications, and knowledge on physical and dietary needs. Items were rated on a 5-point Likert-type scale from 1 (strongly disagree) to 5 (strongly agree), as well as the option to choose ‘not applicable’. Missing data was handled with half rule; using the mean of the answered items in the subscale, if at least half of that subscale had been answered . The English version of the HCCQ is reported to be comprehensive, valid, and reliable. Cronbach’s alpha for the total instrument was 0.95 and the internal consistencies for subscales ranged from 0.80–0.93 [12, 15].
Nordic patient experiences questionnaire (NORPEQ)
NORPEQ is a brief eight-item tool that measures important aspects of patients’ experiences with healthcare interactions . Here, the six items describing concern patients’ experiences with healthcare providers were used. These items assessed whether information provided by physicians was understandable, physicians’ and nurses’ professional skills and nursing care, whether the physicians and nurses were interested in the patients’ problems, and information related to diagnostic tests. Items are rated on a 5-point Likert-type scale from 1 (not at all) to 5 (to very large extent). The scores on these six items are summed to produce an overall subscale score ranging from 0 to 100, where 100 indicates the best possible experience of care . Missing data were handled using the half-rule . NORPEQ has good validity and reliability showing a Cronbach’s alpha of 0.85 and is recommended for cross-national comparisons of healthcare experiences in Nordic countries .
The patient experiences during hospitalisation in somatic hospitals
National surveys are carried out regularly concerning patients having received inpatient specialist health care at Norwegian hospitals. The goal is to obtain information on patients’ experiences when hospitalized in somatic hospitals . In this study, a single item from this survey was used: “Do you feel that the hospital has cooperated well with the general practitioner about what you have been hospitalised for?” The item was rated on a 5-point Likert-type scale, ranging from 1 (not at all) to 5 (to a very large extent).
World Health Organization quality of life abbreviated (WHOQOL-BREF)
The WHOQOL-BREF is a 26-item scale that assesses a person’s perception of quality of life . The World Health Organization (WHO) defines quality of life as follows: “…individuals’ perception of their position in life in the context of the culture and value systems in which they live, and in relation to their goals, expectations, standards and concerns.” One item that is a global measure of overall quality of life was used: “How would you rate your quality of life?” This item was rated on a 5-point Likert scale, ranging from 1 (very poor) to 5 (very good). WHOQOL-BREF has acceptable psychometric properties in the Norwegian population , and has previously been used to assess quality of life in patients with coronary heart disease .
RAND 12-item short form health survey (RAND-12)
RAND-12 is an abbreviated version of the RAND-36 . The 12-item generic self-report instrument was developed to reproduce the physical and mental component summary scores of the RAND-36. The RAND-12 has three to five response levels, with higher scores reflecting better self-reported health. The RAND-12 is a valid and reliable instrument when used in the Norwegian population [29, 30].
Characteristics of the study population
Socio-demographic data (marital status, education, work status) were obtained by self-report, along with questions developed specifically for this study: duration of hospital stay, participation in cardiac rehabilitation (CR), first meeting with a general practitioner (GP) after PCI and how long the patient has been with their current GP.
Expected relationships and subgroup means
The hypotheses were formulated in advance, i.e. before data collection based on an underlying theoretical model, expected differences between subgroups of patients and relationships with similar constructs. It was hypothesized that the sub-scales of the HCCQ instrument would reproduce the dimension structure defined by Haggerty et.al. . Furthermore, it was hypothesized that patients’ CR participation are positively related to HCCQ scores based on Riley et al.’s findings . They found that patients had better perceptions of continuity of care if they participated in CR. Although there is insufficient evidence linking patients’ experiences of continuity of care with other characteristics, it was expected that scores would differ according to the sociodemographics of the patients (e.g., age, educational level, gender and cohabitation status) [14, 31, 32]. It was expected that patients who had longer hospital stays would have lower HCCQ scores . Conversely, it was expected that patients who consulted their GP soon after their hospital discharge would have higher HCCQ scores [34, 35]. As indication of construct validity, HCCQ was expected to have a positive moderate association with NORPEQ and the item measuring patients’ experiences with cooperation between the hospital and their GP. The correlation between the HCCQ and RAND-12 was predicted to be weak (r = 0.10–0.30), as these are thought to assess two different constructs .
Descriptive statistics were used to summarize patients’ sociodemographic characteristics, clinical data, and HCCQ scores. Item means, standard deviations, missing rates and the percentage of “not applicable” for each item were calculated, although items were ordinal, to be able to compare with the original English version. Similarly to Hadjistavropoulos et al.  items that had mean score below 3.75 and at least 25% of the patients indicating negative care experiences (rated as 1 or 2 on the 5-point scale) were identified. Floor and ceiling effects were estimated. Non-parametric tests were used for ordinal variables and parametric tests for continuous variables. Continuous variables were characterised by means and standard deviations. Pearson correlations were used between continuous variables, while Spearman correlations were used when ordinals variables were involved. A strong correlation was operationally defined as r > 0.70, moderate to substantial as 0.30–0.70 and weak as < 0.30, in absolute value .
In general, three different types of validity can be distinguished: content validity, criterion validity and construct validity. Criterion validity involves comparing the newly developed measure to “a gold standard” [18, 22]. Currently there is no validated instrument for comprehensive assessment of continuity of care in Norway. In the absence of such a gold standard, a reasonable alternative is to compare the HCCQ with existing instruments having similar constructs. Three aspects of construct validity were evaluated for the HCCQ: hypotheses testing (convergent/discriminant validity), structural validity, and cross-cultural validation. Convergent validity was tested by correlating the HCCQ instrument with RAND-12 and NORPEQ, using Pearson correlation coefficients. The association between HCCQ and gender was evaluated by using an independent t-test, while between HCCQ and age by using Pearson correlation.
According to Polit and Yang (2016) five types of factorial invariance can be assessed; dimensional, configural, metric, scalar and strict factorial. Such tests require raw data from both groups being compared. However, data from the original English version was not available why dimensional and configural invariance were evaluated using data from the adapted/translated scale. Confirmatory factor analysis (CFA) was used for evaluating the three-factor structure of the original HCCQ instrument. To do this, the following fit indices were used: (a) the root mean squared error of approximation (RMSEA) (preferably < 0.06); (b) Tucker-Lewis index (TLI) (preferably > 0.95); and (c) comparative fit index (CFI) (preferably > 0.95) . A weighted least squares estimator (WLMSV) was used. The WLMSV is a robust estimator that does not assume normally distributed variables and is appropriate for ordinal variables .
There were three hypothesized continuity of care factors, which were included as latent variables underlying the variation and covariation between the observed variables. This hypothesized model for these relationships is presented schematically in Fig. 2. Based on theory and empirical research, the factors are called informational continuity (factor 1), relational continuity (factor 2), and management continuity (factor 3).
Because explorative factor analysis performed by Hadjistavropoulos et al.  showed that item 14 and 15 of the original HCCQ had cross loadings on more than one factor, a second model was constructed to include these cross loadings. In the first model, items are related to one factor only, while in the alternative model two items (item 14 and 15) are both related to informational and relational continuity. The two models were estimated and compared with the results of Hadjistavropoulos et al. . Furthermore, an additional analysis was carried out in which item 14 and 15 were removed from the model. In Fig. 2, the arrows from the factors to the items represent factor loadings. The bi-directional arrow between each of the three factors indicates our assumption that the three factors are intercorrelated.
Internal consistency of the HCCQ was evaluated using Cronbach’s alpha for total continuity of care and for each extracted domains; alpha values of > 0.70 were considered to reflect satisfactory internal consistency. Test-retest reliability was evaluated by using intraclass correlation (ICC) coefficients of 95 patients’ results obtained at a 2-week retest interval . The ICC for agreement was used to get the absolute agreement between repeated measurements. For assessing measurement error, limits of agreement (Bland and Altman method)  and standard error of measurement were used . Reliable change was estimated using the smallest detectable change (SDC).
Little et al.  recommend sample sizes that are a bit larger than 100 observations for single factor structural equation models (SEM). However, several factors influence the sample requirements of an SEM model, including the quality of the data to produce accurate estimates of the sufficient statistics, heterogeneity and representativeness of the sample, precision of the instrument, and model complexity . A sample size of 436 was considered adequate for the CFA. A two-sided p-value of < 0.05 was considered to be statistically significant. SPSS (IBM Corp. Released 2016. IBM SPSS Statistics for Windows, Version 24.0. Armonk, NY: IBM Corp.) was used for calculating summary statistics and correlations, and for conducting statistical comparisons. Mplus (Computer software, 1998–2018, version 7) by B.O. Muthén, L.K. Muthén, was used to perform SEM.
Characteristics of study population
Of the 571 patients included at baseline, 436 (76%) completed the questionnaires at the 2-month post-discharge assessment. These 436 patients ranged in age from 30 to 92 years, with a mean age of 66 years (Table 1). A total of 74% of patients were men, 82% lived with others, and 28% had completed a high school education. With regard to employment, 33% worked full time and 51% were retired. The majority of the patients (57%) were hospitalized for 3 days or less, and 3 out of 4 were discharged to home directly.
Item and sum score analysis of HCCQ
Descriptive statistics of the 33 items of the HCCQ are presented in Table 2. Several items on the HCCQ had mean of < 3.75 (i.e., rated 1 or 2), indicating negative care experiences. For the response categories “strongly disagree” and “somewhat disagree,” on several items of patients’ care experiences stand out as frequent. For instance, 56% of the patients stated that they were not adequately informed about how their heart condition would influence on their lifestyle, and 54% of the patients stated that they were not adequately informed about the types of physical activity they should engage in or avoid. Similarly, 54% of patients reported that their treatment had not been adequately reviewed by their physician following discharge. The mean subscale score on the HCCQ for informational continuity was 3.26, for relational continuity 3.69, and for management continuity 2.49. The total mean score was 3.24 (SD = 0.82) (Table 3). The HCCQ total floor effect was 0% and ceiling was 1.7%. On sub-scales, information continuity floor was 0% and ceiling 4%, relational continuity floor was 0.5% and ceiling 6.8%, and management continuity floor was 4% and ceiling 4.5%.
Table 4 presents results of the CFA of the HCCQ. The result of the first CFA was as follows: Chi-square (χ2)/ Degree of freedom (df) =3047/492; RMSEA = 0.11; CFI =0.90 and TLI = 0.89. Standardized factor loadings ranged from 0.57 to 0.97 (p < 0.001). In the alternative model (Table 4) with items 14 and 15 related to both informational and relational continuity, Chi-square (χ2)/ Degree of freedom (df) was somewhat reduced at 2969/490, but the fit indices did not appreciably improve; the RMSEA = 0.11; CFI = 0.90 and TLI = 0.90. Apart from items, 14 and 15, standardized factor loadings ranged from 0.60 to 0.97; for items 14 and 15, the loadings ranged from 0.28 to 0.43 (all p < 0.001). The fit indices reflected the fact that the structure was not well represented by the hypothesized 3-factor model . The additional analysis, in which item 14 and 15 were removed, did not improve the fit appreciably: Chi-square (χ2)/ Degree of freedom (df) =2712/431; RMSEA = 0.11; CFI =0.91 and TLI = 0.90. Standardized factor loadings ranged from 0.58 to 0.97 (p < 0.001).
Convergent and discriminant validity
Table 5 presents group statistics and correlations between HCCQ domain scores and values on patient-reported variables. It was hypothesized that domain scores might have systematic relationships with certain sociodemographic variables, such as age, gender, education level attained, and cohabital status, etc. [31, 32]. First, a significant difference between genders was found, with men scoring higher on items related to informational and relational continuity. Patients living alone scored lower on informational continuity, although not significant. It was also hypothesized that patients’ CR participation would have a positive impact on HCCQ scores ; the analyses revealed that patients who engaged in CR reported more positive experiences in terms of relational and management continuity compared to those who did not participate in CR.
There was also a weak positive correlation between management continuity and duration of hospital stay (r = 0.16). Patients who met with their GP soon after discharge reported better relational continuity (r = 0.13) and management continuity (r = 0.19). There were also weak-to-moderate positive correlations (r = 0.19 to r = 0.39) between the HCCQ and whether the patients felt they spent enough time in consultation with their GP. Another finding indicated that there were moderate positive correlations between continuity of care and the item identifying cooperation between the hospital and GP (r = 0.40 to r = 0.61). A weak-to-moderate positive correlation between the HCCQ and the score derived from the six items from the NORPEQ (r = 0.16 to r = 0.42) was found. The analyses also revealed a weak positive correlation between informational and relational continuity and quality of life (WHOQL-BREF). Furthermore, weak correlations were found between self-reported health (RAND-12) and informational and relational continuity. There were no significant associations between continuity of care and age, education, and length of treatment with their current GP (p ≥ 0.192).
Cronbach’s alpha for the three domains of informational, relational, and management continuity were 0.93, 0.87, 0,89, respectively; for the total scale Cronbach’s alpha was 0.95. The corresponding Cronbach’s alpha values at the 2-week retest were 0.94, 0.92, 0.92, and 0.96. The mean inter-item correlations within the HCCQ scale scores were 0.44, 0.40, and 0.59; at retest the correlations were 0.50, 0.54, and 0.66. The item-scale correlation was between 0.45–0.70. The ICC for informational continuity was 0.72 (95% CI [0.60, 0.81] p < 0.001); for relational continuity 0.84 (95% CI [0.77, 0.90] p < 0.001); for management continuity 0.82 (95% CI [0.73, 0.88] p < 0.001); and for the total score it was 0.80 (95% CI [0.71, 0. 87] p < 0.001). The mean systematic difference (dashed line in Fig. 3) was - 0.027. The limit of agreement was - 1.025 to 1.025. This difference was not statistically significant (p = 0.964). Standard error of measurement was 0.28 and SDC was ±1.05.
The psychometric properties of the Norwegian version of the HCCQ were evaluated for patients after PCI. The instrument had face and content validity by including relevant topics for cardiac care, and associations between similar and dissimilar scales showed satisfactory figures. However, CFA showed that the hypothesized model was not entirely adequate.
In order to meet the challenge of continuity of care, the healthcare services should involve patients to a greater extent when developing and evaluating the future healthcare [9, 32]. While patient integration contributes to achieving continuity of care, few tools to date have been developed and comprehensively validated to measure continuity of care from the cardiac patients’ perspective [3, 13]. This is especially the case for evaluating continuity of care in Norwegian healthcare systems.
In this study, the aim was to translate and adapt one such tool, the HCCQ [11, 12] and to determine its psychometric properties. The instrument will be helpful in providing new insights into the analysis of continuity of care across and within various healthcare levels. The results complement previous evaluations of the psychometric properties of the HCCQ, in this case for PCI patients interacting with the Norwegian healthcare system. The translation process included forward and backward translations and a pilot test, which produced an accurate translation and readily comprehensible questions that facilitated understanding of the Norwegian context. This comprehensive adaptation and pre-testing effort built a stable foundation for a valid psychometric evaluation. A test-retest ICC evaluation of the HCCQ confirmed that the measures were stable over time. CFA was used to evaluate the 3-factor structure of the HCCQ based on previous analysis . This study showed that the translated and adapted version of the HCCQ would have good internal consistency in terms of informational, relational, and management continuity of care.
The literature search revealed that the HCCQ is suitable for measuring continuity of care in PCI patients. The expert group convened for the present study evaluated the content validity  of the HCCQ and determined it would be adequate for measuring the construct. Face validity  and other feasibility factors (e.g., ease of administration, usefulness) of the HCCQ were determined to be adequate by PCI patients and healthcare providers familiar with cardiac patients. Thus, it was deemed appropriate to translate and adapt the HCCQ for PCI patients speaking Norwegian. Cross-cultural instrument translation is a complex task that cannot be undertaken lightly without the risk of producing poor-quality adaptation . The cross-cultural validation started with a translation process systematically following international guidelines [18, 21].
Importantly, not all words and phrases in the original English version of the HCCQ were easy to translate into comparable Norwegian words and phrases. Thus, the focus was to translate items that reflected the same concepts and were meaningful, clear, and relevant for the Norwegian context. Furthermore, it is important to translate the administrative and role-based hierarchies found in organizations . For example, interdisciplinary primary health care teams have been established in provinces and territories in Canada. Additionally, the pharmacists play a more active role as part of the primary care team to ensure proper adherence to medications in the Canadian than in the Norwegian healthcare system  Therefore, some words in the instrument might be more relevant in the Canadian population. Squires et al.  suggest a more systematic approach for standardizing language translation processes including content validity indexing techniques.
The instrument was pre-tested with PCI patients, and these patients reported that the instrument was comprehensive and acceptable. However, a few patients thought the instrument was too extensive, suggesting that a shorter version may be more useful. In addition, a few patients in the pilot study reported that it was difficult to answer the item on whether healthcare providers communicated effectively after hospital discharge. With the final version of the HCCQ, 47 patients (11%) answered “not applicable” to this effective communication item, and 26 (6%) failed to answer the question. The literature suggests that patient perceptions related to continuity of care are strongly tied to how healthcare providers communicate with one another, both within and across sectors . However, continuity in the delivery of care might not be visible to patients until they experience gaps in the quality of care given . Furthermore, 14% of the patients reported that they had not visited their GP after discharge and were not able to judge these aspects of care. In this study patients were followed up 2 months after discharge in order to get solid experiences of the topic continuity of care and further on to avoid recalling bias. Nevertheless, the optimal time frame for evaluation of post-discharge continuity of care is currently unknown and requires further research.
Structural validity in psychometrics is defined as the degree to which patients’ scores on an instrument are an adequate reflection of the dimensionality of the construct being measured . This can be assessed by factor analysis and CFA is preferred if a priori hypotheses about dimensions of the construct are available based on theory or previous analysis. With regard to the construct of continuity of care, Haggerty et al.  distinguished three dimensions, namely relational, informational, and management continuity. The HCCQ subscales correspond well to this three-factor model . After the specified model for the HCCQ was identified, the model fit was determined. Among the resulting fit indices, the RMSEA was 0.11 (CI = 0.10–0.11), indicating a somewhat poor fit . Two other indices used were CFI and TLI, which both measure the improvement in model fit when comparing the hypothesized model with a less restricted baseline model. Usually, a value greater than 0.90 is considered to be adequate, and 0.95 or greater is considered good. For this analysis of the HCCQ the CFI was 0.90 and the TLI was 0.90; values close to 1.0 indicate a well-fitting model. If a group of indexes provide contradictory indications about model fit, it is usually necessary to carefully re-evaluate the model. In this case, no empirically proposed modifications were regarded as reasonable. That being said, however, most disciplines recognize three types of continuity of care, as one interdisciplinary review of concepts and measures of continuity of care highlighted [10, 46].
An exploratory study of PCI patients showed that the HCCQ might not cover the overall set of items relevant to explaining continuity of care . For example, some patients expressed that healthcare providers showed different levels of concern and interest about their cardiac disease, and they seemed to believe that ongoing relationships based on trust and confidence were important . When the HCCQ was adapted to a generic population, items were added to better address relational continuity, including patients’ perceptions of satisfaction with emotional support, opportunity to discuss and ask questions, confidence in healthcare providers, sense of being understood, and feeling known by healthcare providers . Continuity of care also includes patients’ healthcare experiences over time being perceived as being coherent and linked . The HCCQ focuses on how patients experience the collaboration between specialist healthcare and GPs in primary healthcare . Thus, issues important to the service delivery structure that directly relate to the patient, such as rehabilitation services and other supportive services, are less well addressed by items in the original HCCQ. These issues might also be of relevance in the adaptation and translation of the HCCQ into a Norwegian context and thereby affecting the structural validity.
The basic principle of construct validity is that hypotheses are formulated about associations of scores on the HCCQ with scores on other instruments measuring similar or dissimilar constructs, or differences in the instrument scores between subgroups of patients . In this study, men scored better than women on relational and informational continuity (Table 5). There were no appreciable correlations between the HCCQ and age and education . This finding is inconsistent with an earlier study suggesting that patients with higher education have higher expectations and judge quality of care more critically . The study found a positive association between duration of hospital stay and management continuity of care. The shortened length of stay for patients after PCI, might limit the opportunities to achieve timeliness and complementarity of services .
With regard to hypothesis testing, patients who engaged in CR had more positive scores on management and relational continuity, similar to that in other studies . This highlights that CR provides management continuity by timely, complementary services . The period between hospital discharge and start of CR is very stressful for patients, and consequently CR organized by trained healthcare providers might support the experience of relational continuity . In line with a previous study, the time elapsed between patient discharge and first access to a GP is associated with relational continuity . Another significant finding was that having enough consult time with their GP after hospital discharge positively correlated with informational, relational, and management continuity. GPs are key collaborating partners in the healthcare system, and their ability and willingness to collaborate with patients are affected by organizational conditions . Studies show that patients believe good communication with their physician requires sufficient time and quality of consultations [2, 50, 51]. However, the study did not find a substantial correlation between continuity of care and length of relationship with GPs. A previous study show that not all patients receiving care from a single GP thought they had a good personal relationship with the physician .
As expected, significant correlations between HCCQ and the six items in NORPEQ and the item regarding cooperation between hospital and GP were found. This is not surprising, since these two instruments have similar constructs as the HCCQ with regard to patients’ experiences with healthcare providers in hospital . The study also confirmed previous findings that the HCCQ and RAND-12 are weakly correlated . Generic instruments, such as the NORPEQ, are particularly useful for comparing outcomes for a wide range of patient groups. On the other hand, there is valid concern that they do less well at capturing areas of importance to specific patient populations . The HCCQ is a disease-specific instrument and more clinically relevant for cardiac patients . Moreover, questioning patients directly on actual experiences they had with the healthcare they received seems to describe the quality of care better than asking patients about satisfaction with the care .
The internal consistency of the three domains of the HCCQ, as assessed with Cronbach’s alphas, were comparable with that reported by Riley et al.  and that reported by Hadjistavropoulos et al. . These similarities confirm that HCCQ has satisfactory internal consistency and the instrument is stable over time . The Bland-Altman diagram showed a mean difference line close to zero and one lower and upper limit that are close to the mean difference, indicating substantial precision (less measurement error). Additionally, there were no major outliers and similar differences above and below the mean difference . The standard error of measurement was also calculated and is not as affected as reliability coefficients by the sample within which the estimate is computed .
Individual item analysis of the HCCQ was particularly illuminating for identifying areas of concern to patients that would benefit from institutional review and quality improvement initiatives (cf. Table 2). The study highlights that achieving continuity of care for patients after PCI is challenging. For example, patients are not necessarily receiving adequate information about their medical treatment, possibilities for positive lifestyle modifications, and status of their physical condition. Additionally, patients were not being given consistent information about symptoms and when to contact healthcare providers.
Finally, the importance of patient participation in their healthcare, including sharing information, is highlighted in the most recent ESC/EACTS guidelines on Myocardial Revascularization . Previous research indicates that healthcare providers at hospitals might ignore the critical component of post-hospital discharge care and the transition to home planning process [5, 53]. A global trend is for patients to discharge earlier after procedures, meaning there is reduced time for in-hospital education . Moreover, patients are less receptive to learning. Inadequate education and poor discharge planning also seem to decrease patients’ adherence to treatment and prescribed lifestyle changes [1, 5].
Overall, the present study provides preliminary evidence that the HCCQ can highlight deficiencies in patients’ experiences of continuity of care across and within care levels, valuable information to help identify areas for healthcare improvement.
Limitations of the study
The analysis of the translated and adapted HCCQ was specific to patients who recently underwent PCI and need to be tested for feasibility in self-report format for patients with other chronic cardiac conditions. It may also be beneficial to study the HCCQ using other modes of administration. In this study, the instrument was sent by post, but it is also possible that an equivalent form could be developed for online use, smart phones, or e-mail. CFA was used to determine the measurement equivalence of adapted and original measures. However, other relevant analyses could have been performed if there had been access to comparative data from the original instrument. Moreover, this study mainly focused on cross-cultural approaches for translating instruments and more research is needed to compare the HCCQ across healthcare systems and organizations. Additionally, future research should identify a cutoff score for each of the subscales that would indicate problems in continuity of care. The optimal post-discharge period for evaluating continuity of care can also be assessed systematically in future research. Recall bias at longer periods might distort or limit the amount of information that can be gained from patients. The HCCQ is only appropriate for patients who have no, or only minor, difficulties with communication. Follow-up of the patients is very important in a cohort study, and losses are an important source of bias in these types of designs. However, the CONCARD team thought carefully about the study population, and planning was given priority to avoid errors in sampling.
The present study provides additional information on the psychometric properties of the HCCQ for patients after PCI. Cross-cultural validation was performed according to international guidelines. The internal consistency of the HCCQ was high, and ICC showed good agreement. However, the RMSEA suggests that the fit of data to the hypothesized model is not entirely adequate for fully capturing the theoretical components of informational, relational, and management continuity. Nevertheless, hypotheses about the constructs showed satisfactory results based on existing knowledge. For example, the results point to a positive relationship between continuity and CR participation and sufficient consultation time with the GP after discharge from hospital. Although more research is needed on the psychometric properties of the HCCQ, its use in this study identified problem areas in continuity of care, a critical step towards understanding and improving the quality of care.
Confirmatory factor analysis
Comparative fit index
Heart Continuity of Care Questionnaire
Percutaneous coronary intervention
Root mean squared error of approximation
Structural equation modeling
Weighted least squares estimator
Waibel S, Vargas I, Aller M-B, Coderch J, Farré J, Vázquez ML. Continuity of clinical management and information across care levels: perceptions of users of different healthcare areas in the Catalan national health system. BMC Health Serv Res. 2016;16(1):466.
Haggerty RD, Freeman GK, Beaulieu C. Experienced continuity of care when patients see multiple clinicians: a qualitative metasummary. Ann Fam Med. 2013;11(3):262–71.
Quinn M, Robinson C, Forman J, Krein SL, Rosland AM. Survey Instruments to Assess Patient Experiences With Access and Coordination Across Health Care Settings: Available and Needed Measures. Med Care. 2017;55(Suppl 7 Suppl 1):S84–s91.
Giuliano C, Parmenter BJ, Baker MK, Mitchell BL, Williams AD, Lyndon K, Mair T, Maiorana A, Smart NA, Levinger I. Cardiac rehabilitation for patients with coronary artery disease: a practical guide to enhance patient outcomes through continuity of care. Clin Med Insights Cardiol. 2017;11:1179546817710028.
Redfern J, Briffa TG. The transition from hospital to primary care for patients with acute coronary syndrome: insights from registry data. Med J Aust. 2014;201(10):S97–9.
Neumann F-J, Sousa-Uva M, Ahlsson A, Alfonso F, Banning AP, Benedetto U, Byrne RA, Collet J-P, Falk V, Head SJ, et al. ESC/EACTS guidelines on myocardial revascularization. Eur Heart J. 2018, 2018:ehy394.
Uijen A, Schers H, Schellevis F, van den Bosch W. How unique is continuity of care? A review of continuity and related concepts. Fam Pract. 2012;29:264–71.
Ball LE, Barnes KA, Crossland L, Nicholson C, Jackson C. Questionnaires that measure the quality of relationships between patients and primary care providers: a systematic review. BMC Health Serv Res. 2018;18(1):866.
Schang L, Waibel S, Thomson S. Measuring care coordination: health system and patient perspectives. In: Report prepared for the Main Association of Austrian Social Security Institutions. London: LSH Health; 2013.
Haggerty R, Freeman GK, Starfield BH, Adair CE, McKendry R. Continuity of care: a multidisciplinary review. BMJ. 2003;327(7425):1219–21.
Schultz E, Pineda N, Lonhart J, Davies S, McDonald K. A systematic review of the care coordination measurement landscape. BMC Health Serv Res. 2013;13(1):119.
Hadjistavropoulos HD, Biem HJ, Kowalyk KM. Measurement of continuity of care in cardiac patients: reliability and validity of an in-person questionnaire. Can J Cardiol. 2004;20(9):883–91.
Uijen AA, Heinst CW, Schellevis FG, van den Bosch WJ, van de Laar FA, Terwee CB, Schers HJ. Measurement properties of questionnaires measuring continuity of care: a systematic review. PLoS One. 2012;7(7):e42256.
Riley DL, Stewart DE, Grace SL. Continuity of cardiac care: cardiac rehabilitation participation and other correlates. Int J Cardiol. 2007;119(3):326–33.
Kowalyk KM, Hadjistavropoulos HD, Biem HJ. Measuring continuity of care for cardiac patients: development of a patient self-report questionnaire. Can J Cardiol. 2004;20(2):205–12.
Bautista MA, Nurjono M, Lim YW, Dessers E, Vrijhoef HJ. Instruments measuring integrated care: a systematic review of measurement properties. Milbank Q. 2016;94(4):862–917.
Brislin RW. Back-translation for cross-cultural research. J Cross-Cult Psychol. 1970;1(3):185–216.
De Vet HCW, Terwee CB, Mokkink LB, Knol DL. Measurement in medicine : a practical guide. Cambridge: Cambridge University Press; 2011.
Guillemin F, Bombardier C, Beaton D. Cross-cultural adaptation of health-related quality of life measures: literature review and proposed guidelines. J Clin Epidemiol. 1993;46(12):1417–32.
The World Medical Association. WMA: declaration of Helsinki - ethical principles for medical research involving human subjects. JAMA. 2013;310(20):2191.
Beaton DE, Bombardier C, Guillemin F, Ferraz MB. Guidelines for the process of cross-cultural adaptation of self-report measures. Spine (Phila Pa 1976). 2000;25(24):3186–91.
Polit DF, Yang FM. Measurement and the measurement of change : a primer for the health professions. Philadelphia: Wolters Kluwer; 2016.
Skudal KE, Garratt AM, Eriksson B, Leinonen T, Simonsen J, Bjertnaes OA. The Nordic Patient Experiences Questionnaire (NORPEQ): cross-national comparison of data quality, internal consistency and validity in four Nordic countries. BMJ Open. 2012;2:1-10.
Oltedal S, Garratt A, Bjertnaes O, Bjornsdottir M, Freil M, Sachs M. The NORPEQ patient experiences questionnaire: data quality, internal consistency and validity following a Norwegian inpatient survey. Scand J Public Health. 2007;35(5):540–7.
Bjerkan AM, Holmboe O, Skudal KE, Bjertnæs ØA. Inpatients’ experiences with somatic hospitals in 2012. Methodological perspectives. In: Methodological report from the knowledge Centre, vol. 2. The Norwegian Knowledge Centre for the Health Services: Oslo; 2013.
Hanestad BR, Rustoen T, Knudsen O Jr, Lerdal A, Wahl AK. Psychometric properties of the WHOQOL-BREF questionnaire for the Norwegian general population. J Nurs Meas. 2004;12(2):147–59.
Norekvål TM, Wahl AK, Fridlund B, Nordrehaug JE, Wentzel-Larsen T, Hanestad BR. Quality of life in female myocardial infarction survivors: a comparative study with a randomly selected general female population cohort. Health Qual Life Outcomes. 2007;5:58.
Gandek B, Ware JE, Aaronson NK, Apolone G, Bjorner JB, Brazier JE, Bullinger M, Kaasa S, Leplege A, Prieto L, et al. Cross-validation of item selection and scoring for the SF-12 health survey in nine countries: results from the IQOLA project. International quality of life assessment. J Clin Epidemiol. 1998;51(11):1171–8.
Farivar SS, Cunningham WE, Hays RD. Correlated physical and mental health summary scores for the SF-36 and SF-12 Health Survey, V.1. Health Qual Life Outcomes. 2007;5:54.
Garratt AM, Stavem K. Measurement properties and normative data for the Norwegian SF-36: results from a general population survey. Health Qual Life Outcomes. 2017;15(1):51.
Parker G, Corden A, Heaton J. Experiences of and influences on continuity of care for service users and carers: synthesis of evidence from a research programme. Health Soc Care Community. 2011;19(6):576–601.
Aller MB, Vargas I, Vázquez ML. Available tools to comprehensively assess continuity of care from the patients’ perspective. J Clin Epidemiol. 2012;65(5):578–9.
Hadjistavropoulos H, Biem H, Sharpe D, Bourgault-Fagnou M, Janzen J. Patient perceptions of hospital discharge: reliability and validity of a patient continuity of care questionnaire. Int J Qual Health Care. 2008;20(5):314–23.
Sidhu RS, Youngson E, McAlister FA. Physician continuity improves outcomes for heart failure patients treated and released from the emergency department. JACC: Heart Failure. 2014;2(4):368–76.
Valaker I, Norekvål TM, Råholm M-B, Nordrehaug JE, Rotevatn S, Fridlund B. Continuity of care after percutaneous coronary intervention: the patient’s perspective across secondary and primary care settings. Eur J Cardiovasc Nurs. 2017;16(5):444–52.
Byrne BM. Structural equation modeling with Mplus : basic concepts, applications, and programming. New York: Routledge; 2012.
Bland JM, Altman DG. Measuring agreement in method comparison studies. Stat Methods Med Res. 1999;8(2):135–60.
Little TD. Longitudinal structural equation modeling. New York: Guilford; 2013.
Wolf EJ, Harrington KM, Clark SL, Miller MW. Sample size requirements for structural equation models: an evaluation of power, bias, and solution propriety. Educ Psychol Meas. 2013;76(6):913–34.
Aller M-B, Vargas I, Waibel S, Coderch J, Sánchez-Pérez I, Colomés L, Llopart JR, Ferran M, Vázquez ML. A comprehensive analysis of patients' perceptions of continuity of care and their associated factors. Int J Qual Health Care. 2013;25(3):291–9.
Squires A, Aiken LH, van den Heede K, Sermeus W, Bruyneel L, Lindqvist R, Schoonhoven L, Stromseng I, Busse R, Brzostek T, et al. A systematic survey instrument translation process for multi-country, comparative health workforce studies. Int J Nurs Stud. 2013;50(2):264–73.
Hutchison B, Levesque J-F, Strumpf E, Coyle N. Primary health care in Canada: systems in motion. Milbank Q. 2011;89(2):256–88.
Waibel S, Vargas I, Coderch J, Vázquez M-L. Relational continuity with primary and secondary care doctors: a qualitative study of perceptions of users of the Catalan national health system. BMC Health Serv Res. 2018;18(1):257.
Tarrant C, Windridge K, Baker R, Freeman G, Boulton M. Falling through gaps: primary care patients’ accounts of breakdowns in experienced continuity of care. Fam Pract. 2014;32(1):82–7.
Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Equ Modeling. 1999;6(1):1–55.
Reid RJ, Haggerty JL, McKendry R. Defusing the confusion - concepts and measures of continuity of healthcare. Ottawa, ON: Canadian Health Services Research Foundation; 2002.
Berendsen AJ, Groenier KH, de Jong GM, Meyboom-de Jong B, van der Veen WJ, Dekker J, de Waal MW, Schuling J. Assessment of patient's experiences across the interface between primary and secondary care: consumer quality index continuum of care. Patient Educ Couns. 2009;77(1):123–7.
Haggerty JL, Burge F, Pineault R, Beaulieu M-D, Bouharaoui F, Beaulieu C, Santor DA, Lévesque J-F. Management Continuity from the Patient Perspective: Comparison of Primary Healthcare Evaluation Instruments. Healthc Policy. 2011;7(Spec Issue):139–53.
Hill KM, Twiddy M, Hewison J, House AO. Measuring patient-perceived continuity of care for patients with long-term conditions in primary care. BMC Fam Pract. 2014;15(1):191.
Aller MB, Vargas I, Waibel S, Coderch-Lassaletta J, Sanchez-Perez I, Llopart JR, Colomes L, Ferran M, Garcia-Subirats I, Vazquez Navarrete ML. Factors associated to experienced continuity of care between primary and outpatient secondary care in the Catalan public healthcare system. Gac Sanit. 2013;27(3):207–13.
van Walraven C, Oake N, Jennings A, Forster AJ. The association between continuity of care and outcomes: a systematic and critical review. J Eval Clin Pract. 2010;16(5):947–56.
McKenna SP. Measuring patient-reported outcomes: moving beyond misplaced common sense to hard science. BMC Med. 2011;9:86.
Barnason S, Zimmerman L, Nieveen J, Schulz P, Young L. Patient recovery and transitions after hospitalization for acute cardiac events: an integrative review. J Cardiovasc Nurs. 2012;27(2):175–91.
The authors are grateful to Toril Terum and Jarle Bolstad for forward translation and Charlotte Burges and Rob Pugh for backward translation. Furthermore, Elisabeth Johannesen and Einar Hovlid participated in the expert group. The authors are grateful to the CONCARD team for including the patients, and to the patients who agreed to participate and who shared their experiences.
A major grant from the Western Norway Health Authority supports the CONCARD project (grant no. 912184). The first author was financially supported by Western Norway University of Applied Sciences. The funding bodies had no role in the design of the study, data collection, analyses, interpretation of data or in writing the manuscript.
Availability of data and materials
Ethics approval and consent to participate
Patients gave written informed consent. Confidentiality and the right to withdraw from the study were assured. The study was approved by the Norwegian Regional Committee for Ethics in Medical Research (REK 2015/57).
Consent for publication
The authors declare that there is no conflict of interest.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
- Cardiac care
- Confirmatory factor analysis
- Continuity of care
- Cross-cultural adaptation
- Patient perspective
- Percutaneous coronary intervention
- Psychometric properties