The use of Charlson comorbidity index for observational studies using administrative data in Korea:
A scoping review
Article information
Abstract
The number of large-scale epidemiological studies using administrative databases has rapidly increased in recent years. However, in observational studies, the outcomes of interest are heavily influenced by concurrent or pre-existing comorbidities, which act as confounders, and appropriate adjustment of comorbidities is essential to minimize bias. Although several comorbidity indices are available for this purpose, Charlson comorbidity index (CCI) is the most widely used one. The original version of the CCI was developed for use in longitudinal studies, to classify comorbidities that might alter the risk of mortality. It included 19 comorbidities, with each comorbidity having an assigned weight from 1 to 6; the total CCI score is a simple sum of the weights. Although the original CCI was based on a review of medical records, many modified versions have been proposed in subsequent studies using claims data. The CCI and its modified versions are used in virtually all clinical settings, including oncology, cardiology, respiratory care, emergency care, surgery, intensive care, and geriatrics. Appropriate use of a comorbidity index, such as the CCI, is necessary to minimize bias in observational studies in the era of increasing use of administrative data for healthcare research.
INTRODUCTION
In recent years, there has been a rapid increase in the number of large-scale epidemiological studies supported by availability of administrative databases. These observational studies complement experimental studies, such as randomized controlled trials, and play a crucial role in outcome research. However, in observational studies, the outcomes of interest are heavily influenced by concurrent or pre-existing comorbidities, which act as confounders.
Comorbidities can prevent a person from receiving treatment and can lead to selection bias, affect the prognosis, and also act as a competing cause of mortality. Therefore, appropriate adjustment for comorbidities is essential to minimize bias in observational studies. Several comorbidity indices have been developed for this purpose; however, the most widely used is the Charlson comorbidity index (CCI).
In Korea, there was rapid increase in the use of national health insurance data for clinical epidemiology studies, and CCI was often used to consider comorbidities. In this review, we aimed to provide a general concept of comorbidity, briefly describe the CCI and its modified versions, and describe their use in clinical studies. We then discuss some practical issues in applying CCI in actual research, and its research and clinical implications. We also provide evidence for the use of CCI in predicting mortality in the general population in Korean healthcare settings.
CONCEPT OF COMORBIDITY
In 1970, Feinstein [1] first defined comorbidity as ‘Any distinct additional clinical entity that has existed or that may occur during the clinical course of a disease that is under study.’ The term comorbidity was derived from the combination of two Latin words: ‘co’ which means ‘along with’ and ‘morbus’ which means ‘disease’ [2]. However, co-occurrence is not the only criterion, and comorbidity is usually defined as a condition that affects prognosis and outcomes (i.e., prognostic comorbidity). Otherwise, minor conditions, such as ankle sprain, knee pain, or upper respiratory infection would all be included as comorbidities. Comorbidity is also a concept different from overall health status, self-rated health, or performance status, such as the New York Heart Association criteria or the Eastern Cooperative Oncology Group.
From a clinical perspective, comorbidities can affect treatment selection and interfere with therapeutic plans. Treatment selection is determined or influenced by factors other than the disease itself, and age and comorbid illnesses are other important considerations [3].
Prior to the concept of comorbidities, the comparability of patients was judged primarily based on age, sex, race, and anatomic stage, and patients with any comorbid diseases other than the index disease that may confound the outcomes were excluded from clinical studies [2].
Comorbid illnesses significantly impact the outcomes and prognosis of patients. Comorbidities not only affect the overall mortality but can also influence the prognosis of the outcome of interest. Comorbidities not only impact survival outcomes but also predict a variety of health outcomes, such as functional status, quality of life, complications, readmissions, and health care utilization [4]. Therefore, it is essential to consider prognostic comorbidities in studies that evaluate such outcomes.
Since Kaplan and Feinstein [5] devised a system to account for comorbidities, several comorbidity indices have been developed, such as the cumulative illness rating scale (CIRS) [6], Kaplan–Feinstein classification (KFC) [5], CCI [7], index of coexistent disease (ICED) [8], and Elixhauser index [9], with many modified versions [3,10]. Among these, CCI is the most widely used comorbidity index [2], and this review focusses on the CCI.
CCI AND ITS MODIFIED VERSIONS
The original version of the CCI was developed by Charlson et al. [7] for use in longitudinal studies, as a method for classifying comorbidities that might alter the risk of mortality. It consisted of 19 items corresponding to different comorbid medical conditions, which had different clinical weights based on the 1-year mortality, derived from a cohort of 559 patients admitted to the general internal medicine service of the New York Hospital-Cornell Medical Center. The weight of each comorbidity was determined based on the adjusted relative risk of each item for 1-year mortality. Comorbidities with a relative risk < 1.2 were excluded from the index, those with a relative risk of 1.2–1.5 were given a weight of 1, those with 1.5– 2.5 were given a weight of 2, 2.5–3.5 were given a weight of 3, and those with ≥ 3.5 relative risk were given a weight of 6 [7]. The total CCI score is a simple sum of the weights, with higher scores indicating more severe comorbid conditions, and consequently, a greater risk of mortality. This weighted index was tested for its ability to predict mortality risk of comorbid diseases in a different cohort of 685 patients with breast cancer during a 10-year follow-up period. With each increased in CCI score, the cumulative mortality attributable to comorbid diseases increased in a stepwise manner.
Following the original version of CCI, several adaptations were developed. Charlson developed a model of age and comorbidity as a combined age-comorbidity index (age-CCI) and showed that the relative risk of death from an increase of one in the comorbidity score was similar to that from an additional decade of age [11]. Therefore, in this age-CCI, patients who are ≥ 50 years old are given additional points: 50–59 years old: 1 point; 60–69 years old: 2 points; 70–79 years old: 3 points; and ≥ 80 years old: 4 points. This age-CCI has been frequently used in oncology to predict mortality of various lengths, with a cohort of gastrointestinal [12-14], pancreatic [15], gynecologic [16-18], lung [19], and urological cancers [18,20]. It has also been used in other clinical settings, including orthopedic (hip fracture and revision hip arthroplasty) [21,22] and dialysis patients [23].
While the original CCI was based on a review of medical records, subsequent studies using claims data proposed a modified version of the CCI by applying standardized diagnostic codes, i.e., the International Classification of Disease 9th Revision (ICD-9) (e.g., Deyo CCI, Romano CCI, D’Hoore CCI, and Ghali CCI) [24-26] and ICD-10 data (Quan CCI, updated Quan CCI) [27,28]. In the popular Deyo modification, these different versions of CCI were validated mainly in inpatient settings with various medical and surgical diseases for inpatient or short-term mortality [29-31]. However, many studies comparing chart data with administrative data have found that more comorbidities are coded in charts than in administrative data [32-34], and a review paper suggested that claims data underestimated the CCI by 1 point [35]. In addition, some studies found chart data provided better predictions of 30-day in-hospital mortality than ICD administrative data [32,36].
Several modified CCI have been developed by other researchers. For example, investigators from the Surveillance, Epidemiology, and End Results (SEER) program of the National Cancer Institute (NCI) Medicare team created an index with non-cancer comorbidities, excluding diagnostic codes corresponding to any malignancies, with the same weight as that of the original CCI [37]. However, other investigators modified the index by adding or deleting some comorbidities and/or assigning different weights to certain conditions, adjusting to their own purpose or study population [38-42].
In Korea, Lee et al. [43] developed a modified version for patients receiving chronic renal replacement therapy (mCCI-CRRT) by recalibrating the weights of the comorbidities, and showed that for patients requiring CRRT, mCCI-CRRT better stratified the mortality risk compared to the original CCI. Choi et al. [44] developed a modified version of the CCI for Asians (mCCI-A) using the National Sample Cohort data, showing that recalibration of comorbidity weight slightly improved the prediction of long-term mortality (Table 1).
Charlson comorbidity index and modified versions with different weights
A detailed review of the modified version of CCI is beyond the scope of this study. There are several reviews on different versions of CCIs [10,45,46]. Despite many modifications, the original CCI remains the most widely used [2].
USE OF CCI IN CLINICAL STUDIES
CCI provides a simple, readily applicable, and valid method for estimating the risk of death from comorbid diseases. For example, compared to Elixhauser comorbidity index which include 30 conditions, CCI showed similar C-statistics for predicting mortality [2]. Therefore, in longitudinal studies, it has become the most widely used comorbidity index across a wide range of medical settings (Table 2) [12-23,26,27,30,31,33,47-119]. As of 2021, it has been cited more than 36,000 times [2].
Clinical use of Charlson comorbidity index in various clinical setting
Originally developed to account for comorbidities in patients with breast cancer [11], CCI is most extensively used in oncology settings. However, it is also used in other clinical conditions, including virtually all other specialty areas, and for the older or general population.
Regarding the timeframe, the original version of CCI was developed to predict 1-year mortality and was designed to predict mid-term (1-, 2-, and 3-year) to long-term (5- or 10- year) mortality. However, even though the CCI was not specifically designed to predict very short-term (such as postoperative, in-hospital, or 30 days) or short-term mortality (such as 3 months), it has often been used for this purpose and was found to be useful in most studies. In addition, although the CCI was originally designed to predict mortality, it is also associated with other outcomes such as treatment complications [120], readmissions [121], length of stay, and health care costs [122].
The clinical properties of different versions of the CCI, such as reliability, concurrent validity, sensitivity, and incremental and predictive validity, are thoroughly described in the review article by Charlson et al. [2]. A previous review demonstrated that the CCI is highly sensitive, reliable, and valid according to current standards [123].
PRACTICAL ISSUES IN THE USE AND CONSTRUCTION OF CCI
There are several practical questions researchers face when considering the use of CCI or other comorbidity indices in their research. The first question is whether to use a comorbidity index such as the CCI instead of individual comorbidities. If several comorbidities are important and well-known confounders, their individual effects should be analyzed using a statistical model [45]. However, there could be many comorbidities relevant to the outcome of interest, and controlling for all of these individual comorbidities would result in methodological problems, such as multicollinearity and loss of statistical power [46,124]. Therefore, the use of summary comorbidity measures, as a more general assessment of comorbidity status, is recommended to be included in the statistical model, because the weighted measures of comorbidity represent the overall burden of comorbidity and provide better prediction of mortality than separately evaluated individual conditions [125].
The second question is whether to modify the index specific to the study population. It is approximately 40 years since Charlson developed the CCI, and the management and prognosis of comorbidities have since changed significantly. In addition, the predictive ability would differ for different patient populations, different outcomes of interest, and different time frames (such as short-term and long-term). Modification of the comorbidity index and corresponding weights tailored to a specific population would result in better prediction [43,124]; however, it is cumbersome and may be impractical to recalibrate the weight and create a separate comorbidity index for each specific study, in which comorbidity itself is not an independent variable of interest. In addition, a general CCI, rather than a population-specific CCI, is useful for comparing different patient groups and diseases for health economic evaluations [45]. In our opinion, for adjustment purposes, the use of a generally used version of the CCI (the original version) would suffice, as the difference in predictive ability due to the modification is generally not large [126].
The third issue in constructing CCI with claims data is the ‘look-back’ interval for ascertaining comorbid diseases, i.e., the duration for which comorbidity data should be included. Some studies showed that adding more years of comorbidity data improved mortality prediction [47,127], while another study suggested that including 6 years of prior comorbidity data produced optimal validity and reliability [128]. In practice, many studies have used shorter look-back intervals, such as 2 years [122].
The fourth question is whether to include all inpatient and outpatient claims in constructing CCI. The original version of the CCI used only hospital claims as the data source. However, a previous study showed that comorbidity data from outpatient visits and inpatient admissions agreed poorly [129], suggesting that including both is necessary to obtain comprehensive information on comorbidities [46].
The fifth question is if a competing risk analysis should be used even if the CCI is adjusted. Yes, a competing risk analysis is still necessary, after adjusting for the comorbidity index because even when adjusted, traditional survival analysis methods can overestimate the probability of an event of interest in the presence of competing risks. While adjusting for the comorbidity index is important, it does not eliminate the need for competing risk analysis when studying time-to-event outcomes in the presence of competing events. For example, a study using UK primary care data to predict cardiovascular disease showed that the CRISK-CCI model (which accounts for competing risk and CCI) performed better than the QRESEARCH cardiovascular risk algorithm (QRISK3) model (a traditional prediction model that does not account for competing mortality) and the CRISK model (that accounts for competing risk) [130].
RESEARCH AND CLINICAL IMPLICATIONS OF CCI
Feinstein [1] noted in his book as follows: ‘To compare different modes of therapy, clinicians usually assemble groups of patients in whom the results of the treatments are then observed. For the comparison to be scientifically valid, the groups of patients must be initially comparable—they must have enough resemblance, before treatment, for their out-comes to be similar if treatment was not given. Without this pre-therapeutic similarity in the groups of patients, the different treatments cannot be properly evaluated.’
Therefore, from a research perspective, adjusting for comorbidities is methodologically necessary in any prognostic study. CCI is the most widely used and extensively validated comorbidity index for this purpose. The predictive validity of CCI is well-established for both short-term and long-term mortality in diverse patient populations, including hospitalized patients, patients undergoing surgery, patients with various cancer and non-cancer diseases, and the older population. This is especially important for studies using administrative data with ICD-9 [24] and ICD-10 codes [27].
From a clinical standpoint, the evaluation of pre-therapeutic comorbidities is crucial because their presence affects treatment decisions and prognosis [3]. The use of CCI is likely to provide a valid assessment of the patient’s unique clinical situation by enabling the identification of a constellation of comorbidities in the individual patient. This will improve the prognostic estimation of clinical outcomes for individual patients among seemingly similar patients with the same main diagnosis. However, the clinical usefulness of CCI integration in real clinical settings needs to be further determined. One example of using the CCI in clinical settings is prostate cancer. A United States cohort study showed that men with CCI ≥ 3 had 8.5 times risk of death from causes other than prostate cancer, and the authors suggested that men with high CCI should consider conservative management [131].
VALIDATION WITH KOREAN HEALTH SCREENING PARTICIPANTS
To demonstrate the validity of CCI using the Korean National Health Insurance data, we analyzed the risk of mortality using the CCI score. The claims codes used for calculating the CCI are listed in Table 3. One year look-back period was applied, and both outpatient and inpatient data were used to construct the CCI. Among the 4,234,415 individuals who participated in the general health check in 2009 (40% random sample), 3,956,070 individuals remained after excluding those with missing covariate data. Stepwise increases in mortality were observed with stepwise increases in the CCI, demonstrating the validity of CCI in this population (Table 4).
ICD-10 codes used to calculate Charlson comorbidity index using Korean health insurance data
Mortality risk by Charlson comorbidity index score in Korean general health check participants
CONCLUSION
Although many comorbidity indices are currently available, CCI is the most widely used index. Appropriate use of a comorbidity index, such as the CCI, is necessary to minimize bias in observational studies in an era of increasing use of administrative data for healthcare research.
Notes
CONFLICTS OF INTEREST
Dong Wook Shin is an associate editor of the journal, but he was not involved in the peer reviewer selection, evaluation, or decision process of this article. No other potential conflicts of interest relevant to this article were reported.
AUTHOR CONTRIBUTIONS
Conception or design: DWS, KH.
Acquisition, analysis, or interpretation of data: DWS, KH.
Drafting the work or revising: DWS, KH.
Final approval of the manuscript: DWS, KH.