Validation of the highly sensitive person scale for suicide screening in high-risk groups: A case-control study
Article information
Abstract
Purpose
High sensitivity is among the risk factors that increase psychological burden, increasing the risk of suicide in some people. The aim of the study was to develop a Korean version of the Highly Sensitive Person (HSP) scale and investigate its psychometric properties. Diagnostic validity was evaluated to identify those with high suicidal risk.
Methods
A total of 176 participants were recruited from the 12 hospital-based Crisis Response Centers across South Korea. As a case-control study, we used two groups: suicidal risk group and control group. Those with suicidal ideation or attempts were classified as suicidal risk group, and the control group sample were recruited from the Crisis Response Centers workers. Tests for internal consistency and concurrent validity with other scales were conducted.
Results
Results showed strong item-total correlation and concurrent validity. Receiver operating characteristic curve analysis showed an area under the curve of 0.881. The HSP cutoff score of 13 yielded the best balance between sensitivity (74%) and specificity (87%). Construct validity was supported by a one-factor model using exploratory factor analysis.
Conclusion
Overall, the current study supports the use of HSP in psychiatric settings as a valid screening tool for suicidal risk.
INTRODUCTION
Suicide is a significant burden worldwide [1,2]. An increasing amount of research has investigated possible risk factors leading to suicidal ideation or attempts [3,4]. According to a recent meta-analysis, however, a history of over 50 years of suicide research has not helped to improve our ability to predict who is at risk for suicide [5]. For the last two decades, Korea has had the highest incidence rate of successful suicide among Organization for Economic Cooperation and Development (OECD) countries. As of 2018, the suicidal death rate in Korea (26.6 per 100,000 persons) was higher than the average rate of OECD countries (11.6 per 100,000 persons). In 2004, aiming to meaningfully reduce death due to suicide, the Ministry of Health and Welfare of Korea issued the “National Suicide Prevention Action Plan” [6]. Despite these maximal and permanent policy efforts to reduce suicides; however, rates are still increasing in Korea.
Suicide is a result of complex interactions among environment, temperament, and individual characteristics [7]. According to the stress-diatheses theory of suicide, certain vulnerabilities are activated in at-risk individuals by stressors,resulting in internalization of problems, which eventually increases suicidal risk for these individuals [8]. The concept of high sensitivity as an attribute ofindividual temperament was originally proposed in the work of Aron in 1997 [9]. According to the definition therein, a highly sensitive person (HSP) has a sensitive nervous system, is aware of subtleties in his or her surroundings, and is more easily overwhelmed by highly stimulating environments [9]. While the exact significance of the term is under debate, high sensitivity is understood to encompass an individual’s level of responsiveness to his or her environment [10], together with his or her emotional reactivity [11]. High sensitivity is related to internalization of problems such as anxiety and depression. Moreover, highly sensitive people report lower levels of subjective happiness [12] and life satisfaction [13], poor stress management, higher levels of perceived stress [14], greater work distress, and more frequent somatic complaints than their counterparts without high sensitivity. Taken together, HSPs are vulnerable to psychological distress [15]. Recent studies have aimed to analyze the relationship between sensitivity as a feature of personality and suicidal behaviors. One study shows that sensitivity is associated with a progression of suicidal ideation to suicide attempts, while emotionaltraits such as anger and/or anxiety are not associated with suicidal behavior[16]. HSP, when they go through mental health challenges, may be at higher risk of suicidal behaviors. For example, when depressed, HSP could be vulnerable to suicidal ideation because they feel everything more deeply, with their depth of processing.
The contents of the original HSP scale by Aron and Aron [9] is focused on sensory-processing sensitivity, which is greater reactivity to internal and external stimuli. However, in order to understand the relationship between sensitivity and mental health problems, it is worthwhile to consider the symptoms of sensitivity observed in psychiatric diseases. We aim to develop a scale that includes and captures the clinical features of HSP that contribute to psychological problems and suicidality. The purpose of this study is to identify individuals at-risk for suicide with a psychometrically valid and reliable self-reported questionnaire in a stratified Korean population sample with a wide age range. Additionally, we derive a cut-off score with both high sensitivity and specificity to detect and assess suicidal risk groups.
METHODS
Participants
Participants were recruited from 12 Crisis Response Centers in Korea between October 2020 to March 2021. The Crisis Response Center is operated by university hospitals, providing treatment services for those who visit the emergency room due to a suicide crisis. As a case-control study, we used two groups: suicidal risk group and control group. The control group sample were recruited from the Crisis Response Centers workers. Participants were informed about the purpose and progression of the study and voluntarily signed consent forms for inclusion. Those with degenerative neuropsychiatric diseases (i.e., Parkinson’s disease, Huntington’s disease, chorea, or dementia), intellectual disability, and/or brain damage were excluded. A total of 176 adults was enrolled in the study (58 males and 118 females). Mental health professionals at each clinic underwent online education sessions about overall administration of each assessment tool for suicidal behaviors, and clinical interviews were conducted beforehand. Next, face-to-face interviews about participants’ reasons for referral, symptoms, histories of suicidal behaviors, and structured reviews were conducted. Each participant completed a sociodemographic questionnaire, medical/psychiatric history, self-reports, and suicide assessment tools. The average age of participants was 31.61 years (standard deviation, 11.56). If subjects reported suicidal ideation and/or attempts in the past 6 months,they were classified as the suicidal risk group (n=121). If neither was the case,then subjects were included in the control group (n=55). The Institutional Review Board of the Samsung Medical Center approved this study (IRB No. SMC 2020-04-184).
Tools
Patient Health Questionnaire-9
The Patient Health Questionnaire-9 (PHQ-9) is a self-report questionnaire originally developed by Kroenke et al. [17] in 2001. It is a nine-item depression module from the full PHQ, which is an instrument for criteria-based diagnoses of depression and other mental disorders commonly encountered in primary care. In measuring the severity of depression, PHQ-9 scores range from 0 to 27, as each of the nine items can be scored from 0 (not at all)to 3 (nearly every day). At 10, a PHQ-9 score has a sensitivity of 88% and a specificity of 88% for major depression or clinically relevant depression. Importantly, PHQ-9 score of 15 usually signifies the presence of clinically significant major depression [17]. A validation study by Park et al. [18] shows that the Korean version of the PHQ-9 has high reliability and validity. Also, total scores on the PHQ-9 have been shown to differentiate depressive patients from normal samples in Korean populations [19,20].
Beck Scale for Suicide Ideation
The Beck Scale for Suicide Ideation (BSSI) is a self-reported questionnaire developed by Beck that consists of a total of 21 questions attempting to measure suicidality and the severity thereof. Question content covers several topics, including relative desire for life and death, frequency of suicide incidents, perceived sense of control in committing suicide, and degree of actual preparation.Based on participants’ experience within recent past weeks, a three-point Likert scale (0 to 2 points) was used. Questions 1 to 5 are screening questions, asking whether participants have an active or passive desire for suicide, in which three items evaluate the participants’ desire to live or die and two items evaluate their desire to attempt suicide. If a participant shows any suicidal desire, then the remaining items of the questionnaire are administered. Total scores of the questionnaire range from 0 to 38 points. Validation research by Choi et al. [21] suggests a need for additional support from more diverse clinical groups and samples including teenagers and the elderly to better validate the tool for effective assessment of suicide behaviors in respondents.
Highly sensitive person scale
The HSP scale describes the symptoms of a HSP may experience. The HSP is developed under the same latent construct ‘high sensitivity’ from the work of Aron, but clinical symptoms of a HSP are emphasized in our scale. The content of each item is presented in Table 1. Together with the standard criteria of the Diagnostic and Statistical Manual of Mental Disorders (DSM-5), potential HSP symptoms are included in the assessment tool. The HSP scale was developed to reflect characteristics of high sensitivity—including sensitivity to external stimuli and vulnerability to neuroticism. The HSP scale consists of 28 yes/no questions examining the characteristics of respondents.
Highly sensitive person scale items
Statistical analyses
Descriptive statistics, correlation analyses, and receiver operating characteristic (ROC) curve analyses were performed using the statistical program SPSS Statistics version 23 (IBM Co., Armonk, NY, USA). Factor analyses were conducted using R programming (R Foundation for Statistical Computing, Vienna, Austria). The “psych” package was utilized to perform an exploratory factor analysis (EFA). Because HSP is a binary variable, we used the “polycor” package to convert it to a polycorrelation matrix. Results of Kaiser-Meyer-Olkin (KMO) tests showed a sampling adequacy of 0.85, and results from Bartlett’s test of sphericity were significant (chisq=1,538, df=351, P<0.001), suggesting that the intercorrelation among variables was sufficient for factor analysis. With the criteria met for factor analyses, EFA was conducted with orthogonal rotations. We conducted ROC curve analysis to examine the clinical utility of HSP to differentiate between individuals at high risk for suicidal behaviors and individuals not atrisk. The chosen cut-off point maximizes the area under the curve (AUC), as well as sensitivity and specificity. An AUC value of 8.0 or greater was established as excellent[22].
RESULTS
Demographics
Table 2 provides descriptive statistics of sociodemographic information and self-reported measures among subjects. There were significant differences between the suicidal risk group and control group in age, years of education, marital status, employment, and income (t=2.28, P<0.05; t=8.60, P<0.001; t=–2.58, P<0.05; t=3.35, P<0.05, respectively). There was no significant difference between the groups in terms of sex. Subjects in the suicidal risk group reported significantly higher scores on each of the PHQ-9, BSSI, and HSP scales (t=–20.65, P<0.001; t=–15.99, P<0.001; t=–17.96, P<0.001;t=–10.71, P<0.001;respectively).
Descriptive statistics of demographic characteristics, past suicidal ideas, plans, or attempts, and self-reported questionnaire scores
Reliability
Coefficient alpha and corrected item-total correlations were analyzed. The Cronbach’s alpha value for total items was 0.89. Corrected item-total correlation values were in the range of 0.28 and 0.70. Cronbach’s alpha if item deleted was between 0.88 to 0.89. All of the values were acceptable (at a value of 0.7 or above).
Factorial validity
We explored the factor structure of HSP using EFA. Based on the results of parallel analysis, which is regarded as one of the most accurate methods to determine the number of factors, a one-factor model was suggested. Factor loadings for most of the items were over 0.3, with the exception of three items (item numbers 5, 6, 25). Factor loadings are presented in Table 3. Because the HSP is a newly developed scale, we tested a two-factor model for exploratory purposes. Most of the items were cross-loaded, and the reliability of the two factors was low (Cronbach’s alpha=0.7). Additionally, no theoretical background for sub-factors was involved in the process of HSP development. Accordingly, we adopted the one-factor model.
Exploratory factor analysis factor loading values for the 28-item highly sensitive person scale
Concurrent validity
Results of the correlation analyses among the HSP scale and other self-reported measures are presented in Table 4. All of the intercorrelations were significant(P<0.01). The results remained significant even after controlling for sex and level of education in subjects (all P<0.01). The highest correlation was observed between depressive symptom measures and the HSP scale, supporting concurrent validity.
Correlations between the HSP scale and other suicide-related measures
ROC analysis
Results of ROC analysis are presented in Fig. 1. Table 5 presents the values of sensitivity and specificity for different cutoff points that may be useful when using the test. The AUC was 0.881 (95% confidence interval, 0.829 to 0.933), and a cut-off point of 12.5 was shown to provide an optimal balance between sensitivity and specificity (Youden’s index=0.608). We suggest a cut-off score of 9 to minimize false negatives in screening those who might be at risk for suicidal behaviors.
Receiver operating characteristic curve for the highly sensitive person scale.
Sensitivity and specificity of the highly sensitive person scale
DISCUSSION
This study investigates the psychometric properties of the newly developed HSP scale as a novel tool for assessment of suicide risk. The results support HSP as a reliable scale, with a Cronbach’s alpha value of 0.8. The HSP scale has high convergent validity, with high correlation between the HSP and PHQ-9 scales, as well as the BSSI. The observed AUC was 0.881, which indicates high diagnostic validity. A cut-off point of 13 effectively differentiated the group of subjects at high risk for suicide from the low suicidal risk group with adequate sensitivity and specificity. Moreover, construct validity was supported with a unidimensional model.
Item 9 of the PHQ-9 (which evaluates level of suicidal ideation) is widely used to assess risk of suicide attempt or death. According to previous research,the PHQ-9 suicide item is a strong predictor of suicide attempts and deaths in a sample of more than nine million adult outpatients in the United States [23]. In that research, based on electronic health records, patient response to PHQ-9 item 9 was a critical predictor of suicide risk among 313 demographic and clinical characteristics [23]. This finding has encouraged routine assessment of suicide risk, contributing to further referral for intensive therapies and outreach prevention strategies as part of the U.S. National Action Alliance for Suicide Prevention [24]. Because PHQ-9 item 9 asks responders directly about suicidal ideation and frequency; however, its sensitivity is relatively high, while its specificity is low [25]. While high sensitivity is considered critical in suicide risk evaluation in terms of identifying potentially missed cases, labeling a patient as suicidal just because of a positive response on a suicide item could lead to misallocation of limited health care resources [26]. In our study, a cut-off HSP score of 13 has a specificity of 87.3%, which is higher than the normally reported specificity of the PHQ-9 suicide item [25,26]. Patients who report few or mild depressive symptoms are likely to be overlooked in real psychiatric settings, whereas clinicians are more likely to pay attention to patients with severe depression.
Differences in HSP scores between the high risk and control groups were significant. In our sample, more than 90% of subjects in the high suicidal risk group reported past psychiatric visit(s). While the HSP scale cannot be a stand-alone measure of suicidal risk, its addition to assessment can help clinical decision-making. The HSP scale can be helpful in identifying patients in need of treatment such as pharmacotherapy, outpatient short-term intervention services, and intensive psychotherapies including cognitive behavioral therapy or dialectical behavioral therapy. We propose that the HSP scale will serve as a baseline assessment tool to measure psychological distress, such as mood symptoms. Prevalent psychological distress is associated with suicidal behaviors, and suicide can be precipitated or aggravated by emotional problems [27]. Ample evidence has shown that use of outcome measurements to guide treatment decisions in clinical care—a practice known as measurement-based care—is effective in improving clinical outcomes among patients with mood disorders [28,29]. Insofar as the results of our research support the clinical utility of HSP, we propose that the HSP scale can serve as one effective patient-reported rating scale to monitor mood symptomatology.
This study has several limitations. First,the total sample size is relatively small, which could have reduced statistical power. Second, inter-rater reliability and test-retest assessment were not explored, which could limit accuracy and validity. Third, the suicidal risk group is heterogeneous in nature insofar as sample recruitment was conducted in crisis centers, and was subject to selection bias. We used a Korean population only, so recapitulation in East Asian countries that share similar cultural backgrounds will help increase our understanding of the utility of the HSP scale. Fourth, high sensitivity is related to anxiety and certain personality traits such as neuroticism, but scales that measure these constructs are not used in our study. Further studies with measures of anxiety and personality are recommended to increase convergent validity. Moreover, the age range of our sample is limited. The total sample comprised mostly younger adults, with subjects older than 60 years of age comprising less than 3%. In Korea, the suicide rate of older adults is substantially high [30]. Further validation is needed using older populations. In addition, the control group was limited to the workers at the Crisis Response Centers, which limits generalizability. Lastly, the correlation coefficient between the HSP scale and the PHQ-9 was strongly high. This suggests that the HSP scale basically measures depressive symptoms, and that the ability of HSP for screening high suicidality reflects depression. Further studies are necessary to delineate the construct nature of HSP.
To conclude, this initial study examines the psychometric properties of the Korean version of the HSP scale. Our study confirms that the HSP scale is a reliable and valid tool for assessment of suicide risk. Moreover, the HSP scale has clinical utility, identifying individuals at high risk for suicide with a cut-off score at 13. The HSP scale has the potential to be used as a screening tool in patients who visit psychiatric clinics to identify those at high risk for suicide in need of immediate support. In using the HSP scale as a screening tool, individuals with positive screening results should undergo clinical interviews to rule out possible false positives for health care systems to optimize resources and protect those most in need of support against suicide risk.
Acknowledgements
This study was supported by a grant from the Korean Mental Health R&D Project, funded by the Ministry of Health & Welfare, Republic of Korea (HL19C0001; PI, Hong Jin Jeon), and by a grant from the Korea Health Technology R&D Project through the Korea Health Industry Development Institute (KHIDI), funded by the Ministry of Health & Welfare, Republic of Korea (HR21C0885).
Notes
Hong Jin Jeon has been editorial board of P&FM since October 2022. He was not involved in the review process of this original aricle.
AUTHOR CONTRIBUTIONS
Conception or design: YC, HJJ.
Acquisition, analysis, or interpretation of data: YC, VY, YHC, MJP, HK, HJJ.
Drafting the work or revising: YC.
Final approval of the manuscript: HJJ.