0
We're unable to sign you in at this time. Please try again in a few minutes.
Retry
We were able to sign you in, but your subscription(s) could not be found. Please try again in a few minutes.
Retry
There may be a problem with your account. Please contact the AMA Service Center to resolve this issue.
Contact the AMA Service Center:
Telephone: 1 (800) 262-2350 or 1 (312) 670-7827  *   Email: subscriptions@jamanetwork.com
Error Message ......
Original Investigation |

A New Scoring System for Upper Airway Pediatric Sleep Endoscopy FREE

Dylan K. Chan, MD, PhD1; Bryan J. Liming, MD3; David L. Horn, MD2; Sanjay R. Parikh, MD2
[+] Author Affiliations
1Division of Pediatric Otolaryngology, Department of Otolaryngology–Head and Neck Surgery, University of California, San Francisco
2Division of Pediatric Otolaryngology, University of Washington School of Medicine, Seattle Children’s Hospital, Seattle
3Division of Otolaryngology, Department of Otolaryngology–Head and Neck Surgery, Madigan Army Medical Center, Tacoma, Washington
JAMA Otolaryngol Head Neck Surg. 2014;140(7):595-602. doi:10.1001/jamaoto.2014.612.
Text Size: A A A
Published online

Importance  Sleep-associated upper-airway obstruction in children is a significant cause of morbidity. Development of a simple, standardized, quantitative technique to assess anatomic causes of sleep-related breathing disorder is important for surgical planning, clinical communication, and research.

Objective  To design, implement, and evaluate a scoring system to quantify airway obstruction in pediatric drug-induced sleep endoscopy.

Design, Setting, and Participants  This study was a retrospective case series conducted at a tertiary pediatric hospital. The patients were children with sleep-related breathing disorder who underwent polysomnography and drug-induced sleep endoscopy.

Interventions  Flexible fiber-optic laryngoscopy was performed. Endoscopic examinations were recorded on video and assessed by 4 independent raters based on a scoring template.

Main Outcomes and Measures  Five locations in the upper aerodigestive tract (adenoid, velum, lateral pharyngeal wall, tongue base, and supraglottis) were evaluated on a 4-point scale for minimum and maximum obstruction. Internal reliability was assessed by calculating interrater and intrarater intraclass correlation coefficients (ICCs). For external validation, aggregate and site-specific scores were correlated with preoperative polysomnographic indices.

Results  Videos recorded of sleep endoscopies from 23 children (mean age, 2.2 years) were reviewed and rated. Children had an average apnea-hypopnea index of 24.8. Seventy percent of interrater and intrarater ICC values (7 of 10 for each set) were above 0.6, demonstrating substantial agreement. Higher total obstructive scores were associated with lower oxygen saturation nadir (P = .04). The scoring system was also used to quantitatively identify children with multilevel airway obstruction, who were found to have significantly worse polysomnographic indices compared with children with single-level obstruction (P = .02).

Conclusions and Relevance  The proposed scoring system, which is designed to be easy to use and allow for subjectivity in evaluating obstruction at multiple levels, nonetheless achieves good internal reliability and external validity. Implementing this system will allow for standardization of reporting for sleep endoscopy outcomes, as well as aid the practicing clinician in the interpretation of sleep endoscopy findings to inform site-directed surgical intervention in cases of complicated obstructive sleep apnea.

Figures in this Article

Obstructive sleep apnea-hypopnea syndrome (OSAS) affects approximately 1% to 4% of children.1 Consisting of repetitive arousals and/or oxygen desaturations secondary to partial or complete airway obstruction, OSAS has well-described neurocognitive, behavioral, cardiovascular, and inflammatory sequelae.2 Adenotonsillectomy has been the treatment of choice for pediatric OSAS. It has demonstrated efficacy with regards to objective and subjective outcomes.35 However, 20% to 74% of children will have persistent symptomatic or polysomnographic evidence of OSAS after adenotonsillectomy.4,6 Sleep-induced upper airway obstruction is a complicated phenomenon involving multiple anatomic levels and physiologic causes.

Drug-induced sleep endoscopy (DISE) consists of flexible fiber-optic assessment of the airway from nares to carina under a state of induced simulated sleep using a combination of anesthetic agents and sedatives. Spontaneous ventilation is maintained. First described in 1991,7 DISE is safe,8 quickly learned, and uses equipment already available in most otolaryngology operative suites. Upper airway collapse is a highly dynamic state, and real-time endoscopic assessment of the upper airway can provide valuable information beyond the polysomnographic and clinical examinations. DISE seems to be an effective technique for diagnosis of site of obstruction and for surgical planning in children.9,10 It can be used in both children in whom treatment with adenotonsillectomy has failed8 and in children whose physical examination findings are inconsistent with adenotonsillar hypertrophy as a cause of their sleep-related breathing disorder .9 Several validated classification systems for defining the location and severity of obstruction have been described in adult patients.1113 Studies in adults have demonstrated moderate to substantial test-test reliability14 and interrater reliability.15The VOTE classification11 in adults addresses the velum, oropharynx, tongue base, and epiglottis but does not evaluate 2 common sites of airway obstruction in children: choanal obstruction by adenoid hypertrophy and arytenoid collapse into the glottic airway as seen in sleep-induced, late-onset laryngomalacia. There has not been an equivalent scoring system described in the pediatric literature that encompasses all of these potential levels of upper-airway obstruction, and it is unknown if sleep endoscopy has the same level of reliability in pediatric patients.

In this study we sought to design, implement, and evaluate a scoring system to quantify airway obstruction in pediatric DISE. An ideal diagnostic assessment will have excellent internal reliability and external validity for the desired outcome measures. Developing such a system will allow for standardization of reporting for sleep endoscopic outcomes. Practicing clinicians who implement such a system will be able to better interpret their sleep endoscopic findings and define their surgical plan.

Patient Selection and Data Collection

This study was approved by the institutional review board of Seattle Children’s Hospital. The investigators adhered to the policies for protection of human subjects as prescribed in 45 CFR 46. Waiver of written informed consent was obtained and approved by the Seattle Children’s Hospital institutional review board. All patients undergoing DISE by pediatric otolaryngologists at Seattle Children’s Hospital for airway evaluation in cases of suspected obstructive sleep apnea over a 2-year period were collected in a quality-improvement database. Our indication for performing sleep endoscopy during this period was the presence of sleep-related breathing disorder in a child of any age that could not be explained by simple adenotonsillar hypertrophy on clinical examination. This included children with persistent symptoms after adenotonsillectomy, children with sleep-related breathing disorder and normal adenotonsillar size, and children with additional medical comorbidities, history, or anatomic features associated with synchronous sites of airway obstruction in addition to the tonsils and adenoids. At the time of the study, the database was queried retrospectively. Patients were excluded if a video of the sleep endoscopy was not available and if a complete polysomnogram was not available for primary review.

Because the aim of the study was to present a widely generalizable DISE scoring system, no additional clinical or demographic exclusion criteria were applied. All included patients underwent medical chart review. Demographic data, operative details, and polysomnographic indices were collected. Polysomnograms were all performed at the sleep laboratory at Seattle Children’s Hospital and were scored according to standard American Academy of Sleep Medicine scoring criteria for pediatric polysomnography.16 Sleep architecture, including sleep stages, cortical arousals, awakenings, limb movements, and arousal indices were assessed by standard techniques. Respiratory disturbances were defined as follows: (1) obstructive apnea: at least 90% reduction in airflow for 2 respiratory cycles in the presence of continued thoracic or abdominal effort; (2) hypopnea: 50% to 89% reduction in airflow for 2 respiratory cycles, accompanied by an electroencephalographic (EEG) cortical arousal, awakening, and/or at least 3% oxygen desaturation; (3) central apnea: at least 90% reduction in airflow with no respiratory effort, lasting for at least 2 respiratory cycles, and accompanied by an EEG arousal, awakening and/or at least 3% oxygen desaturation. The apnea-hypopnea index (AHI) was defined as the total number of apneas and hypopneas averaged per hour of total sleep time. The oxygen saturation nadir is defined as the lowest oxygen saturation recorded during the polysomnogram.

Scoring System for Drug-Induced Sleep Endoscopy

All patients underwent transnasal flexible fiber-optic laryngoscopy in the operating room in a supine position, spontaneously ventilating under propofol anesthesia. Endoscopy was performed by 3 of us (D.K.C, D.L.H., and S.R.P.). No specific instructions were given as to how to perform the procedure, only that the intent of the endoscopy was to identify sites of airway obstruction during drug-induced sleep. Videos were recorded and reviewed post hoc without knowledge of the patient’s medical history or subsequent surgical intervention. All videos were initially screened and edited to remove all patient-identifiable information. They were assigned 2 random numbers, 1 for use for each of the 2 independent ratings performed. Once all assignments were made, the study database was purged of all patient-identifiable information. All raters were thus blinded to the identities of the patients to whom each video belonged. The scoring template and guidelines were given to each of 4 raters (a postgraduate year-3 resident, a pediatric otolaryngology fellow, and 2 pediatric otolaryngology attending physicians) (Figure 1). Minimal and maximal obstruction was assessed on a 4-point ordinal scale (0, 1, 2, and 3) at 5 subsites within the upper aerodigestive tract: the adenoid, velum, lateral pharyngeal wall (LPW), tongue base, and supraglottis. Each rater reviewed the complete set of 23 videos in 1 sitting. One month later, each rater reviewed the independently randomized complete set of the same 23 videos. Rating forms were collected, and data analysis was subsequently performed.

Place holder to copy figure label and caption
Figure 1.
Scoring Form and Guidelines

The form (A) and guidelines (B) used for post hoc scoring of drug-induced sleep endoscopy are presented. Some data fields in the scoring form, including turbinate and tonsil size, and use of adjunct airway, were not included in our analysis. LPW indicates lateral pharyngeal wall.

Graphic Jump Location
Post Hoc Scoring Analyses

From the raw site-specific obstructive scores, post hoc calculations were performed to generate indices used for further analyses. For each video, an aggregate obstructive score (“total obstructive score”) was calculated as the sum of the 5 site-specific maximal obstructive scores.

Scoring of DISE was also used as an objective measure to identify multilevel obstruction. We used our scoring system for quantitative determination of the presence of multilevel obstruction. Patients were grouped as follows:

  • •Group 1: Single site of obstruction (no or 1 site with a maximum obstructive score ≥2.0);

  • •Group 2: Two or more sites of obstruction, but only within either the adenoid-velum-LPW complex or the tongue base–supraglottis complex; or

  • •Group 3: Two or more sites of obstruction encompassing both the adenoid-velum-LPW and tongue base–supraglottis complexes.

These groupings were based on prior descriptions of grouping of obstruction at the velum and LPW10 as well as a priori anatomic relationships of the adenoid, velum, and palatine tonsils as 1 interactive obstructive unit, and the tongue base and supraglottis as a second.

Statistical Analysis

We assessed the intraclass correlation coefficient (ICC) and 95% confidence interval (95% CI) using a 1-way random effects analysis of variance (ANOVA) model to calculate interrater and intrarater reliability for each rating, as well as selected aggregate ratings that were each calculated at the individual patient level. An ICC of 1 indicates perfect correlation; 0 indicates no correlation. The ICC is comparable with κ values in assessing interrater and intrarater reliability for ordinal data; analogous to κ, the degree of agreement between 0 and 1 is interpreted roughly as follows17: less than 0, poor agreement; 0.01 to 0.20, slight agreement; 0.21 to 0.40, fair agreement; 0.41 to 0.60, moderate agreement; 0.61 to 0.80, substantial agreement; and 0.81 to 1.00, almost perfect agreement.

To assess the association between polysomnographic features and total obstructive scores, we performed simple linear regression between selected parameters (total obstructive score vs AHI and total obstructive score vs oxygen saturation nadir). No additional variables were considered in the model. Linear regression best-fit slopes and 95% CIs were assessed. Pairwise comparisons with 95% CIs that did not encompass a slope of zero were considered to have a significant association; in addition, the P value for the association was calculated. For reporting of individual-level aggregate data, means and 95% CIs are presented. Comparison of nonparametric data was conducted using the Mann-Whitney rank-sum test. All statistical analyses were performed using STATA software (version 12.1; StataCorp LP).

Patient Demographics

A total of 23 children underwent drug-induced sleep endoscopy for suspected obstructive sleep apnea (Figure 2). Their mean age was 2.2 years (95% CI, 1.8-2.6; range, 0.5-8.9 years). Sixteen of the 23 (70%) had a medical comorbidity that has been previously associated with obstructive sleep apnea18; these included chromosomal abnormalities, hypotonia, and encephalopathy. All children underwent preoperative polysomnography, with a mean AHI of 24.8 events per hour (95% CI, 18.3-31.3; range, 0.7-95) and oxygen saturation nadir of 78.1% (95% CI, 75.1%-81.1%; range, 43.0%-94.1%).

Place holder to copy figure label and caption
Figure 2.
Patient Profile

Individual patient obstructive, polysomnographic, and demographic profiles are shown. Mean obstructive score at each of the 5 evaluated sites (adenoid, velum, lateral pharyngeal wall [LPW], tongue base, and supraglottis) is graphically represented as either less than 2.0 (white box) or 2.0 or more (black box). Multilevel obstruction group (1, 2, or 3) was defined as follows: Group 1: No or 1 site with obstructive score greater than 2; group 2: 2 or more sites with obstructive score greater than 2 but confined to upper (adenoid, velum, LPW) or lower (tongue base, supraglottis) airway complex; group 3: 2 or more sites with obstructive score greater than 2, encompassing both upper and lower airway complexes. Total indicates total obstructive score (sum of obstructive scores at each of the 5 sites) from sleep endoscopy. AHI indicates apnea-hypopnea index from preoperative polysomnography. Oxygen nadir is low oxygen saturation from preoperative polysomnography. Age in years, body mass index (BMI; calculated as weight in kilograms divided by height in meters squared), prior sleep surgery (N, none; A, adenoidectomy; or TA: adenotonsillectomy), and medical comorbidities (neuromuscular, syndromic, pulmonary, or chromosomal) are shown.

Graphic Jump Location
Scoring of DISE

Representative images for each of 4 ordinal scores (0, 1, 2, 3) at 5 levels of obstruction are shown (Figure 3). Obstructive scores at minimum and maximum obstruction were obtained at each site for each patient by 4 independent raters, for a total of 80 independent ratings per patient and 8 independent ratings for each unique scoring unit (minimum or maximum obstructive score at a single site). Mean obstructive scores are shown together with preoperative polysomnography parameters in Figure 2.

Place holder to copy figure label and caption
Figure 3.
Representative Images

Each of the 4 ordinal scores at each of the 5 upper-airway sites are shown.

Graphic Jump Location
Internal Reliability

Interrater and intrarater reliability was assessed by calculating the ICC using a 1-way random-effects ANOVA model (Figure 4). Overall, interrater reliability for individual sites of obstruction ranged from fair (0.24) to substantial (0.71), with most (7 of 10 sites) exhibiting substantial agreement (0.6-0.8). Minimum velopharyngeal opening and minimum LPW obstruction showed lower levels of agreement (0.24 and 0.41, respectively). In general, assessment of maximum obstruction was more reliable (mean score, 0.65) than the minimum obstructive score (mean score, 0.51); all sites had reliability scores for maximal obstruction of 0.61 to 0.71.

Place holder to copy figure label and caption
Figure 4.
Internal Reliability

Interrater (A) and intrarater (B) reliability assessed with the intraclass correlation coefficient (ICC) are shown for the 5 sites of upper airway obstruction scored for videos of drug-induced sleep endoscopy. Ratings for minimum (light blue) and maximum (white) obstruction at each site are shown. 1.0 indicates perfect correlation; 0, no correlation. Error bars indicate 95% CIs for each measurement. LPW indicates lateral pharyngeal wall.

Graphic Jump Location

Intrarater reliability was assessed and in general yielded higher ICC values following a similar site-specific pattern (Figure 4). Again, the reliability of minimum obstructive scores was lower than for maximum obstruction. The velum was the least reliably assessed site. Overall, 7 of 10 of the site-specific obstructive scores (70%) had substantial intrarater reliability.17 When taking all 10 measurements in aggregate (maximum and minimum scores at 5 sites), the overall intrarater ICC values for each of the 4 reviewers were statistically indistinguishable between the 2 trainees (ICC, 0.75 [95% CI, 0.69-0.81] and 0.76 [95% CI, 0.70-0.82]) and 2 attending physicians (ICC, 0.78 [95% CI, 0.73-0.84]) and 0.79 [95% CI, 0.74-0.84]).

External Validity

To assess external validity, total obstructive scores, calculated as the sum of the 5 maximum site-specific obstructive scores, were correlated with preoperative AHI and oxygen saturation nadir by simple linear regression. Association was seen between higher total obstructive score and worse polysomnographic indices (higher AHI and oxygen saturation nadir). This association was statistically significant for oxygen saturation nadir (P = .04) but not for AHI (P = .12).

Scoring of DISE was also used as an objective measure to identify multilevel obstruction as follows (Figure 5):

Place holder to copy figure label and caption
Figure 5.
Multilevel Obstruction

Multilevel obstruction was determined based solely on objective site-specific obstructive scores. Apnea-hypopnea index (AHI) and low oxygen saturation values for patients in the 3 multilevel obstruction groups were compared. Multilevel obstruction group (1, 2, or 3) was defined as follows: group 1, no or 1 site with obstructive score greater than 2; group 2, 2 or more sites with obstructive score greater than 2 but confined to upper (adenoid, velum, lateral pharyngeal wall [LPW]) or lower (tongue base, supraglottis) airway complex; group 3, 2 or more sites with obstructive score greater than 2, encompassing both upper and lower airway complexes.aP = .02.

Graphic Jump Location

  • •Group 1: Single site of obstruction (no or 1 site with a maximum obstructive score ≥2.0) (n = 9 children);

  • •Group 2: Two or more sites of obstruction, but only within either the adenoid-velum-LPW complex or the tongue base–supraglottis complex (n = 7); or

  • •Group 3: Two or more sites of obstruction encompassing both the adenoid-velum-LPW and tongue base–supraglottis complexes (n = 7).

Groups 1 and 2 had indistinguishable AHI values (mean events per hour, 16.7 [95% CI, 8.5-24.9] vs 15.1 [95% CI, 8.2-22.0], respectively; P = .30; Mann-Whitney rank-sum test). When these 2 groups were pooled and compared with group 3, AHI values were statistically significantly different (16.0 [95% CI, 10.6-21.4] for groups 1 and 2; 44.8 [95% CI, 29.2-60.4] for group 3; P = .02). On the one hand, oxygen saturation nadir was lower in group 3 compared with groups 1 and 2, but this difference was not statistically significant (70.4% [95% CI, 63.8-77.0] for group 3 vs 81.4% [95% CI, 78.4-84.4] for groups 1 and 2; P = .07). On the other hand, the total obstructive score was significantly different between group 1 and group 2 (5.4 [95% CI, 5.0-5.8] vs 7.8 [95% CI, 7.2-8.4]; P = .006) but was not different between group 2 and group 3 (7.8 [95% CI, 7.2-8.4] vs 8.6 [95% CI, 8.1-9.1]; P = .15).

This study is a preliminary attempt to develop a standardized scoring system for DISE that can be used for all cases of upper-airway evaluation in children. Rather than narrow our focus to the group of children who traditionally undergo DISE (those with persistent OSA after adenotonsillectomy), we wished to broaden the applicability of DISE and this scoring system by examining a wider cohort of children, including those with sleep-related breathing disorder but with an AHI of less than 1, infants younger than 1 year, and those who had not yet undergone adenotonsillectomy but had normal adenotonsillar size on clinical examination. We identified 5 sites of airway obstruction (adenoid, velum, LPW, tongue base, and supraglottis) and provided instructions on how to rate obstruction on a 4-point ordinal scale.

Four raters with different levels of experience scored the videos of the endoscopic examinations on 2 separate occasions. Because our indications for DISE and inclusion criteria were broad, the study was designed to maximize generalizability, with the acknowledgment that this might compromise reliability owing to the following attendant limitations: (1) no specific instructions were given as to how to perform the endoscopy; (2) the patient population represented a wide range of disease, including normal children with persistent sleep apnea after adenotonsillectomy to infants with multiple medical comorbidities; (3) no clinical context was given to the raters at the time of scoring; and (4) the 4 independent raters had a wide range of experience levels in performing and viewing sleep endoscopy. These limitations of the study design should lead to an underestimate of the potential reliability of the scoring system itself and stimulate the ongoing development and optimization of a reliable, simple, and reproducible method for performing and evaluating DISE in children.

Despite these limitations, interrater and intrarater reliability are generally substantial for all anatomical sites, and are comparable with those achieved in adults for DISE evaluation of site of airway obstruction,14,15 as well as those described for the validated Golding-Kushner scale for quantification and objective description of velopharyngeal insufficiency.19,20 The internal reliability in our study is undoubtedly limited by the variability in quality of the sleep endoscopy videos; many of the videos provided only fleeting glimpses of the areas of obstruction in question. Standardizing the DISE procedure should improve this. We propose 5 views, based on our scoring scheme: (1) adenoid: posterior view from nasal cavity; (2) velum: inferior view from nasopharynx; assessing anterior-posterior obstruction; (3) LPW: inferior view from velum; assessing LPW-tonsillar obstruction; (4) tongue base: inferior view from oropharynx; assessing anteroposterior obstruction; and (5) supraglottis: inferior view with tongue base (if obstructing) out of the way, with and without jaw thrust.

Representative views shown in Figure 3 can be used as references for the ordinal degree of obstruction. As long as these standard views are achieved during the recorded sleep endoscopy, post hoc scoring can be performed robustly. Additional investigation can be performed at the time of sleep endoscopy to further evaluate the subjective degree and location of obstruction.

Obstruction was rated at both the minimum and maximum points of obstruction in the respiratory cycle. We included a measure of minimum and maximum obstruction to allow for possible future analyses of dynamic airway change associated with sleep apnea. Although previous rating schemes have focused only on maximum degree of obstruction,11 there is no a priori reason to exclude the level of minimum obstruction as an independent contributor. We intend to use this scoring scheme, whereby both minimum and maximum obstruction at each site are calculated, to investigate this possibility in the future. Should we find that the minimum obstruction scores are either consistently unreliable or that they do not contribute to the evaluation of sleep-related breathing disorder, they may be removed in later simplified iterations of the scoring system. In this preliminary study, we found that evaluation of maximum obstruction is more reliable, with minimum obstruction at the velum being the least reliable score. Although assessment of both phases of the respiratory cycle may reveal a meaningful effect of dynamic motion within those airway sites, further improvement of the reliability of minimum obstructive ratings is necessary before such dynamic evaluation can be performed.

Validation of any scoring system for DISE is difficult, owing to the lack of a gold standard for assessing degree and site of airway obstruction in OSA. We attempted to address external validity by using the scoring system to objectively determine the severity of obstruction (the total obstructive score) and presence of multilevel obstruction, which were each correlated with polysomnographic indices of severity of obstruction (AHI and oxygen saturation nadir). Although direct correlation between polysomnographic parameters and objective obstructive scores using a standardized rating system has been shown in adults,21,22 association of worse polysomnographic scores with higher-degree or multilevel obstruction has only been suggested in children.10 We demonstrate herein that there exists a significant correlation between total maximum obstructive score and oxygen saturation nadir. An association between total obstructive score and AHI is suggested but is not statistically significant.

When the scoring system was used to objectively detect multilevel obstruction, this association was substantially stronger. Our scoring system was useful for detecting multilevel obstruction; we found that multiple sites of obstruction within a region such as the adenoid-velum-LPW or tongue base–supraglottis complex is not associated with worsened sleep apnea, but that having obstruction in both of these larger complexes is. Merely having 2 of the 5 sites with obstructive scores of at least 2.0 did not associate with worse polysomnographic parameters; only when obstruction was noted in both the adenoid-velum-LPW and tongue base–supraglottis complexes did multilevel obstruction correlate significantly with higher AHI and low oxygen saturation. This is consistent with previous descriptions of the effect of multiple obstructive levels on sleep apnea indices.10 Taken together, these associations provide evidence to support the external validity of this scoring system.

Limitations of this study include the small sample size and demographic heterogeneity, with wide variation in patient age and OSA severity, and a high percentage of children with comorbidities. The number of patients, however, is comparable with that used for validation of the VOTE scoring system for scoring adult sleep endoscopy,14 and the narrow variance with substantial interrater and intrarater reliability exhibited in spite of the small sample size attests to the robustness of the scoring system. Likewise, the external validation, particularly with respect to the correlation of AHI with multilevel obstruction, is statistically significant despite the small sample size; however, we were unable to control for potential confounders in this analysis. Finally, the large proportion of children with comorbidities in this cohort could limit applicability of this system to less complicated community populations.

Development of a validated method to quantify degree of obstruction in a site-specific manner is important for standardization of research, communication between clinicians, and most important for reliable and reproducible clinical assessment of DISE to determine subsequent therapeutic intervention. The system proposed herein demonstrates both internal reliability, with comparable interrater and intrarater reliability compared with other similar systems; and external validity, with significant correlation to preoperative polysomnographic indices. We intend this to be a widely applicable system; despite the range of experience of the 4 raters, with 2 attending physicians and 2 trainees, there was no statistically significant difference in the intrarater reliability of the 4 raters, nor was there any systematic difference in the overall total obstructive score determined by the raters (data not shown). Our next step will be to expand the validation study to include raters from multiple institutions and backgrounds in order to confirm the universality of this system and its reliability across different clinicians.

Standardizing a robust technique for performing and scoring DISE will be an iterative process; establishment of a scoring system will aid in the optimization of the DISE technique and vice versa. One particularly important aspect of DISE is the selection of an appropriate anesthetic agent. Dexmedetomidine, together with ketamine, has been proposed as an alternative to propofol and fentanyl, owing to reported benefits in pharyngeal tone and respiratory drive.9,23 An objective scoring system, such as that proposed herein, could be used to compare these regimens in a controlled manner. Most important, a robust method for performing and interpreting sleep endoscopy will provide an organized system for evaluating and treating challenging cases of pediatric obstructive sleep apnea.

Our proposed scoring system has substantial interrater and intrarater reliability for maximum obstructive scores at 5 sites in the upper airway. When these individual-site scores are aggregated, or used to determine multilevel obstruction, a greater degree of obstruction correlates with worsened polysomnographic parameters. We propose that this scoring system be used together with a standardized DISE method for reliable, quantitative, and reproducible evaluation of sleep endoscopy for use in surgical planning, communication between clinicians, and research.

Corresponding Author: Dylan K. Chan, MD, PhD, Division of Pediatric Otolaryngology, Department of Otolaryngology–Head and Neck Surgery, University of California, 2233 Post St, Third Floor, PO Box 1225, San Francisco, CA 94115 (dylan.chan@ucsf.edu).

Submitted for Publication: September 5, 2013; final revision received March 6, 2014; accepted March 17, 2014.

Published Online: May 8, 2014. doi:10.1001/jamaoto.2014.612.

Author Contributions: Dr Chan had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: All authors.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: Chan, Liming, Horn.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Chan, Horn.

Administrative, technical, or material support: All authors.

Study supervision: Chan, Horn, Parikh.

Conflict of Interest Disclosures: None reported.

Disclaimer: The views expressed herein are those of the authors and do not reflect the official policy of the Department of the Army, the Department of Defense or the US Government.

Additional Contributions: Babette Saltzman, PhD, Craniofacial Center, provided statistical guidance, and Maida Chen, MD, Pulmonary and Sleep Medicine (both at Seattle Children’s Hospital), contributed to the polysomnographic findings. They were not reimbursed for their contributions.

Lumeng  JC, Chervin  RD.  Epidemiology of pediatric obstructive sleep apnea. Proc Am Thorac Soc. 2008;5(2):242-252.
PubMed   |  Link to Article
Marcus  CL, Brooks  LJ, Draper  KA,  et al; American Academy of Pediatrics.  Diagnosis and management of childhood obstructive sleep apnea syndrome. Pediatrics. 2012;130(3):e714-e755.
PubMed   |  Link to Article
Goldstein  NA, Fatima  M, Campbell  TF, Rosenfeld  RM.  Child behavior and quality of life before and after tonsillectomy and adenoidectomy. Arch Otolaryngol Head Neck Surg. 2002;128(7):770-775.
PubMed   |  Link to Article
Friedman  M, Wilson  M, Lin  HC, Chang  HW.  Updated systematic review of tonsillectomy and adenoidectomy for treatment of pediatric obstructive sleep apnea/hypopnea syndrome. Otolaryngol Head Neck Surg. 2009;140(6):800-808.
PubMed   |  Link to Article
Teo  DT, Mitchell  RB.  Systematic review of effects of adenotonsillectomy on cardiovascular parameters in children with obstructive sleep apnea. Otolaryngol Head Neck Surg. 2013;148(1):21-28.
PubMed   |  Link to Article
Bhattacharjee  R, Kheirandish-Gozal  L, Spruyt  K,  et al.  Adenotonsillectomy outcomes in treatment of obstructive sleep apnea in children: a multicenter retrospective study. Am J Respir Crit Care Med. 2010;182(5):676-683.
PubMed   |  Link to Article
Croft  CB, Pringle  M.  Sleep nasendoscopy: a technique of assessment in snoring and obstructive sleep apnoea. Clin Otolaryngol Allied Sci. 1991;16(5):504-509.
PubMed   |  Link to Article
Durr  ML, Meyer  AK, Kezirian  EJ, Rosbe  KW.  Drug-induced sleep endoscopy in persistent pediatric sleep-disordered breathing after adenotonsillectomy. Arch Otolaryngol Head Neck Surg. 2012;138(7):638-643.
PubMed   |  Link to Article
Truong  MT, Woo  VG, Koltai  PJ.  Sleep endoscopy as a diagnostic tool in pediatric obstructive sleep apnea. Int J Pediatr Otorhinolaryngol. 2012;76(5):722-727.
PubMed   |  Link to Article
Ulualp  SO, Szmuk  P.  Drug-induced sleep endoscopy for upper airway evaluation in children with obstructive sleep apnea. Laryngoscope. 2013;123(1):292-297.
PubMed   |  Link to Article
Kezirian  EJ, Hohenhorst  W, de Vries  N.  Drug-induced sleep endoscopy: the VOTE classification. Eur Arch Otorhinolaryngol. 2011;268(8):1233-1236.
PubMed   |  Link to Article
Vicini  C, De Vito  A, Benazzo  M,  et al.  The nose oropharynx hypopharynx and larynx (NOHL) classification: a new system of diagnostic standardized examination for OSAHS patients. Eur Arch Otorhinolaryngol. 2012;269(4):1297-1300.
PubMed   |  Link to Article
Gillespie  MB, Reddy  RP, White  DR, Discolo  CM, Overdyk  FJ, Nguyen  SA.  A trial of drug-induced sleep endoscopy in the surgical management of sleep-disordered breathing. Laryngoscope. 2013;123(1):277-282.
PubMed   |  Link to Article
Rodriguez-Bruno  K, Goldberg  AN, McCulloch  CE, Kezirian  EJ.  Test-retest reliability of drug-induced sleep endoscopy. Otolaryngol Head Neck Surg. 2009;140(5):646-651.
PubMed   |  Link to Article
Kezirian  EJ, White  DP, Malhotra  A, Ma  W, McCulloch  CE, Goldberg  AN.  Interrater reliability of drug-induced sleep endoscopy. Arch Otolaryngol Head Neck Surg. 2010;136(4):393-397.
PubMed   |  Link to Article
Iber  C, Ancoli-Osrael  S, Chesson  AL  Jr,  et al. The AASM Manual for the Scoring of Sleep and Associate Events: Rules, Terminology, and Technical Specifications. Westchester, IL: American Academy of Sleep Medicine; 2007.
Landis  JR, Koch  GG.  The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159-174.
PubMed   |  Link to Article
Chan  DK, Jan  TA, Koltai  PJ.  Effect of obesity and medical comorbidities on outcomes after adjunct surgery for obstructive sleep apnea in cases of adenotonsillectomy failure. Arch Otolaryngol Head Neck Surg. 2012;138(10):891-896.
PubMed   |  Link to Article
Yoon  PJ, Starr  JR, Perkins  JA, Bloom  D, Sie  KC.  Interrater and intrarater reliability in the evaluation of velopharyngeal insufficiency within a single institution. Arch Otolaryngol Head Neck Surg. 2006;132(9):947-951.
PubMed   |  Link to Article
Sie  KC, Starr  JR, Bloom  DC,  et al.  Multicenter interrater and intrarater reliability in the endoscopic evaluation of velopharyngeal insufficiency. Arch Otolaryngol Head Neck Surg. 2008;134(7):757-763.
PubMed   |  Link to Article
Bachar  G, Feinmesser  R, Shpitzer  T, Yaniv  E, Nageris  B, Eidelman  L.  Laryngeal and hypopharyngeal obstruction in sleep disordered breathing patients, evaluated by sleep endoscopy. Eur Arch Otorhinolaryngol. 2008;265(11):1397-1402.
PubMed   |  Link to Article
Bachar  G, Nageris  B, Feinmesser  R,  et al.  Novel grading system for quantifying upper-airway obstruction on sleep endoscopy. Lung. 2012;190(3):313-318.
PubMed   |  Link to Article
Mahmoud  M, Gunter  J, Donnelly  LF, Wang  Y, Nick  TG, Sadhasivam  S.  A comparison of dexmedetomidine with propofol for magnetic resonance imaging sleep studies in children. Anesth Analg. 2009;109(3):745-753.
PubMed   |  Link to Article

Figures

Place holder to copy figure label and caption
Figure 1.
Scoring Form and Guidelines

The form (A) and guidelines (B) used for post hoc scoring of drug-induced sleep endoscopy are presented. Some data fields in the scoring form, including turbinate and tonsil size, and use of adjunct airway, were not included in our analysis. LPW indicates lateral pharyngeal wall.

Graphic Jump Location
Place holder to copy figure label and caption
Figure 2.
Patient Profile

Individual patient obstructive, polysomnographic, and demographic profiles are shown. Mean obstructive score at each of the 5 evaluated sites (adenoid, velum, lateral pharyngeal wall [LPW], tongue base, and supraglottis) is graphically represented as either less than 2.0 (white box) or 2.0 or more (black box). Multilevel obstruction group (1, 2, or 3) was defined as follows: Group 1: No or 1 site with obstructive score greater than 2; group 2: 2 or more sites with obstructive score greater than 2 but confined to upper (adenoid, velum, LPW) or lower (tongue base, supraglottis) airway complex; group 3: 2 or more sites with obstructive score greater than 2, encompassing both upper and lower airway complexes. Total indicates total obstructive score (sum of obstructive scores at each of the 5 sites) from sleep endoscopy. AHI indicates apnea-hypopnea index from preoperative polysomnography. Oxygen nadir is low oxygen saturation from preoperative polysomnography. Age in years, body mass index (BMI; calculated as weight in kilograms divided by height in meters squared), prior sleep surgery (N, none; A, adenoidectomy; or TA: adenotonsillectomy), and medical comorbidities (neuromuscular, syndromic, pulmonary, or chromosomal) are shown.

Graphic Jump Location
Place holder to copy figure label and caption
Figure 3.
Representative Images

Each of the 4 ordinal scores at each of the 5 upper-airway sites are shown.

Graphic Jump Location
Place holder to copy figure label and caption
Figure 4.
Internal Reliability

Interrater (A) and intrarater (B) reliability assessed with the intraclass correlation coefficient (ICC) are shown for the 5 sites of upper airway obstruction scored for videos of drug-induced sleep endoscopy. Ratings for minimum (light blue) and maximum (white) obstruction at each site are shown. 1.0 indicates perfect correlation; 0, no correlation. Error bars indicate 95% CIs for each measurement. LPW indicates lateral pharyngeal wall.

Graphic Jump Location
Place holder to copy figure label and caption
Figure 5.
Multilevel Obstruction

Multilevel obstruction was determined based solely on objective site-specific obstructive scores. Apnea-hypopnea index (AHI) and low oxygen saturation values for patients in the 3 multilevel obstruction groups were compared. Multilevel obstruction group (1, 2, or 3) was defined as follows: group 1, no or 1 site with obstructive score greater than 2; group 2, 2 or more sites with obstructive score greater than 2 but confined to upper (adenoid, velum, lateral pharyngeal wall [LPW]) or lower (tongue base, supraglottis) airway complex; group 3, 2 or more sites with obstructive score greater than 2, encompassing both upper and lower airway complexes.aP = .02.

Graphic Jump Location

Tables

References

Lumeng  JC, Chervin  RD.  Epidemiology of pediatric obstructive sleep apnea. Proc Am Thorac Soc. 2008;5(2):242-252.
PubMed   |  Link to Article
Marcus  CL, Brooks  LJ, Draper  KA,  et al; American Academy of Pediatrics.  Diagnosis and management of childhood obstructive sleep apnea syndrome. Pediatrics. 2012;130(3):e714-e755.
PubMed   |  Link to Article
Goldstein  NA, Fatima  M, Campbell  TF, Rosenfeld  RM.  Child behavior and quality of life before and after tonsillectomy and adenoidectomy. Arch Otolaryngol Head Neck Surg. 2002;128(7):770-775.
PubMed   |  Link to Article
Friedman  M, Wilson  M, Lin  HC, Chang  HW.  Updated systematic review of tonsillectomy and adenoidectomy for treatment of pediatric obstructive sleep apnea/hypopnea syndrome. Otolaryngol Head Neck Surg. 2009;140(6):800-808.
PubMed   |  Link to Article
Teo  DT, Mitchell  RB.  Systematic review of effects of adenotonsillectomy on cardiovascular parameters in children with obstructive sleep apnea. Otolaryngol Head Neck Surg. 2013;148(1):21-28.
PubMed   |  Link to Article
Bhattacharjee  R, Kheirandish-Gozal  L, Spruyt  K,  et al.  Adenotonsillectomy outcomes in treatment of obstructive sleep apnea in children: a multicenter retrospective study. Am J Respir Crit Care Med. 2010;182(5):676-683.
PubMed   |  Link to Article
Croft  CB, Pringle  M.  Sleep nasendoscopy: a technique of assessment in snoring and obstructive sleep apnoea. Clin Otolaryngol Allied Sci. 1991;16(5):504-509.
PubMed   |  Link to Article
Durr  ML, Meyer  AK, Kezirian  EJ, Rosbe  KW.  Drug-induced sleep endoscopy in persistent pediatric sleep-disordered breathing after adenotonsillectomy. Arch Otolaryngol Head Neck Surg. 2012;138(7):638-643.
PubMed   |  Link to Article
Truong  MT, Woo  VG, Koltai  PJ.  Sleep endoscopy as a diagnostic tool in pediatric obstructive sleep apnea. Int J Pediatr Otorhinolaryngol. 2012;76(5):722-727.
PubMed   |  Link to Article
Ulualp  SO, Szmuk  P.  Drug-induced sleep endoscopy for upper airway evaluation in children with obstructive sleep apnea. Laryngoscope. 2013;123(1):292-297.
PubMed   |  Link to Article
Kezirian  EJ, Hohenhorst  W, de Vries  N.  Drug-induced sleep endoscopy: the VOTE classification. Eur Arch Otorhinolaryngol. 2011;268(8):1233-1236.
PubMed   |  Link to Article
Vicini  C, De Vito  A, Benazzo  M,  et al.  The nose oropharynx hypopharynx and larynx (NOHL) classification: a new system of diagnostic standardized examination for OSAHS patients. Eur Arch Otorhinolaryngol. 2012;269(4):1297-1300.
PubMed   |  Link to Article
Gillespie  MB, Reddy  RP, White  DR, Discolo  CM, Overdyk  FJ, Nguyen  SA.  A trial of drug-induced sleep endoscopy in the surgical management of sleep-disordered breathing. Laryngoscope. 2013;123(1):277-282.
PubMed   |  Link to Article
Rodriguez-Bruno  K, Goldberg  AN, McCulloch  CE, Kezirian  EJ.  Test-retest reliability of drug-induced sleep endoscopy. Otolaryngol Head Neck Surg. 2009;140(5):646-651.
PubMed   |  Link to Article
Kezirian  EJ, White  DP, Malhotra  A, Ma  W, McCulloch  CE, Goldberg  AN.  Interrater reliability of drug-induced sleep endoscopy. Arch Otolaryngol Head Neck Surg. 2010;136(4):393-397.
PubMed   |  Link to Article
Iber  C, Ancoli-Osrael  S, Chesson  AL  Jr,  et al. The AASM Manual for the Scoring of Sleep and Associate Events: Rules, Terminology, and Technical Specifications. Westchester, IL: American Academy of Sleep Medicine; 2007.
Landis  JR, Koch  GG.  The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159-174.
PubMed   |  Link to Article
Chan  DK, Jan  TA, Koltai  PJ.  Effect of obesity and medical comorbidities on outcomes after adjunct surgery for obstructive sleep apnea in cases of adenotonsillectomy failure. Arch Otolaryngol Head Neck Surg. 2012;138(10):891-896.
PubMed   |  Link to Article
Yoon  PJ, Starr  JR, Perkins  JA, Bloom  D, Sie  KC.  Interrater and intrarater reliability in the evaluation of velopharyngeal insufficiency within a single institution. Arch Otolaryngol Head Neck Surg. 2006;132(9):947-951.
PubMed   |  Link to Article
Sie  KC, Starr  JR, Bloom  DC,  et al.  Multicenter interrater and intrarater reliability in the endoscopic evaluation of velopharyngeal insufficiency. Arch Otolaryngol Head Neck Surg. 2008;134(7):757-763.
PubMed   |  Link to Article
Bachar  G, Feinmesser  R, Shpitzer  T, Yaniv  E, Nageris  B, Eidelman  L.  Laryngeal and hypopharyngeal obstruction in sleep disordered breathing patients, evaluated by sleep endoscopy. Eur Arch Otorhinolaryngol. 2008;265(11):1397-1402.
PubMed   |  Link to Article
Bachar  G, Nageris  B, Feinmesser  R,  et al.  Novel grading system for quantifying upper-airway obstruction on sleep endoscopy. Lung. 2012;190(3):313-318.
PubMed   |  Link to Article
Mahmoud  M, Gunter  J, Donnelly  LF, Wang  Y, Nick  TG, Sadhasivam  S.  A comparison of dexmedetomidine with propofol for magnetic resonance imaging sleep studies in children. Anesth Analg. 2009;109(3):745-753.
PubMed   |  Link to Article

Correspondence

CME


You need to register in order to view this quiz.
Submit a Comment

Multimedia

Some tools below are only available to our subscribers or users with an online account.

1,556 Views
0 Citations

Related Content

Customize your page view by dragging & repositioning the boxes below.

Articles Related By Topic
Related Collections
Jobs
JAMAevidence.com

The Rational Clinical Examination: Evidence-Based Clinical Diagnosis
Original Article: Can the Clinical History Distinguish Between Organic and Functional Dyspepsia?

The Rational Clinical Examination: Evidence-Based Clinical Diagnosis
Original Article: Can the Clinical History Distinguish Between Organic and Functional Dyspepsia?

×