0
We're unable to sign you in at this time. Please try again in a few minutes.
Retry
We were able to sign you in, but your subscription(s) could not be found. Please try again in a few minutes.
Retry
There may be a problem with your account. Please contact the AMA Service Center to resolve this issue.
Contact the AMA Service Center:
Telephone: 1 (800) 262-2350 or 1 (312) 670-7827  *   Email: subscriptions@jamanetwork.com
Error Message ......
Original Article |

Generation of Consensus in the Application of a Rating Scale to Nasendoscopic Assessment of Velopharyngeal Function FREE

David D. Tieu, MD; Mark E. Gerber, MD; Henry A. Milczuk, MD; Sanjay R. Parikh, MD; Jonathan A. Perkins, DO; Patricia J. Yoon, MD; Kathleen C. Y. Sie, MD
[+] Author Affiliations

Author Affiliations: Division of Pediatric Otolaryngology (Drs Tieu, Parikh, Perkins, and Sie) and Childhood Communication Center (Dr Sie), Seattle Children's Hospital, University of Washington School of Medicine; Pediatric Otolaryngology–Head and Neck Surgery, Northshore University Health System, Chicago, Illinois (Dr Gerber); Department of Otolaryngology–Head and Neck Surgery, Oregon Health and Sciences University, Portland (Dr Milczuk); and Department of Pediatric Otolaryngology, Children's Hospital Colorado, University of Colorado, Aurora (Dr Yoon).


Arch Otolaryngol Head Neck Surg. 2012;138(10):923-928. doi:10.1001/archotol.2013.203.
Text Size: A A A
Published online

Objective To generate consensus ratings of velopharyngeal function on nasendoscopy (NE) with the goal of creating a video instruction tool.

Methods The American Society of Pediatric Otolaryngology Velopharyngeal Insufficiency Study Group convened to identify NE segments to be included in an instructional video. Of 24 segments reviewed, 11 were selected based on the quality of the examinations and spectrum of closure patterns. Participating otolaryngologists independently rated NE segments using the Golding-Kushner scale. The participants then convened and rated each of the NE segments as a group. Thirty-nine members of the American Society of Pediatric Otolaryngology met and agreed with the group ratings, creating a consensus standard.

Results Individual scores for palate and lateral wall motion showed high variability, ranging from 0 to 6 points difference from the consensus. Variability was also seen for the following qualitative findings: the Passavant ridge, aberrant pulsations, and dorsal palatal notch. The individual ratings are presented graphically to demonstrate the range of individual responses as well as to compare responses to the consensus ratings. No further changes were made to the proposed consensus ratings when reviewed by the larger group.

Conclusions Rating of NE evaluations of velopharyngeal function was variable among a group of pediatric otolaryngologists experienced in treating velopharyngeal insufficiency. These results highlight the need to develop a standardized method of reporting NE findings for velopharyngeal insufficiency. Despite this, consensus ratings were achieved that will facilitate development of a video instruction tool.

Figures in this Article

Velopharyngeal (VP) closure has an important role in normal speech production. Incomplete closure of the velopharyngeal complex, known as velopharyngeal insufficiency (VPI), is characterized by hypernasal resonance and nasal air emissions or turbulence during speech production. This dysfunction may be a result of anatomical abnormalities, cleft palate, or neuromuscular dysfunction.

The diagnosis of VPI remains a subjective determination on the basis of perceptual speech assessment.1,2 Perceptual speech assessment involves the evaluation of all the components of speech production (including voice), articulation (including compensatory misarticulations, motor speech, oral motor sequencing), velopharyngeal function, and overall speech intelligibility. Description of VP function includes ratings of resonance, nasal emissions or turbulence, and overall VPI severity. Features, such as hypernasal resonance and nasal rustle, were shown to be predictive of relative VP gap size in the absence of instrumental assessment.3 Although perceptual evaluation is performed by trained speech language pathologists, it is subjective. Furthermore, the degree of VPI does not reliably predict the degree of VP closure.

Estimating the amount of VP closure is important when surgical intervention is being considered. Alteration of the VP complex has the potential to compromise upper airway patency, leading to hyponasal resonance, nasal obstruction, or obstructive sleep apnea. Because the degree of VP closure varies among individual patients, other authors have advocated for surgical approaches that are tailored to each clinical situation.46 For example, individuals who have a larger VP gap require more tissue augmentation to effectively treat the VPI.

Nasendoscopy provides a useful tool for direct visualization of the VP during speech, allowing for a 2-dimensional view of the VP complex. This allows the endoscopist to evaluate lateral wall and palatal motion during speech; determine the presence of a notch on the nasal surface of the palate; see the Passavant ridge and aberrant pulsations; and estimate gap size and shape. For example, the presence of a notch on the nasal palate surface suggests sagittal orientation of the levator veli palatini, which can be corrected with palatal procedures. Patients with evidence of transverse orientation of the levator veli palatini musculature may be candidates for pharyngeal or palatopharyngeal speech procedures. The nasendoscopic (NE) findings may be helpful in selecting the type of surgical procedure in addition to providing the surgeon with an estimate of degree of VP augmentation required.

An international working group, which was convened to address the description of VP function, published its recommendations in 1990.7 The group described a rating system in which the maximal VP movement during speech is rated relative to the resting VP gap. The scale subjectively rates right and left lateral wall movement on a scale from 0 to 0.5, with 0 denoting no movement and 0.5 denoting movement to the midline; right and left palate movement on a scale from 0 to 1.0, with 0 denoting no movement and 1.0 denoting movement to the posterior pharyngeal wall; and posterior wall movement on a scale of 0 to 1.0, with 0 denoting no movement and 1.0 denoting movement to meet the soft palate (Figure 1).8 Lateral wall motion and palatal motion are rated separately, and other characteristics of the closure pattern, such as gap shape and notch on the nasal surface of the palate, are indicated. This rating scale is intended to be applied to nasendoscopic and fluoroscopic examinations. Although instrumental assessment of VP function is more objective than perceptual speech assessment, there continues to be wide variation in the description of nasendoscopic assessment of VP function. Previous studies examining the reliability of NE evaluation have suggested that providing feedback during the evaluation or allowing group participation statistically improves reliability.9 Studies of intrarater and interrater reliability have generally shown that intrarater reliability is moderately good regardless of the level of training or amount of experience in managing patients with VPI. The interrater reliability was greatest among 2 experienced attending otolaryngologists at a single institution.10 In a study comparing the use of the Golding-Kushner scale among 16 otolaryngologists at 8 institutions, the interrater reliability was weaker.11

Place holder to copy figure label and caption
Graphic Jump Location

Figure 1. Sample of an nasendoscopic view of the velopharynx and use of the Golding-Kushner scale. Velopharynx at rest (A), during speech production (B), at rest with superimposed grid (C), and during speech production with superimposed grid (D). In this sample, the palate movement is rated as 0.2; the posterior pharyngeal wall is rated 0.25; and the right and left lateral pharyngeal walls are rated 0.15 and 0.2, respectively. (For an explanation of the rating system, see the introductory section of the article.) The gap size is estimated to be 60%. Reproduced with permission from Sie and Chen.8

On the basis of the findings of previous studies and the need to have a reliable method of communicating the nasendoscopic findings of VP function for patient care, clinical training, and research, the American Society of Pediatric Otolaryngology (ASPO) VPI Study Group set out to create a video instruction tool. The ASPO VPI Study Group proposed a series of NE segments with consensus ratings of lateral pharyngeal, palatal, and posterior pharyngeal wall movement, and descriptions of the nasal surface of the palate, Passavant ridge, aberrant pulsations, and pattern of closure. A minor modification of the Golding-Kushner scale was used. Specifically, instead of assigning a decimal rating between 0 and 1 to the palate and lateral walls, whole numbers between 0 and 10 were used to simplify calculations. The goal of the NE video instruction tool is to facilitate a standardized approach to describing VP function. This may become an important tool in patient care, clinical training, and clinical research.

Institutional review board exemption was obtained at Seattle Children's Hospital, Seattle, Washington. The ASPO VPI Study Group convened at the 2009 Annual ASPO meeting to identify NE segments to be included in a teaching video. Twenty-four endoscopic examinations were reviewed for possible inclusion. Eleven segments were selected on the quality of the examination and characteristics of VP closure pattern. After the 11 segments were identified, the participating otolaryngologists independently rated each segment using the Golding-Kushner scale. The group then reconvened and rated each of the segments as a group to generate consensus ratings. Items rated included right and left lateral pharyngeal wall movement, right and left palate movement, posterior pharyngeal wall movement, and an estimate of gap size. Raters were asked to judge the presence or absence of the Passavant ridge, midline palatal notch, aberrant pulsations, and shape of VP closure.

Finally, members of ASPO were invited to attend a meeting of the ASPO VPI Study Group at that 2010 Annual ASPO meeting. Thirty-nine ASPO members participated. The 11 segments and the consensus ratings generated by the 2009 participants were presented. The items are presented graphically to demonstrate individual responses as well as the consensus ratings. All 39 participants agreed with the consensus ratings proposed, thus creating a consensus standard. Descriptive statistics were used to describe individual ratings for each characteristic of the Golding-Kushner scale used.

Using the 11 selected nasendoscopy segments, 5 otolaryngologists (M.E.G., H.A.M., S.R.P., P.J.Y., and K.C.Y.S.) rated each component of VP function using the Golding-Kushner scale. Prior to generation of the consensus ratings, these ratings appeared to have a broad distribution. For example, ratings of palatal and lateral wall motion were variable among each rater (Figure 2). There was a range of up to 7 points when compared with the consensus rating (Figure 3). This variability was also present when rating qualitative factors, such as posterior pharyngeal wall motion and gap size. Using descriptive statistics to compare individual ratings with consensus ratings, there continues to be a broad range of ratings for each component of the Golding-Kushner scale (Table 1 and Table 2).

Place holder to copy figure label and caption
Graphic Jump Location

Figure 2. Sample of individual and consensus ratings for palatal and lateral wall motion. Right palate motion (A), left palate motion (B), right lateral wall motion (C), and left lateral wall motion (D).

Place holder to copy figure label and caption
Graphic Jump Location

Figure 3. Individual ratings for each of the 11 video segments using the Golding-Kushner scale in comparison to the consensus ratings. Right palate motion (A), left palate motion (B), right lateral wall motion (C), and left lateral wall motion (D).

Table Graphic Jump LocationTable 1. Descriptive Statistics of Each Qualitative Componenta
Table Graphic Jump LocationTable 2. Description of Differences Between Individual Ratings and Consensus Ratings

Ratings of the presence of carotid pulsations, the Passavant ridge, and a notch on the nasal surface of the soft palate showed high agreement with the consensus ratings, with 85%, 76%, and 78% in agreement with the consensus, respectively (Table 3). Interestingly, the characterization of the shape of the gap closure demonstrated the lowest percentage agreement with consensus ratings with 54% in agreement with the consensus. This variability may be due to the more subjective nature of this rating. As a result of this variability among otolaryngologists, we showed a series of video segments to highlight the qualitative components of the Golding-Kushner scale and to propose a teaching video to facilitate these evaluations (Figure 4).

Place holder to copy figure label and caption
Graphic Jump Location

Figure 4. Example of velopharyngeal closure with various features from the American Society of Pediatric Otolaryngology Velopharyngeal Insufficiency Study Group teaching tool. A and B, Velopharyngeal complex during rest. C, The presence of the Passavant ridge during closure. D, Presence of the notch on the nasal surface of soft palate during closure.

Table Graphic Jump LocationTable 3. Descriptive Statistics for Presence of Velopharyngeal Gap, Aberrant Pulsations, Passavant Ridge, and Notch on Nasal Surface of Soft Palate

When assessing the range of VP function included in the 11 segments, descriptive statistics of the consensus ratings were evaluated. When examining right and left palatal motion and gap size, there appears to be a broad range of ratings from 2% to 10% and 5% to 80%, respectively. Specifically, when examining the gap size, consensus ratings ranged from 5% to 80%, suggesting that the selected video segments are collectively reflective of a broad range of degree of closure.

There is no objective clinical measure of VP function that closely correlates with perceptual speech assessment. It is important to understand the reliability of the clinical tools commonly used in the evaluation of patients with VPI. Because the evaluation of VP function is relevant for clinical and surgical planning as well as outcomes assessment, otolaryngologists who routinely treat patients with VPI emphasize the importance of developing a standardized reporting system.7

A 2008 multicenter study, consisting of a large group of otolaryngologists who routinely treat children with VPI, used this standardized reporting system to evaluate the interrater and intrarater reliability.11 Results demonstrate good intrarater reliability when using the Golding-Kushner scale for rating VP function on the basis of NE. Moreover, the reliability of rating qualitative characteristics, such as the presence of the Passavant ridge, aberrant pulsations, and notch on the nasal surface of the soft palate, was variable.

The Golding-Kushner scale is the most detailed scale described for the assessment of the multidimensional mobility of the VP complex. The scale is appealing because it can be applied to both nasendoscopic and radiographic tests of VP function, but interrater reliability needs improvement as seen in both single10 and multi-institutional studies.11

The results of our study suggest that rating of NE evaluations of VP function was variable even among a group of experienced otolaryngologists who treat VPI. During the group discussion, participants expressed their previous misconceptions of the Golding-Kushner rating scale. The results highlight the need to develop a standardized method of describing the nasendoscopic assessment of VP function. The fact that the larger ASPO VPI Study Group in 2010 reviewed the consensus ratings without any modifications suggests that the ratings were broadly accepted.

We present ratings for 11 video segments, generated from a group of experts, that were then reviewed and agreed on by a larger group of pediatric otolaryngologists. These ratings will be used to produce an ASPO VPI video instruction tool to demonstrate application of the Golding-Kushner scale to NE examinations. This could be used as a self-administered tutorial available online.

Follow-up studies are required to demonstrate the effect of this video instruction tool on the variability of this rating system. Overall, standardized reporting of VPI is important to determine disease impact, VPI severity, and the effectiveness of therapy. Studying the effect of this video instruction tool will allow us to determine whether the Golding-Kushner scale can be a reliable rating system.

In conclusion, there was wide variability in application of the Golding-Kushner scale to nasendoscopic examination of VP function among a small group of pediatric otolaryngologists. This group proposed consensus ratings for 11 videoendoscopic segments that were reviewed and accepted by a larger group of pediatric otolaryngologists, thereby generating consensus standard ratings. Generation of the consensus standard is the first step in the development of a video instruction tool for evaluation of VP function on NE.

Correspondence: Kathleen C. Y. Sie, MD, Division of Pediatric Otolaryngology, Seattle Children's Hospital, University of Washington School of Medicine, 4800 Sandpoint Way NE, Seattle, WA 98105 (kathleen.sie@seattlechildrens.org).

Submitted for Publication: April 25, 2012; final revision received June 14, 2012; accepted August 12, 2012.

Author Contributions: Drs Tieu, Yoon, and Sie had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Tieu, Milczuk, Perkins, and Sie. Acquisition of data: Tieu, Gerber, Parikh, Yoon, and Sie. Analysis and interpretation of data: Tieu, Parikh, Perkins, and Sie. Drafting of the manuscript: Tieu and Parikh. Critical revision of the manuscript for important intellectual content: Tieu, Gerber, Milczuk, Parikh, Perkins, and Yoon. Statistical analysis: Sie. Administrative, technical, and material support: Tieu, Parikh, Perkins, and Sie. Study supervision: Yoon and Sie.

Financial Disclosure: None reported.

Previous Presentation: This article was presented at the American Society of Pediatric Otolaryngology Annual Meeting; April 20, 2012; San Diego, California.

Additional Contributions: Rose Mary S. Stocks, MD, provided input during generation of consensus ratings; Babette Saltzman, PhD, performed the statistical analysis and compiled descriptive statistics; and Carrie Wachob, BS, generated the figures and multimedia.

Willging JP. Velopharyngeal insufficiency.  Curr Opin Otolaryngol Head Neck Surg. 2003;11(6):452-455
PubMed   |  Link to Article
Rudnick EF, Sie KC. Velopharyngeal insufficiency: current concepts in diagnosis and management.  Curr Opin Otolaryngol Head Neck Surg. 2008;16(6):530-535
PubMed   |  Link to Article
Kummer AW, Briggs M, Lee L. The relationship between the characteristics of speech and velopharyngeal gap size.  Cleft Palate Craniofac J. 2003;40(6):590-596
PubMed   |  Link to Article
Peat BG, Albery EH, Jones K, Pigott RW. Tailoring velopharyngeal surgery: the influence of etiology and type of operation.  Plast Reconstr Surg. 1994;93(5):948-953
PubMed
Shprintzen RJ, Lewin ML, Croft CB,  et al.  A comprehensive study of pharyngeal flap surgery: tailor made flaps.  Cleft Palate J. 1979;16(1):46-55
PubMed
Sloan GM. Posterior pharyngeal flap and sphincter pharyngoplasty: the state of the art.  Cleft Palate Craniofac J. 2000;37(2):112-122
PubMed   |  Link to Article
Golding-Kushner KJ, Argamaso RV, Cotton RT,  et al.  Standardization for the reporting of nasopharyngoscopy and multiview videofluoroscopy: a report from an International Working Group.  Cleft Palate J. 1990;27(4):337-348
PubMed   |  Link to Article
Sie KC, Chen EY. Management of velopharyngeal insufficiency: development of a protocol and modifications of sphincter pharyngoplasty.  Facial Plast Surg. 2007;23(2):128-139
PubMed   |  Link to Article
D’Antonio LL, Marsh JL, Province MA, Muntz HR, Phillips CJ. Reliability of flexible fiberoptic nasopharyngoscopy for evaluation of velopharyngeal function in a clinical population.  Cleft Palate J. 1989;26(3):217-225
PubMed
Yoon PJ, Starr JR, Perkins JA, Bloom D, Sie KC. Interrater and intrarater reliability in the evaluation of velopharyngeal insufficiency within a single institution.  Arch Otolaryngol Head Neck Surg. 2006;132(9):947-951
PubMed   |  Link to Article
Sie KC, Starr JR, Bloom DC,  et al.  Multicenter interrater and intrarater reliability in the endoscopic evaluation of velopharyngeal insufficiency.  Arch Otolaryngol Head Neck Surg. 2008;134(7):757-763
PubMed   |  Link to Article

Figures

Place holder to copy figure label and caption
Graphic Jump Location

Figure 1. Sample of an nasendoscopic view of the velopharynx and use of the Golding-Kushner scale. Velopharynx at rest (A), during speech production (B), at rest with superimposed grid (C), and during speech production with superimposed grid (D). In this sample, the palate movement is rated as 0.2; the posterior pharyngeal wall is rated 0.25; and the right and left lateral pharyngeal walls are rated 0.15 and 0.2, respectively. (For an explanation of the rating system, see the introductory section of the article.) The gap size is estimated to be 60%. Reproduced with permission from Sie and Chen.8

Place holder to copy figure label and caption
Graphic Jump Location

Figure 2. Sample of individual and consensus ratings for palatal and lateral wall motion. Right palate motion (A), left palate motion (B), right lateral wall motion (C), and left lateral wall motion (D).

Place holder to copy figure label and caption
Graphic Jump Location

Figure 3. Individual ratings for each of the 11 video segments using the Golding-Kushner scale in comparison to the consensus ratings. Right palate motion (A), left palate motion (B), right lateral wall motion (C), and left lateral wall motion (D).

Place holder to copy figure label and caption
Graphic Jump Location

Figure 4. Example of velopharyngeal closure with various features from the American Society of Pediatric Otolaryngology Velopharyngeal Insufficiency Study Group teaching tool. A and B, Velopharyngeal complex during rest. C, The presence of the Passavant ridge during closure. D, Presence of the notch on the nasal surface of soft palate during closure.

Tables

Table Graphic Jump LocationTable 1. Descriptive Statistics of Each Qualitative Componenta
Table Graphic Jump LocationTable 2. Description of Differences Between Individual Ratings and Consensus Ratings
Table Graphic Jump LocationTable 3. Descriptive Statistics for Presence of Velopharyngeal Gap, Aberrant Pulsations, Passavant Ridge, and Notch on Nasal Surface of Soft Palate

References

Willging JP. Velopharyngeal insufficiency.  Curr Opin Otolaryngol Head Neck Surg. 2003;11(6):452-455
PubMed   |  Link to Article
Rudnick EF, Sie KC. Velopharyngeal insufficiency: current concepts in diagnosis and management.  Curr Opin Otolaryngol Head Neck Surg. 2008;16(6):530-535
PubMed   |  Link to Article
Kummer AW, Briggs M, Lee L. The relationship between the characteristics of speech and velopharyngeal gap size.  Cleft Palate Craniofac J. 2003;40(6):590-596
PubMed   |  Link to Article
Peat BG, Albery EH, Jones K, Pigott RW. Tailoring velopharyngeal surgery: the influence of etiology and type of operation.  Plast Reconstr Surg. 1994;93(5):948-953
PubMed
Shprintzen RJ, Lewin ML, Croft CB,  et al.  A comprehensive study of pharyngeal flap surgery: tailor made flaps.  Cleft Palate J. 1979;16(1):46-55
PubMed
Sloan GM. Posterior pharyngeal flap and sphincter pharyngoplasty: the state of the art.  Cleft Palate Craniofac J. 2000;37(2):112-122
PubMed   |  Link to Article
Golding-Kushner KJ, Argamaso RV, Cotton RT,  et al.  Standardization for the reporting of nasopharyngoscopy and multiview videofluoroscopy: a report from an International Working Group.  Cleft Palate J. 1990;27(4):337-348
PubMed   |  Link to Article
Sie KC, Chen EY. Management of velopharyngeal insufficiency: development of a protocol and modifications of sphincter pharyngoplasty.  Facial Plast Surg. 2007;23(2):128-139
PubMed   |  Link to Article
D’Antonio LL, Marsh JL, Province MA, Muntz HR, Phillips CJ. Reliability of flexible fiberoptic nasopharyngoscopy for evaluation of velopharyngeal function in a clinical population.  Cleft Palate J. 1989;26(3):217-225
PubMed
Yoon PJ, Starr JR, Perkins JA, Bloom D, Sie KC. Interrater and intrarater reliability in the evaluation of velopharyngeal insufficiency within a single institution.  Arch Otolaryngol Head Neck Surg. 2006;132(9):947-951
PubMed   |  Link to Article
Sie KC, Starr JR, Bloom DC,  et al.  Multicenter interrater and intrarater reliability in the endoscopic evaluation of velopharyngeal insufficiency.  Arch Otolaryngol Head Neck Surg. 2008;134(7):757-763
PubMed   |  Link to Article

Correspondence

CME
Also Meets CME requirements for:
Browse CME for all U.S. States
Accreditation Information
The American Medical Association is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians. The AMA designates this journal-based CME activity for a maximum of 1 AMA PRA Category 1 CreditTM per course. Physicians should claim only the credit commensurate with the extent of their participation in the activity. Physicians who complete the CME course and score at least 80% correct on the quiz are eligible for AMA PRA Category 1 CreditTM.
Note: You must get at least of the answers correct to pass this quiz.
Please click the checkbox indicating that you have read the full article in order to submit your answers.
Your answers have been saved for later.
You have not filled in all the answers to complete this quiz
The following questions were not answered:
Sorry, you have unsuccessfully completed this CME quiz with a score of
The following questions were not answered correctly:
Commitment to Change (optional):
Indicate what change(s) you will implement in your practice, if any, based on this CME course.
Your quiz results:
The filled radio buttons indicate your responses. The preferred responses are highlighted
For CME Course: A Proposed Model for Initial Assessment and Management of Acute Heart Failure Syndromes
Indicate what changes(s) you will implement in your practice, if any, based on this CME course.
Submit a Comment

Multimedia

Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging & repositioning the boxes below.

Articles Related By Topic
Related Collections
PubMed Articles