0
Original Article |

Speech Evaluation After Palatal Augmentation in Patients Undergoing Glossectomy FREE

Viviane de Carvalho-Teles, PhD; Luiz Ubirajara Sennes, MD, PhD; Ingrid Gielow, MSc, PhD
[+] Author Affiliations

Author Affiliations: Rehabilitation Department, Fundação Oncocentro de São Paulo and Instituto do Câncer Arnaldo Vieira de Carvalho (Dr de Carvalho-Teles), Department of Otorhinolaryngology, University of São Paulo (Dr Sennes), and Federal University of São Paulo (Dr Gielow), São Paulo, Brazil.


Arch Otolaryngol Head Neck Surg. 2008;134(10):1066-1070. doi:10.1001/archotol.134.10.1066.
Text Size: A A A
Published online

Objective  To assess, in patients undergoing glossectomy, the influence of the palatal augmentation prosthesis on the speech intelligibility and acoustic spectrographic characteristics of the formants of oral vowels in Brazilian Portuguese, specifically the first 3 formants (F1 [/a, e, u/], F2 [/o,ó, u/], and F3 [/a,ó/]).

Design  Speech evaluation with and without a palatal augmentation prosthesis using blinded randomized listener judgments.

Setting  Tertiary referral center.

Patients  Thirty-six patients (33 men and 3 women) aged 30 to 80 (mean [SD], 53.9 [10.5]) years underwent glossectomy (14, total glossectomy; 12, total glossectomy and partial mandibulectomy; 6, hemiglossectomy; and 4, subtotal glossectomy) with use of the augmentation prosthesis for at least 3 months before inclusion in the study.

Main Outcome Measures  Spontaneous speech intelligibility (assessed by expert listeners using a 4-category scale) and spectrographic formants assessment.

Results  We found a statistically significant improvement of spontaneous speech intelligibility and the average number of correctly identified syllables with the use of the prosthesis (P < .05). Statistically significant differences occurred for the F1 values of the vowels /a, e, u/; for F2 values, there was a significant difference of the vowels /o,ó, u/; and for F3 values, there was a significant difference of the vowels /a,ó/ (P < .001).

Conclusions  The palatal augmentation prosthesis improved the intelligibility of spontaneous speech and syllables for patients who underwent glossectomy. It also increased the F2 and F3 values for all vowels and the F1 values for the vowels /o,ó, u/. This effect brought the values of many vowel formants closer to normal.

In Brazil, the estimated gross rate of oral cancer per 100 000 inhabitants in 2006 was 10.31 for men and 3.50 for women, with the tongue being one of the most common sites.1 The treatment of tongue tumors typically consists of surgery, followed by radiotherapy as required. Tongue resections, with or without other surgical procedures such as mandibulectomy, pelvectomy (resection of the floor of the mouth), and palatectomy, lead to important changes regarding swallowing function and oral communication.2

The tongue plays an essential role in speech, being the main articulator of vowels and, to a large extent, consonants. Vowels are identified by their formants, ie, the natural resonance frequencies of the vocal tract in the articulatory position of the spoken vowel. Although there is an infinite number of formants, the first 3 are the most important for the acoustic description and identification of the vowels. The first 2 formants (F1 [/a, e, u/] and F2 [/o,ó, u/]) are related to the vertical and the horizontal displacements of the tongue, respectively, and the third formant (F3 [/a,ó/]) is related to the size of the oral cavity.3 After tongue resections, changes in configuration and volume of the vocal tract generate resonant and articulatory alterations, consequently affecting the intelligibility of these patients' speech.4

Rehabilitation of oral communication in patients undergoing glossectomy (hereafter referred to as glossectomy patients) has traditionally involved speech therapy, with the objective of maximizing the use of the remaining structures for the development of compensatory articulations.5 With speech therapy, treatments such as the palatal augmentation prosthesis seem to contribute to the articulation of sounds, thus improving the oral communication of the patients.6 However, very few works have assessed the effect of the palatal augmentation prosthesis on the communication function of glossectomy patients. Most studies considered limited and heterogeneous samples, which makes it difficult to reach conclusions based on their findings.4,710 In some of these studies, the improvement of communication was evaluated only by the subjective impressions of the physicians, patients, or both, without the use of tools or specific protocols for this purpose, thus not allowing an intersubject and/or intrasubject comparison of these data.11

The objective of this study was to assess the influence of the palatal augmentation prosthesis on the speech intelligibility and acoustic spectrographic characteristics of the formants of oral vowels in Brazilian Portuguese, specifically the first 3 formants, in glossectomy patients.

We analyzed the medical files of all patients who had received a palatal augmentation prosthesis at the Fundação Oncocentro de São Paulo, São Paulo, Brazil, from July 7, 1998, through October 31, 2005. Of the 45 patients identified, 5 did not return for consultation or could not be found, 2 were dead, and 2 had undergone glossectomy associated with total laryngectomy and, therefore, could not be selected for the survey. The remaining 36 patients constituted the study sample, which met the following inclusion criteria: (1) total or subtotal glossectomy or hemiglossectomy with or without other surgical procedures in the oral cavity and mandible and (2) use of a stable and well-adapted palate-lowering prosthesis for at least 3 months.

The ages of the 36 subjects (33 men and 3 women) ranged from 30 to 80 years. Fourteen underwent total glossectomy; 12, total glossectomy plus partial mandibulectomy; 6, hemiglossectomy; and 4, subtotal glossectomy. All of the patients underwent reconstruction of the floor of the mouth using the pectoralis major flap, except one who underwent a hemiglossectomy with a reconstruction using the platysma myocutaneous flap. Thirty-five patients underwent radiotherapy with doses ranging from 6000 to 7020 Gy, and only 2 had postoperative chemoradiotherapy. All 36 patients underwent speech therapy for at least 3 months before prosthesis placement, and they received oral feedings. The mean (SD) time from the surgery to adaptation of the prosthesis was 22.3 (21.2) months, and the mean (SD) time of prosthesis use was 9.3 (10.6) months. Twenty-three patients used only a maxillary prosthesis, including a complete prosthesis in 20 and a removable partial prosthesis in 3; 11 used a maxillomandibular prosthesis; and only 2 patients used a removable partial maxillary prosthesis. The last 2 patients had a complete set of lower natural teeth. Any lower prosthesis with little stability was retained with a denture adhesive. No patient presented with a mandibular implant (Table 1).

Table Graphic Jump LocationTable 1. Demographic Data of the Study Patients

The subjects were fully informed of the nature of the study and agreed to participate. In addition, informed consent was obtained from each subject before the study. This research was approved by the Committee on Ethics in Research of the São Paulo Medical School, University of São Paulo (process 75/2004).

The patients' voices were recorded inside a silent room using commercially available software (Computerized Speech Laboratory [CSL] 4300B; Kay Elemetrics Corp, Lincoln Park, New Jersey). They were asked to stand up, and the microphone (model SM48S; Shure Brothers, Inc, Niles, Illinois) was then positioned 12 cm from the mouth of each patient.

The following 3 types of speech samples were obtained, with and without prosthesis: (1) 30 seconds of spontaneous speech; (2) repetition of 18 syllables with plosive, fricative voiced and voiceless sounds, which are nasal and liquid sounds together with the vowel /a/; and (3) sustained emission in the usual frequency and intensity of the vowels /a, e,é, i, o,ó, u/.

The records of speech samples 1 and 2 were edited on a CD, with a random distribution of the speech samples from the patients with and without prosthesis. The edited material was then presented to 2 speech-language pathologists with experience in the treatment of patients with head and neck cancer, and they were asked to assess the intelligibility of the spontaneous speech and the syllables (the presentation to a third pathologist was not considered, because the assessments were completely discordant when compared with those of the other two). The recorded speech samples, with and without the prosthesis, were randomly presented, so that the listeners did not know whether the patients were using the prosthesis.

For the assessment of the spontaneous speech intelligibility, patient speech samples of approximately 8 seconds with and without the prosthesis were presented. Each sample was edited from the previously recorded 30 seconds of spontaneous speech, with priority given to the best parts of the conversation.

The spontaneous speech intelligibility was assessed on a 4-category scale as normal, mildly impaired, moderately impaired, and severely impaired. For the assessment of the syllable intelligibility, the sequence of presentation of the 18 syllables from each patient with and without the prosthesis was also randomized. The speech-language pathologists were asked to transcribe exactly what they had understood immediately after listening to each speech sample.

Finally, we performed a spectrographic assessment of the formants of the 7 vowels of Brazilian Portuguese, with and without the prosthesis. For this assessment, the mean values of the first 3 formants were extracted from the most stable part of each vowel, with a duration of approximately 5 seconds, using a broad-band spectrogram generated by the CSL module with a 300-Hz filter, which is indicated to identify the sound formants.

The quantitative variables were described by means and standard deviations, whereas the categorical variables were described by frequency. We checked adherence to the normal distribution with the Kolmogorov-Smirnov test. We evaluated the interrater agreement by means of the κ coefficient. Wilcoxon signed rank tests and 1-way analysis of variance were used; P < .05 in 2-tailed tests was considered statistically significant.

No statistically significant difference was found among the 4 types of surgery or among the different types of prosthesis, with consideration of any variable assessed in this study. Therefore, data from all of the patients were pooled for analysis. The only exception found was the comparison between the hemiglossectomy and total glossectomy plus mandibulectomy procedures, where a statistically significant difference was found on F2 values for the vowels /i/ and /u/. The mean values of F2 in the group of hemiglossectomy patients were higher for the vowel /i/ and lower for the vowel /u/ when compared with the group undergoing total glossectomy plus mandibulectomy. A statistically significant consistency was noted between the listeners concerning the assessment of (1) the spontaneous speech intelligibility without (κ = 0.58) and with the prosthesis (κ = 0.41; P < .001) and (2) the syllable intelligibility without (Kendall a = 0.46, b = 0.51) and with the prosthesis (Kendall a = 0.40, b = 0.45; P < .001).

Table 2 gives the results of the assessment of spontaneous speech intelligibility with the prosthesis, with an increased percentage of samples demonstrating the categories of normal speech and mild impairment. The improvement was statistically significant (Wilcoxon signed rank test, P = .01).

Table Graphic Jump LocationTable 2. Mean Percentages of Spontaneous Speech Intelligibility in 36 Patients With and Without Palatal Augmentation Prosthesis

A statistically significant difference was noted between the mean (SD) number of correctly identified syllables, with and without the prosthesis (9.4 [2.5] and 8.8 [2.8] syllables, respectively; Wilcoxon signed rank test, P = .03).

The values of the vowel formants with and without the prosthesis were different for most of the vowels (Table 3). However, the differences were statistically significant for F1 values of the vowels /a, e, u/ (P < .001), and there was a statistical trend of difference for the vowel /o/ (P = .09). For the F2 values, there was a significant difference for the vowels /o,ó, u/ (P < .001) and a statistical trend for the vowels /e, i/ (P = .06 and P = .08, respectively). For the F3 values, there was a significant difference for the vowels /a,ó/ (analysis of variance, P < .001).

Table Graphic Jump LocationTable 3. Mean Values With and Without Palatal Augmentation Prosthesis in 36 Study Patients

Without the prosthesis, for each of the vowels assessed, F1 (except for the vowel /i/), F2 (except for vowels /a, o,ó, u/), and F3 values were below the normal range of vowel formants of the Brazilian Portuguese spoken in the city of São Paulo.12 With the prosthesis, there was an increase in the F2 and F3 values for all vowels, whereas for the F1 value this occurred only for the vowels /o,ó, u/.

Speech rehabilitation is essential for reestablishing the interpersonal communication of glossectomy patients.4,13 In this study, judgment of speech intelligibility of the 36 patients without the prosthesis ranged from normal to severely impaired, with a predominance of moderate impairment. On the other hand, with the prosthesis, mildly impaired speech intelligibility predominated, with an increased level of normal intelligibility. Other authors2,14 have also observed, after adaptation of the prosthesis, a more marked improvement in the speech intelligibility of patients undergoing total glossectomy or patients with important limitations of the tongue movements, which shows that the palate augmentation device could maximize the use of the remaining tongue and other nearby structures during articulatory adaptation.

The average number of syllables correctly identified by the judges also increased with the prosthesis. Davis et al4 have also observed an improvement of 20% and 30% in the intelligibility of the sounds /t/, /d/, and /k/, /g/, respectively, after use of a palate augmentation prosthesis in a glossectomy patient. Similar results were found by other researchers.7,13,15

The F1 values without the prosthesis were not too different from the reference values.12 With the prosthesis, the F1 values for most vowels were close to normal, except for the vowel /a/, for which they were distant, and the vowels /e, i/, for which there were no changes. The vowels /a, e/ and /o, u/ had their values reduced and increased, respectively.

According to Baken,16 the F1 formant is not only related to the vertical movements of the tongue, but also influenced by closing of the mouth and narrowing of the pharynx. In this study, anterior vowels were, most likely, produced with a tighter closure of the mouth, thus reducing the F1 values. This articulatory posture may have been used by patients in the attempt to bring together the augmentation prosthesis and the floor of the mouth, as shown in another study.4 On the contrary, the increased F1 values for posterior vowels may have occurred as a result of the lowering device filling the oropharyngeal cavity, thus narrowing this space and consequently increasing the values of the formant.

The F2 values increased with use of the prosthesis for all vowels, with results farther from normal for the vowels /o,ó, u/ and closer for the vowels /e, i/. The increase in F2 values is related to the anterior position of the tongue or to its lowering in the posterior region.3 The palate augmentation prosthesis used by all of the patients in this study covered the whole area of the palate plate, in an attempt to compensate for the absence of the tongue in the anterior and posterior regions. Therefore, the increase of the F2 values most likely occurred because, in the case of the more anterior vowels, the prosthesis worked as if the tongue had been placed in the anterior position. For the posterior vowels, it worked as if the tongue had been lowered, because the palate augmentation prosthesis usually has a larger vertical dimension in the posterior region. For this reason, with the prosthesis, the intermediate vowels /o,ó/ and the high vowel /u/ became low vowels; thus the F2 values for these vowels became closer to normal for the low vowel /a/, deviating from their reference values.

After the prosthesis placement, the F3 values increased toward normal for all vowels. These increased F3 values are related to the reduction of the oral cavity dimensions,16 which was caused in this study by the presence of the palatal augmentation device. However, a statistical difference was observed only for the open vowels /a,ó/. To produce this type of vowel, there is an enlargement of the oral cavity but, with the prosthesis, the effect was probably less marked.

Without the prosthesis, there was no statistically significant difference between the types of surgery concerning the values of the formants for the 7 vowels tested. Likewise, Vale17 did not find statistically significant differences in the values of the first 4 formants of these same vowels, when comparing the glossectomy patients with those undergoing total glossectomy plus mandibulectomy. However, with prosthesis, differences in F2 values were found only for /i/ and /u/, when hemiglossectomy and total glossectomy plus mandibulectomy were compared; this may have occurred because production of these vowels requires more marked horizontal movements of the tongue. When the tongue and other adjunct structures are absent, as in the case of glossectomy plus mandibulectomy, the difficulty of developing compensatory articulations is greater, thus explaining why the F2 values found in this study for these vowels and after this type of surgery have remained stable, even in the presence of the prosthesis. On the other hand, hemiglossectomy patients had F2 values for the vowels /i/ and /u/ closer to normal, perhaps because the palatal augmentation prosthesis worked with the remaining structures for the articulatory compensation of these vowels.15

The results of this study have shown that subjective and objective assessments not only help to recognize the impact of the altered speech on oral communication, but may also be used for better speech therapy or prosthetic treatment planning. The limitations of our study include the small number of patients, heterogeneity among them (ie, type of previous surgery and type and time of prosthesis use), and the evaluation performed by only 2 expert listeners. Future studies need to address these limitations.

The palatal augmentation prosthesis improved the intelligibility of spontaneous speech and syllables in glossectomy patients. The prosthesis also increased the F2 and F3 values for all vowels and the F1 values for the vowels /o,ó, u/. This effect brought the values of many vowel formants closer to normal intelligibility.

Correspondence: Viviane de Carvalho-Teles, PhD, Fundação Oncocentro de São Paulo, 2396, Oscar Freire, São Paulo, Brazil 05409-012 (vivicarvalhofono@yahoo.com.br).

Submitted for Publication: October 12, 2006; final revision received July 16, 2007; accepted July 29, 2007.

Author Contributions: All of the authors had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Sennes and Gielow. Acquisition of data: de Carvalho-Teles. Analysis and interpretation of data: de Carvalho-Teles, Sennes, and Gielow. Drafting of the manuscript: de Carvalho-Teles and Sennes. Critical revision of the manuscript for important intellectual content: Sennes and Gielow. Study supervision: Sennes and Gielow.

Financial Disclosure: None reported.

Additional Contributions: Maria Cecilia Lorenzi, MD, performed the statistical analysis.

Ministério da Saúde, Instituto Nacional do Câncer, Estimation of the Incidence and the Mortality of Cancer in Brazil [in Portuguese].  Rio de Janeiro, Brazil INCa2006;
Cantor  RCurtis  TAShipp  TBeumer  J  IIIVogel  BS Maxillary speech prosthesis for mandibular surgical defects. J Prosthet Dent 1969;22 (2) 253- 260
PubMed
Fant  G Acoustic Theory of Speech Production. 2nd ed. Paris, France Mounton1970;
Davis  JWLazarus  CLogemann  JHurst  PS Effect of a maxillary glossectomy prosthesis on articulation and swallowing. J Prosthet Dent 1987;57 (6) 715- 719
PubMed
Furia  CLKowalski  LOLatorre  MR  et al.  Speech intelligibility after glossectomy and speech rehabilitation. Arch Otolaryngol Head Neck Surg 2001;127 (7) 877- 883
Aramany  MADowns  JABeery  QCAslan  Y Prosthodontic rehabilitation for glossectomy patients. J Prosthet Dent 1982;48 (1) 78- 81
PubMed
Leonard  RGillis  R Effects of a prosthetic tongue on vowel intelligibility and food management in patient with total glossectomy. J Speech Hear Disord 1982;47 (1) 25- 30
PubMed
Christensen  JMHutton  JEHasegawa  AFletcher  SG Evaluation of the effects of palatal augmentation on partial glossectomy speech. J Prosthet Dent 1983;50 (4) 539- 543
PubMed
McKinstry  REAramany  MABeery  QCSansone  F Speech considerations in prosthodontic rehabilitation of the glossectomy patient. J Speech Hear Disord 1982;47 (1) 25- 30
PubMed
Robbins  KTBowman  JBJacob  RF Postglossectomy deglutitory and articulatory rehabilitation with palatal augmentation prosthesis. Arch Otolaryngol Head Neck 1987;1131214- 1218
Marunick  MTselios  N The efficacy of palatal augmentation prosthesis for speech and swallowing in patients undergoing glossectomy: a review of the literature. J Prosthet Dent 2004;91 (1) 67- 74
PubMed
Monteiro  MC A Spectral Computerized Analysis of the Oral Vogals of the Brazilian Portuguese Spoken in São Paulo [in Portuguese].  São Paulo, Brazil Universidade Federal de São Paulo1995;
Leonard  RJGillis  R Differential effects of speech prosthesis in glossectomy patients. J Prosthet Dent 1990;64 (6) 701- 708
PubMed
Martins  MVVale-Prodomo  LPCarrara-de Angelis  E Effect of palatal augmentation prosthesis in swallowing and speech articulation in a patient submitted to total glossectomy: case report [in Portuguese]. Revista Fonoaudiologia Brasil 2004;5 (1) 1- 3
Gillis  RELeonard  RJ Prosthetic treatment for speech and swallowing in patient with total glossectomy. J Prosthet Dent 1983;50 (6) 808- 814
PubMed
Baken  RJ Clinical measurement of speech and voice. Berry  WRClinical Dysarthria. San Diego, CA College Hill Publication1987;121- 154
Vale  LP Spectral Analysis of the Formants of Oral Vowels in Patients Submitted to Total Glossectomy [in Portuguese] [master's thesis]  São Paulo, Brazil Universidade de São Paulo2003;

Figures

Tables

Table Graphic Jump LocationTable 1. Demographic Data of the Study Patients
Table Graphic Jump LocationTable 2. Mean Percentages of Spontaneous Speech Intelligibility in 36 Patients With and Without Palatal Augmentation Prosthesis
Table Graphic Jump LocationTable 3. Mean Values With and Without Palatal Augmentation Prosthesis in 36 Study Patients

References

Ministério da Saúde, Instituto Nacional do Câncer, Estimation of the Incidence and the Mortality of Cancer in Brazil [in Portuguese].  Rio de Janeiro, Brazil INCa2006;
Cantor  RCurtis  TAShipp  TBeumer  J  IIIVogel  BS Maxillary speech prosthesis for mandibular surgical defects. J Prosthet Dent 1969;22 (2) 253- 260
PubMed
Fant  G Acoustic Theory of Speech Production. 2nd ed. Paris, France Mounton1970;
Davis  JWLazarus  CLogemann  JHurst  PS Effect of a maxillary glossectomy prosthesis on articulation and swallowing. J Prosthet Dent 1987;57 (6) 715- 719
PubMed
Furia  CLKowalski  LOLatorre  MR  et al.  Speech intelligibility after glossectomy and speech rehabilitation. Arch Otolaryngol Head Neck Surg 2001;127 (7) 877- 883
Aramany  MADowns  JABeery  QCAslan  Y Prosthodontic rehabilitation for glossectomy patients. J Prosthet Dent 1982;48 (1) 78- 81
PubMed
Leonard  RGillis  R Effects of a prosthetic tongue on vowel intelligibility and food management in patient with total glossectomy. J Speech Hear Disord 1982;47 (1) 25- 30
PubMed
Christensen  JMHutton  JEHasegawa  AFletcher  SG Evaluation of the effects of palatal augmentation on partial glossectomy speech. J Prosthet Dent 1983;50 (4) 539- 543
PubMed
McKinstry  REAramany  MABeery  QCSansone  F Speech considerations in prosthodontic rehabilitation of the glossectomy patient. J Speech Hear Disord 1982;47 (1) 25- 30
PubMed
Robbins  KTBowman  JBJacob  RF Postglossectomy deglutitory and articulatory rehabilitation with palatal augmentation prosthesis. Arch Otolaryngol Head Neck 1987;1131214- 1218
Marunick  MTselios  N The efficacy of palatal augmentation prosthesis for speech and swallowing in patients undergoing glossectomy: a review of the literature. J Prosthet Dent 2004;91 (1) 67- 74
PubMed
Monteiro  MC A Spectral Computerized Analysis of the Oral Vogals of the Brazilian Portuguese Spoken in São Paulo [in Portuguese].  São Paulo, Brazil Universidade Federal de São Paulo1995;
Leonard  RJGillis  R Differential effects of speech prosthesis in glossectomy patients. J Prosthet Dent 1990;64 (6) 701- 708
PubMed
Martins  MVVale-Prodomo  LPCarrara-de Angelis  E Effect of palatal augmentation prosthesis in swallowing and speech articulation in a patient submitted to total glossectomy: case report [in Portuguese]. Revista Fonoaudiologia Brasil 2004;5 (1) 1- 3
Gillis  RELeonard  RJ Prosthetic treatment for speech and swallowing in patient with total glossectomy. J Prosthet Dent 1983;50 (6) 808- 814
PubMed
Baken  RJ Clinical measurement of speech and voice. Berry  WRClinical Dysarthria. San Diego, CA College Hill Publication1987;121- 154
Vale  LP Spectral Analysis of the Formants of Oral Vowels in Patients Submitted to Total Glossectomy [in Portuguese] [master's thesis]  São Paulo, Brazil Universidade de São Paulo2003;

Correspondence

CME
Meets CME requirements for:
Browse CME for all U.S. States
Accreditation Information
The American Medical Association is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians. The AMA designates this journal-based CME activity for a maximum of 1 AMA PRA Category 1 CreditTM per course. Physicians should claim only the credit commensurate with the extent of their participation in the activity. Physicians who complete the CME course and score at least 80% correct on the quiz are eligible for AMA PRA Category 1 CreditTM.
Note: You must get at least of the answers correct to pass this quiz.
You have not filled in all the answers to complete this quiz
The following questions were not answered:
Sorry, you have unsuccessfully completed this CME quiz with a score of
The following questions were not answered correctly:
Commitment to Change (optional):
Indicate what change(s) you will implement in your practice, if any, based on this CME course.
Your quiz results:
The filled radio buttons indicate your responses. The preferred responses are highlighted
For CME Course: A Proposed Model for Initial Assessment and Management of Acute Heart Failure Syndromes
Indicate what changes(s) you will implement in your practice, if any, based on this CME course.
NOTE:
Citing articles are presented as examples only. In non-demo SCM6 implementation, integration with CrossRef’s "Cited By" API will populate this tab (http://www.crossref.org/citedby.html).
Submit a Comment

Multimedia

Some tools below are only available to our subscribers or users with an online account.

Web of Science® Times Cited: 6

Related Content

Customize your page view by dragging & repositioning the boxes below.

Articles Related By Topic
Related Topics
PubMed Articles