A New Blended Learning Concept for Medical Students in Otolaryngology FREE

Over the past few years the concept of e-learning has become a significant part of medical education.¹ Particularly, institutions like the Medical University of Vienna, Vienna, Austria, which has changed its medical curriculum from traditional teacher-centered lecturing to an integrated module system, are obliged to deal with this new, very promising teaching method for a number of reasons. Basically, the advantages of e-learning are (1) the significantly increasing interactivity between teacher and students, (2) the students' immediate feedback, (3) individually adapted teaching, and (4) examinations under objective circumstances.^2- 4 Blended learning concepts allow for the combination of traditional lecturing and a web-based forum.^5- 8

In combination with lecturing, the Unified Patient Project (UPP) is a new web-based, case-oriented, blended concept that allows students to discuss topics among themselves while promoting peer-aided e-learning as well.^9- 12 The application is free and accessible on the web, offering development, searching, and viewing of medical case reports. Radiology data are accessible as well, and medical cases can be downloaded for offline use with a “Pack&Go” function. When logged in, students are able to discuss each case, ask questions, and finally answer all the teacher's questions. On the other hand, lecturing in otolaryngology includes presentation and discussion of symptom-oriented cases in addition to hands-on teaching in the form of clinical internships and seminars in otolaryngology.^13- 14

The aims of this study were to gain feedback on satisfaction related to the perceived utility of the UPP and to test the increase of knowledge in students using the newly installed UPP blended learning concept compared with students taught by traditional face-to-face teaching.^15- 16

This prospective study, approved by the research ethics committee, was performed between August 2008 and February 2010 in a comparison group design of the quasiexperimental type with (1) preintervention test (pretest), intervention (UPP), and postintervention (posttest) (experimental group [n = 117]) and (2) pretest, alternative intervention (lecturing alone), and posttest (comparison group [n = 119]). Tests and questionnaires could not be conducted for both groups in 1 room simultaneously for organizational reasons. Therefore, 6 groups, approximately 45 students each, were formed. The authors had no influence on the formation of the groups. To avoid a cohort effect, the 6 groups were randomly selected and stratified into the experimental and comparison groups. Personal computers (PCs) and access to the Internet were guaranteed for all students participating the study at home and on campus.

Students participating in the study were introduced and gave their written informed consent. Each student had to complete 5 questionnaires during the study. Questionnaires were distributed in paper form and completed under the guidance of 2 persons in the classroom. Finished questionnaires were collected by a responsible person on location, thus ensuring a 100% return rate.

The pretest had to evaluate a student's basic knowledge about otolaryngology, gained in the 2 previous years, and included 24 multiple choice questions of the single best answer type. The results were calculated as the proportion of correctly answered items and were illustrated in a box plot diagram.

At the beginning of the study, a questionnaire regarding the use of PCs and attitude toward and experience with e-learning was handed out (Figure 1).

The questionnaire consists of the following 6 constructs with a total of 33 items: (1) enabling technology (items 1, 3-7, and 23), (2) student's attitude toward multimedia learning (items 2 and 22), (3) effective scope of PC use (items 8-20, 25, and 26), (4) private use of PC (item 21), (5) problems in using e-learning programs (item 27-33), and (6) case studies already discussed in the UPP (item 24). The items for these constructs were merged to scales by averaging the values per person.

After that, 1 group was taught using the UPP (17 case studies), with face-to-face teaching only for the introduction and some questions at the end of a case—not more than 10 minutes per case—whereas the control group was exposed to lecturing alone (17 case studies, 45 minutes each).

On the last day of the study, posttesting was performed using 25 multiple-choice questions of the single best answer type referring to the content of the lectures. The results were calculated as the percentage of correctly answered items and were illustrated in a box plot diagram.

In addition, a questionnaire to evaluate students' satisfaction with the UPP (Figure 2) or lecture (Figure 3), with 5 constructs and 34 items regarding (1) satisfaction with the teaching activity (items 1-5), (2) UPP suitability, specifically, lecturer's competence (items 6-15), (3) workload manageability (items 16-20), (4) student's attitude to learning in the ear, nose, and throat (ENT) tertiary (items 21-29), and (5) general assessment of the ENT tertiary (items 30-34), was administered. This questionnaire used a 6-point Likert scale from 1 (strongly agree) to 6 (strongly disagree). The category “don't know” enabled students to encode certain circumstances that did not occur for them or were not to be answered.

Three open questions at the end of the questionnaire—“What did you like in particular?” “What did you not like?” and “Suggestions for improvement”—did not enter in this study because they were not answered completely.

The questionnaires were constructed with no difference between the 2 types of teaching. Questions were only adapted when specific to the UPP or lecture. The items for these constructs were concentrated to scale by averaging the values per person.

Another questionnaire (Figure 4) was then administered, with 8 constructs and 33 items focusing particularly on judgments about general advantages and disadvantages of e-learning and targeting (1) self-dependent learning (items 1, 6, 18, 20, and 21), (2) motivation (items 16, 19, 22, and 23), (3) learning outcome (items 2 and 13), (4) communication (items 3, 11, 12, 14, and 29), (5) relation to application (items 5, 9, 10, and 24), (6) relation to practical experience (items 4, 7, 8, 25, and 30), (7) effort of time and costs (items 27, 28, and 31), and (8) general assessment (items 15, 17, 26, 32, and 33). The items for these constructs were concentrated to scales by averaging the values per person. For each construct a representative item was selected. For evaluation the categories “I don't agree” and “I partially agree” in a 3-stage Likert scale were summarized.^16- 17

Nominal and ordinal variables were presented as absolute and relative frequencies. Continuous variables were presented as median, interquartile ranges, and minimums and maximums because of nonnormal distributions. Categorical variables were compared between groups using multivariable logistic regression models to adjust for the effect of potential confounders. For the same purpose, multivariable analysis of covariance (ANCOVA) models were used to compare continuous variables between groups (in the case of right-skewed distribution, the continuous dependent variable had been log-transformed). To adjust the comparison of posttest results between the 2 groups for pretest results and potential influences of other variables (ie, age and place of morning clinical internship [university clinic vs extramural hospitals]), an ANCOVA model was calculated incorporating an interaction between group indicator and pretest result. Data were recorded in an electronic database using the program SPSS version 15.0 (SPSS Inc) and analyzed with the software package SAS version 9.2 (SAS Institute Inc). All P values were 2-sided, and values ≤.05 were considered statistically significant and were not corrected for multiplicity owing to the explorative character of secondary questions.

At the beginning of the study, 120 medical students were included in the experimental group (UPP) and 125 in the control group (lecturing alone). Of these 245 medical students, 236 finished (dropout rate of 3.7%) the study. In particular, the dropout rate was 2.5% (n = 3) and 4.8% (n = 6) in the experimental and comparison groups, respectively.

The sex distribution of the students was balanced in the experimental group (51.3% female vs 48.7% male), while in the comparison group, the male to female ratio leaned toward female students (68.9% female vs 31.1% male). The median age was 24.6 years in the entire cohort (interquartile range [IQR], 23.8-25.6 years), with no significant difference in both groups.

The primary research question, general satisfaction with lecturing in otolaryngology, was rated using the 6-level Likert scale: 1, strongly agree; 2, agree; 3, partly agree; 4, partly disagree; 5, disagree; 6, strongly disagree; and “don't know.” In the comparison group, median general satisfaction was 1.0 (IQR, 1.0-2.0) compared with 2.0 (IQR, 1.0-2.0) in the experimental group without reaching statistical significance (P = .26). Five predefined theoretical constructs—satisfaction with teaching activity; suitability of the UPP program, specifically, teacher’s competence in face-to-face teaching; workload manageability; student's attitude to learning in the ENT tertiary; and general assessment—were rated on the same scale. For all these constructs, with the exception of “student's attitude to learning in the ENT tertiary,” the rating was better in the control group, reaching statistical significance in 3 of the 5 categories (Table 1).

The constructs concerning advantages and disadvantages of e-learning—independent learning, motivation, learning outcome, communication, relation to application, relation to practical experience, effort of time and costs, and general assessment—were rated with a 3-stage Likert scale: “agree,” “partly agree,” “disagree,” and “don't know.”

In all of these constructs except “effort of time and costs,” the experimental group (UPP) rated with higher agreement (Table 2), mostly with statistical significance.

Analyzing the practicability of the UPP, we observed that the average time 1 student spent on one of the 17 cases was approximately 20 minutes. In addition, the short attendance time of 10 minutes per case in the classroom must be calculated, which is held immediately after a workshop with obligatory attendance. Therefore, students did not have to attend a separate lecture at the campus for the case presentations. Nearly all of their learning was conducted solely electronically. Face-to face teaching for the comparison group was performed in 17 lessons, 45 minutes each, with the same learning matter. The time of learning in the UPP group was approximately one-third less than in the lecture group.

The project manager (M.Ch.G.) had to spend approximately 35 hours on the creation of the 17 case studies, for both the UPP and the face-to-face teaching. Once the UPP was implemented it took approximately 20 minutes per case to review. He therefore spent approximately 30 minutes on each of the 17 cases including the introduction and final feedback. On the other hand, he saved 45 minutes for each of the 17 lessons by not teaching at the campus—in summary, a benefit of a third of his teaching time. The application specialist needed approximately 70 hours for the preparation of the 17 cases and approximately 6 hours for ongoing support.

The distributions of pretest and posttest results for both groups are shown in Figure 5. In spite of the aforementioned quasirandomized study design, the pretest distribution in the comparison group was generally lower than in the experimental group. For this reason, the group comparison in respect to the posttest results was adjusted for pretest results. Thereby, a fair comparison between the groups is nevertheless possible.

All students using the UPP generated an increase of their knowledge. Students with higher preknowledge profited from the UPP, while students with lower preknowledge did not (P = .01). For a 75% preknowledge, the posttest results in the experimental group exceeded those of the comparison group, on average by 8.7 percentage points (eg, 54.2% vs 45.5% for a student aged 25 years; P = .03). On the other hand, for a 25% preknowledge, the posttest results fell below those of the comparison group by 8.1 percentage points (eg, 67.2% vs 75.3% for a student aged 25 years; P = .03). At approximately 51% preknowledge, the posttest results for the 2 groups coincided on average.

This effect was also demonstrated in Figure 6, where students in both groups were classified into thirds according to their group-specific tertile in the pretest. While the increase of knowledge for the middle third was similar in the 2 groups, there was a higher increase for the upper third in the experimental group and for the lower third in the comparison group.

The aim of this study was to evaluate teaching of senior medical students in otolaryngology at the Department of Otolaryngology, Medical University of Vienna. We investigated the effects of the new adapted web-based blended learning concept UPP by teaching with a case series in otolaryngology and compared the students' knowledge with a comparison group that was taught with traditional lecturing.

Students in the comparison group were more satisfied with their type of teaching than those in the experimental group, as previously shown in other studies.¹⁸ A possible explanation for this observation could be that students are generally skeptical about new methods of teaching, especially those that do not have a detailed catalog of educational objectives at hand yet, and that lecturing is known to be a reliable method for acquiring knowledge.¹⁹ However, when advantages and disadvantages of e-learning on its own without reference to otolaryngological teaching were queried at the end of the study, students in the experimental group gave a more positive rating in all 8 constructs compared with those in the comparison group. In 6 of these 8 constructs a statistically significant rating was observed. Both groups were familiar with the UPP through approximately 20 cases presented in a grand forum of up to 450 students the year before.

With respect to time resources, the UPP approach saves about a third of learning time. The UPP is a more flexible, self-directed, off-campus form of learning for students and shifted time resources from teaching faculty to technical support personnel. Teachers can save about a third of teaching time compared with face-to-face teaching. An additional administrative benefit results because fewer classrooms are needed owing to the reduced attendance time.

We could demonstrate a positive effect on gaining knowledge through the blended learning approach with the UPP. Students in the experimental group surprisingly started with a higher pretest score on average and showed an increase of knowledge in the whole range of pretest items, potentially because of the influence of the UPP used in the blended learning concept.

Limitations of our study include that it was a field study comprising many uncontrolled influences with different groups concerning preknowledge. Also, other potentially important influencing factors, such as complexity of the study materials or collaboration among student participants on study materials, were not measured. While using experimental research as we did is a powerful investigative method, additional qualitative data would indeed be helpful and should be considered for subsequent studies. Moreover, the study was performed only in 1 subject matter. Although a quasirandomized study design was chosen, the pretest distribution in the comparison group was generally lower than in the experimental group. For this reason, the group comparison in respect to the posttest results was adjusted for unequal pretest results. By using this adjustment it could be shown that students with higher preknowledge benefit more from the blended learning approach using the UPP, while students with lower preknowledge do not (P = .01).²⁰ A possible cause could be that students with a primarily better knowledge are more interested in new forms of teaching.

As documented in the literature,^21- 22 high-performing students with adequate prior knowledge profit more from e-learning than poorly performing ones. A determining criterion thereby is the level of prior knowledge to be acquired in the domain. This is reflected in our results using the UPP. In consequence, a diagnostic module to determine students' prior knowledge before using our UPP blended learning approach should be considered.

As possible reasons for this lower learning outcome in students with low preknowledge, it is assumed that (1) they are less able to develop independent solutions; (2) their solving organization functions (structuring, time schedule, information search, and transfer) are inferior; (3) their processing of given data are inferior; and (4) they do not have a high level of self-organization and learning motivation. These students need special promotion programs to enhance their preknowledge.

Students' satisfaction with the blended learning concept UPP was, although not statistically significant, lower than in the comparison group with face-to-face teaching.

Our study shows that the new web-based UPP leads to improved knowledge in clinical otolaryngology for all students. Students with lower preknowledge profit more from face-to-face teaching, while students with higher preknowledge profit more from the UPP. This implies that when the UPP approach is expanded to further subject domains at the Medical University of Vienna, students with poor preknowledge will need special promotion programs.