Original Article

Evaluation of an Undergraduate Oral Surgery Curriculum – A Pilot Study

*Olufemi Kolawole Ogundipe¹, Lianne Keiller², Olawumi Adedoyin Fatusi¹

¹ Department of Oral Maxillofacial Surgery, Obafemi Awolowo University, Ile Ife, Nigeria.

²Department of Information Technology, Stellenbosch University, South Africa.

DOI: https://doi.org/10.60787/nmj-64-6-374

Background

Following curricular revisions at the Faculty of Dentistry of the Obafemi Awolowo University, no formal evaluation of its alignment and implementation has been carried out.

This study aimed to evaluate the alignment of an Oral surgery curriculum and to determine students’ learning experiences and perceptions of the curriculum.

Methodology

This was a mixed-method study conducted at Obafemi Awolowo University, Ile-Ife, Nigeria. This ethically approved study adopted a mixed-method descriptive approach to data collection and analysis. Data collection followed a sequential, phased approach comprising document analysis, curriculum mapping, questionnaire survey, and focus group discussions.

Quantitative data collected via questionnaires were analyzed descriptively. Qualitative data from focus group discussions were analyzed using a hybrid deductive-inductive thematic analysis by the researchers to generate relevant themes.

Results

The curriculum data reveal that the oral surgery course is well-aligned. Furthermore, each course outcome utilizes different methods of assessments and teaching/learning activities (TLAs). Focus group discussions yielded four themes and two sub-themes. The main themes included curriculum alignment, implemented teaching and learning activities, feedback, and curriculum renewal, while the subthemes were recommended TLAs and student learning experiences.

Conclusions

The curriculum evaluation identified strengths of the course as multiple student assessment methods and potential areas for improvement. These improvements included updating the content of the course and including more variety in teaching and learning methods. The authors identified potential learning gaps that require curriculum renewal. Evaluation of the entire program is recommended for comparison.

Keywords: Curriculum Alignment, Curriculum Renewal, Dental Education.

*Correspondence: Dr. Olufemi Ogundipe, Obafemi Awolowo University

Email. [email protected]

How to cite: Ogundipe OK, Keiller L, Fatusi OA. Evaluation of an Undergraduate Oral Surgery Curriculum – A Pilot Study. Niger Med J 2024; 64 (6):825 - 837

Introduction

Curriculum evaluation (CE) is the systematic process of collecting, analyzing, synthesizing, and interpreting information related to the design, implementation, and outcomes of a program of studies, a field of study, or a course of study for the purpose of monitoring and improving its quality and effectiveness (1)^. This evidence-based iterative process is used to make value judgments about whether a planned (written) curriculum is delivered (taught curriculum) and produces desired results (experienced/learned curriculum). The intended, enacted, experienced, and assessed curriculums should be properly aligned in order to achieve the education goal of an institution, school, college, department, program, or course(1). Fraser and Bosanquet(2) identified four levels of curriculum alignment. The content and structure at unit and program levels represent a product-based alignment while the other two levels i.e. the student’s learning experience(3) and the co-construction of knowledge between student and teacher represent a process-based approach to curriculum alignment. Curriculum alignment at the program level is the constructive coherence between teaching, learning, and assessment and is crucial for the quality of teaching(1).Obstacles to ensuring alignment in curricula in higher education include a lack of communication between teachers and constant changes in programs, modules, and staff over time(4).

Every curricular element and design of an educational program should be adequately aligned to detect gaps, redundancies, or inconsistencies in content, pedagogies, and student assessments thereby optimizing students’ learning. This requires the mapping of each course’s learning outcomes to program-level outcomes, student assessment methods, and teaching and learning activities. The level and depth of each association can also be recorded using a mapping scale (5). A curriculum map is an assessment tool that provides a visual and spatial representation of selected curricular elements. They are either used as tools for planning, communication, or curriculum analysis (6). Curriculum mapping typically involves the gathering of data from course documents, curriculum developers, teachers, and students to check and monitor their congruence with the objectives of a course or a program and to ensure the congruence is maintained(7).

Context

In 2012, The Federal Ministry of Health in Nigeria introduced an integrated, system-based, person-centered, community-oriented, and competency-driven national template designed to optimize students’ learning experience with the intention for it to serve as a framework for medical/dental schools in Nigeria to modify their curriculum(8). Based on the aforementioned government-driven framework, and the need for better harmonization with the medical programs, a new dentistry curriculum was implemented in 2015 at Obafemi Awolowo University (OAU). The new curriculum requirements were developed by the Faculty of Dentistry Board of Studies and approved by the OAU Senate in conformity with the standards set by the National Universities Commission (NUC) and the Medical and Dental Council of Nigeria (MDCN).

Changes to the old curriculum included the addition of new courses such as aesthetic dentistry, dental practice management and administration, and introduction to oral health research methods. In addition, a major rearrangement of courses affected the oral surgery course, which was previously introduced in year 4 and completed in year 6 but is now consolidated into a single phase in year 6. The oral surgery course – Principles of Oral Surgery (CHD 601), is an accredited core course of the dentistry program with a structured curriculum, taught by several instructors as weekly three-hour classes over a 36-week period in the old curriculum.

Given the recent curriculum revision at the faculty necessitated by policy changes as well as changes in the faculty’s staff structure, there was a need to evaluate the alignment of the new curriculum at regular intervals with all staff involved. Without a formal evaluation, questions regarding the alignment of the curriculum and optimal application of the curriculum to facilitate learning arise. Furthermore, it is unclear what the experiences of stakeholders have been since the implementation of the revised curriculum. We situated the question of curriculum alignment in the dynamics between program structure and students’ learning to determine the internal alignment of the curriculum and explore the perception of the students on their learning. This evaluation would serve as a pilot study to possibly inform the full evaluation of the entire dentistry program curriculum at OAU and a model for other African dental schools. Furthermore, it is hoped that the outcome of this study will provide important feedback necessary for the improvement of the CHD 601 course.

Methodology:

This study adopted a mixed-method descriptive approach to data collection for the evaluation of an oral surgery course at the Obafemi Awolowo University and was approved by the local research ethics committee (IPHOAU/12/1554). All staff and student participants provided informed consent before their involvement in the study.

Data Collection

Data collection followed a sequential, phased approach. Phase one included a review of the literature by the principal investigator to identify a suitable curriculum evaluation model. As most models identified were applied to full degree programs, the authors adopted a modification of a curriculum evaluation method outlined by the Taylor Institute for Teaching and Learning⁹for the course evaluation. This procedure included curriculum mapping and data collection procedures, which were recognized as critical components of curriculum evaluation.

Phase two included a review of the faculty and departmental handbooks as well as student logbooks in oral surgery for a collaborative curriculum mapping exercise. The principal investigator utilized an online electronic mapping tool – Curriculum Links (5) to capture program and course level outcomes from faculty and departmental handbooks as well as student logbooks. Because program-level outcomes (PLOs) were not written for the oral surgery course alone but for the entire dentistry program, a decision was taken to use the course content/topics in place of PLOs to make the mapping relevant to the oral surgery course.

The course instructors were given verbal and written instructions to guide the mapping process and were thereafter invited to match course-level outcomes to course contents. Where an association existed, the instructor decided if the course outcome addressed the course content at an introductory, developing, or advanced level. Instructors were also required to match the corresponding student assessments, and TLAs with each course outcome.

In Phase 3, the authors sought to establish the students’ learning experiences and perceptions of the curriculum. To identify learning gaps in the curriculum, final-year students and interns were invited via email to complete anonymous questionnaires (Google Forms ®). The questionnaire was pre-tested for content validity by a team of faculty experts using content validity ratio for individual questions (CVR) and content validity index (CVI) for the average of the 6 questions and used to collect data on students’ perceptions regarding the alignment of the curriculum.

Ordinal measurement levels of the quantitative questions were coded as follows: “Agree” and “Disagree”.

The final phase of data collection included two focus group discussions on the perception of the curriculum (final year and interns) each lasting 45 minutes via Zoom. This data allowed for triangulation with the findings from Phase 2 and 3. The second author, as an independent and experienced expert familiar with the health professions education research and the study, facilitated the focus group discussion. Each focus group discussion was observed by at least 2 independent health professions educators from external institutions to provide feedback to the authors on the process, objectivity, and fairness of the discussions. Each participant was given an identification number to maintain anonymity during the discussion. No cameras were used during the online discussion. The facilitator prompted discussion about the curriculum using semi-structured questions related to the research question, with a number of primary and follow-up questions, after which probing questions were asked based on the participants’ responses. Each focus group discussion was recorded and transcribed verbatim. The authors reviewed the transcripts independently to confirm the accuracy of the data collected. The transcripts were coded and analyzed using a hybrid inductive/deductive-thematic analysis as outlined by Braun and Clark(9). The approach comprised six steps including: 1) familiarizing with the data; 2) generating initial codes; 3) searching for themes; 4) reviewing themes; 5) defining and naming themes; and 6) producing the report. Coding and re-coding were conducted iteratively on Excel by each author. The authors engaged in consensus-building discussions throughout the analysis of the data. Qualitative data was reviewed alongside the questionnaire and curriculum mapping data jointly by the first and second authors.

Results:

Curriculum mapping data:

CHD 601 is an accredited course with 9 mandatory course learning outcomes, 13 course contents/focus areas, and 8 skills and competencies. The number of course outcomes at each mapping level showed that basic principles of oral surgery, medically compromised/ hospitalized patients, Interpersonal skills, medical ethics, and general attributes had the highest number of course outcomes and are therefore more heavily emphasized. IT and entrepreneurship, lifelong learning, principles of orthognathic surgery and salivary gland diseases have relatively less emphasis (Table 1).

Table 1: The frequency of course level outcomes to each course content and competencies/skills

The data demonstrates that the oral surgery course is well-aligned. Tables 2 and 3 present the alignment of each course level outcome with each of the 13course contents/focus areas and 8 skills and competencies. This map indicates how specific course outcomes contribute to each attribute.

Table 2: The alignment of course-level outcomes to each skill/competence.

Competencies & Skills:			General Graduate Attributes	Diagnose and manage oral disorders	Management and communication	Evaluate information	IT and entrepreneurship	Interpersonal skills	Lifelong learning	Medical Ethics
Course Outcomes
C1- Evaluate the general condition of patients presenting in oral surgery		D		D	NA	I	I	D	I	D
C2- Appropriately assess and diagnose oral surgery conditions, order proper investigations, and treat or refer the case as necessary.		D		D	I	I	NA	D	D	I
C3- Control pain and anxiety		D		D	I	I	NA	D	NA	D
C4- Demonstrate proficiency in basic surgical skills including exodontias, raising mucoperiosteal flap, extraction of retained roots and root fragments.		D		D	I	D	NA	D	NA	D
C5- clearly Describe the process of surgical of surgical extractions		I		NA	NA	NA	NA	D	NA	D
C6- Evaluate facial bone fracture, orofacial infections, oral neoplastic lesions, and cranio-maxillofacial deformities		D		D	NA	I	NA	D	I	D
C7- Manage intra and post-operative complications arising in clinical practice, medical and surgical complications, and necessary lifesaving and airway management procedures		I		D	I	I	NA	I	I	D
C8- Demonstrate critical thinking and a holistic approach to patient care		I		I	I	I	NA	D	I	D
C9- Participate actively in a year internship program regardless of ultimate career goal		D		D	D	D	NA	D	D	D
C10-C10: Basic knowledge on how to pursue a career in private dental practice		D		D	D	D	D	D	D	D
C11- Basic knowledge in law and ethics as regards dental practice		I		NA	I	NA	NA	D	NA	I

Table 3: The alignment of course-level outcomes to each course contents.

Course Content:

Basic principles of Oral surgery I & II

Local anaesthetic technique I & II

Principles of Exodontia I & II

Oro-facial-infection

Maxillofacial Trauma

Congenital anomaly-Cleft lip & Palate, Jaw defects

Cysts &Tumors of the Jaws

TMJ diseases

Diseases of the salivary glands

Dental Implantology

Effects of irradiation & chemotherapy on oral tissues/ laser in oral surgery

Medically-compromised and Hospitalized patients

Principles of Orthognathic surgery I

Course Outcomes

C1

D

A

A

D

D

I

D

I

D

I

I

D

I

C2

I

A

A

D

D

I

D

I

D

I

I

I

I

C3

D

A

A

D

D

NA

D

I

NA

I

I

D

NA

C4

D

A

D

D

NA

NA

NA

NA

NA

I

NA

I

NA

C5

D

D

D

NA

NA

NA

D

NA

NA

NA

NA

D

NA

C6

D

NA

NA

D

D

I

D

NA

D

NA

NA

I

D

C7

D

D

D

I

I

I

D

I

NA

I

I

I

I

C8

D

A

D

D

D

D

D

D

D

D

D

D

I

C9

D

A

D

D

D

I

D

D

D

I

D

D

I

C10

D

A

D

D

D

I

D

I

D

I

I

D

I

C11

D

A

D

D

D

I

D

I

D

I

I

D

I

Introduced (I), Developing (D), Advanced (A), Not Aligned (NA)

Curriculum mapping highlighted that multiple assessments were used for each course outcome.

Furthermore, most course outcomes have multiple teaching and learning methods except

“Basic knowledge on how to pursue a career in private dental practice” and “basic knowledge in law and ethics as

regards dental practice” which were taught as lectures only.

Figure 1: Method of assessment and TLAs

Student Questionnaire Responses

Electronic questionnaires were sent to 52 interns and students altogether. Nineteen interns and 21 final year students responded giving a response rate of 76.9% (40/52). Respondents perceived the Oral surgery course to be well-aligned and relevant (Figure 2).

Figure 2: Student responses

Focus group discussions provided the final data set for data triangulation. Two focus groups, one for interns (n=10) and one for final year students (n-12), yielded four themes and two sub-themes (Figure 2). The main themes included curriculum alignment, existing teaching and learning activities, feedback, and curriculum renewal, while the sub-themes were recommended (Teaching and Learning activities) TLAs and student learning experiences. Illuminating quotes from dental students and interns illustrate their perceptions of the predominant strengths and weaknesses of their educational experience, organized into main themes to demonstrate the outcome of the thematic analysis.

Theme 1: Curriculum Alignment

Students and interns emphasized in both focus groups that the oral surgery course is well-aligned.

“I feel it's important the way the course outline is being designed. Because it covers basically most of the things we have not yet encountered in other courses and Oral Surgery being a specialty on its own, has a lot to teach and the curriculum is quite encompassing to the best of my knowledge”. (FG2, P13)

Another student added; “I agree with the fact that topics are well arranged it's easier to flow with the topics so we can get to understand every topic and put it into everyday use, like topics are very easy to understand and you can easily practice as we learn.” (FG2, P7)

Theme 2: Curriculum Renewal

Students and interns expressed their dissatisfaction with course overload preferring more scaffolding and more time allocated to the course.

“… I feel it was an overload in the context of the fact we did so much of what we needed in a very short space of time for certain topics. For certain topics, I would say it would have been preferable if it could have spanned for like three lectures, two lectures, or four lectures rather than being just one lecture…” (FG1, P7)

“While the uh curriculum schedule is perfect, I think the duration for oral surgery is a little bit short I think if it was longer a bit maybe from our post phase years, all through the final year. I think it would be better that way I think if the number of years we oral surgery do is increased, I think it'd be better that way because it's so much like a big world, so I think if we go little by little step by step.” (FG2, P11)

Theme 3: Feedback

Students and interns understood feedback to relate directly to assessments within the course.

“We received feedback to a very good extent so after each assessment each lecturer will come back in with our script or as the case may be and then the areas noticed that some people perform poorly and ask us why we didn't, was it that we didn't understand it or we generally just chose, maybe people have dislike for the particular topic or aspects then if we if it was something we needed further clarification on it, he or she would take his time to go over it again and when he did again either in class only clinic by asking those questions to see if really we understood that and if we performed well we are told and then we are further encouraged to do more.” (FG1, P2)

This experience was further confirmed by another student who stated:

“To agree with what number two said we actually got feedback and another thing that was done is most of the time after assessment our scripts are returned to us and most of the time the scripts are, there are comments within the script that will make us realize areas in which we had lapses and areas we can improve on. I think one thing that can be improved on is that the assessment should not just be solely concentrated at the end of the course because at that time that is the only time we are able to get feedback on learning…” (FG1, P5)

Theme 4: Teaching and learning activities

Students and interns discussed their experiences of the teaching and learning activities in response to questions on the suitability of the course design.

“I’m saying if, uhm, a video in line with what we've been taught could have been added, that would have been an added advantage because most of the time we have to wait and hope that during the course of our clinical rotation uhm would have, uhm something similar to what we've been taught and at times that would take a long time….”(FG 1, P6)

“We agree with all they've said. I agree with the fact that we need more clinical illustrations in learning, for example, number five said that we need to try out some procedures on mannequins such as setting of lines and suturing also.” (FG 1, P1)

“What is memorable about the oral surgery course for me it's one the mode of teaching this is this is something that I cannot really overemphasize actually because they go in-depth their method of teaching is,that they go in depth to explain and use anything that is around to explain... one of the best lectures I’ve had so far, like the lecturer that brought in instruments to class to show us to explain to us that this is what he is talking about and he used slides and presentation slides which were able to refer to and read at our own pace as well… I remember those things it's sticks, and I remember what I’ve been taught quite well in class.” (FG 2, P5)

Discussions

This study is a pilot study ahead of the full dental program curriculum evaluation at OAU. The study evaluated the oral surgery course (CHD 601) by mapping the intended curriculum using the faculty handbook, which contained the learning objectives for the oral surgery course. The taught curriculum was examined using Instructors within the course who mapped the taught curriculum, while the experienced curriculum was explored using questionnaires and focus group discussions with students and graduates/interns. This method acknowledged the value of co-creation of curricula going forward and the need for constructive alignment in a research-driven effort to encourage internal accountability and for Departmental developmental purposes. Obtaining information about the oral surgery curriculum from more than one source ensured reliability by minimizing the risk of misinterpretation(10–13).

In this study, we have utilized a mixed-method approach which focused on the perceptions and priorities of students and staff. Goldie(12) affirmed that students are important stakeholders in the curriculum and are competent to evaluate the design and the delivery, that is, the attributes of teachers and methods used. Our approach assumed that those who regularly teach a course are in a potentially better position to evaluate a curriculum rather than a board of specialists.

There is a dearth of scientific literature on oral surgery curriculum evaluation(14). However, few studies regarding dental students' perception of their oral surgery education/Self-Perceived Competency/teaching have been reported where qualitative data have been used in addition to open and closed-ended questionnaires. Exploring the student perceptions through this qualitative analysis facilitated the triangulation of data in this study, informing the researchers of nuances and specific opportunities for recommendations to the Institution for curriculum renewal.

According to the curriculum mapping data, all program-level outcomes are explicitly addressed in at least,3 or more course outcomes, 8 out of the 21 PLOs, 10 of 21 PLOs, and 2 of 21 PLOs are addressed at introductory, developing, and advanced levels respectively. Although the mapping data showed multiple assessment methods for each course outcome, the teaching and learning activities were limited to only lectures and clerkships with no online tutorial. Most of the teaching and learning activities were teacher-centered rather than learner-centered. This leaves much to be desired considering that the Senate at OAU intended a learner-centred curriculum. Previous studies have emphasized that students learn better when they are at the center of learning.

The mapping data showed that the intended curriculum is well aligned with what was being taught in the classrooms in line with the intended curriculum approach by the Ministry guidelines. The result of this pilot study can inform how other faculties of dentistry in Africa can conduct CE for the standardization of curricula in the field of dentistry.

Through the findings of this study, the researchers have initiated a process of co-creation of the curriculum, recognizing the value of student and intern contributions and recommendations to teaching approaches.

Limitations and Recommendations:

Using the insight from this pilot study, the authors recommend that the process be repeated with each of the core topics in the Dentistry curriculum at OAU. This should include qualitative discussions and/or interviews with lecturers for further validation of the recommendations from students for curriculum alignment.

Conclusions: The evaluation of this curriculum has identified many strengths of the course including, multiple student assessment methods and potential areas for improvement such as updating the content of the course and including more variety in teaching and learning methods. In addition, the extent to which the course is in alignment with its stated learning outcomes has been demonstrated. Potential learning gaps were identified that may be better targeted in future versions of this program. Finally, the approach adopted in this CE demonstrates a potential approach for other institutions as they respond to calls for standardization and internationalization of dental curricula.

References:

1. Biggs JB, Tang C. Teaching for quality learning at university. (4th Edn.). Vol. 50, Innovations in Education and Teaching International. 2011. 422–423.

2. Fraser SP, Bosanquet AM. The curriculum? That’s just a unit outline, isn’t it? https://doi.org/101080/03075070600680521 [Internet].;31(3):269–84. Available from: https://www.tandfonline.com/doi/abs/10.1080/03075070600680521

3. Fink LD. Creating Significant Learning Experiences : An Integrated Approach to Designing College Courses [Internet]. San Francisco, CA: Jossey-Bass; 2003. Available from_https://www.researchgate.net/publication/31776742_Creating_Significant_Learning_Experiences_An_Integrated_Approach_to_Designing_College_Courses_LD_Fink

4. Allen M. Assessing Academic Programs in Higher Education. San Francisco: Jossey-Bass. Jossey-Bass, editor. 2004 Available from: https://www.wiley.com/en-us/Assessing+Academic+Programs+in+Higher+Education-p-9781882982677

5. DyjurP, GrantK, Kalu, F. Introduction to curriculum review. Taylor Institute for Teaching and Learning. Calgary: University of Calgary. (2019)Available from https://taylorinstitute.ucalgary.ca/sites/default/files/Introduction%20to%20CR%20UPDATED%202019.pdf

6. Dyjur P, Lock J. Three strategies for moving curriculum mapping online. Educ Dev. 2016;2(17):15–9.

7. Harden RM. AMEE Guide No. 21: Curriculum mapping: a tool for transparent and authentic teaching and learning. Med Teach [Internet]. 2001 ;23(2):123–37. Available from: https://pubmed.ncbi.nlm.nih.gov/11371288/

8. Federal Ministry of Health of Nigeria. Nigeria Undergraduate Medical and Dental Curriculum Template [Internet]. 2012. Available from: https://www.hfgproject.org/nigeria-undergraduate-medical-dental-curriculum-template/.

9 Dyjur, P., & Kalu, F. (2016). Introduction to curriculum review.Retrieved from http://curriculummapping.weebly.com/uploads/1/4/9/0/14908434/introduction_to_curriculum_review.pdf

10. Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006;3(2):77–101.

11. Chelimsky E, R.Shadish W. Evaluation for the 21st Century: A Handbook. SAGE Publications, Inc., https://doi.org/10.4135/9781483348896. (1997).

12. Goldie J. AMEE Education Guide no. 29: evaluating educational programmes. Med Teach 2006;28(3):210-24. doi: 10.1080/01421590500271282.

13. Labouta, H. I., Kenny, N. A., Dyjur, P., Li, R., Anikovskiy, M., Reid, L. F., & Cramb, D. T. Investigating the alignment of intended, enacted, and perceived learning outcomes in an authentic research-based science program. The Canadian Journal for the Scholarship of Teaching and Learning, (2019).10(3). https://doi.org/10.5206/cjsotl-rcacea.2019.3.9462

14. Olasoji HO. Rethinking the Approach to Curriculum Review in Medical and Dental Education in Nigeria. J Educ Pract [Internet]. 2014;5(32):82–7. Available from: https://www.iiste.org/Journals/index.php/JEP/article/view/16718