I recently finished compiling the evaluation results of the pharmacology learning object.
Method
The participants consisted of 2 health science faculty and 10 students from the CLOE consortium. I used the Learning Object Review Instrument (LORI), version 1.4 developed by Belfer, et al. (2002) to collect faculty's individual assessments of the quality of the pharmacology learning object. Faculty were asked to assess the learning object in 8 categories using a five point rating scale ranging from low to high and to provide a rationale for their score.
A survey with 25 questions was distributed to students in order to carry out a learning impact study based on their use of the learning object. The student questionnaire items were combined into four sub-scales: Learning Value; Value Added by the Learning Object; Usability of Learning Object; Usability of Technology
For a more detailed description select evaluation methodology.
Results
The results from the faculty evaluation indicate that the learning object scored high in the categories of content quality, motivation and instructor/ student guides. It was also rated very good to high in learning goal alignment, presentation design and reusability. Interaction/ usability and feedback/ adaptation received lower scores ranging from good to very good and satisfactory to very good respectively.
Student Questionnaire results appear in Table 3. These results show that 75.7% and 77.5% of the possible responses within the Learning Value subscale and the Value Added subscales, respectively, were in the Agree and Strongly Agree categories. The responses for Usability of the Learning Objects and Usability of the Technology were in the Agree and Strongly Agree categories 80.0% and 92.0% of the time, respectively.
Student comments
The responses to the following questions that expressed the most common themes were recorded.
What did you like about this LO?
"I like the use of animations. I am a visual learner so any type of diagram is helpful, but animations are sort of novel and made the module more interesting."
"I liked the way it was broken down into the simplest components so you could figure it out yourself even if you still weren't sure about the material. Also, the fact that it is always there and you don't have to go out and get a CD."
"The interactive process and the use of the graphs related to drug effect on the body."
How could this LO be improved?
"Need to make output continue to relate to variables chosen by the learner. Should label diagrams a bit better for use beyond the demos. Should be able to have 'type of drug' selection as well, not just specific drugs."
"Sounds, more use of animation to support use of formulas. Narration. Glossary. Drug dictionary to describe what it is for.
Please use this space for any further comments you’d like to make about the LO including clarification of any of your responses.
"I would not prefer just a lecture on the topic. I feel that a lecture and a seminar using the LO would work well together."
"The equations page was kind of useless for a basic knowledge of pharmacology. It doesn't tell you what it's for and how it is tied in."
"I found that the LO improved understanding of pharmacokinetic principles."
Discussion of the Findings
The evaluations of the quality of the learning object completed by faculty indicate that it will be a valuable resource for instruction. It is interesting to note that the content quality and the instructor/ student guides received the highest rating and reflects the amount of work that was done to select the appropriate strategies for the learning object. The high ratings for motivation reinforce the constructivist notion that activities which engage the learner are perceived as being more relevant and therefore more applicable for students. The ‘very good’ rating for reusability was somewhat surprising given the amount of modifications that were identified in order for it to be used in other contexts. The ‘low score’ by the one evaluator in the area of feedback/ adaptation may indicate that more direction is necessary for generating the graphs.
The ‘learning impact study lite’ served to reinforce the assumptions about the target audience, and the prior knowledge required by students in order to use the learning object. The results of the ‘usability of technology’ subscale (Agree and Strongly Agree 92.0%) reflects the prevalent role that technology plays in student learning. The 75.7% and 77.5% (Agree and Strongly Agree) responses for the Learning Value subscale and the Value Added subscale suggests that the learning object will help students to better understand the therapeutic principles of drug administration. Most significantly, this reinforces the design decisions about the scope and sequence of the learning object. This also supports the rationale for developing a learning object that could convey the impact that a number of variables have on the overall effect of the drug and the achievement of optimum therapeutic effect.
References
Belfer, K., Nesbit, J.C., Archambault, A., & Vargo, J. (2002) Learning object review instrument (LORI). Version 1.4. http://elera.matchbox.surrey.sfu.ca/eLera/Home
No comments:
Post a Comment