VOLUME 1 · ISSUE 1 · 2017
Conference
Proceedings
DREXEL.EDU/ACONF
BEST PRACTICES IN ASSESSMENT:
A STORY OF ONLINE COURSE DESIGN AND EVALUATION
By
Gulbin Ozcan-Deniz,
Assistant Professor of Construction Management
Thomas Jefferson University
ABSTRACT
The demand and increase in online courses have brought the need to research on online course
assessment methods. As the success of course delivery and the quality of education can only be measured
by proper assessment, best practices in online assessment requires specific strategies in course design and
assessment activities. Due to the unique nature of online delivery that lacks prompt feedback and face-
to-face interaction, the challenge in assessing online student learning is apparent. This study focuses on
the best practices to assess online courses to optimize online course delivery, while using the most out of
the readily available tools such as Blackboard as the Learning Management System (LMS). The discussions
will help readers to decide on how they can successfully assess their own online courses.
Key words: Online Course Design, Online Course Assessment, LMS, BB
INTRODUCTION
Online courses use the online environment to set up and deliver all class material and activities. With the
lack of physical attendance, it is very hard to track students’ progress and learning of course topics.
Assessment methods come into the picture at this stage, however, a general misconception includes
creating exams on an LMS, and assuming it will perform the assessment on instructor’s behalf. Traditional
assessment activities of giving students assignments and midterm/final exams do not work in a fully
online environment, as the course design and students’ expectations are very different from a face-to-
face (FTF) setting. Ideally, the online course should still have the same learning outcomes as an on campus
version of the class with enhanced assessment activities.
There are various studies on matching assessment techniques with learning outcomes. Bloom’s
Taxonomy has been the base of learning and education objectives since 1960s. It describes a hierarchical
level of learning from knowledge to evaluation by increasing the difficulty of the tasks involved (Cannon
& Feinstein, 2005). Learning outcomes are created based on six levels in the Bloom’s taxonomy, such as
remember, understand, apply, analyze, evaluate, and create. Assessment methods for remember or
understand may include answering knowledge-based questions in an assignment, while assessment for
evaluate and create may be a final report or project. The improvement in the levels results in increasing
the difficulty of the task and its content. Previous studies suggest sample assessment techniques associated
with Bloom’s taxonomy levels as given in Table 1 (Thede & Sewell, 2009).
Table 1. Sample Assessment Techniques
28
Suggested Assessment Techniques (Thede & Sewell, 2009)
Bloom’s Level
Remember Flash cards, games, quizzes
Understand Simulations, animations, tutorials
Apply Interactive tutorials, simulations, instructional games, case
studies
Analyze Virtual labs, computer simulation models, case studies,
Evaluate multiple choice questions
Create
Suggested assessment techniques need to be evaluated and modified as needed to assess online
courses successfully. This paper will give an overview of online course design to come up with the best
practices to assess online courses.
ONLINE COURSE DESIGN AND ASSESSMENT
In online courses, instructors need to design their courses to line up interactive learning and assessment
activities with the learning outcomes (Sewell, Frith, & Colvin, 2010). Therefore, a successful online course
design is extremely essential to make sure students can reach the information conveniently, and
experience a welcoming environment in the online portal. The path from learning outcomes to assessing
students’ learning lies in hands of the instructor based on how they will set up their course design with
the tools as given in Figure 1.
Learning Outcomes
Course Online
Content Course
Layout
Instructor Feedback
Syllabus and Loop
Weekly Assessment
Schedule Tools
Assessment Plan for Student
Learning
Figure 1. Learning Outcomes vs. Assessment Plan
Students should be able to find the course content and weekly schedule in an optimal setting in
the LMS system. Student learning outcomes should be identified clearly at various levels of the Bloom’s
taxonomy and should be given to students in the Syllabus. A suggested approach is to set weekly learning
29
outcomes that feed into the course learning outcomes, and that eventually ties up to program and
institution’s learning goals (Figure 2).
Weekly Institutional
Course student learning Program
learning learning
outcomes goals
outcomes goals
Figure 2. Progression of Learning Outcomes from Weekly to Institutional
The author’s suggested online course design starts with the Course Policies (Figure 3). The Syllabus
and Weekly Schedule should be updated to serve to the purposes and conditions of the online delivery. It
should also be noted that the online student body might be different from the on campus student body,
as adult learners, who work for part-time or full-time and who have family responsibilities at the same
time, tend to take online courses frequently. The policies as well as the learning outcomes should be clear.
These learning outcomes will be the basis of the online course assessment, so they will set the standards
on what to assess and how to assess. Secondly, course material need to be created and/or updated to be
suitable for online delivery. At this point, the instructor has the decision to go with synchronous or
asynchronous lectures. Synchronous teaching means live lectures with a real-time online setting, whereas
asynchronous lectures include recorded videos. Jefferson’s Masters for Construction Management
(MCM) Online Program preferred asynchronous setting as many of our students work full time and
prefer the flexibility of watching online lectures wherever and whenever.
Online Course Design
Course Policies Course Material Blackboard (BB)
Organize
Syllabus Video Lectures
Upload material
Weekly Outside Sources
Schedule Asses student work
Assessment Tools
Give feedback
Figure 3. Online Course Design Chart
On campus, assessment tools should be reviewed to see if they are suitable for the online setting.
As mentioned before having midterm and final exams that hold a great percentage of students’ overall
evaluation in one setting may not be suitable for online delivery. Having an exam of 20-30% of grade
will not allow the instructor to track students’ progress successfully throughout a time frame. Besides, it
can create extra stress on students due to the high percentage grade associated. It will be beneficial to use
activities that are associated with small grade points throughout the semester for online courses. There
are two main assessment techniques as given in Figure 4. Summative assessment activities evaluate
students’ learning at the end of a learning period. For example, midterm exams assess students’
knowledge at the mid-term stage. Formative assessment is an ongoing evaluation of students’ learning
through providing continuous feedback in smaller increments of time. While many on campus courses
30
can benefit from summative assessment, formative techniques work best for online students, whom we
need to make sure the interaction, feedback, and evaluation is continuous. The pros and cons of online
assessment activities will be discussed in detail in the next section.
Assessment
Methods
Formative Summative
• Start-of-semester, mid-semester, • Assignments
and end-of-semester surveys • Exams
• Polls • Final Projects
• Discussion Boards • Final Presentations
• Quizzes
Figure 4. Formative and Summative Assessment Methods
At Jefferson’s MCM Online Program, we use the same course template on Blackboard (BB) in all
on campus and online courses to give all students the same look and feel in the BB environment. This
creates a consistent organization for all our courses and makes it easier for students to navigate and find
the information. After creating the video lectures and supplementary material, all should be uploaded to
BB. Additionally, the assessment activities should be created with room for feedback. It should be noted
that the assessment methods should be updated after each online course offering based on students’
feedback and the assessment results. Some activities may work well for a certain course type, while some
others should be eliminated. Details of assessment methods will be discussed next.
ONLINE COURSE ASSESSMENT METHODS (PROS AND CONS)
SURVEYS AND POLLS
Surveys and polls can be used to collect students’ perceptions in the class content and online delivery at
any time during the semester. Surveys given at the beginning, at the middle and at the end of the semester
will allow instructors to evaluate students’ experience throughout the semester better. Same or similar
questions can be used in these surveys to see the knowledge of students at the beginning and at the end
of the class. This will allow the instructor to see which topics were delivered successfully and which topics
or areas need further attention. Whenever an innovative approach is used in class, a short assessment
poll can show the instructor whether the new technique works or not. It is beneficial to keep surveys and
polls anonymous to make sure students answer questions honestly. There are many tools available to be
used to create the surveys and polls. BB has its own surveys, however, the analysis of the survey results
may not be easily documented. Using, Socrative, Survey Monkey, or Polldaddy can be helpful due to the
ease of creating and distributing surveys, as well as saving the results of the analysis. The cons of surveys
are due to the fact that most of the time they are optional and may result in a low response rate. The
author prefers to assign small grade to surveys and polls to increase its effectiveness. As the surveys are
anonymous, students are asked to upload the thank you page of survey tools such as Survey Monkey to
BB as a proof of completion to receive the grade associated.
31
DISCUSSION BOARDS
Discussion Boards are preferred in online courses due to its potential to trigger in depth conversations
with easy access and moderating. BB had a discussion board tool, where all posts are recorded for grading.
Although the discussion boards are flexible in their nature, it should be noted that students need clear
directions on when to post and what to post. An online grading rubric will answer these questions and
will help the instructor to grade student work fairly. With the full participation of the class students,
instructors will be able to assess students’ learning as well as communication skills with online discussion
boards.
QUIZZES AND EXAMS
Formative assessment is suggested for online courses, however summative assessment techniques cannot
be ignored. Quizzes and exams provide opportunities to assess if students have learned a considerable
part of the class material. They can be created in BB as graded tests or quiz questions can be embedded
in video lectures if the format allow. For example, if the video lectures are recorded by using MS
PowerPoint with voice over PPT option, Office Mix add-on can be used to create quiz questions in between
slides. This type of quizzes helps the video lecture to be more active, rather than student watching videos
without interacting. When quizzes and exams are created on BB, the system allows instructors to create
a pool of questions, which can be distributed randomly to students. The order of questions and how the
questions will be seen (one at each time or all together) can also be decided by the instructor. One of the
cons is that it is very time consuming to create and set up these exams on BB. Another common concern
with giving tests in an online environment is the test security. Previous studies mentioned the ways of
promoting honesty, while simultaneously using other sources to control the test security (Sewell, Frith,
& Colvin, Online Assessment Strategies: A Primer, 2010). As online exams are open book exams for sure,
fixed answer questions may be limited to include more short answer questions to receive unique answers
from students. Additionally, third-party tools such as Respondus LockDown Browser can be used in
connection with BB to prevent students from using other documents during the exam. It locks out students
from all other programs and internet, and only allows them to see the exam on their laptop/PC. Some
institutions are implementing the webcam recording requirement together with the Respondus
LockDown Browser to have students record themselves during the exam. The reliability of the assessment
data will surely increase with implementing lockout applications for these online unproctored exams.
FINAL PROJECTS
Final Projects are great opportunities to collaborate in both on campus and online courses. As online
students do not get to know each other in a physical class setting, teamwork assignments to create a final
project will challenge students to work together. In this case, students should be comfortable with BB and
should find other online means of communication. They can use Dropbox or Google Drive to share files,
and can perform conference calls to discuss the details and share their word. As final projects mostly use
real case studies, they assess critical thinking and students’ response to the teaching strategies in the
online class. They also require specific rubrics to be evaluated effectively.
32
SUMMARY OF BEST PRACTICES IN ONLINE ASSESSMENT
Per the online course design strategies and assessment methods discussed before, the top three best
practices in online assessment can be summarized as:
• Identify clear standards
• Include a variety of assessment methods
• Think of students’ background and skills
Online assessment requires clear standards. The instructor should have the learning outcomes defined
clearly and updated for online delivery. The topics of the class, as well as communication skills can be
assessed in online classes. Assessing written communication skills will be easier, however if one of the
learning outcomes requires assessing oral communication skills, the instructor should find alternative
tools for students to record their presentations for evaluation. When the learning outcomes are decided,
online assessment methods should be defined with a target level. On campus, target levels may not work
for online courses, and the levels of achievement expectation should be updated accordingly.
When the assessment methods are in consideration, both formative and summative assessment
activities should be included. Surveys and polls are great for formative assessment, but they allow
collecting indirect data. If the surveys and polls are not mandatory, it will not be possible to evaluate the
whole class and a small sample may not reflect the status of the whole. Discussion Boards can also be
used for formative assessment, if they are distributed evenly throughout the semester. Assignments and
exams work as summative assessment methods, as they tie a considerable portion of students’ grade to
the knowledge and application of several topics. Final Projects are also a summative in a sense that they
allow students to create the output project based on their cumulative knowledge of topics. Using three or
more methods will bring variety and will increase the reliability in the online assessment process.
Students’ background and skills play an even more important role in online courses compared to on
campus courses. In on campus courses, the instructor has the ability to judge his/her audience with the
help of the FTF contact. In online courses, the backgrounds of students are more diverse. This diversity
may be based on students’ technical knowledge or technological knowledge. Some students may be very
successful in using online systems like BB and other tools, while some students may have serious technical
issues. Learning cannot take place in online courses if the technology becomes a problem. The institutions
Information Technology (IT) department, as well as the instructor should work hand in hand to solve
technical emergencies. Additionally, students, who have not been involved in online courses before may
have misconceptions that should be clarified before the learning can take place. The most common
misconception is to see the online course easier that an on campus course, which is not the case. Online
courses require self-motivation and self-organization of students. Students should take the responsibility
of learning in hand and be sure to communicate with the instructor immediately when in need. As
instructors, we can see the need of a student in an FTF setting. However, as we do not share a physical
classroom with them, we will not be able to see, if the online student is in need unless they contact with
us directly. Virtual attendance and formative assessment activities help in this process up to some point.
Still, they do not allow us to track students’ understanding before they submit the activity. Therefore,
online students should be reminded of their responsibilities, and even invited to take an online course
preparedness questionnaire to see if they are ready to take a class online.
33
CONCLUSIONS
This paper summarized the basics of online course design and assessment, together with best practices
for online assessment. The course design steps at Jefferson MCM were shared with readers to tell them
the extents of developing an online course. Various formative and summative assessment methods were
explained and evaluated on their suitableness for online courses. It was suggested to use at least three
formative and summative assessment methods to evaluate students learning throughout the semester. BB
was also evaluated in its extents to be used with certain online assessment methods. In some cases, like
surveys, BB was found ineffective due to its limits in documentation of results. As an important portion
of assessment included evaluating the results, documentation, and ease of creating assessment activities
become very important from an instructor and an assessor’s perspective.
Three top best practices in online course assessment were created based on previous studies and
experience. It was suggested (1) to identify clear standards for learning outcomes assessment, (2) to
include a variety of formative and summative assessment methods for sustaining the reliability of the
assessment data, and (3) to think of students’ background and technical skills during the development of
the online assessment activities. Using various tools and software packages will help in adding variety to
assessment techniques to reach out to a diverse student population. Names of some software packages
were mentioned previously to help readers on this topic.
Successful assessment –online or on campus- still lies beneath matching learning outcomes with
the most suitable assessment methods. Formative assessment helps instructors to assess in pieces, and
summative assessment methods allow instructors to evaluate the whole class content and delivery.
Constant interaction with online students via emails or other means of communication will surely
increase learning. Online students need timely feedback, which can only occur when the instructor is
responsive. Prompt feedback may be missing due to the lack of a physical setting, however continuous
feedback on formative assessment activities are needed to achieve successful learning and assessment.
Best practices in online assessment have been created with generic wording so that instructors in
various disciplines can adopt and use these suggestions. Future research opportunities can include
specializing in certain types of online courses for best practices, as well as detailing the assessment process
such as questioning data, data validation, and the desired level of learning for online delivery.
REFERENCES
Cannon, H., & Feinstein, A. (2005). BLOOM BEYOND BLOOM: USING THE REVISED TAXONOMY TO
DEVELOP EXPERIENTIAL LEARNING STRATEGIES. Developments in Business Simulations and
Experiential Learning, 32, 348-356.
Sewell, J., Frith, K., & Colvin, M. (2010). Online Assessment Strategies: A Primer. MERLOT Journal of
Online Learning and Teaching, 6(1), 297-305.
Thede, L., & Sewell, J. (2009). Informatics and nursing: Competencies and applications (3 ed.). Lippincott
Williams & Wilkins.
34
ABOUT THE AUTHOR
Dr. Deniz teaches core courses and electives in undergraduate and
graduate Construction Management (CM) programs at Jefferson with a
variety of delivery methods such as on campus, hybrid, and online. She is a
LEED Accredited Professional, a member of Construction Management
Association of America (CMAA) and Sigma Lambda Chi ETA V Chapter,
and has experience in commercial and LEED certified buildings. She has
been developing the Online Masters of CM (MCM Online) Program in
collaboration with the CM faculty, and is currently working as the
Assessment Leader of the CM program in College of Architecture and the
Built Environment (CABE). She is a member of the CABE College
Assessment Committee that is responsible to collect and evaluate Middle
States assessment reports. She has been developing program and course learning outcomes in relation
to the institution’s student learning outcomes. Currently, she is working on establishing, revising, and
assessing the Accreditation Board for Engineering and Technology (ABET) student learning outcomes
for the CM program.
Dr. Deniz has performed research in Building Information Modeling (BIM) and sustainable design and
development, as well as in construction education. She has published and presented related to teaching,
learning, and assessment in various national and international peer-evaluated conferences. She has
served as a reviewer of education-related journals and conferences. She has been working on project
grants related to innovative capstone course experiences, Virtual Reality (VR) in construction education,
and best practices in delivering online independent study courses. Her studies continue in the areas of
BIM and VR use in construction education and online education and assessment.
35
