Online Participation as an Instrument of Learning and Assessment - DE Oracle

DE Oracle @ UMUC An Online Learning Magazine for UMUC Faculty Center for Support of Instruction Online Participation as an Instrument of Learning and Assessment Stella Porto Program Director Graduate School of Management and Technology Published: September-October 2006 Category: » Online-pedagogy » Classroom-communication Introduction Asynchronous computer conferencing has enabled a more interactive form of distance education, retaining the initial flexibility of time and space (Bullen, 1998). Engagement in an online course depends on the use of a system that promotes continuous student interaction (Edelstein & Edwards, 2005). Such interaction is commonly and effectively implemented through the use of 'threaded discussions' The instructor will be the one to design and manage such discussions "to direct students in achieving the intended learning outcomes" (Edelstein & Edwards, 2005, p.1). Requiring students' participation in the online discussion through regular contributions is a way to determine if students are keeping pace with the ongoing learning activities, their commitment with the learning community and their necessary involvement with course content. This article discusses the goals and learning objectives of online discussions, the instructor's role in such interactions and provides insights, guidelines and examples of how to assess students' performance in online conferencing. The goals of threaded discussions and instructor's role Before discussing how we should assess our students' participation in the online classroom, we should have in mind that during those 10 or 15 weeks of class, we need to create an "effective online educational community" (Anderson, 2004, p.273). Such 'community of learning model' is built upon three essential kinds of 'presence': cognitive presence, social presence and teaching presence. The cognitive presence refers to the "study of a particular content" (p.274); the social presence refers to the existence of an environment where student feel confident enough to express themselves freely and openly; and finally, the teaching presence represents the formal education or instruction. The discussions that take place in the online environment should allow for social presence, but should aim at the opportunities for teaching presence. This is the dimension, which addresses the effective learning that results from interaction among students and between students and instructor throughout the course. Anderson et al. (2004) list three roles that an instructor needs to perform in order to achieve effective teaching presence, namely: the "design and organization of the learning experience" (p. 274); encouraging "discourse between and among students" (p.274); and going beyond the e-moderation to sharing expertise through direct instruction. Lecture notes, personal reflections and insights, full descriptions of one's experience, can all be characterized as part of the teaching presence category of interaction between instructor and students. These serve to stimulate and motivate students. In many cases, especially at the university level, students will take turns in assuming a role of teaching presence as well, sharing their professional and personal experiences. The myth of the 'guide on the side' might not serve well to make the point to instructors that the teaching presence is the kind of interaction that students are most eager to hold with instructors. The 'guide' symbol was brought into play as a reaction to the 'sage on the stage', which has served as the negative paradigm associated to traditional face-to-face teaching practices. These are two extremes of the same spectrum, none of them in fact reflecting best teaching practices, be it online or in class. Nonetheless, Online Participation as an Instrument of Learning and Assessment - DE Oracle 'the sage on the stage' tends to better fit the traditional prototype, while the 'guide on the side' evokes the image of 'a companion through the journey of learning&', which has been used frequently in the more superficial recent literature in distance education. We need to have in mind that the end result should be the same, students' learning. It is true, however, that online education does require more autonomy from the student. The instructor should be an instrument towards this autonomy, not losing sight of the teaching presence opportunities that a classroom based scenario provides, even if the classroom is a virtual environment. Formative and summative participation assessment Besides the roles mentioned above, in a formal education environment, there is the role of evaluation and assessment, which substantiates the institution with its own role of certifying students' learning. In essence, with the use of online education, assessment has not changed its conceptual framework of verifying if learning objectives are met and determining students' performance in achieving those objectives. As part of the overall assessment, in the online environment, we confront ourselves with assessing students in their participation in the online community of learning, through their own 'presence' in the threaded discussions. Thus, there is an intrinsic connection between our assessments of students' participation, how we organize the threaded discussions and our own participation model, which we define as we interact with students online. The main issue that needs to be considered when designing threaded discussions is to determine how threaded discussions will contribute to the completion of other assignments and students' overall learning. Any assessment should be an integral part of the learning process. Assessment should be an evaluation of students' performance through their "demonstration of the ability to apply knowledge in real-life settings" (Edelstein & Edwards, 2005, p.2). The goals need to be clearly communicated to the learner, so s/he understands how s/he will be evaluated. These principles should serve the instructor when creating the structure for the threaded discussion. One important guideline during this process is to assess how much time students will need to participate effectively. By the same token, the demand on the instructor's time for participating and assessing students' participation authentically should be considered. Thus, it is important to consider how important the threaded discussions are in the learning process, so workload is balanced and time is used efficiently. Requiring student participation is a question that does not have a solid single answer, although more frequently than not, participation plays some part in students' final grade in every course. A possible rule of thumb is to consider participation between 15% and 20% of the grade (ATC, (http://www.wpi.edu /Academics/ATC/Collaboratory/Idea/gradingdiscussions.html) nd.). However, this does not constitute a research-based guideline, and thus should not be taken too strictly. Depending on the course, these values might be justifiably higher, and thus participation even more carefully graded. The context of the course, and the activities proposed as part of the online discussions can change the assessment dramatically. Another often found rule is that instructors should participate in the discussions at the same level or above the one requested from students. Instructors should be defining the benchmark of participation with their own behavior. The lack of instructor presence (Winter, 2002) in the discussions is very often negatively mentioned in students' evaluations as a trait that shows lack of involvement and concern with students' learning. Instructor's presence, as we have discussed before, should be a mix of social and teaching presence. There is certainly no one single assessment method. But, in all cases, having a clear set of criteria and expectations, sharing these with students and using them consistently during the assessment procedure are basic requirements in the process. The use of rubrics can be a significant tool to achieve these goals. It helps in defining and describing what is being assessed, it supports clarifying expectations when communicating with students and streamlines the grading process, reducing the workload associated to this task for instructors. Rubrics are a scoring tool that helps assess the performance of executing a certain task or completing an assignment. In this tool, performance levels Online Participation as an Instrument of Learning and Assessment - DE Oracle are associated to scores, streamlining the process of grading the execution of this particular task (Porto, 2005). Rubrics also serve as effective feedback to students about their performance. Rubrics used for online participation vary. One important differentiator is if the rubric will be used in summative form, after the course has ended, or it will be done in a formative way, throughout the semester, and with intermediate results reported back to the student. Other differences relate to the quantitative and qualitative sets of criteria, as well as the level of student involvement in the assessment process. The next section lists several examples of such rubrics. You will certainly find some good ideas to build your own. Grading participation: criteria and rubrics Assessment criteria for online participation follow mainly one of two major categories, namely quantitative and qualitative. Under the quantitative dimension, rubrics include elements such as minimum number of contributions (including self originating posts and responses to others) per learning module, frequency of postings during a certain timeframe, the size of posts, and timeliness of postings. As part of the qualitative criteria, rubrics will include aspects related to writing quality, research reference, quality of expression, relevancy of the content and how much the post supports the learning community, the dialogue and the collaboration among students (Edelstein & Edwards, 2005). An interesting technique is one that where students are directly involved in the process of assessing their own participation and analyzing their contributions to the conference area. (Murphy & Jerome, 2005) The best way to develop a more personalized rubric is to use examples and then customize them according to your own learning objectives. Below you will find a list of annotated rubrics from distinct sources. These will certainly provide enough input to help you build your own. Rubrics are also a dynamic tool in the sense that they change with our own experience and use of the tool. So, be ready to update your rubric every term. Here are the examples: Example of use of general criteria not in rubric format (http://www.stedwards.edu/ursery/discuss.htm) . In this rubric, the author instead of providing a standard table provides a holistic rubric in which levels of overall participation performance are provided. Each level is carefully described in paragraph mode, and is applied several times during the course, instead of a one time grade at the end of the term. 1. Example of students' self-assessment of participation (http://www.usq.edu.au/electpub/e-jist /docs/vol8_no1/commentary/stu_contrib_ansynch.htm) (Murphy & Jerome, 2005). Students receive clear criteria, and are provided with instruction in the beginning of the course about how they will be graded. They are also informed that after the course is complete they will have to work on an analysis of their contributions according to the described criteria. The analysis provided by students includes both qualitative discussion as well as quantitative considerations about their work. This assignment, in itself, should also be graded as an integrated part of participation or separately. (Other versions of this kind of approach involve students writing pieces of work about their learning in which they cite exclusively their own contributions to the discussion area.) This kind of approach may alleviate the workload of grading participation, but will need clarity with instructions and purposefulness in the activity in order to engage students. 2. Assessing effectiveness of student participation in online discussions (http://www.westga.edu /~distance/ojdla/spring51/edelstein51.html) (Edelstein & Edwards, 2005). This example includes general assessment guidelines, some of them mentioned previously. Also includes an example of this type of rubric with criteria based on: (i) Promptness and initiative; (ii) Delivery of post; (iii) Relevance of post; (iv) Expression within the post and (v) Contribution to the learning community. You will also find an example of the use of such rubric in a specific student case. 3. Grading online discussions from Academic Technology Center (http://www.wpi.edu/Academics /ATC/Collaboratory/Idea/gradingdiscussions.html) (ATC, nd). Here one finds general guidelines and examples of both formative and summative approaches. A few are based on scoring weekly contributions, which can 4. Online Participation as an Instrument of Learning and Assessment - DE Oracle be very detailed. Then there is a summative approach using several general performance levels. Grading Online Discussion Participation from The Educational Technology Center (http://www.edtech.neu.edu/teach/use_blackboard/managing_your_course/documents/sample_rubric.pdf) (Northeastern University Academic Technology Services, nd). One holistic summative rubric represented. 5. Porto's participation rubric (Porto, 2005). A sample of a participation rubric, which can be qualified as being summative and analytical, considering several aspects of students contributions throughout a full semester. It can be used, however, in a formative mode, per learning module. 6. Conclusion It is clear that participation has gained a different meaning within the online education model. 'Showing-up for class' is not enough for students to demonstrate their involvement. This represents a significant opportunity to re-design the learning process, to include online threaded discussion as an activity that contributes to students' overall learning, as well as a way to counteract the lack of 'high-touch' that might result from the asynchronous online education model. There needs to be some thought and planning to utilize threaded discussions as an integral part of the course. Students in most cases should be required to participate and should be told how they will be assessed in their online contributions. Rubrics are the most direct and objective way of communicating with students on how they will be assessed, and they provide a way to streamline the process of grading students as well. Participation rubrics should consider both quantitative and qualitative aspects of online discussions, with all the specifics that will be used during the grading process. The best way to create a rubric is to start out from a few examples and change them according to one's needs. Then after a few iterations of using such tool, tweaking can be done as necessary to dynamically adapt the tool to one's convenience. References Anderson, Terry (2004). Teaching in an online learning context. Chapter 11 in Theory and Practice of Online Learning. Retrieved from http://cde.athabascau.ca/online_book/ch11.html (http://cde.athabascau.ca/online_book/ch11.html) on August 1st, 2006. ATC (nd). Grading online discussions. WPI Teaching with technology collaboratory. Retrieved from http://www.wpi.edu/Academics/ATC/Collaboratory/Idea/gradingdiscussions.html (http://www.wpi.edu/Academics/ATC/Collaboratory/Idea/gradingdiscussions.html) on August 25th, 2006. Bullen, Mark (1998). Participation and Critical Thinking in Online University Distance Education. Journal of Distance Education. Retrieved from http://cade.athabascau.ca/vol13.2/bullen.html (http://cade.athabascau.ca/vol13.2/bullen.html) (http://cade.athabascau.ca/vol13.2/bullen.html) on August 1st, 2006. Edelstein, Susan and Edwards, Jason (2005). If you build it, they will come: building learning communities through threaded discussions. Online Journal of Distance Education Administration. Retrieved from http://www.westga.edu/~distance/ojdla/spring51/edelstein51.html (http://www.westga.edu/~distance/ojdla/spring51/edelstein51.html) on August 1st, 2006. Murphy, Elizabeth and Jerome, Tami (2005 ). Assessing student's contributions to online asynchronous discussions in university-level courses. E-Journal of Instructional Science and Technology. Vol.8, no.1. Retrieved from http://www.usq.edu.au/electpub/e-jist/docs/vol8_no1/commentary /stu_contrib_ansynch.htm (http://www.usq.edu.au/electpub/e-jist/docs/vol8_no1/commentary /stu_contrib_ansynch.htm) on August 25th, 2006. Northeastern University Academic Technology Services (nd). Grading online discussion participation: a sample rubric. Retrieved from http://www.edtech.neu.edu/teach/use_blackboard /managing_your_course/documents/ Online Participation as an Instrument of Learning and Assessment - DE Oracle Contact Site Manager Created and Maintained by the Center for Support of Instruction © University of Maryland University College Powered by ArticleMS from ArticleTrader.com discussion_guidelines_students.pdf (http://www.edtech.neu.edu/teach/use_blackboard/managing_your_course /documents/discussion_guidelines_students.pdf) on August 25th, 2006. Porto, Stella C.S. (2005). An introduction to the power of rubrics. DE Oracle Retrieved from http://deoracle.org/online-pedagogy/rubrics/an-introduction-to-the-power-of- rubrics.html. Winter, Robert H. (2002). How much communication is enough in online courses? – exploring the relationship between frequency of instructor-initiated personal email and learners' perceptions of and participation in online learning. International Journal of Instructional Media v29 i4 p377 (18). Rating: Not yet rated Comments No comments posted. You must be logged in and be a member of the UMUC community in order to comment. If you are a member of the UMUC community and do not have an account, please register for a FREE one. If you have a guest account but are Faculty/Staff of UMUC please send an email to the DE Oracle Site Manager (mailto:degrad@umuc.edu?subject=Please Update my DE Oracle Guest Account) so that your guest account can be updated. Online Participation as an Instrument of Learning and Assessment - DE Oracl