Page Comparison

Topics Covered In This Article

• Workshop Overview
  • Key features
  • Peer assessment
  • Self-assessment
• Workshop Phases
  • Setup phase
  • Submission phase
  • Assessment phase
  • Grading evaluation phase
  • Closed
• Workshop Grading
  • Grade for submission
  • Grade for assessment (grading grade)
  • Groups and Workshop
• Workshop Grading Strategies
  • Accumulative grading
  • Comments
  • Number of errors
  • Rubric

This document will provide an overview of the Moodle Workshop peer assessment activity. You may also find the step-by-step workshop article, which goes through setting up and facilitating a Workshop peer assessment activity, to be informative.

Workshop Overview

Workshop is a multi-phase peer assessment activity that generates two separate grades for students: (1) a grade for submission based on peer evaluation, and (2) a grade for assessment based on students' ability to assess.

Workshop is one of the more challenging Moodle activities to gain proficiency with, but it's also one of the most rewarding to students. Evaluation, which is at the top of Bloom's Taxonomy, engages higher levels of learning. 

Key features

Workshop is similar to the Assignment activity activity, extending its functionality in many ways. It is recommended that both course instructor and course participants (students) have at least some experience with the Assignment activity before using Workshop.

• As in the Assignment, course participants submit their work during the Workshop activity. All course participants submit their own work. The submission may consist of text and attachments. Therefore, the Workshop submission merges both Online text and Upload file types of the Assignment activity. Workshop does not support students working teams and submitting a single submission per group.
• The submissions are assessed using a structured assessment form defined by the instructor. Workshop supports several types of assessment forms. All of them allow multi-criteria assessment, in comparison to the Assignment activity where only one grade is given to a submission.
• Workshop supports the peer assessment process. Course participants may be asked to assess a selected set of their peers' submissions. The activity coordinates the collection and distribution of these assessments.
• Course participants receive two grades in a single Workshop activity and both of these grades appear in the gradebook:
• A grade for participants' submissions (i.e., how good their submitted work is)
• A grade for participants' assessments (i.e., how well they assessed their peers)
• The process of peer assessment and understanding the assessment form can be practised in advance on example submissions. These examples are provided by the instructor together with a reference assessment. Workshop participants can assess these examples and compare their assessments with the reference assessments.
• The course instructor can select some submissions and publish them so they are available to the other participants at the end of Workshop activity (in comparison to the Assignment activity where, with the default permissions, submitted work is available only to the author and the instructor).

Peer assessment

If this feature is enabled, students will be allocated submissions from peers to assess. Students will receive grades for the accuracy of their assessments, which will be added together with the grades for their own submissions and will be used as the final grades for the assignment.

This is the key feature of Workshop: to encourage students to assess the work of their peers and learn from each other. Through this, they will see the strengths of their classmates’ submissions and have a better understanding of how to perform well at the assignment. In addition, the advice they get from their peers will give them a more comprehensive view of their own work: The comments from their peers will point out the weakness of their work, which is generally difficult to discover by themselves.

Self-assessment

If this option is turned on, students will assess their own work, possibly in addition to assessing the work of others. The grade received from self-assessment will be counted in the grade for assessment, which will be added together with the grade for submission and used to calculate the final grade for this assignment.

This setting enables instructors to see whether students can determine the strengths and weaknesses of their own submissions and judge them objectively. It is a good way to help students think more comprehensively.

Workshop Phases

The workflow for the Workshop activity can be viewed as having five phases. The typical Workshop activity can cover days or even weeks. The instructor switches the activity from one phase to another manually or this process can be automated.

The typical Workshop follows a straight path from Setup to Submission, Assessment, Grading/Evaluation, and ending with the Closed phased. However, alternatives may be used such as multiple submission/assessment phases to allow students to respond to feedback.

The progress of the activity is visualized in the Workshop planner tool. It displays all Workshop phases and highlights the current one. It also lists all the tasks the user has in the current phase with the information of whether the task is finished, ongoing, or even failed.

Setup phase

In this initial phase, Workshop participants cannot do anything (neither modify their submissions nor their assessments). Course instructors use this phase to change Workshop settings, change the grading strategy, or modify assessment forms. You can switch to this phase any time you need to change the Workshop setting and prevent users from modifying their work.

Submission phase

In the submission phase, Workshop participants submit their work. Access control dates can be set so that even if the Workshop is in this phase, submitting is restricted to the given time frame only. Submission start date and time and/or end date and time can be specified.

Assessment phase

If the Workshop uses the peer assessment feature, this is the phase when Workshop participants assess the submissions allocated to them for the review. As in the submission phase, an assessment start date and time and/or end date and time can be specified.

Grading evaluation phase

The major task during this phase is to calculate the final grades for submissions and for assessments and provide feedback for authors and reviewers. Workshop participants cannot modify their submissions or their assessments in this phase. Course instructors can manually override the calculated grades. Also, selected submissions can be set as published so they become available to all Workshop participants in the next phase.

Closed

When a Workshop is transitioned to the Closed phase, the final grades calculated in the previous phase are entered into the course Gradebook. This will result in the Workshop grades appearing in the Gradebook and in the Workshop. Participants may view their submissions, their submission assessments, and eventually other published submissions in this phase.

Workshop Grading

Grades for a Workshop activity are obtained gradually at several stages and then finalized. A Workshop creates two grade items in the Gradebook:

• Submission: a grade for the participant's submission based on the assessment from peers
• Assessment: a grade for the participant's assessment of other submissions

During the Grading evaluation phase, the course instructor can allow the Workshop to calculate the final grades automatically. Note that grades are stored in the Workshop activity until the activity is switched to the final, closed, phase. Therefore it is safe to modify the grades until they are acceptable and then close the Workshop to push the grades to the Gradebook. The Workshop can be switched to another phase at any time, even after it is closed. So after switching to the closed phase, the grades can still be recalculated and/or overridden (grades can always be overridden in the Gradebook).

During the grading evaluation phase, the Workshop grades report provides you with a comprehensive overview of all individual grades. The report uses various symbols and syntax:

Grade for submission

The final grade for every submission is based on the grades given by all reviewers of this submission. More specifically, it's calculated as a weighted mean of these assessments.

Course instructors can influence the grade for a given submission in several ways:

• by providing their own assessment, possibly with a higher weight than usual peer reviewers have
• by changing the weight of the peer reviews
• by overriding the grade to a fixed value

Grade for assessment (grading grade)

The grade for assessment estimates the quality of the assessments that the participant gave to peers.

Workshop estimates the quality of a participant's assessment by comparing that participant's assessments to the average of the assessments provided by the other participants. The further away an assessment is from the average of assessments the worse that assessment is considered. To obtain meaningful estimates it is recommended that there are at least three peer reviews for each submission. During the Grading Evaluation phase, Comparison of assessments can be set from very lax to very strict to determine how close an assessment should be from the average of the assessments to get full marks. Note that the description in this paragraph has been simplified and is not technically correct from a math perspective, but it gives you the general idea of how things work.

Groups and Workshop

When a Workshop is used in a course using separate or visible groups and groupings, it is possible to filter by group in a drop-down menu at the Assessment phase, manual allocation page, grades report and so on.

Workshop Grading Strategies

The selected grading strategy determines how the assessment form will look and how the grade for a submission given by a certain assessment is calculated based on the assessment form. Workshop has four standard grading strategies: Accumulative grading, Comments, Number of errors, and Rubric.

Accumulative grading

In this case, the assessment form consists of a set of criteria (or aspects). Each criterion is graded separately using either a number grade (e.g., out of 100) or a scale (using either a site-wide scale or a scale defined in a course). Each criterion can have its weight set. Reviewers can comment on each criterion.

Comments

This assessment form is similar to the one used in the accumulative grading strategy but no grades can be given, just comments. The total grade for the assessed submission is always set to 100%.

This strategy can be effective in iterative workflows when the submissions are firstly just commented by reviewers to provide initial feedback to the authors. Then Workshop is switched back to the submission phase, and the authors can improve it according to the comments. Then the grading strategy can be changed to another one using proper grading and submissions are assessed again using a different assessment form.

Number of errors

This assessment form consists of several assertions, each of them can be marked as passed or failed by the reviewer. Various words can be set to express the pass or failure state (e.g., Yes/No, Present/Missing, Good/Poor, etc).

Rubric

The rubric assessment form consists of a set of criteria. For each criterion, several ordered descriptive levels are provided. A numerical grade is assigned to each of these levels. The reviewer chooses which level answers/describes the given criterion best.

The assessment form can be displayed in either grid form or list form. It is safe to switch the representation of the rubric any time.

Levels are assigned point values. To obtain the expected results, it is recommended that there is always a level with a value of zero (otherwise points will be normalized within the min-max range).