knowledge-kitchen / education-automation / course-notes

Automated Grading (within an Introduction to Computer Programming Course)

The Introduction to Programming course is designed to include automation of many mundane aspects of grading and administration in order to allow more focus on human elements of teaching and grading.

Quizzes

Quizes are issued to students as Google Forms. Submissions are automatically graded against an answer key entered into the form’s administrator interface.

The Google Forms admin interface contains a Responses tab, where student submissions are visible. This page also shows an icon that can be clicked to generate a Google Sheet showing each student’s time of submission and score.

Generate spreadsheet from Google Forms

Usage

  1. All scores are kept as integers between 0 and 100.

  2. Scores are automatically pulled from the spreadsheet associated with the form into the main course gradebook spreadsheet. Students’ form submissions are automatically joined to their gradebook data via the email address, which thus must be the same in both spreadsheets.

  3. Once an assignment is past due and late quiz submissions are no longer accepted, click the button to Release scores from within the Google Form Responses page - this sends students an email containing their score and a link to view the answer key of the Quiz.

    Release Scores

Integration with Google Sheets Gradebook

If using a Google Sheets gradebook, a column in the gradebook can be set up to pull the score from the Google Sheet with the students Quiz scores, including automatic calculation of late penalties.

For example, given the following gradebook structure, which includes students’ email addresses (column C), a link to the Google Form Quiz results spreadsheet (cell AE2), and the due date and time of the Quiz (cell AE1): Google sheet gradebook

The following formula, entered into cell AE38, would pull the score from the Google Form Quiz results spreadsheet for the student with email address in that row:

=IFNA(ARRAYFORMULA(VLOOKUP($C38,importrange(AE$2,"B2:G1000"),2,False)), 0)

And the following formula, entered into cell AF38, would calculate a late penalty of 10 points per 24 hours late for the student in the same row, with a 0 given for submissions more than 72 hours late:

=IF(XLOOKUP($C38, importrange(AE$2,"B1:B1000"), importrange(AE$2,"A1:A1000"), "No submission")="No submission", 0, IF((XLOOKUP($C38, importrange(AE$2,"B1:B1000"), importrange(AE$2,"A1:A1000"), "No submission")-AE$1)<=0, 0, IF((XLOOKUP($C38, importrange(AE$2,"B1:B1000"), importrange(AE$2,"A1:A1000"), "No submission")-AE$1)<=1,-10, IF((XLOOKUP($C38, importrange(AE$2,"B1:B1000"), importrange(AE$2,"A1:A1000"), "No submission")-AE$1)<=2, -20, IF((XLOOKUP($C38, importrange(AE$2,"B1:B1000"), importrange(AE$2,"A1:A1000"), "No submission")-AE$1)<=3, -30, -AE38)))))

Assignments

Assignments are issued to students as GitHub repositories using GitHub Classroom. Each student receives a copy of a code repository containing instructions and any relevant starter code.

There are tests directories in most assignments. These includes Python-based unit testing code that automatically analyzes students’ submissions to determine whether they satisfy the requirements of the assignment.

Usage

To use these automated tests, do the following:

  1. Install Python if you don’t already have it, and then install pytest. You can install pytest from the command line by running pip install -U pytest

  2. Open a student’s repository in Visual Studio Code, open the integrated Terminal within VSCode, and run the command, pytest. This should show you how many tests passed and how many failed.

  3. Develop a rubric for each assignment that determines how many points each test should be worth, and subtract points for any tests that fail. All assignments are graded from 0 - 100 and all tests should be equally weighted. Take off points for any lateness, as per the course late work policy - the commit history of any repository on GitHub shows the date and time of each submission.

  4. Once this automatic analysis of student submissions works on your machine, focus your human efforts on looking over students’ code and giving useful feedback and criticism. Take off some points for stylistic code problems like poor indentation of code, inconsistent capitalization, unnecessary type conversions, and other sloppy practices discussed in the course, even if the code works functionally.

  5. Leave short but clear helpful comments for every assignment in the main course gradebook. Explain in human terms why each student lost any points. There is no need to refer the tests in these comments - explain concisely in human terms the problems identified in the code either by the automated tests or by yourself.

Batch assignment grading

A separate project contains tools for automatically calculating grades for all submissions to a particular assignment based on the number of automated tests that pass. That repository also contains a tool to check how much time each student took to complete the work, which can be useful for giving late penalties.

Exams

Each exam is structured as a combination of a quiz and an assignment, so grading exams combines the techniques for both types.

Main Gradebook

The main course gradebook is a Google Sheet. This spreadsheet contains a Google Apps Script that will send students an email containing all their grades when run.

Usage

A time-based Trigger is set up in the script to send these emails automatically a few times per week. Nothing further is needed.

However, if necessary, it is possible to manually send grading emails to students.

Send grades from gradebook