Wednesday, February 14, 2018

Traditional mathematics assessment... but enhanced online

Online mathematics assessment... how do you do it?

Most of the time in mathematics when we hear "online" and "assessment" we're thinking or talking about multichoice quizzes in a learning management system, or perhaps the more mathematically sophisticated options offered by WeBWorK or Numbas. That's all great, but it's not what I'm talking about here. This post is about:
  • students submitting handwritten or typeset work online
  • academics grading and providing feedback on that work, again, online, and
  • returning graded student work to them via online means.
Until 2016, my University mandated the use of a centrally managed assignment submission system. The system was a physical location where students could print out (if needed), hand in, and obtain proof of submission for assessment items. While I'm sure some academics still asked students to submit in class, by email or via "drop boxes" located in departments, none of this was really "allowed". In 2016, the submission system was closed in preparation for a compulsory online submission process.

Now in theory, this is great. Online submission is trackable and proof of submission is fairly easy. Plagiarism detection is significantly automated through, for example, turnitin. Typed feedback and rubric-based grading is also often built into these systems. But what about problem/solution style quantitative disciplines like mathematics, physics and engineering? My colleagues and I found great difficulty in adhering to this new system given that we wanted students to submit worked solutions to mathematical problems (often entirely symbolic/graphical representation), to handwrite feedback to them (again in symbolic/graphical representations), and to get this feedback to them as quickly and seamlessly as possible.

There are certainly issues here, but not insurmountable ones:

  • Handwritten work can fairly easily be scanned to PDF through photocopiers or "scanned" via smartphones. Online submission is then feasible. ADDED PROBLEMS: File formats, unfamiliarity with technology.
  • Handwritten feedback can be provided by printing out PDFs or by using PDF annotators on smart devices/tablets. ADDED PROBLEMS: Printing costs $$$ and is environmentally troublesome. Tablets/devices are costly and generally speaking not owned by all academics. The ability to use the PDF annotators requires training.
  • Feedback can be returned to students by re-scanning the printed copies or transferring annotated files back to the students. ADDED PROBLEMS: Scanning takes time ($$$). Transfer of files is not seamless and again takes time ($$$).

One of the most successful ways to get over most of these issues, that I've found, has been to use Microsoft's OneNote with its ClassNotebook add in. At its heart, OneNote is essentially freeform note taking taking software. It allows both text based and inking input (ie using a pen on a tablet), as well as the capability to import the contents of a file (say a PDF) into a note space. In the maths classroom, this is great because these three things allow students to input their mathematics in authentic ways: handwritten straight onto the screen, typed (just like in MS Word docs), or handwritten and scanned to a PDF and imported to the notebook. So it seems that one side of the story (the student) is catered for.

But what about the academic and the feedback process? Well that's where ClassNotebook comes in. The addin is in some sense a way to provide learning management system-like capabilities from within the OneNote notebooks you might want to create, as well as the sharing of notebook spaces. There's lots of things that can be done here including simply co-authoring notes, communicating synchronously and asynchronously, locking spaces, and so on. But back to the issue of online assessment in maths, here's what I did.

First, I was able to share a OneNote notebook with all of my students so that they could see the same notebook as me. This can either be done manually, or if your LMS is setup appropriately, by linking to that LMS and the contained classlist. I had notes and so forth in the notebook that I shared with the students. They could see these note pages, make copies of them and then edit them as much as they liked (I retained a clean copy). Students each had their own personal spaces in the notebook that only they and I could see. We could interact through these as well. I also had my own hidden space where I could keep notes etc.

Now for the key part: I could share an assessment item handout with my students and automatically set up a space where each student would attempt the assessment. When the due date and time came around, I could grab each of these (electronically speaking) and keep what amounts to a timestamped copy of their work, which I would then edit, mark, annotate etc. I could do this by typing, or because I have a tablet by inking straight onto the work itself. If I wanted to I could even make audio and video annotations if that made it easier to explain something. When I was ready, I could return a copy of this to each individual student's workspace with a couple of mouse clicks.

Fantastic! But this is just one idea. I'm really interested to know what other people are doing in the electronic/online assessment space. Let me know!

Class response systems and (re-)shaping the lecture

This post is kind of in response to Amie's tweet above, and kind of something I probably should have done ages ago. Also, I make no promises about this being novel or interesting!

In 2015 I started running my first year introductory engineering mathematics course in a fairly structured, blended (some would say flipped) manner (see references below for further detail). One of the big parts of that was trying to extract the "process-heavy" elements from my f2f lectures, moving them to videos and out of class activities. This extraction allowed for more interactive learning/more extended/broader learning in the f2f classes. But it also meant I needed ways to ensure that people were progressing with the outside of class activities AND getting something out of them.

What I'll describe below is essentially one of the ways I went about this checking or diagnosing if you will. In the end, it became a way for me to both check in on students' progress and understanding as well as to really shape the f2f learning and teaching to suit the students present at the time. So here we go.

---------------------------------------

Prior to each f2f lecture class, I would create 3 or 4 multi choice questions on GoSoapbox which related to the process/content videos and activities that I had asked students to work on over the prior 7 days. Sometimes these would be process questions themselves (eg carry out this operation, differentiate this function etc), while sometimes they would address understanding of concepts. Generally though, they were not overly deep questions and meant to be something that be completed in around a minute or two each for a prepared student.

Prior to commencing the lecture, the GoSoapbox details were provided on the screen and students were directed to attempt the questions using whatever device they had with them. I would allow around 10 mins (plus what ever pre-lecture time was available) for students to attempt the questions, before calling the class to a stop. At that point, I would project the results for each question - GoSoapbox provides percentage bars for poll-type questions for example. Here's two examples.



Now it's at this point that the results would determine what was to come! I usually planned for an "ideal world" scenario. In that world, the majority of those present would successfully complete the problem. I would show the results for a question, indicate the correct response, explain away any distractors, and provide references to materials etc for those who had troubles. Then we would move on to extension activities or interactive activities etc, building on the pre-lecture work.

Of course, the world wasn't always ideal! It's in these cases that things are able to get interesting and learning opportunities raise their heads. For example, I could simply go about presenting a worked solution to a question live in front of the class. That's ok, sure, but there is more that can be done. For example, when a significant percentage of the class answered a question incorrectly, we would focus on their misconceptions and attempt to unravel them and reform a better/more correct conception. Or if there were two responses that students favoured (one correct, one incorrect), we would look at the differences and why one was correct and the other incorrect.  Sometimes we even got interactive and students would discuss with their neighbours how they arrived at a particular answer, and try to figure out who was right and why.

Regardless of what happened though, the great thing about this was that I wasn't just guessing where my class was at - I was collecting the data in real-time to know (well, close to it) where they were at. We could move on if it made sense, or stick around and iron out the creases if need be. The experience was meaningful, and hopefully useful, to those present.



Shameless plug for our papers...

Czaplinski, Iwona & Mallet, Dann G. (2016) Preparing future graduates to become lifelong, expert learners: Lessons and considerations from a blended learning engineering mathematics unit. In The Australian Conference on Science and Mathematics Education, 28-30 September 2016, The University of Queensland, Brisbane, Qld.
Czaplinski, Iwona, Mallet, Dann G., Burrage, Pamela, & Psaltis, Steven (2015) Preparing engineering graduates for the knowledge economy through blended delivery of mathematics. In HERDSA 2015 Conference: Learning for Life and Work in a Complex World, 6-9 July 2015, Melbourne, Vic.
Czaplinski, Iwona & Mallet, Dann G. (2015) Investigating the effectiveness of an ecological approach to learning design in a first year mathematics for engineering unit. In ascilite 2015. Globally Connected, Digitally Enabled, 29 November - 2 December 2015, Curtin University, Perth, W.A.