What's new? Assessing Learners with Enterprise (pm)

This afternoon focussed on SafeAssign, the Grade Center and Scholar (not sure what this has to do with assessing, but I suppose it's an opportunity to push the tool).

A useful debate in the presence of Blackboard about the usability, value and robustness of SafeAssign. The urgent checking option isn’t really an option at all because all instructors will want checking to be completed as soon as possible rather than queued in the ‘slower’ option. SafeAssign does not yet accept Office2007 formats and it is not known when this will be resolved. Pepijn referred to the reports as giving an actual judgement of plagiarism. We explored copying and submitting web sources; the reports rarely identified the correct sources or simply gave partial matches, if they identified any matches at all. Jennifer expressed concerns about this and is going to report this issue to their Product Development Team.

A fantastic time exploring and getting to grips the Grade Center, although Pepijn and Jennifer must have experienced question fatigue from the amount of questions I was throwing at them. Again it was good to be able to hear reasons why things work the way they do from a Blackboard perspective, and being able to suggest enhancements.

And finally we were given a quick over of Scholar. I didn’t really take much away from this, not least after 7 hours of sitting in a warm pc room.

What's new? Assessing Learners with Enterprise (am)

Quick report on the morning's session of the post-pre-conference workshop on 'What's new? Assessing Learners with Enterprise'. Arriving tired after being kept awake by police and ambulance sirens during the night's city centre violence, the workshop aims to cover the Self and Peer Assessment and SafeAssign tools, the Grade Center and Scholar.

The session is facilitated by Pepijn (Pippin) Kalis (Blackboard Training Manager) and supported by Jennifer Matthews (Director of Blackboard Training). There are 3 attendess: a teacher from Delft University (which burnt down last week - http://www.dumpert.nl/mediabase/105731/46ccb8cd/bouwkunde_gebouw_delft_stort_in.html), a Blackboard systems administrator from the United Arab Emirates and myself. Interesting mix.

So far we've looked at the Self and Peer Assessment tool. We've explored it from the student perceptive and it seems easier to use and more fun than I anticipated. The Instructor view and creating self and peer assessment items is much more complex and involves a number of discrete steps. And Jennifer herself admitted it can be quite difficult and easy to miss out crucial steps. The session is not all about point-and-click, but there's plenty of opportunity for discussion and sharing experiences which works well being such a small group.

Will be starting with SafeAssign after lunch.

My final parallel of the day for me (before our 'graveyard' slot) was by Julie Andrews (not the one of Mary Poppins or the Sound of Music fame) from Manchester presenting on the benefits to staff and students of online formative and summative assessment (so I had to attend, naturally). Eee, and I'm glad I did. Slide no.2: Why should we be thinking about incorporating more assessment and feedback into learning and teaching? And I was referenced on the slide, and Julie said 'I'd given a great talk earlier in the week'. Yes, me on a University of Manchester slide. Highlight of the presentation. Nothing more worth mentioning/hadn't heard before/hadn't written about - benefits of online formative assessment, good reasons for online summative assessment, and how to get staff buy-in. Oh, and she overran into our joint session ...

Respondus

I like Respondus. I want Respondus. I went to the demo by the Account Mgr for Respondus. He demonstrated the 3 (soon to be 4) products of the suite and I have to say that I was actually impressed. We all know about Respondus 3.5 offering an offline assessment authoring tool, but this now makes publishing and deploying an assessment in Blackboard seamless (and you can even publish and deploy the same assessment in several Blackboard sites at the same time) - might be useful for surveys. Respondus allows the user to import questions (even containing images) from Word and publish directly into Blackboard. And Test Banks from several publishers including Thomsons, Pearson and McGraw-Hill can be downloaded for free.

StudyMate Author is a tool that allows Flash activities and multi-player games to be produced and again seamlessly published in Blackboard and they can also be viewed on mobile devices, phones and iPods. These are generated via wizards and templates and once deployed student scores can be extracted via SCORM. I was thinking that this might be useful to generate interesting and interactive activities to support pre-enrolment and applicants, which staff often struggle to develop ideas for - crosswords, jeapody, flashcards.

The next product demonstrated is not yet available - StudyMate Class Server. This plugs into Blackboard and enables staff and students to work collaboratively in generating Flash-based assessment activities, which can be converted into quizzes and linked with the Gradebook. Finally, we were shown the Respondus Lock-Down Browser. This secure browser enables any assessment in Blackboard to be 'forced' into a locked-down browser in which all browser menu functionality (apart from Refresh, it would seem), keyboard shortcuts, the systems tray and taskbar are disabled, screen-monitoring and IM are also disabled. Blackboard and website navigation (unless part of the test) is also locked. Once in the test environment students must submit the test; they cannot close or exit without. The Lock-Down Browser needs to be installed locally, and students can download this on their own machines and it is possible to perform a cluster installation. There is a two month free pilot available and I wonder whether it is worth pursuing this, along with the other products.

Useful Feedback and Flexible Submission: Innovative Assignment Management through Blackboard

Session by some guy called Stuart Hepplestone from Sheffield Hallam. Fantastic. Must have
been 100-120 people in the audience. Lots of nodding. Questions as expected. Integration
with Turnitin? Applying quotas to student upload? And specifying format of files permitted
to be submitted? Looking forward to tomorrow's joint paper.

Formative and summative assessment: integrating Blackboard software and Questionmark

I didn't really want to go to this session, but having not seen QMP for about three years I
thought it time to catch up with any developments.
John Dermo (online assessment specialist apparently) from Bradford asked the audience who
currently uses QMP - only two admitted it.

Drive to online assessment is part of Bradford's 'ecoversity' agenda to reduce paper wasted
on printing hardcopy MCQs, in addition to NSS + feedback, HEFCE strategy on 'enabling
e-assessment', Bradford's LTA and e-strategies, and student expectations. Working on a
Pathfinder Project to 'embed support processes for e-assessment' See:
www.brad.ac.uk/lss/tqeg/projects/caa.php. Outcomes of this project - rules and regs,
training, item banking and learning spaces for QMP. John then spent the rest of his session
explaining the procedures for setting up formative assessments in Blackboard via the QMP
connector, a question type comparison, accessing results in the Gradebook etc, and the
procedures for summative assessment which simply allow assessments to be rescheduled in
event of power/server failure.

QMP still looks the same as the last time I saw it, and is as
difficult to use as ever, and lots of moans from the audience that the connector nor the QM
secure browser doesn't work properly. Really pleased we don't use it.

Using Blackboard for student-led podcasting and group assessment

Chris Stokes - University of Sheffield - talked about an 'engaging way to present a group project' and replace group essays and rather than using yet another wiki (apparently there's wiki fatigue in the Dental School) Chris decided to get 3rd year students to make podcasts (or in reality, audio files) which can be peer assessed. 80 students in 5/6 (self-select) groups were given 2 weeks to produce a 5min 'podcast' based on the critcal evaluation of a published article. Students used Audacity or GarageBand and converted the files to mp3 before uploding to a MOLE discussion forum (My Online Learning Envt) in Moodle called the 'Podcast Wall'). On paper, each group assessed 2 other 'podcasts', the assessment criteria based on HEFCE standards for peer
assessment.

Chris demonstrated some examples; each one had a 'big opening' before becoming more serious
once they had got that out of their system. They were playing to the people who were
assessing them. Examples played were scripted, used clever editing and roleplay, used
backtracking tracks, presented as a radio show, or simply over the top.

Students found that it was difficult to condense their work into a 5min audio clip, but
thought it was an equitable and fair way of assessing as everyone in the group had to
contribute. Concerns raised by students included cross-over into informal space.

Next steps: peer assessment through Blackboard, full RSS feed for podcasts, possibly move to
producing video.