| 
  • If you are citizen of an European Union member nation, you may not use this service unless you are at least 16 years old.

  • You already know Dokkio is an AI-powered assistant to organize & manage your digital files & messages. Very soon, Dokkio will support Outlook as well as One Drive. Check it out today!

View
 

SALT Case Study: Taking exams on line: does it work and does it save time in marking

Page history last edited by Chris Hall 10 years, 9 months ago
Dr Sam Webster

Academic: Cornelia Tschichold - College of Arts & Humanities
Librarians: Dr. Sam Oakley, Dr. Ian Glen and Susan Glen
SALT Team: Chris Hall

 

 

 

Context

 

The lecturer who normally teaches the 10 credit module MLTM05 - Study Skills and Linguistics for Translation MA Students was on sabbatical and as Cornelia Tschichold teaches the 20 credit module ALEM25 – Describing English, which covers similar ground, the question of whether the two modules could be taught together was raised. Teaching the two modules together was easy to organize, but the assessment required more thought, both in terms of adjusting for the different credits for the two modules, and of dealing with the extra marking.  Could the tools in Blackboard be used to provide a separate 10 credit assessment that would not greatly increase the marking load? 

 

MLTM05 - Study Skills and Linguistics for Translation MA Students

Studying translation involves a number of special skills. They include, for instance, making optimal use of mono- and bilingual dictionaries, and writing a well-structured commentary explaining the translation strategy or the use of specialized software. Some general skills are also needed, such as setting up a bibliography for a parti­cular topic and formatting documents to a professional standard. Apart from these skills, this module will also give students some introductory know­ledge of, for instance, the difference between terms and words and the organization of Computer-Assisted Translation tools. Finally, this module will introduce basic linguistic concepts and terminology, which can be used in discussing translation work. This last aspect is what was assessed in the exam described here.

 

ALEM25 – Describing English

This module will equip students with the terminology and skills necessary to describe and analyse the structure of English at the word, clause, sentence and text levels. Attention will be given to both spoken and written language samples, and classes will include training in the application of linguistic description to text samples from different media and registers.

 

Our approach.

 

The solution arrived at was a separate two part exam for the MLTM05 - Study Skills and Linguistics for Translation MA Students, which was made up of a multiple choice question exam and a short essay. These would both take place over two hours in a PC lab under exam conditions. The multiple choice exam would be administered through the Blackboard test tools and the essay would be written in MS Word an uploaded to Turnitin. 

 

The first thing to check was whether it was possible with Blackboard and whether anyone had used Blackboard for this type of activity before. The answer from the Blackboard team was positive, as it had been successfully used in the past. When talking to the students about the format of the exam they were quite comfortable with the process, as the MA in translation makes extensive use of IT and they were used to uploading work to Turnitin.

 

Before setting up the Blackboard part of the exam, the multiple choice questions were drafted in Word. This had a number of benefits – it’s more straightforward to write the questions in Word separately from creating the Blackboard test, it’s an easier format to send to the external examiner for approval, and the Word document also formed the ‘Plan B’ in the event of any technical problems on the day. This task turned out to be the most time consuming part of the process. Writing good multiple choice questions is not easy and writing appropriate distractors takes lot of thought. There is more on this in the literature search to the right. Once the questions were written the positions of the correct answers for each question were randomised with the help of a spreadsheet function to avoid an obvious pattern appearing.  

 

The next stage was to create the test in Blackboard. Initially it seemed easiest to cut and paste from the Word document into the Blackboard test tool, but unfortunately that caused a number of formatting issues. In the end the quickest way was to type the questions, which was not too arduous as most of the questions were quite short. Much of the knowledge that needed to be assessed can be tested through well-constructed text-only multiple-choice questions. But syntactic tree structures are almost impossible to read when transferred to a text-only format. So, in order to test students’ knowledge of syntax trees, pictures (see below) were created and uploaded into the Blackboard test tool.  This type of multiple choice question is very powerful and has subsequently been used really effectively in lectures where clickers are used for feedback.

 

 

The complete list of multiple-choice questions were split into three sections to allow for saving answers at more than just one point, in case there were any technical difficulties. In order to make sure that students couldn’t see the questions before the exam, the test was only made available in Blackboard for the time of the exam. In addition, there was a separate password for the test that was given to the students at the beginning of the exam. The whole marking process, including marking the essays, took place on-line (but not using Grademark).

After the exam for the MLTM05 - Study Skills and Linguistics for Translation MA Students the multiple choice test was used as revision material for the ALEM25 – Describing English students, who had their paper-based exam about a week later. The majority of the students on this module have used it as a revision tool and there is a good correlation between results achieved on the Blackboard test and exam result.

 

The outcomes.

 

The main finding is that the test worked well, and there were no technical hitches. You can actually sit and watch the results come in on the Gradecentre in Blackboard as the students work their way through the questions – a bit like election night! All that is left to do after the students finish their exam is to convert the total number of points achieved on the Blackboard exam into a pegged mark. As this part of the exam tested purely declarative knowledge, the scale of marks goes up to 85. 

As mentioned earlier, the students were all comfortable with using Blackboard, Word and Turnitin during the exam. In fact many students preferred typing to writing an essay by hand, as this is how they normally produce written work. Writing by hand for a length of time is beginning to be something only done in exams. Using Word also enabled the students to easily keep a track of the required word count. With the multiple choice test, there were a few students who failed to press the save/submit button at the end of each section. This didn’t mean that their answers were lost but the total score for that student was not transferred to the Gradecentre and had to be calculated by hand. In future, a practice test in class before the exam to familiarise the students with the process would be useful.

 

There is some criticism that multiple choice questions make it easier for students to achieve good marks than traditional essay exams.  This is an issue that can be resolved by careful writing of exam questions. In this instance, there was a wider spread of marks for this exam than for similar but essay only exams, with some failures, some low marks and some very high marks.

 

Additionally, there was some discussion about whether students would be able to use the internet to help with their answers for the essay. However, the questions were written in such a way and with a the time limit of about 30 minutes for the essay that if they started searching for answers they would have very little time to complete the essay. The fact that students submitted through Turnitin would also allow for any plagiarism to be detected.

 

So in answer to the questions of the title - yes, exams can be taken on-line with the right preparation and yes, it does reduce quite dramatically the amount of marking required. In the end, it’s a question of where you put in the work. With an online multiple choice test the work is largely in creating the exam and much time is saved on the marking. The more students you have in a class the more worthwhile in terms of time it is to use an online multiple choice test. 

 

 

Cornelia Tschichold's University Page http://www.swan.ac.uk/staff/academic/artshumanities/ell/tschicholdc/ 

 

 

 

 

 

 

 

 

Comments (0)

You don't have permission to comment on this page.