Context

Dr Darren Oatley (Chemical Engineering) is the module leader for EG-M85 – Strategic Project Planning. It is a cross disciplinary engineering module where Masters Level students come together to work with each other in mixed discipline teams to consider a variety of engineering issues and projects. It is unlikely that the students will know each other due to their diverse backgrounds and the module basically looks at the issues surrounding operating and planning project work as part of a multi disciplinary team – the norm in industry.

The module assessment for each team is to plan, manage, run and complete a project, paying close attention to the team work involved. There had not been any peer assessment in previous years as a suitable method of carrying it out had not been identified.

What was done ?

WebPA was used to collect “after action review” following completion of the assessed project work for the module. Darren had always wanted to do this previously, but a suitable tool had not been found to allow him to do it satisfactorily. He considered it to be a vital part of the process as he felt the students needed to be made more aware of the fact that this was how it would actually be done in the workplace. Large, interdisciplinary teams, combining their engineering cultures and skill sets to bring projects to successful completion in a timely and cost effective fashion, followed up with an after action review.

The review in industry is used to evaluate the performance of not just the team, but also the individual members of the team. To find out who works well with who. To decide who gets a bonus and who gets promoted.

So the students were put together in groups of eight to 1ten. They then spent six weeks on the project task. Then they completed the WebPA Assignment that Darren had set for them to assess the contributions made to the success (or otherwise) of their project. After Darren had taken account of the WebPA feedback in his marking, he then made the results available to the students, so that they could see how it was that their peers had (anonymously) rated them on the criteria.

What actually led to using WebPA ?

On asking round the College to find out what tools were available to do this sort of After Action Review, Darren heard of the WebPA project and communicated his interest in being involved to the project board.

What worked ?

The simplicity of the system was a winning factor. The ease of set up, from the lecturer’s point of view was key. Dr Oatley felt that the developers of the system had obviously remained focused on their desired output and generated an intuitive interface that did “what it says on the tin”.

Getting the peer assessment to the students was also a simple process and all the students found the system easy to use. They also got a considerable amount from the process. Although there was not a high completion rate for the student evaluation survey for this particular group of students, Dr Oatley was able to provide a considerable amount of anecdotal evidence. The students enjoyed using the system and also appreciated the opportunity to, in some cases, vent some frustrations they had with their peers and, in other cases, to give their well performing a “pat on the back”.

Darren also noted the way that showing the students their peers rating of them was able to demonstrate to some high performing, but overly self critical students how greatly they were valued by their team ... and at the other end of the spectrum showed some less active, but more self praising students how they were perceived by their peers !

All in all, Darren felt that the system contributed greatly to the increase in the students awareness of the need for strong team working skills in the workplace and how the after action review process would be used and how it would impact on them.

What didn’t work ?

Not things that didn’t work ... instead two things that Dr Oatley would have like to have seen in the system. Everything on the system worked very well.

Firstly, Darren wished there had been some method of automatically making the anonymous ratings available to the students as he been required to go into the system and do it all manually.

Secondly, although he admits it isn’t actually what the system had been written to do, he would have liked the ability for the students to rate the performance of the group as a whole on each of the criteria.

Will it happen again next year ?

Yes, definitely. Because the idea of the after action review is so important and so relevant to the students working lives, now that a way of doing it convincingly has been found, it will certainly be done again. Darren is responsible, for one set of the Strategic Project Planning, students and his colleague (responsible for the rest) also used it and will be using it again next year too. Darren intends to extend the use into some of his undergraduate units next year too.

He also hopes to encourage more of his colleagues to look at peer assessment and feel that WebPA might well help to get over some of their issues with it.

Will anything be done differently ?

It was felt that some of the questioning had been a bit too simple, so Darren feels he needs to give that some attention for next year – maybe asking some similar questions a couple of times to cross reference for consistency in the students responses. He is also intending to be more descriptive in his ranking values.

Darren will also be experimenting with a strategy to get over the absence of the ability for students to rate the group as a whole. He has it in mind to set up an extra “student” in each group, but to assign these extras as the “Whole Group”, so by rating the extra member, the students can rate the performance of the group overall.