Assessment and WordPress

[GraphDesProject]

I’m talking about WordPress blogs as a site of assessment. I use these with my f2f learners – some of whom now only use these and have given up the traditional sketchbooks or journals. I also use these with my online learners as the sole site of discussion and assessment.

In my graphic design teaching summative assessment usually takes the form of a whole design project, with a final design piece and a body of work to show evidence of how the learning outcomes where achieved – these can be practical or theoretically applied to practice. Formative feedback will be ongoing in tutorial or studio style. Blogs fit in very effectively with all the learning outcomes for both f2f and online courses, as they make excellent places for research and analysis, showing ideas development and problem solving, as well as presenting final pieces and making them into portfolios (so, in effect archiving as well as presenting for professional purposes). F2f learners do also present real world solid final pieces, but will have described and reflected – hopefully at length – in the blog. The only difference is that I am not able to undertake full interim crits (formative feedback) with online learners through blogs, where peer feedback is used (the online learners are studying asynchronously).

The feedback issue is interesting! I feedback to online learners entirely in blogs because it is a logistical necessity and they’ll start conversations in the comments section, perhaps asking for clarification of feedback. Feedback is always some positive comment on strengths as well as areas for development. I have told online learners that I could potentially send a voice message by email or just an email (perhaps if their work was so desperate that it needed a real lot of development – see below…). For f2f learners I do comment in blogs, but as I suppose I am aware that they are mostly in the public domain I often also feed back in person, or if there is something that I need to catch straight away I’ll email them. I guess I am more likely to put good feedback in their blogs as I don’t want to be mean and put criticisms, however tactfully couched, in public view. The comments can of course usually be moderated. I have only just become aware of this habit of mine and wonder if it is a good one or if I should just go ahead and use the blog comments more. One of the reasons I do not do this more is that it would take so much time I’d never actually talk to the f2f learners themselves – they were always asking me to comment on every post, which is a lot!

I’ve also noticed that the stronger learners especially take up on blog comment feedback as well as f2f feedback (ie chats or formal crits) and rework these into their posts as targets and developmental areas. This, I feel, is really good practice for them as it shows them learning and taking criticisms on board to action them. Occasionally they are able to refute feedback and justify why this is the case. This is also excellent practice. I see more of this in blogs than I did in sketchbooks, where practical developments were recorded but often without the stop and pause-to-reflect-on-why moments.

Learners prefer to be assessed “on the job” (Headington et al 2012) and blogging makes it easy to do that as they move through the project – the learners can also record evidence while on the job, so it’s really convenient.
What technologies support this assessment? Apart from blogs I have used in the past the VLE Joomla for online learners, which was then quite clunky but is better now. There was a problem in the site I was asked to use in which you could not see the portfolio and make comments on the same page. The technology for graphics assessment needs to be one where imagery can be seen as well as the commentary and also have the a place where assessor can comment. Digital portfolios like Digication or Mahara might be ok, but I still think blogs are more convenient and allow more ownership/personalisation. I suppose you could use emails, but that’d be slow; or Skype, but hard to see imagery clearly. I understand that learners really like audio-message feedback so that could be a way forward, but blogs, for now are still the most efficient way of feeding back formatively throughout the project (which is most developmental – Shrand and Eliason 2011) and of recording learning outcome evidence for summative grading.

Am I too sold on blogs? My universal Panacea! I wonder if they’d work so well for other disciplines?? Also, I should note that my colleague on the f2f course finds them hard to get time to look at and doesn’t support them as much as I, or many of the learners, do. I’d like to hear some stories of how they didn’t work…….

Sancha (@GraphDesProject)

[Linda Creanor]

Hi Sancha, thanks for starting off this topic with your excellent post on blogs for assessment and feedback in a design context. This is a really interesting area and you’ve raised some useful questions, including,

can blogs be used for summative as well as formative assessment?
does the public nature of blogs encourage less critical feedback?
how can tutors manage their time with regard to student expectations of feedback?

I look forward to hearing other experiences and views!

Linda

 

[Leonie]

Hi Sancha

Formative assessment of blogs is a really interesting idea – particularly the issue you raised about where to provide feedback: the difficulty of providing criticisms in public, yet would students learn to interpret no public feedback indicating that a blog is bad.

And I wondered how you judge a good blog? Do you have set criteria or guidelines for students?

I’m also thinking, as a student, whether being assessed would affect how I write on a blog. E.g. would my writing style be more formal? Would I be less open about mistakes and confusion? Or would I be more likely to let my blogging slip if it wasn’t assessed?

Just wonderings out loud having not given much thought to this before – would be really interested in how you (or others) have tackled these sorts of questions.

 

[GraphDesProject]

Leonie,

I guess a learner might interpret no feedback in the blog as bad if they weren’t getting verbal feedback a lot of the time. I have also heard that school students judge their blogging only by the number of comments they get back!

I judge a good blog probably in a very similar way as to how I also judge a good sketchbook (or any other supporting work). I look for evidence of research and analysis that is meaningful in that it is particular (not meandering) and helps inform the learner about whatever is needed – ie styles, techniques, theory, inspiration, audience and so on. I also look for conclusions, ideas, evaluation and reflection. These are generic to any project, but I’d also be looking for evidence that specific learning outcomes have been addressed usefully (or indeed, sometimes at all!).

I look for ability to self-regulate their design process, to take on board feedback and to reflect. I don’t look for “good” writing or spelling in the posts (only in the going-in-front-of-the-public pieces of design) because I don’t want to stifle the learning, or especially the creativity, by stopping to correct all the time. However, this is also a throw back to what went on in private sketchbooks and now I do have to remind learners about curating their public persona and spelling etc is, of course, part of that in a professional sense. So this raises another issue! Blogs are however, good for rehearsing writing styles and getting uses to making a rounded post. I also like to see learners taking ownership and using their blogs for linking to other personal or professional blogs or portfolio websites, or adding other topics and design portfolios within their blogs. The latter in particular is useful in a professional development sense.

Your last point about letting blogs slip is also very pertinent. Some people take to it naturally and fly with it. Others do, admittedly, slack if not prompted.  I have also started blogging with groups from other disciplines that I am not with all the time (ie a fashion group and a television group) and these only used their blogs patchily or even not at all. There was an article published recently (linked in #ocTEL Tweets – but I have forgotten the reference right now) which said that learners only use technology when prompted to by tutors. I think this may well be true as where my learners have been really persistently encouraged to they do use blogs well (not all do – they have a choice) and when not always prompted or when they don’t see the use, they quite often leave them. On my f2f graphics course all the staff refer to the blogs and they are used directly in summative assessment, whereas on these other courses only one or two staff did so they were not seen by learners as “necessary”.

Hope this helps.

Sancha (@GraphDesProject)

[AliSheph]

Interesting posts.  This would a very radical approach in my discipline.  We’re trying reflective logs for the first time, and my aim would be to turn these into a blogging requirement eventually, but cultural shift would be needed, to say the least.

I’ve done some critique of some other methods at:

http://alicesadventuresinedtech.wordpress.com/2013/05/19/octel-week-6-assessment-and-feedback-if-you-do-only-one-thing/

[PeterHartley]

If you want to consider other technologies that might also be useful then I think you should consider how you might incorporate audio and video in your feedback.

There is a fair bit of evidence that audio feedback can be very useful as it allows you to give much more and more personal feedback in the same time as a tutor – this is my own experience and there are studies which back me up.

For the use of video you might want to look at the work of James McDowell from Huddersfield – http://www.jamesmcdowell.com/Epigeum/ . A very different subject area but the idea of commenting directly on students’ visual productions could be adapted to your area.

Some years ago I was associated with postgraduate work in Design where we replaced the traditional paper-based portfolio with an assessment where students has to present their ‘design story’ within the constraint of 6 Powerpoint slides (using the notes field for personal reflections). Students found this very demanding but much more satisfying than the ‘bung everything in’ strategy many of them applied to conventional portfolios. I don’t know how (or whether) they have updated this approach to use different technologies.

 

[ElizabethECharl]

I am taken with the issue about using blogs and where the feedback is given – it is a real concern.  This approach will suit some subjects much more easily than others, it is more a case of experimenting and being very clear about the criteria required.  I am aware of a film course where students who put their first attempt at film on YouTube were castigated by the public.  Thereafter the VLE was used as a more secure environment for students to show what they had learnt.

[James Kerr]

I think if I were in Sancha’s position I would be slightly more conservative about the use of blogs as the primary means of assessment.  I think it is a good component to include, as it would be an accurate representation of how their work will be assessed by consumers and the public later in their fields, but for academic assessment, I would provide the rubric-guided assessment notes private, with the option of posting a “critical” review on a more public blog space.  The assessment would be directly related to the assignment, but the review would be more related to the “spirit” of the assignment, in terms of an artistic commentary.

[GraphDesProject]

Hi Elizabeth and James,

Yes the public/private issue is really fascinating and needs to be watched. However, let me clarify that blogs are not the primary means of asessment with the f2f students. The Learning Outcomes have a proportion (variously 70%/30% and 60%/40%) in favour of the end product (ie a magazine spread, book jacket, signage etc), so most of the assessment goes towards the actual design itself. The blogs provide evidence of the journey with both practical and theoretical material. Strictly academic pieces, like essays and dissertations have written pieces. My learners don’t tend to use their blogs very much to help with these, despite there being good evidecne that dissertation in particular can be thought about and discussed through connectivist blogging.

 

Let me stress that the learners DO get lots of feedback rom us, usually verbally, so they are not either stranded or suffering public humiliation (most of the blogs are pretty good anyway!!). When surveyed the learners who use blogs all stressed that they much prefered it (inc the online learners). The main concern of my f2f learners in regards to the public/private issue is that they worry about the public stealing their artwork. They have got round this by adding watermarks to finished pieces. Though this, in  a negative sense, also makes them feel part of the community of practice as it is quite a common event.

 

Moreover, since they have been blogging more they have also been connecting more to other online sites for portfolio sharing (like Behance) and they often voluntarily ask for feedback from peers/strangers on Facebook etc etc. They also have aslo engaged in other online forums where the feedback can be ferocious and indeed pretentious. Harsh feedback can be part of design life (I’m not saying that is a good thing) and the learners seem aware of that. We have – hopefully – taught them to separate private and professional feelings.

 

And just before anyone mistakes what I am saying – our own feedback is always developmental. There was a research project at the Uni of Kent about the language of feedback, stressing that is should not be a fait accompli, but with strong or weak work should always show how a learner can take their practice forward. (CPD…)

Sancha

 

[Rachel Temple]

Hi All,

Thank you Sancha for the excellent description of your assessment process using blogs and the pros and cons involved.  Somehow it is more interesting reading these descriptions and discussions than formal texts on assessment.

The link provided by Peter to the James McDowell project on video feedback is amazing, so thank you for sharing this.  The clip is about 15 minutes long, and shows clearly how a revised format of feedback and dialogue was embedded into a whole process of curriculum re-design.  It’s also great fun to watch because it’s about computer game design!

I also agree with Elizabeth’s point about the necessity to be Discipline specific and going with what fits in with that Discipline.

Rachel

[imogenbertin]

I have used wordpress portfolios for assessment, and was concerned about whether, in relation to visual design tasks, there was an element that was less than fair to students who had stronger graphical than verbal/written skills. I tried to design tasks to avoid this but I think blogs can be extremely difficult for those who find writing hard.

If you are the admin of the WordPress site, I think there is a way around the public private issue. First of all, create the blogs as “multisite” (ie a group of “child” blogs belonging to a specific site). Second, don’t put them online in public view (visibility option) during the “formative” phase. Include the comments, record them as needed for your exam process if required. Then let the students decide which comments if any they want to be visible, then make the blogs public on a pre-arranged date.

[GraphDesProject]

Hi Imogen,

Thanks for raising the point about writing.  One of the main reasons that we started to push blogging was specifically as we had found that those who struggled with writing in a journal or sketchbook found it much easier to write in blogs. In fact before we started, our dyslexic students often asked it they could blog instead of hand writing stuff. And indeed we have found, as research indicates, that writing levels do improve with blog use. This is partly because bloggers have to choose a public “voice” and partly because they have to make “rounded posts” (Kirkup) rather than what you might call “random” notes. Those with challenges to writing when surveyed do state that they prefer blogging. However, I do have one excellent student who is very aware of his awful spelling and he prefers to use a sketchbook. I must say, on a note of convenience, that it is much easier for us to read blogs than plough through some terrible hand-writing and this too has helped clarity and indeed has enabled grades to rise.

 

Some of our students do keep thier blogs on private at all times and staff can be invited by email to be allowed to see the content, so no-one has to work in the public eye. Most bloggers though, do prefer to keep their blogs public as they are seeking comments from all and sundry and feel that the how-to tutorial parts of their blogs help others as they are helped by tutorials. And yes, in most blogs the comments are moderated so don’t appear in the blog until the blogger has allowed them. But it is this very public-ness that enables professionalisation of identities (Turner), which is why in-house institutiuonal blogs (Moodle, for instance) don’t work in that sense.

 

Sancha

[Author]

Posts