Designing in features of evaluation gives you a chance to see how your TEL resources are being used, and also provides evidence that might help people to buy into what you’re doing (or tell you if you’ve got it totally wrong).

One of my objectives is to enhance our offering with some specific new applications (we won’t be redesigning entire programmes). These resources should help students understand concepts that they tend to find difficult, thereby improving student performance and retention rates.

For formative evaluation, we plan to run focus groups where we present prototypes to faculty and student groups. The Napier University guidlines on evaluating TEL give some good tips about how to structure these.

Here are some ideas about summative evaluation.

Quantitative

Students self-rating their understanding of a concept before and after using the new resources might be a better option than a simple 1–5 stars rating system. Materials that are pitched at a very high level might get 5 stars from the most able students, but would be completely useless for those who are struggling. So rating understanding before and after would allow us to track the effect the resources are having, particularly for the people who are need them most. This data could then be analysed in a number of ways: by study route / country / language etc, or assessed against individual or cohort exam performance.

Qualitative

Student questionnaires would allow us to get feedback from a large sample of students, but this type of survey is always limited to the questions that we want to ask. So maybe focus groups would be useful too, so that students can put across the points they want to raise. Areas to cover include the design, content, academic level and ease of use (including accessibility, particularly for anyone with support needs). The week 10 webinar gave good tips about timing these – ideally just after students sit their exams, so they have more time.

We should also interview faculty to see how easy the resources are to create, how much support they have to give and how helpful they think they are.

So that brings me to the end of ocTEL. Next week I’ll post about what’s gone well and what could have gone better; it’s been a great experience though. Well done the Association for Learning Technology.