You are browsing the archive for Evaluation.

Expectations taster

29/04/2013 in About this course, Course information, Evaluation

Last week we asked you to tell us what your expectations are for taking part in ocTEL. If you haven’t already, please do add your views via the expectations questionnaire!

We are really hoping to hear from more of you, but from the responses so far, here’s a taster of what the ocTEL community looks like, and what you hope to achieve.
Pie chart of location of respondents
As you can see from this pie chart, most respondents are based in the UK.

The majority also indicated that they work in Higher Eduction, but some cover other sectors including the NHS, private and third sectors. Over half are learning technologists or lecturers, but other roles such as manager and teacher are also represented.

We’re also interested in what experiences participants had prior to taking part in ocTEL. Responses so far suggest most have some experience of using technology for teaching, and of using social media (SM) for their own learning and for networking. Perhaps reassuringly for the course team (!), there are also participants who have joined ocTEL with a view to broadening their understanding in these areas. In the words of one respondent “I know I have big gaps in my knowledge regarding pedagogical theory and instructional design. I’d like to start ‘shading in’ these areas and find out what else I need to know for the future.”

Read the rest of this entry →

What are your expectations of ocTEL?

21/04/2013 in Course information, Evaluation

As noted in Friday’s overview, the team is keen to hear your feedback about ocTEL, and to act on it! We also have fairly detailed plans for a broader evaluation of ocTEL, which includes investigating impact on course participants. I have written in more detail on my blog about the thinking behind the evaluation, as well as the key evaluation questions we’re looking to address. These are organised in four themes:

  • Impact on staff competency (value in practice)
  • ocTEL Content and Design
  • ocTEL Discourse and Knowledge
  • ocTEL Community and Sustainability

There is also a sub-theme looking at the collaborative approach to open authoring and the impact of ocTEL on the course team.
Read the rest of this entry →

There’s nothing new in MOOCs

10/01/2013 in About this course, Evaluation

Towards the end of last year I started working with the ocTEL team on how to evaluate the Massive Open Online Course or MOOC they’ll be running in 2013. There is a *huge* amount of discussion on MOOCs at the moment from tips for those taking a MOOC to the future of MOOCs and HE. This article from the Chronicle provides a rather US centric timeline of developments and responses to MOOCs. There is also the recent launch of the slightly mysterious Futurelearn in the UK. There has been some very interesting discussion on this on the ALT mailing list. The list is members only, but quoting from a recent list posting by Diana Laurillard “Everyone in the field knows there’s nothing new in MOOCs” but we do need to meet the “massive demand for education, across all sectors from primary to lifelong, all over the world… It can’t be done without technology… It’s time to start looking seriously at what those models could be.

For me at the moment, the focus is on evaluating just one MOOC, which is still under construction. Nonetheless, when I started asking the ocTEL team what they hope the MOOC will provide, one of the themes that emerged quite strongly is the desire for a sustainable community to come out of the MOOC. This has led me back to work I undertook over ten years ago on the theory and practice of online learning communities. At that time, I was influenced by Etienne Wenger’s insights into Communities of Practice and Jenny Preece’s perspective of designing usability and supporting sociability in online communities. Figure 1 summarises the dimensions of practice Wenger attributes to ‘community’, and Figure 2 shows the relative similarities in Preece’s online community key features.

Dimensions of practice as identified by Etienne Wenger

Figure 1 – Dimensions of practice as the property of a community
(Wenger, 1998, p73).

Online Community Features as identified by Jenny Preece
Figure 2 – Key features of an online community, with associated characteristics
(Adapted from Preece, 2000)

These models will be reviewed and elements drawn into the ocTEL evaluation framework. I’ll also be looking at the concept of sustainability in relation to community. This has been addressed before by Bell et al in their 2007 book chapter entitled Evaluation: a link in the chain of sustainability. They highlight how the lack of persistence could come down to coordination failure or when the “costs of participation exceed the perceived benefits“.

So perhaps there is nothing new in MOOCs, or maybe it’s just a reminder to put old lessons into practice.

Bell, F et al (2007) Evaluation: a link in the chain of sustainability. In Lambropoulos, N & Zaphiris, P (Eds.) User-Centered Design of Online Learning Communities. Idea Group Inc.
Harris, RA & Niven, J (2002) Retrofitting theory to practice – a reflection on the development of an e-learning community. In Banks et al (Eds). Proceedings of Third International Conference on Networked Learning. Sheffield University.
Preece, J (2000) Online Communities: Designing Usability, Supporting Sociability. Chichester, John Wiley and Sons.
Wenger, E (1998) Communities of Practice. Learning, meaning and identity.Cambridge University Press, Cambridge.

** Adapted from a post on the Inspire Research blog. **

Skip to toolbar