- Check-in Badge
- This week’s aims
- If you only do one thing…
- TEL One Badge
- Come and join the webinar
- Webinar Badge
- Activities for this week
- Resources and more to watch, read and research
- Be a TEL Explorer
- TEL Explorer Badge
- Notes and commentary
We’ve covered technology in ocTEL; we’ve covered learning. This week we’re focusing on enhancement – what shapes it takes, and how to achieve it – and evaluation – how we can decide that our interventions have been successful.
As you will all be at different stages in terms of introducing/working with TEL, we offer different routes through the week depending on which core activity you undertake.
Click on this badge link to 'check-in' and see who else is also active in this topic.
This week’s aims
This week will help you achieve the aims you prioritise from the following:
- identify and analyse the essential elements of successful educational enhancement initiatives based on TEL
- analyse the cost effectiveness of a particular learning experience and use TEL to improve this effectiveness
- appreciate the importance of project and course evaluation
- understand methodologies for carrying out evaluation
- critically review evaluation instruments or tools
- reflect on how your experience of the ocTEL course has compared with expectations, and how you can best build on this experience.
If you only do one thing…
Choose between focusing on enhancement or evaluation.
Each week we award a TEL One badge for completing 'If you only do one thing…'. Click on this badge title for full instructions
How to enhance the effectiveness and efficiency of particular TEL approaches
- this 13 minute video about the Saylor Foundation and how they have developed 241 degree-level courses, available free of cost, free of accreditation and largely free of professors,
- or this 9 minute video about one MOOC model (this is usually referred to as the xMOOC model, in contrast to the cMOOC model, which ocTEL is closer to).
Write down in short bullet point form a list of
- elements of the Saylor or xMOOC approach that you think could be applicable to your context (what you’re involved in teaching, to whom, with what goals and constraints)
- problems you might anticipate with the approach
Think of different situations where efficiencies might be important in
- reaching more people or providing a richer experience for the same cost
- reducing tutor costs by encouraging more elements of the learning experience to be peer-based or self-organised by learners
- reducing production and infrastructure costs by using free resources and technology
- taking the learners’ perspective in getting a sound, rounded, education with minimum financial outlay.
Describe those situations and identify any potential tensions or conflict in these situations between the goals of effectiveness and efficiency. Share your thoughts on your blog or via Twitter (using #ocTEL tag), on this forum topic, or via the JiscMail list.
The Whys, Hows and Whats of Evaluation
Think about a TEL initiative you’ve been involved with.
- How did you know it had been successful or provided benefit?
- Was the initiative evaluated? If not, why not?
- If yes, what evaluation took place, what did you and your stakeholders need to know and how were the findings used?
Reflect on the different ways you have come across for evaluating the effectiveness and impact of TEL, using the headings below. What are the advantages and disadvantages of each approach?
- What do you need to know? e.g. how learners rated the support and how this differed according to their circumstances or demographics
- What evaluation methods could help you find out? e.g. online survey
- What advantages do these methods have? e.g. easy to collect information from large of people
- What disadvantages do these methods have? e.g. often require incentives to increase response rates, disaffected learners may be less likely to respond
Compare the methods you have identified with the methods and approaches used by the contributors to this week’s webinar
Share your thoughts on your blog or via Twitter (using #ocTEL tag), on this forum topic, or via the JiscMail list.
Come and join the webinar
Enhancement and evaluation of TEL
Monday June 16 at 12.30 – 1.30 (BST).
This webinar will include three guest speakers who will talk about the enhancement and evaluation of a major example of TEL which they have been involved in. Our three examples will include one at module level, one innovation based on introducing new devices, and one based on an overall institutional strategy.
Our three speakers:
Mark Kerrigan is the Director of Teaching, Learning and Assessment in the Faculty of Health Social Care and Education for Anglia Ruskin University. He will be talking about the enhancement and evaluation of the iPads in Science initiative, a project by the University of Greenwich (where he used to work). More information and a case study on the project can be found here:
James McDowell is University Teaching Fellow in the School of Computing and Engineering at the University of Huddersfield. He will be talking about the development of work which won him the 2011 ALT-Epigeum Award for the Most Effective Use of Video. This award recognized the important contribution he has made to the enhancement of student learning through his use of video feedback – http://eprints.hud.ac.uk/19915/ .
Keith Smyth is a Senior Teaching Fellow and Senior Lecturer in Higher Education at Edinburgh Napier University, where he leads the MSc Blended and Online Education and works on TEL-related strategic initiatives. Keith will be talking about the development, dissemination and evaluation of the 3E Framework for technology-enhanced learning, teaching and assessment. Since being introduced as the basis for Edinburgh Napier’s Benchmark for the Use of Technology in Modules, the 3E Framework has been adapted and implemented by over 25 institutions within and beyond the UK. http://3eeducation.org/3e-framework/
Watch the webinar using Blackboard Collaborate: http://go.alt.ac.uk/octel2014-week6-recording
Watch the recording of Blackboard Collaborate live stream on YouTube: https://www.youtube.com/watch?v=7c2QzMVq3b8
Tune in or watch the weekly recording of each week's webinar to achieve the this badge. Click this badge link to view Instructions for getting this badge.
Activities for this week
Activity 6.1: Analysing the cost-effectiveness of new learning technologies
- Read why productivity and sustainability in HE is an issue, as suggested by The Economist’s take on “Baumol’s cost disease”.
- Check out a few of the resources listed below (e.g. on evaluation and costing).
- Do some searching to see if you can find further resources that link this general area to your specific concerns and context (use Google, also ask people you think might know of useful materials, collaborate with other ocTEL participants). Try and answer the question, “What can I take away from this area and apply in my teaching-related work?”
- On your blog or via Twitter (using #ocTEL tag), on this forum topic, or via the JiscMail list, note your initial impressions about which strategies look promising or unpromising to you.
Activity 6.2: What do you think about ocTEL?
We invited you at the start of this course to think about big questions and we hope that you have kept in mind what your question(s) were. As we approach the end of the course, we’d like to encourage you to revisit your ‘big TEL question’ and see how your view of it has been changed and reframed by the course (and indeed whether it is still the same!). Share your thoughts on your blog or via Twitter (using #ocTEL tag), on the original forum post if you made one or this forum topic, or via the JiscMail list.
We also recommend that you take a bit of time to look through the weeks and bookmark or otherwise keep a record of the best bits for you. What is most relevant for your practice? Which resources could you use or re-use?
The materials for the course including the recordings of live sessions and the forums will remain on the ocTEL website so that you can refer back to them later, if you have missed something or not had time to complete any of the activities.
- Post your reflections and contributions using the #ocTEL tag via your blog, Twitter, the online forums or other channels
- Join the webinar/watch the recording
- Check out the TEL Explorer activities
- Tweet about your experience of #ocTEL and find other participants on Twitter
Resources and more to watch, read and research
- Graham Attwell (ed.) (2006) Evaluating E-learning: A Guide to the Evaluation of E-learning. Evaluate Europe Handbook Series Volume 2. http://www.pontydysgu.org/wp-content/uploads/2007/11/eva_europe_vol2_prefinal.pdf
- Learning Technology Dissemination Initiative: Evaluation Cookbookhttp://www.icbl.hw.ac.uk/ltdi/cookbook/recipes.html
- Oliver, M (2000a) Evaluating online teaching and learning, Information Services and Use, vol 20 (2/3) http://eprints.ioe.ac.uk/3082/1/Oliver2001Evaluating83.pdf
- Oliver, M (2000b) An introduction to the Evaluation of Learning Technology. Educational Technology & Society 3(4). http://www.ifets.info/journals/3_4/intro.html
- Warwick Learning and Development Centre – Guide to Evaluating E-learning Developments http://www2.warwick.ac.uk/services/ldc/resource/eguides/elearning/
- What the Curriculum Design projects tell us about evaluationhttp://jiscdesignstudio.pbworks.com/w/page/61872351/What%20the%20Curriculum%20Design%20projects%20tell%20us%20about%20evaluation
Enhancement through increasing reach and/or quality for same cost
- Carol Twigg’s tools and approaches available onhttp://thencat.org
- Diana Laurillard’s 2013-01-31 ALT webinarhttp://repository.alt.ac.uk/2244/ (watch in particular from about 24 mins in to end).
- David Noble’s (1998) contrarian viewhttp://firstmonday.org/htbin/cgiwrap/bin/ojs/index.php/fm/article/view/569/490
Enhancement through reducing tutor cost
- Seven compelling arguments for peer learning, by Donald Clarkhttp://donaldclarkplanb.blogspot.co.uk/2011/10/7-compelling-arguments-for-peer.html
- Sugata Mitra ALT-C keynote talk (54 minute video)http://www.youtube.com/watch?v=Ps8MwyJH8Zo
- Mitra et al‘s (2010) Self-Organised Learning Environment School Support Packhttp://repository.alt.ac.uk/2208/
Enhancement through reducing production and infrastructure costs
- 2.5 min animated video about Open Educational Resourceshttp://www.youtube.com/watch?v=-xGRztrWv-k [may annoy for it’s US-centric superficiality, may be instructive despite or because of this]
- Chapter 1 of Curtis Bonk’s The World is Openhttp://media.wiley.com/product_data/excerpt/06/04704613/0470461306.pdf or in kindle format by clicking “Send sample now” athttp://www.amazon.co.uk/dp/B002JMV6KK/
Being a low-cost learner
- Nicholas Negroponte on learning by themselves (15 minute video)http://www.youtube.com/watch?v=CNRaM2GgQuA
- Edupunk’s Guidehttp://edupunksguide.org/
- David Jennings’ 2011 ALT Newsletter article on “Agile learning: How ‘making do’ can evolve into ‘making good’”http://newsletter.alt.ac.uk/2011/05/agile-learning-how-%E2%80%98making-do%E2%80%99-can-evolve-into-%E2%80%98making-good%E2%80%99/
Be a TEL Explorer
This is additional content and an activity to enable you to explore topics that are really relevant to your own practice. Explorer activities are part of the course, so we’d like to encourage you to share your ideas on your blog or via Twitter (using #ocTEL tag), on this forum topic .
Activity 6.3: Exploring enhancement and evaluation in practice
- Find an example of innovation in TEL which interests you.
You can use other ocTEL participants as a resource here as they may know of innovative practice in whatever curriculum area you wish to explore. Other sources include websites from JISC, HEA etc. and awards for innovative practice such as NTFS and those from ALT, the Guardian and THES.
- Contact the innovator and arrange a short conversation to explore (at least) the following issues:
– what was the impetus and rationale behind the innovation?
– what has the innovation achieved so far in terms of student learning?
– what has its impact been on the staff involved and the wider institution?
– how do you know it has had this impact? What evaluation strategies and methods have been used?
- Write a brief commentary on the outcomes of this conversation and publish it via the blog or other methods you have been using on this course so far. We recommend you check your summary with the innovator before publishing to make sure that you have interpreted everything correctly.
Each week we award the TEL Explorer badge for completing activities outlined in the course material. Click on this badge title for full instructions
Notes and commentary
The challenge of evaluation
Our webinar will look at examples of enhancement in TEL in higher education and compare them in terms of their objectives, the relative success of the intervention, and how it was evaluated. These examples and their discussion will mean different things to you depending on where you’re starting from and what ambitions you have for your educational work. Here are some of the challenges that you might like to tackle this week.
- How can we most efficiently help more people access learning experiences?
- How can we take advantage of TEL and create a richer learning experience within resource constraints (e.g. by redesigning the curriculum and/or cutting overheads)?
During this week you will identify commonly used methods of evaluation and review existing evaluation tools. We hope that you will be able to find time to finish the course by discussing your own experiences of ocTEL. This is not only about you and the course, but also about how we can make things better next time round. As you know, the course was designed, written and largely supported by Members of ALT (see who’s who on our ocTEL team page) and your feedback can make a big difference to how we develop and run the course in future.
Evaluating what you have implemented is important in understanding how effective your course or project has been. Oliver (2000a) identified five approaches for carrying out an evaluation of online learning:
- Formative – carried out by members of the course team for the benefit of the course team as a way of identifying revisions and improvements.
- Summative – carried out by an external auditor intending to provide judgement about whether a project has met its aims or a course has met its learning outcomes.
- Illuminative – an approach which seeks to understand the perceptions of the participants in terms of what is important to them through phases of observation, inquiry and explanation.
- Integrative – a combination of the summative and illuminative approaches.
- Evaluation for Quality Assurance
Once you’ve decided the type of evaluation, it’s important to consider the following aspects of the evaluation cycle, from defining aims, through data collection and analysis, to drawing conclusions, and reflecting back on the aims. For more information about the evaluation cycle see Embedding Learning Technologies – Evaluation Cycle (pdf)
The efficiency debate
The idea of getting maximum learning for minimum cost is met with wary scepticism in many quarters. It smacks of putting managerial values ahead of teaching, or of cutting costs to the point where the learning process becomes brittle and no longer has the agility and resilience to deal with the unpredictable demands of teaching (see David Noble’s paper, linked above, for an example of this critique).
Issues of deskilling teaching work or filleting the soul of education institutions are not trivial, and you can focus on them this week if you wish. But this is also an opportunity to ask about the different ways that learning and teaching are enhanced in Technology-Enhanced Learning. Do they result in learning outcomes that are richer in themselves, accessible to a wider range of people in a wider range of contexts, easier to achieve, or cheaper to accomplish?
Some enhancements are incremental, some are transformational. Sugata Mitra’s work on using technology as part of “Self Organised Learning Environments” in contexts where there is a lack of good quality teachers hints at radically new models for supporting learning.
Realising the enhancements that may be possible depends on an understanding of learning context. We appreciate straight away that it’s cheaper and safer for pilots, dentists and surgeons to learn their skills using simulations, but other contexts of learning present different challenges. This week you have the opportunity to select a context of your choice and explore what kinds of enhancements might be possible to make the learning more productive for more people. You can make your own value judgements.