Think about a TEL initiative you’ve been involved with.  How did you know it had been successful or provided benefit?
When the outcomes match the aims, or when significant progress has been made in a related, though not necessarily specified, area.
Was the initiative evaluated? If not, why not?
Yes
 If yes, what evaluation took place, what did you and your stakeholders need to know and how were the findings used?
The findings were used to ascertain whether additional monies would be found for further implementation of the project.  To establish this, we needed to know how both teachers and learners were using the technology, as well as how they felt about such use.
Reflect on the different ways you have come across for evaluating the effectiveness and impact of TEL, using the headings below. What are the advantages and disadvantages of each approach?  What do you need to know?
Where this is agreement, and where there is divergence, between the views held by both teachers and learners.
 What evaluation methods could help you find out?
Interviews, focus groups, statistical analysis, online survey
 What advantages do these methods have?
A triangulated viewpoint from a range of perspectives, providing information on a range of different aspects.
What disadvantages do these methods have? E
Technology dependent; time consuming
 
Activity 6.3: Exploring enhancement and evaluation in practice
Find an example of innovation in TEL which interests you.
The use of electronic badges.
Contact the innovator and arrange a short conversation to explore (at least) the following issues: – what was the impetus and rationale behind the innovation?
Gamification
what has the innovation achieved so far in terms of student learning?
For me, as a course attendee, the badges have acted as a driver.  They’ve been useful in looking at what is required, then ensuring I’ve done what is necessary.
what has its impact been on the staff involved and the wider institution?
Implementation of badges in my own college.
how do you know it has had this impact? What evaluation strategies and methods have been used?
Discussions, meetings, agreement of ways forward.
Through discussion and screen sharing with Martin Hawksey I gained some ‘inside view’ of the practical implementation of electronic badges and the ways in which they are, or are not, compatible with different platforms.  As a result of this discussion, I elected to implement badges through Moodle’s own system, rather than use an external one.  Most helpful, Martyn, thank you!

@mhawksey ah right, thanks. I’ll take a look at that then. #octel— Fern Faux (@FernFaux) June 18, 2014

#octel using badges in an institutional setting – must all recipients have their own backpack account? Or can it be automated for them?— Fern Faux (@FernFaux) June 18, 2014

 
Who were your stakeholders?
The Project in question aimed to provide a web-based resource sharing tool.  This was primarily aimed at teaching staff but the focus of the project was on the creation of the resource.
What resources were used?
Self-created resources as part of the project.
How clear/achievable was the project plan?
Very clear and very achievable in terms of resource creation, less clear regarding how to get teaching staff to actually use the end product.  We had assumed they would be pleased to have such a resource and had not taken into account an unwillingness to share ‘their’ resources.
What methods did you use to evaluate your project?
A varied selection but, crucially, none which included end user uptake!
How did you measure project success?
Creation of the project resource and measurement of its use.
Did you celebrate your success and did this encourage further developments?
Yes we did, both internally and externally, but whilst this resulted in some minor, and temporary, external use, there was no uptake internally from teaching staff.
Reflection
From the outset, the project should have focussed on the end users, as much as the resource creation.  Had this been built into the project design it may be that greater success would have been achieved in user buy-in to the resulting resources.
 
Explorer Activity 5.7: Student involvement in TEL (how students in your institution engage with and influence TEL developments)
All TEL developments are initiated through the lens of what benefits they will bring to students.  It is student need which drives such developments.  However, with students who have a range of learning difficulties and cognitive impairments, it is often difficult to ensure their true understanding of project information, and project ideas rarely come from the students themselves.  That said, students are included in all subsequent stages of project work.  This includes user testing, product development, and the provision of both formative and summative feedback.  Inclusion into project work holds value for the students in many ways, not least their perception of its value and status.

@mhawksey it tells me I’m about to send an #octel badge, then asks me to accept it, then it fails http://t.co/MrApc6GnAD— Fern Faux (@FernFaux) June 12, 2014

oops … #octel badges – what am I missing? http://t.co/cztPlFnXXt— Fern Faux (@FernFaux) June 12, 2014

Any course which asks me to watch a 4-minute Dilbert parody video on why projects fail, has to be a great course! #octel— Fern Faux (@FernFaux) June 12, 2014

Just testing #octel. Check out my OpenBadges at http://t.co/nk3oABPv6c via @OpenBadges— Fern Faux (@FernFaux) June 12, 2014

@romieh and not just MOOCs, I’m sure! #octel— Fern Faux (@FernFaux) June 12, 2014

Notice reduced attendee numbers each successive week in #octel webinars (which remain excellent!). Compensated by recorded session viewers?— Fern Faux (@FernFaux) June 12, 2014

#octel webinar: Leaders’ hormones change when in leadership to exhibit less sympathy and empathy. How interesting!— Fern Faux (@FernFaux) June 12, 2014

Looking forward to hopefully being able to (simply!) export #octel badges to wordpress in the future— Fern Faux (@FernFaux) June 05, 2014

 

http://dilbert.com/strips/comic/2010-10-03/
 
Why did/would you choose a particular type of e-assessment? Describe why you think it is effective and how it can help deepen knowledge and understanding.
InStep is an Open Source video database assessment tool. Whilst InStep can be used for a range of learners, in diverse settings and contexts, it was originally conceived as a tool to provide reliable assessment measures for non-accredited learning for learners with learning difficulties and/or disabilities (LLDD), with an emphasis on learners being supported to recognise and record their own progress and achievement (RARPA). Many of these small steps of progress go un-measured, yet it is essential that learners are able to evidence progress which they have made. InStep combines an assessment continuum with a video database, where pedagogical aspects of implementing such a tool enhance learner development.
In your experience, what type of approach creates an environment conducive to self-directed learning, peer support and collaborative learning? How might technology help?
Traditional assessment methods are rarely appropriate for learners with LLDD, nor do they recognise that these learners make progress in many ways – particularly in increasing their confidence, in posture, social and life skills. Given the important role of appropriate and relevant assessment in special needs education – with stroke patients, those in rehabilitative recovery, and within speech and language therapy, for example – it is essential to develop additional forms of assessment which capture the achievements and attainments of such learners with authenticity, fairness and validity. Various e-portfolios offer a video database, however these are not combined with formalised assessment measures and, for learners with LLDD, they are largely inaccessible, being text-based. Whilst the use of video technologies in assessment is gaining momentum, and there is some clear evidence of the benefits and potential in using these devices as a tool for learning and teaching, further discussion around the development and implementation of video assessment is required.
What opportunities and challenges does this approach present to tutors?
Overall, the assessment continuum is seen as a valuable educational tool, though staff confidence in making assessment judgments based on video footage varies. Whilst the focus of developing InStep had been on the use of video evidence and the development of an assessment continuum, technical aspects played a more dominant role than had been anticipated. Initially, some staff had difficulty in capturing the required steps of progress within the limitation of 50 seconds per video clip and others had difficulty in taking quality, reliable video footage and performing basic editing and trimming procedures independently. However, all staff commented that as the project proceeded they learned how to limit their filming to that which was immediately relevant and pertinent. This is important because the ability to make an assessment judgment against a video clip is largely dependent on the quality of the video clip itself.
(Edited extract from: Fern Faux, David Finch, Lisa Featherstone in Computers Helping People with Special Needs (2012). InStep: A Video Database Assessment Tool)
 

silly question time! #octel badges have points associated with them – for what purpose? (Do points make prizes? Sorry!)— Fern Faux (@FernFaux) June 03, 2014