Assessment using e-learning - Kwik-Fit Training Academy

Overview

“When we really looked into why 20% of our short-course learners weren’t passing we realised that it wasn’t that they were failing to cope with the subject, they were just failing to cope with the tests. This took us on a journey that started with solving a dyslexia problem, but ended with a Who Wants to be a Millionaire type approach to assessment. Staff took to our new voting-button system straight away. We all had to invest a bit of time at the start, but now we find it really easy to use.

"Refuse to allow people to fail. Do whatever it takes to support them to succeed."

The impact has been tremendous. Pretty much everyone passes now and the trainers like it to the point where we’d probably struggle without it – it’s become a tool very central to what we do.”

Advice for others? “Refuse to allow people to fail. Do whatever it takes to support them to succeed.”

Steve Blount – technical trainer

The good practice in detail

Have you watched any of the quality improvement programmes on the EGTV channel?
Click the link below to watch Steve and his students bring this good practice study to life:

e-Assessment – Kwik-Fit Training Academy

‘Refuse to allow people to fail. Do whatever it takes to support them to succeed.’ This phrase captures Steve’s moral commitment to his learners, and is the driving force for quality improvement.

Why not download the Q-box Action Plan for this example and make notes while you read?

The Kwik-Fit training environment at the Derby academy is dominated by one-day and one-week courses. And with 1400 Kwik-Fit mechanics passing through the centre every quarter, a 20% failure rate was leaving the company under qualified and Steve and his team dissatisfied.

“I was aware that we’d had some dyslexic students who’d failed because they couldn’t read the test papers rather than because they didn’t know the subject,” recalls Steve. “So we had a day’s training from a consultant and began to make changes. One of the most significant of which was to use a sans serif font on a pastel background. And the easiest way to do this was to deliver the test using PowerPoint rather than paper – though learners still had to write their answers down.”

Achievement rates leapt to 95%. But the journey away from passing tests to being accredited for a level of understanding had only just begun – and Steve didn’t want any of his learners to fail.

How could you sum up your ambition for your learners? If this was your advertising slogan would learners sign up for your provision? Is your ambition sufficiently challenging? If the challenge level was to increase, what implications would this have for your delivery?

"You press one button, press another button, and it’s done."

He was very aware that his staff were giving fairly standard off-the-shelf lessons irrespective of the learners’ abilities on entry. With hindsight, insufficient emphasis on initial and formative assessment meant that the trainers taught to the middle ground. In their review, Steve and his team considered the written assessments and verbal Q&A sessions used at the start of the course to be flawed – whilst being sufficient for confident learners, they were inappropriate for those who were not.

Management recognised the bottom-line impact of Steve’s initiatives and so set about searching for technology to help him reach the next stage – completely paperless assessment. They eventually invested in a computer-based voting button system. Staff immediately engaged with the technology and whilst it did take a little time to get use to the interface, the program’s built-in PowerPoint function gave some immediate familiarity.

With this simple approach, unconfident learners were no longer exposed by the assessment process, but could contribute risk-free at the press of a button – anonymous to all but the trainer. Initial and formative assessment then became fun.

Daniel Mundy, learner: “I hate writing answers down. With this, you press one button, press another button, and it’s done. It’s a lot easier.”

“We started by collecting together all of the questions we’d each used in our training,” says Steve, “and then set about writing more. That done, it was quite easy to build all of the quizzes we needed. We just had to:

• make a standard template (staying with the dyslexia-proof colours)
• type the questions and answers into the PowerPoint slides, and then
• set the parameters for each question, e.g. multi-choice, true/false, etc.

It takes just a matter of minutes, once you’re use to the interface.”

Initial assessment

"I couldn't train without it now."

“We use the test at the start of all our courses, which are all either one-day or one-week block release. It’s a great ice-breaker as it gets all of the students involved straight away. But it’s more than that – in the past, if you asked an open question of the group, only the confident ones would answer. And the ones with no confidence could get locked into under-performing from then on. Worse still, we sometimes found ourselves asking the confident ones not to answer – but then we wouldn’t know whether they were getting it right or not, and it put an awkward pressure on the shy ones. The new system gets round all of these issues. I couldn’t train without it now. It’s so much easier and more effective. The system provides a level playing field – everyone gets to answer every question and, initially at least, it’s anonymous. This means they’ve got a secure environment in which to build their confidence and learn.”

How have you considered the impact of the way you carry out initial assessment on your learners? Are you disadvantaging any of your learners?

Formative assessment

The trainers make notes on what learners get wrong in the first tests and adjust their training plans accordingly. This now means that courses are never delivered the same twice – they are always customised to meet the unique needs of each group. “The beauty is that I absolutely know whether they’re getting it or not,” says Steve, “so the chance of them getting to the final test and failing have all but gone.”

Learners enjoy using the Who Wants to be a Millionaire style voting buttons. Craig Hughes, student: “It doesn’t let you fall behind because it gives your instructor a chance to know that not everyone knows the answer.” Once all learners have selected what they feel is the most appropriate response, the results are shown as a bar chart on the main classroom data screen. If the answers given are not unanimous, the trainer uses the disagreement as an opportunity to revisit the learning point and reinforce the right answer through debate or illustration.

“On the longer programmes,” says Steve, “we can look across all of the quizzes each learner has taken to see how their understanding is developing. This helps us pick up on any individual support needs we might have missed and also acts as a check on our approach.”

How confident are you that all of your learners understand all of what they need to, all of the time? What additional strategies could you use to measure the on-going impact of your training?

Summative assessment

An added benefit of the system is the improved spirit in which the final test is taken. As the learners have used the voting button approach throughout the whole of their programme, “When they get to the final test that accredits them,” says Steve, “they’re no longer fazed by it.”

Systems

Roger Smith, Technical Training Manager: “There are lots of different systems on the market and they all operate in slightly different ways. We picked the one that seemed best for our situation. It’s wasn’t expensive, which was great, so I would urge anybody who’s in a similar situation to us to have a play with these systems and see what they think. There’s also some very good free, open source technology such as Moodle which, in a networked computer suit, can produce the same sort of results.”

How might you explore the available e-assessment solutions?

Impact

When one of Kwik-Fit’s long-serving trainers was introduced to the voting buttons, with it’s very practical emphasis on initial, summative and formative assessment, he said to Steve afterwards that he’d suddenly realised how to become a trainer. But it wasn’t the technology that had inspired him, it was the sudden realisation that you have to check a learner’s understanding throughout the whole of their programme. His absolute confidence in knowing, at all times, what his learners had and hadn’t understood ensured that his sessions were never delivered the same twice and always met individuals’ needs. Little wonder that Kwik-Fit’s motor vehicle provision was given a grade 1, outstanding, in its last inspection report. The key strengths inspectors highlighted were:

• excellent training resources
• strong company commitment to training
• particularly well planned and managed training programme.

Uploaded - June 2009

What could you do next to improve your provision?

Answering all of the italicised questions in the Q boxes above will help you begin to health-check your current practice. Download all of the linked documents, compare them with your own or adapt them for your own use. Write a short action plan to get you from where you are now (what is good and what needs improving) to where you want to be.

The Building Better Practice (BBP) web resource is a great place to start if you want to benchmark yourself against other providers. It will show you the most common inspection strengths and weaknesses for each issue or topic, an analysis of the good practice found on inspection and a series of health-check questions to help you establish how you compare to others. Look specifically at how you could use BBP to improve your assessment practice and the internal verification in NVQ programmes.

Actions for Quality Improvement (AQI) is a set of activities with resources around which you can run staff development sessions with your teams. The activities cover all aspects of the learner’s journey and will help your staff embed quality improvement in the heart of your provision. Look specifically at how you could use AQI to improve your assessment practice.

If you need more help, ideas and resources for the process of self assessment then look at the Learner-Centred Self Assessment (LCSA) materials. This is a web-based or hard-copy resource to help you generate a rich source of evidence for your self-assessment report through professional discussion rather than the completion of lengthy forms.

9 Data Projects to Improve Your Provision is a set of projects which help you use data to explore all aspects of the learner’s journey for improvement themes.

The Self-Assessment Surgery Projects have proved very popular at the Preparing for Inspection events. They will help you determine whether or not your SAR is fit for purpose.

Interpreting the Common Inspection Framework (CIF) is essential guidance on how to interpret the CIF for your remit and is now contained in the appendices of the Ofsted inspection handbook for work-related and adult and community learning.

The Inspection Toolkit contains step-by-step guidance on how to prepare for inspection and covers such topics as choosing the right person to be nominee, using data and self assessment.