White Paper: The Case for Decentralised, Personalised Assessments

QA has been working to develop a new take on assessments, moving away from the standardised testing approach and more towards a decentralised, personalised learning and assessing experience.

In this white paper, we will be exploring how a decentralised approach to assessment enables improved outcomes for both learners and employers, and how it is supported by academic research.

In the rear-view mirror: standardised assessments

The testing culture has become a significant part of today’s educational landscape. In the words of Sir Ken Robinson and Lou Aronica (2018), "high-stakes" testing was introduced as an attempt to raise educational standards. Often taking the form of multiple-choice questions, they significantly simplified the administrative burden. Enabled by automated optical scanners generating streams of data, their results became easily compiled into charts and league tables.

However, besides this positive set of ideals, Sir Ken Robinson argued that this approach is now, more than ever before, being challenged (Strauss, 2020) – is "high-stakes" testing an effective way of gauging learners’ achievement? The late British author and international advisor on education highlighted a major flaw with this approach:

"Assessments are currently very heavy on comparisons, and overly light on description. Learners are being labelled against a rank, rather than being provided with an insight on the qualities of their work. This subverts an assessment’s deepest purpose" (Robinson, 2016).

Other academics, such as Sawyer (2008), argued that standardised high-stakes assessment methods are particularly inadequate for the new generation of technological enhanced courses, which offer opportunities for personalisation. Two critical flaws are:

It assumes repeatable learning experiences: Standardised types of assessment assume that all learners will complete the same learning experience. This makes it a blunt tool to measure learning in courses with customised/personalised components.
It achieves superficial assessment of knowledge: Standardised types of assessment, almost by definition, focus on (and assess) decontextualised and compartmentalised knowledge. This means that they do not assess deep knowledge. If you think about it, it is optimistic to expect that, ahead of time, one could deliver a one-size-fits-all test that accurately predicts how well each learner will apply new knowledge, skills, and behaviours at work.

However, even if considered as "inadequate" by many, the use of high-stakes assessment methods is likely to continue to be reinforced for as long as educational institutions continue to be ranked and evaluated on how well their learners are performing in such tests (Sawyer, 2008).

In Robinson’s view, to break this cycle, the necessary change will not come from incremental improvements; instead, we need new thinking to revolutionise the way we approach assessment (Cortez, 2018).

The case for a decentralised approach to assessment

With this backdrop in mind, QA has been working to develop a new take on assessment. The focus has been:

Decentralisation, to enable personalised learning experiences: Learners express what success looks like through the definition of bespoke goals, to be delivered as part of everyday workplace projects. Employers validate their performance as an integral part of their normal workflow.
Meaningful application, to enable a descriptive in-depth assessment: Using each learner’s complex real contexts, our new approach to assessment emphasises scaffolded application, wrapped around real projects. Our focus is to lead learners to deliver tangible business impact earlier, in a structured way, supported by immediate and direct feedback.

In summary, assessment becomes an integral part of the whole learning experience, rather than a point in time. This is far from a complex process, and can be easily implemented with three simple steps:

Lift-off: Workplace application plan

Learners will create a plan to apply their newly found skills at their workplace. They will name which key course concepts to use, how these will help them to work toward their business goals, as well as professional development plans.
Mid-flight: Implementation

With essential QA guidance, learners start implementing their plan at work. QA provides some structure to their approach to workplace application, and a way for them to report their process and findings as they progress.
Landing: Reflection and validation

Finally, learners reflect on their experience, highlight lessons learned, and the impact of their work. They receive confirmation of their output by their line manager, as part of their day-to-day workflow.

Once this takes place, QA will validate the professional interaction, making it publicly available via digitally verifiable credentials, accompanied by descriptive metadata.

A working framework

Our final question was: when should this new type of decentralised approach to assessment be used? Given that our courses’ learning architecture is defined by the type of learning goals, it made sense to do the same with the assessment.

In a nutshell, literature (Mayer & Clark, 2011) categorises learning goals as follows:

To inform: the objective of this type of learning is to build awareness or provide information.
To perform procedures (procedural goals): the objective of this type of courses is to prepare learners to perform procedures in a defined way, across a defined set of situations. These courses will typically focus on near-transfer skills (i.e., the transfer from the learning environment to the workplace application is direct).
To perform tasks (strategic goals): the objective of this type of courses is to prepare learners to perform tasks with no predefined process or procedure, requiring them to use judgement in determining how to accomplish what’s required. These courses will typically focus on far-transfer skills (i.e., the transfer from the learning environment to the workplace application is indirect and complex).

QA courses typically focus on performance goals (i.e., goal types 2 and 3 highlighted above):

Courses with procedural goals typically offer standardised, more repeatable learning experiences. Learners will perform procedures which will be compared against a defined formula or process.
Courses with strategic goals frequently offer personalised learning experiences. Learners will have to dynamically adjust procedures or processes to complex workplace contexts.

It is unlikely that the decentralised assessment is a one-size-fits-all approach. For that reason, we have decided to incentivise the adoption of the new approach in courses whose goals are strategic, as these would benefit much more from a personalised learning experience, and a descriptive in-depth assessment. However, our decisions at this level will ultimately be data driven, product specific, and follow the wider business strategy.

A long-overdue reform

From an employer’s perspective, this means managers and learners will decide how to use our courses to improve their performance, efficiently getting just enough time off the job to master their role. For them, this means better and faster return-on-investment.

From a learner’s perspective, they will be inspired by their engagement in meaningful work, empowering them to apply their newly found skills. They will think about strategic outputs as they learn, efficiently getting just enough learning to master their role. As a result, they will also achieve verified digital credentials that confirm they not only know the theory, but have done it for real.

For QA, this means a new partnership with our learners and their managers. This is especially relevant in today’s agile world where, in many cases, the contexts of application may not yet exist at the time of course creation.

This open-ended design philosophy allows our courses to remain relevant for longer, as they enable possibilities which go beyond what would have been conceivable at any one time.

All the while, providing a compelling alternative to the standardised "high-stakes" assessment culture, and paving the way for further reform.

Find out more about QA Total Learning™

Read about QA's new Total Learning offering, focusing on real-world workplace outcomes.

Also read: Total Learning: Designing learning for outcomes, by Ben Sweetman.

References

Cortez, M. B., 2018. FETC 2018: Sir Ken Robinson Urges Educators to Drop the Standardized Approach. Vernon Hills, EdTech.
Mayer, R. E. & Clark, R. C., 2011. e-Learning and the Science of Instruction: Proven Guidelines for Consumers and Designers of Multimedia Learning. Third Edition. ed. San Francisco: Pfeiffer.
Robinson, S. K., 2016. Sir Ken Robinson and assessment [Interview] (16 May 2016).
Robinson, S. K. & Aronica, L., 2018. You, Your Child and School: Navigate Your Way to the Best Education. s.l.:Penguin Books Ltd.
Sawyer, K., 2008. Optimising Learning: Implications of Learning Sciences Research. s.l., CERI - Centre for Educational Research and Innovation.
Strauss, V., 2020. It looks like the beginning of the end of America’s obsession with student standardized tests, s.l.: The Washington Post.