Laying the foundation for refined decision-making
Ten years ago, few people dared to talk about evidence in social politics. The qualitative approach dominated common practice among social workers, teachers and pedagogues. 'People cannot be treated as objects and measured as digits and data. Quantitative methods simplifies a very complex reality', the predominant mantra sounded.
In recent years, however, the increasing role of evidence-based policy-making and performance has changed many practitioners' attitude. Evidence justifies political decisions. Evidence measures performance and results in organisations. Evidence documents the effect of any given social effort. In other words, evidence provides the ever so important knowledge base to develop the best solutions.
Nicolaj Ejler, senior director in Ramboll Management Consulting, is an expert in developing the Danish public sector, widely regarded as one of the most highly-developed and modern welfare states there are. With 20 years of experience in evaluations and analyses of welfare initiatives within education, labour market and social management in mind, he remains convinced that the evidence-based approach has become a determining factor for decision-makers at several levels.
- As I see it, evidence plays three very important roles: Ethically, a welfare state is obliged to help the citizens in the best way possible. We need to know what works for those we want to help and support. Economically, we can't prioritise everything so we have to find the solutions that work the best and are the most efficient to ensure value for money. And professionally, practitioners are proud of creating the best effects, they're in their jobs to help others in the best way possible and they don’t want to waste their time working on a programme or with a method that doesn't work. The better your knowledge is, the better your decisions will be, he argues.
Stepping up the ladder of evidence
The definition of evidence is broader than most. Picture yourself a ladder. On the lower steps, you will find qualitative methods such as expert opinions and case studies. As you climb, the method design become less loose and more detailed, and you edge closer to the documented effect of your work. On the highest steps, you will stumble upon quantitative approaches such as randomised controlled trials (RCTs) that compare treatment groups with control groups not receiving treatment to test the effectiveness of a certain action.
- The best knowledge lies in the high end of the ladder. But if that knowledge isn't there, you have to step down and find the best available. The low end represents knowledge based on assumptions, experiences and feelings while the high-end methods document an effect and represent a value that policy-makers can use, says Nicolaj Ejler.
Critics confront the Golden Standard
Hundreds and hundreds of RCTs are conducted in social science around the world. In popular terms, RCTs are referred to as the 'Golden Standard' in evidence-based policy-making and research. However, not everyone joins the cheerleading squad to applaud experimental designs and RCTs as the best road to validity.
In the book 'The Battle of Evidence', professor in Evaluation Hanne Kathrine Krogstrup raises the opposing voice:
"Critics of randomised controlled trials stress that human acts are not activities in the same way as the physical, but instead influenced by meaning, constructed by humans in time and space, in interaction with others and the surroundings. Social knowledge that derives from interventions is not rational and unambiguous but interpreted, contextual, dynamic, temporary, inclusive value and ideology so complicated that it is impossible to procure precise causal explanations through experimental design."
Public welfare is so complicated that certain knowledge between intervention and effect does not exist. Nonetheless, the problem is not as serious, because probability knowledge is regarded as fully adequate to adjust practice, Krogstrup argues.
Nicolaj Ejler reiterates that you have to be well aware of the complexity that surrounds the evidence-based approach in social policy.
"Social workers and other practitioners often oppose to these methods, because their minds aren't tuned in on quantitative measures. I agree that you lose some complexity along the road. You shouldn't use RCTs if you don't know the complexity of the context as you will simplify reality. But studies have repeatedly proved that some methods work while some don't. If you methodically prepare an intervention and collect enough observations, you will get better results because the law of large numbers will cancel out any variation," he learns.
Systematic wordplay excites kids
For 30 minutes, twice a week, five kids and their day care teacher come together. The teacher reads aloud, using special techniques that engage the children and is targeted their individual language level. The kids learn how books work. They play with words. They rime. They get to know the difference between left and right, between capital and small letters.
13 day care centres in Guldborgsund Municipality have participated in the SPELL project in the first half of 2013. As a former nursery class teacher and speech therapist, educational consultant Eva Helver coordinates the language programme for the 3-5 year olds. First hand, she has experienced how the kids react to an evidence-based approach.
"I've seen how the children cue up in front of the door, because they're so excited. Their enthusiasm very much depends on the pedagogues' attentiveness and presence, but the structured and focused approach is a new way of thinking pedagogically. It brings the pedagogues and children closer together, it's very intense and it creates a recognisable routine that makes the way to learning shorter. The evidence-based approach contains an integrated flexibility that takes the individual kid into consideration, and this method is crucial if we want to improve our professionalism and quality," she claims.
To systematically measure the impacts of the project, the participating nurseries are involved in different ways. Some only get access to books and learning material, others receive training in differentiated language development while a third group of day care centres also engage parents. The control group continues unchanged.
As a part of the evaluation, consultants from Ramboll Management Consulting collect large amounts of data. Through surveys, video observations and interviews about the organisation of the nursery, children and employee background, attitude and so on, they identify the most determining factors for a positive lingual development. Altogether, new insights will contribute to international research in terms of how universal learning cultures can thrive in day cares.
Read more about SPELL – a research project carried out by University of Southern Denmark, Danish Technological Institute and Ramboll Management Consulting.
In recent years, Ramboll has conducted 14 RCTs in partnerships with acknowledged and leading university professors in Denmark.