|
Development Roast https://inesad.edu.bo/developmentroast/2013/01/fighting-poverty-effectively-experiments-are-not-just-for-scientists/ |
Fighting Poverty Effectively – Experiments Are Not Just For Scientists Here's a question for you: how do we know that all the aid given to Africa has had any effect? The tricky thing about trying to answer this question, and analyzing problems such as global poverty, is that the world cannot be treated as a laboratory experiment. We cannot create two identical Africas and give aid to one and not the other, keeping all the other variables such as political conditions, climate, and population, constant. What happens in fields such as medicine, when we want to know whether a drug works or not? Suppose we've developed a new pill which, we think, prevents headaches. We find two chronic headache-sufferers who are identical in terms of gender, age, and general health, and X takes the pill every day but not Y. After a month it turns out that X no longer suffers headaches but Y does. What can we conclude? Well, nothing at all. Because we have no way of knowing what would have happened if X hadn't taken the medicine – maybe his headaches would have disappeared anyway because he finally started wearing his glasses. Medicine circumvents this problem by using 'Randomized Controlled Trials' (RCTs). These are experiments where, in their simplest form, a large number of participants are divided randomly into two groups. The people in one group are told to take the drug (the 'test group') but not the other group (the 'control group'). The important points are that we have a large number of participants that are divided at random into two groups. In this way, although no two individuals are exactly identical, the average characteristics of the two groups are very nearly identical. This enables us to isolate the effect of the drug so that we can be sure that any difference between the two groups is caused only by the drug. We compare the overall behavior of the two groups, not of any two individuals. The larger the group sizes, the more 'identical' they will be and hence the more reliable the study. Social policy is not a science, but that is no reason why scientific methods such as RCTs cannot to be used to test and develop them. By doing so, we can use objective, quantitative data to determine the best course of action, rather than just relying on our assumptions. This is the work undertaken by Ester Duflo, the co-founder and director of the Abdul Latif Jameel Poverty Action Lab 1 (J-PAL) at the Massachusetts Institute of Technology 2, and her colleagues. The principle in this context is identical to that in drug-testing: randomly divide your participants into two groups then trial a strategy on one group, such as providing a small incentive for parents to bring their children for vaccinations 3, and compare how that group compares to the control group. Trials such as these sometimes yield surprising results, proving that relying only on our intuition is not always the best course of action. For example, child education is unarguably a necessity for development, yet school absenteeism is a big problem in many countries. You might guess that the most effective way to improve attendance would be by providing financial incentives – reducing the cost of school necessities such as uniforms and stationery, or providing scholarships. But actually, what turned out to be overwhelmingly more effective in Kenya was providing free deworming treatment at schools 4, which improved the health, concentration, and school attendance of the children. Contrary to common belief, it wasn't the cost or distance from school which had the biggest impact on attendance, but intestinal worms. Like all experiments, RCTs are not perfect 5. They are especially difficult to conduct in a social context such as development, because we cannot analyze people's thoughts in the same way that we can manipulate the chemical composition in a test tube. However 'identical' we try to make the test group and the control group, there will always be some bias: the people who volunteer to take part in the trial may have a different outlook to those who do not volunteer, the results might depend upon the time that the trial was undertaken, and so on. Added to which, there is sometimes a practical limit as to how randomly you can divide your participants. In the above example about testing the effect of offering small incentives to encourage child vaccinations 6, the test group was one entire village, and the control group another separate village. The two groups may appear to be statistically identical, but what if there was one particularly persuasive mother in the test group who persuaded all her neighbors about the benefits of vaccination, so that they would have taken their children to be vaccinated anyway, regardless of whether there was any additional incentive or not? It is impossible to control factors like these, so the results will not always be correct. Plus, RCTs can be expensive and time-consuming to perform and evaluate 7. However, in view of the huge sums of money spent on international aid – around US$50 billion per year in the case of Africa 8, as well as the fact that the welfare of millions of people is at stake, the cost of performing these trials surely cannot be that high? It seems bizarre that the process of giving and distributing aid currently comes down mainly to guesswork rather than quantitatively proven strategies. You wouldn't risk your health and money by buying a medicine that hadn't been tested thoroughly in a scientific manner, so why do we risk billions of dollars and other people's health by carrying out aid programs that are not properly tested? It was Ester Duflo who asked, in her TED talk 9, whether the aid given to Africa had done any good. The answer is that we don't know and we never will because we have no way of knowing what would have happened if we hadn't given the aid – things might have turned out worse, but they might have turned out better. Randomized Controlled Trials may not be perfect and the conclusions not one hundred percent accurate, but they seem to be one of the most effective tools that we have available at the moment. Without doubt they will give us some information about what works and what doesn't, and an indication as to the best course of action to take. Which will be an improvement over merely keeping our fingers crossed and hoping for the best. Do you know of any examples of surprising results from Randomized Controlled Trials? Please leave your reply below. Tracey Li is a Research and Communications Intern with INESAD. Like this article? Sign up at the top of this page for email updates. For your reference: Duflo, E February 2010, TED talk, Social experiments to fight poverty. <http://www.ted.com/talks/esther_duflo_social_experiments_to_fight_poverty.html 10> Green D May 7th 2010, From Poverty to Power, Randomized Controlled Trials: panacea or mirage? <http://www.oxfamblogs.org/fp2p/?p=2478 11> Moyo, D March 21st 2009, The Wall Street Journal, Why Foreign Aid Is Hurting Africa. <http://online.wsj.com/article/SB123758895999200083.html 12> J-PAL Policy Briefcase November 2011, Incentives for Immunization. <http://www.povertyactionlab.org/publication/incentives-immunization 13> J-PAL Policy Bulletin March 2011, Deworming: a Best Buy for Development. <http://www.povertyactionlab.org/publication/deworming-best-buy-development 14> |
Links:
|
