‘I keep saying the sexy job in the next ten years will be statisticians. People think I’m joking, but who would’ve guessed that computer engineers would’ve been the sexy job of the 1990s?’ That was Hal Varian last year in the McKinsey Quarterly.
It’s been hard to miss this kind of talk lately. A few weeks ago The Economist chimed in with a ‘special report’ on ‘the data-centred economy’.
Unfortunately, one prominent growth sector seems to involve hucksters selling meaningless information to anyone dumb, cynical or rich enough to buy. This can be done in several ways, but a few are especially common:
- Claiming to detect an effect where the coefficient isn’t statistically significant: the observed result could have occurred by chance rather than genuine association (for example, evolutionary psychologist Satoshi Kanazawa purporting to show that ‘beautiful parents have more daughters’ while tall people and engineers have more sons).
- Failing to control for potential confounds, and thus finding spurious correlations. As previously discussed here, this one is beloved of blowhard journalists who, for example, tie crime figures to ethnicity (look how Sudanese offenders are over-represented! look how Indians are disproportionately victims!) without adjusting for relevant variables like age, gender, occupation and locality.
- Selection bias, where the sample drawn is unrepresentative of the target population, with some people more likely to be included than others. If you see market research in the popular media, it’s almost guaranteed to have this flaw. Respondents to online surveys usually self-select. They may choose to participate because they have some axe to grind, while people without strong opinions – or older people who don’t use the web – may not bother taking part.
It’s always been easy to sell equivocal results to a credulous media, which is eager to hype and sensationalise quantitative findings if they involve sex, gender or race. But, increasingly, public and private-sector managers seem willing to pay for noise presented as signal. Consider this week’s report in Melbourne newspaper The Age. Apparently ‘research done for the government’ shows residents of the inner-western suburbs are worried about a proposed freeway tunnel. The research referred to seems to be a local community survey hosted online at SurveyMonkey. Fill it out, if you like; it takes only 15 minutes of your time, and is almost completely worthless. And – hey! – your tax dollars, by way of Sweeney Research, have already paid for it.