1200 hrs. local
Fair warning: I’m about to get a little wonkish, so if you don’t care about market or academic research, skip on down to my last post.
In an otherwise superb review of Doh Chull Shin’s Confucianism and Democratization in East Asia, in Foreign Affairs, (“Confucius and the Ballot Box“, July/August 2012) Andrew Nathan digresses from his eloquent explanation of why “Asian Values” and democracy are not incompatible to defend Shin’s methodology.
Critics view the survey-based study of culture as flawed in three ways. First, if culture is something shared by all members of a society, treating it as a distribution of values and attitudes among individuals ignores the way it works as a shared experience. Second, by reducing culture to a series of questionnaire items, the survey method oversimplifies complex, multilayered attitudes. Third, the questionnaire format forces respondents to choose among rigid response categories that cannot adequately reflect their beliefs.
For all that, the survey method remains indispensable. No other approach does as good a job of finding out what large numbers of people actually believe. And it is less reductive than the older method of gesturing in the direction of an entire nation and claiming that all its members share some vaguely defined set of norms.
This all may seem like a shot in an esoteric battle of academic nit-picking, but it is telling that Nathan feels compelled to conduct this preemptive defense in a review written for a mainstream (rather than purely academic) journal. Dr. Nathan protests overmuch, and in so doing gives us a view to his fear that while they may be one of the few methods available to make the social sciences as scientific as possible, surveys have some innate flaws that call for some extra scrutiny on Shin’s thesis specifically, and the conduct of research in Asia in general.
My first concern is general: does the survey method travel well? The survey method was developed in the West by sociologists who were operating in a specific culture and political environment, one in which people feel relatively comfortable about voicing their honest opinions without worrying about political reprisals or pleasing the person doing the survey. This is important because if you stop to think about it, the value of any answer to a survey absolutely depends on those two conditions.
Yet common sense and a passing knowledge of Asian cultures call into question whether these conditions exist to the same degree in Asia as the West, and, equally important, whether they exist to the same degree among Asian cultures. Indians, for example, might be very ready to answer frankly and at length. Would Koreans be as willing? Would Singaporeans? Would Japanese? Would there not be differences between how Americans would answer and how any Asians would answer?
I would be less concerned if Shin had conducted his survey in the context of a single culture, because he could have designed an survey and delivery method that compensate for the variances between that country’s culture and the outspoken West. One firm I know in China, for example, has learned how to compensate for cultural factors by making the interviewer someone who is known and trusted by the interviewee, and focuses on interviewer training to work the variances out of the responses.
But Shin’s survey cuts across sixteen different cultures and nationalities, and this brings me to my second concern. By using the survey method across a range of cultures and polities where there are likely to be huge variances in the willingness to speak out and the cultural desire to please, is it even possible to wind up with comparable results?
I have no answers that I can prove scientifically, but the questions need to be addressed before we accept Shin’s findings or those of any survey in Asia – whether academic of for business research. If we are genuinely interested in defensible research, we need to question our implicit assumption that the tools created in one culture work just as well in any other, and then test our answers.
I would feel a lot better about Dr. Shin’s book if either Shin – or Dr. Nathan, whom I admire greatly – had hinted that they had even considered whether the survey method is culturally appropriate outside of America at all, and whether and how it can be meaningfully administered across cultures.
They did not, and I believe that they did not because they cannot. The minute they start questioning the precious few tools on which their peer-reviewed research rests, they lose the ability to conduct the kind of research their professions and employers expect them to conduct. When our best scholars are shackled to weak tools, what can we expect but debatable outcomes?
Without more substantive answers, I’m led to the conclusion that for the sake of marketers, academics, and policymakers, we need to re-engineer research. The realities of globalization demand new tools. It is time to create them.