Also Read: Science Behind Opinion Polls
Noted mathematician, statistician and psephologist, Prof. Rajeeva Karandikar, delivered a lecture on March 6th, 2014, at The Hindu Centre for Politics and Public Policy on ‘The Science Behind Opinion Polls’.
Mr. N. Ram, Chairman of Kasturi and Sons Ltd., when introducing Prof. Karandikar, said that the lecture was an educative exercise, aimed at bringing clarity on the issue of how opinion polls were conducted, in an environment where the credibility of opinion polls was being contested. In the current political climate, the Election Commission seemed to be allergic to both opinion and exit polls, and there was a suspicion that polls lacked integrity and that pollsters are open to extra scientific and extra professional considerations, he said.
The Hindu Centre’s lecture, an attempt to discuss all these issues and serve as a knowledge sharing exercise, was attended by A. Sabarullah Khan, Deputy High Commissioner, Sri Lanka, Prof. C.S. Seshadri, Director-Emeritus of the Chennai Mathematical Institute, journalists, academicians, students and members of the general public.
Prof. Karandikar started his discussion by talking about the scientific basis of opinion polls, their power as well as the limitations. The primary aspect of an opinion poll is the sample survey, and was used to get a truly representative sample in a country with a voting population as large as India’s, roughly 700 million people. He illustrated an experiment to the audience using probability theory and said that opinion polls use a basic probability calculation to estimate the likelihood of voters’ preferences based on which they can predict who will win in an election.
Illustrating his point, he said that if one were to do a survey in one constituency in Chennai, for example, and if the contest was very close between two parties, no survey could credibly predict who would win. All that could be done at that time would be to report how many were voting for whom. Opinion polls try to estimate who will get a majority on the list of sampled voters that we pick, he said. For example, if the electoral roll was taken for one constituency in Chennai, and all possible lists of 4,001 voters were taken and the majority opinion on the chosen list would very likely reflect the dominant opinion of the electorate in that constituency. There would be no guarantee that the exercise would give you a winning result but the probability would be high, he said.
Sample surveys and statistics were tools to conduct opinion polls, and policy decisions were meant to be taken on the basis of statistical information, though how much of this actually happens in the government’s decision making was a matter of debate, Prof. Karandikar added.
Pointing out the limitations posed by resources to conduct large sample surveys in a country’s of India’s size, he said, surveys cost about Rs. 500 per head and therefore increasing the sample size would end up being prohibitive in terms of costs. There was also the problem of trained and reliable manpower to conduct the surveys. He said that the U.S., India and the U.K, faced different sets of challenges for conducting rigorous opinion polls.
In the U.S,, for instance, pollsters would estimate who will be President in a winner-take-all system, which functions at the level of the federal State. In India the attempts were to estimate who will win in the constituency in a slightly different winner-take-all system.
While in India, less that 10% of seats could be considered as ‘safe’ to make predictions, in a country like the UK, this ‘safe’ figure rose to 60% - 70% of the seats as socio-economic factors drove voting intentions across generations. On the contrary, in India, leaders hop parties and parties hop alliances, leading to instability in voter preferences, he pointed out. Referring to previous surveys done, he said that voters change their preferences across dates, and staggered polling dates can also affect voter preferences.
Low predictive power
While there was no ban on conducting opinion polls, there was a ban on making the results public during the poll dates. Referring to nationwide polling conducted by CNN-IBN and the Centre for the Study of Developing Societies (CSDS), Prof. Karandikar said that in the previous surveys, more than 50% of the voters surveyed in the first round, conducted in January 2014, had changed their preferences when surveyed in the second round held in February 2014. This was an indication that despite the polarisation of the electorate as reported in the media, people have still not made up their minds on how to vote, he said.
The predictive power of any opinion poll, irrespective of the methodology, was rather low given the time gap between polling dates, he said, as ultimately what counted was the opinion of those who actually vote at elections, which was only 60% - 70% of the population actually voting. The better-educated and richer people are less likely to vote, he said. The other limitation of a survey done well ahead of actual polling day is that though the survey measures the opinion of the whole population, it does not factor in those who do not vote. Moreover, Prof. Karandikar pointed out that the propensity to vote is much lower among the urban, upper middle class and upper class, college educated, high income groups.
The other factor is that in India, voting intentions can undergo massive swings as voting day approaches. These two factors mean that the predictive power of any opinion poll done weeks ahead of the poll is limited. All it can measure is the mood of the nation at the time the poll was conducted.
He said that support for the two main contenders in the General Elections, the Bharatiya Janata Party and the Congress, was tied to socio-economic status. The higher the socio-economic status of the individual, the more support for the BJP and lower the socio-economic status, the higher the support for the Congress.
“When we think of predictions we are not worried about how people of a certain caste are going to vote, but are more interested in how the people in a constituency vote,” Prof. Karandikar said. In India, while there was good data at the administrative level, these could not be used at the constituency level, because of the differences between the demarcation of administrative units and constituencies. Even if one were to use similar socio-economic parameters we would need to look at historical data, how the constituency has voted in the past, the leadership and so on, which make a difference to the voter while making a decision about voting.
Referring to the mathematical model that he developed, Prof. Karandikar spoke about how to convert vote share into seat projections. Suppose in a constituency ‘A,’ a party, say, the Dravida Munnetra Kazhagam (DMK), has 51% vote share and the All India Anna Dravida Munnetra Kazhagam (AIADMK) has 49%. In Constituency ‘B’, the DMK has 55%, and the AIADMK has 45%. In the first case we would say that the vote it was too close to call and in the second case we would convert the estimated vote share into estimated probability. The vote to seat conversion would take into account how close or how far the estimated votes were from each other.
Lauding the role of the CSDS and the Lokniti network in designing and conducting properly randomized surveys, Prof. Karandikar said that most other agencies that conducted opinion polls based on methodologies used in market research whereby the enumerators are given a profile on a few attributes such as gender, education, religion, caste, income and are asked to somehow identify voters on the basis of this pre-assigned profile. The hope is that if the sample matches the population profile on these important attributes, it will match on attributes of interest as well. However, there is no statistical guarantee that this will be so, since there is no randomization.
Prof. Karandikar said that when reporting the findings it was important to stick to the data and not get swayed by expert opinion. He shared the example of the recent Uttar Pradesh Assembly Elections, where the data showed the Samajwadi Party to be leading with a comfortable majority over the rest of the parties. However, experts and commentators insisted that this was wrong. The results finally showed that the Samajwadi Party had won.
Prof. Karandikar also discussed exit polls and post polls, referring to the fact that many media channels conduct exit polls, where sampling is done among voters soon after exiting the polling booth. However, during an exit poll, it is difficult to implement a strict sampling scheme, and the choice of respondents would have to be left to the enumerator since it would not be feasible to identify respondents out of a pre-selected list. Instead, a post poll survey, conducted two or three days following the actual casting of the vote in a given constituency, is a better indicator of the voting patterns. While the post-poll includes people who have not voted, the enumerators record if the ink mark is visible and then to arrive at seat prediction, take into account only the opinion of those with the ink mark visible. Post polls, rather than exit polls, were a better tool to estimate the number for major parties and alliances, he said.
Mr. P. Ramajayam, Assistant professor at the Centre for Study of Social Exclusion and Inclusive Policy, Bharathidasan University, Tiruchirapalli, and State Coordinator, SCDS-Lok Niti in Tamil Nadu, elaborated on the methodology of how the Lokniti surveys are conducted, with properly trained field investigators and enumerators, who maintain confidentiality of the respondents, are transparent about how the data will be used.
A lively discussion that followed the lecture with questions raised several questions, including the effect of undecided voters causing a swing vote, the feedback effect of opinion polls, projection of vote share into seats, and voter behaviour.
Prof. Karandikar was of the opinion that the feedback effect of opinion polls had reduced, given the multitude of opinions that are available freely through the media and expert commentary and said that he welcomed self-regulation by the media houses, with the Press Council of India drawing up well-thought out disclosure norms.
Please Email The Hindu Centre