Flawed Opinion Poll in NarcoNews

The website “Narco News” has made available an opinion poll made in Honduras in August regarding the political crisis. According to Narco News it shows that the majority of Hondurans want Zelaya to be reinstated, and they are against the military coup. Let us look at the report.

First, in the introduction the author explains that the purpose of the poll is “to know, in a trustworthy way, the political situation in the country after the coup d’état against president Manuel Zelaya Rosales on June 28th. As is known, Roberto Micheletti Bain is presiding over a de facto government, which, since the moment it took the power, has received forceful rejection from the majority of the Honduran population, which daily expresses itself by means of demonstrations by the National Resistance directed by prestigious popular leaders and honest politicians”.

Let us stop right there. Is this report biased or what? Only a complete fool can believe that anything that follows that statement can be unbiased, let alone trustworthy. That sentence is a strong declaration of support for one side in the poll.

From now on, we are no longer looking for signs of bias. We know it is there. The only thing left to look for is how they allowed their bias to influence the results.

They interviewed 1470 persons, one per household, in 54 neighbourhoods distributed over 16 out of the 18 “departamentos” in the country. The neighbourhoods were allegedly selected randomly, but the households within each neighbourhood was selected as every 4th or 5th. About 11% had university eduction, 12% had no education whatsoever, the ages ranged from 18 years to above 60, and about half were presently employed. These types of questions (including sex, religion) suggest that there is, at least to me, no obvious bias in the sample.

However, the devil is in the questions.

The first question is “¿Está en favor o está en contra del Golpe de Estado del 28 junio pasado al Presidente Manuel Zelaya Rosales?” (Are you for or against the coup d’état of June 28 against president Manuel Zelaya Rosales?) This is a very biased question, designed to get a specific result. It is like asking a jury “Do you think the defendant was right or wrong in killing his wife?” instead of asking “Do you think the defendant killed his wife?”

Furthermore, many are of the opinion that it was ex president Manuel Zelaya Rosales who planned to do a coup d’état on June 28th. Thus, a small minority might actually have misunderstood the question altogether, and replied that they were “against the coup”, when the correct answer as the question was posed would have been “for”. In Spanish the difference between the two questions is “pasado al” (against), and “pasado del” (of), so when read out they sound virtually the same.

Again, there is no credibility in the unbiasedness of the poll. The fact that only 17.4% responded “for” does not mean much when the question is so obviously biased, and many surely did not dare to be honest in the violent environment that existed, when they knew what the opinion of the pollster was. Most of the 29.9% who did not respond must therefore be suspected to be sympathetic to the democratic institutions who deposed of Zelaya.

The second question was if the Micheletti government should stay in power or leave power. Here 60.1% said that he should leave, and 22.2% said he should stay. Given that he has to leave by January 27th, it is not clear how the respondents would have interpreted the question. There are quite a few people who wish Micheletti could stay beyond January 27th. Was that how they understood the question?

On the third question it gets more substantive: Do you support or not support the return of Manuel Zelaya Rosales to the presidency?

51.6% support, 33.0% do not support it, and 15.4% did not answer. It is mostly the youngest and the oldest who are in favor of his return, and men more than women. But then something strange appears: People with the highest education level are most in favor of his return. This runs contrary to all my experience so far, and it suggests to me that there is something fishy about the data that goes beyond biased sampling, biased questions, and biased attitudes in asking the questions.

The sixth question is, “Do you agree with the repression or do you condemn the repression that the military and police have carried out against the National Resistance?” Again, a clearly biased question. The unbiased version would have been, “Do you agree with the way in which the security forces have strived to uphold the order in the face of street protests?”

In the seventh question the respondents were asked to point out which group was behind the coup: politicians, business interests, military, foreign interests, all of the above, none, or don’t know/no answer. Again, what is the respondent to reply who does not view it as a coup, but a constitutional succession? It’s like asking, “Have you stopped beating your wife?”

In question eight they ask for the favorability rating of Ramon Custodio, ombudsman for human rights, CONADEH. They find it to be 21.4% good or very good, and 39.3% bad or very bad. However, in question eleven, where they ask for favorability of a number of persons, they again include Ramon Custodio, who now, strangely, only has 12.4% good or very good rating.

In the questions on from which radio and TV channels the respondents get their news, the bias in the sample becomes clear. In both cases the pro-Zelaya channel got the highest percentage, even though they are not the largest channels in the country. The pollsters spin this to mean that people have changed viewing habits since they know that the pro-government channels are just lying.

On the question if the general and constitutional elections should be held or canceled, 66.4% said they should be held, and 23.8% that they should not.

Conclusion

It is evident from the questions that the respondents are not a random representation of the population. Those who are favorable to Zelaya are seriously over-represented. We see this from what radio and TV channels they watch.

How can that be, if they selected the respondents randomly?

The simple answer is, they didn’t. Remember that they selected the neighborhoods randomly, but not the houses and persons in the houses. They took about every 4th to 5th home, they wrote. They had 1500 surveys filled out and discarded 30 for being filled out erroneously. However, a figure that does not appear anywhere is how many persons they approached in total. It is impossible that they only approached those 1500, as any pollster can tell you. Most people refuse to participate, which is the main reason for the bias in the results in any poll.

In this case those who designed the poll had built the bias into it on purpose. Remember the first question? Many who got that as the first question would most likely slam the door – or something worse – on the pollster. By starting out with such biased language they assured themselves a seriously biased result in their favor.

And that is all there is to it. Lies, damned lies, and opinion polls.

9 thoughts on “Flawed Opinion Poll in NarcoNews”

  1. Mycket bra analys.

    Antropologiprofessorn, som enligt bade min egen analys samt mina kallor (jurister vid Cambridge) hade totalt fatt om bakfoten hur juridiska prejudikat fungerar och alltsa FEL angande kongressens ratt att tolka konstitutionen, skriver triumferande om opinionsundersokningen har:

    http://hondurascoup2009.blogspot.com/2009/10/why-holding-opinion-poll-in-honduras.html

    Ga in och skriv en kommentar med din analys. Ska bli mycket intressant och se om hon publicerar den – gor hon det inte ar det totalt fiasko, sarskilt med tanke pa hennes agerande i diskussionen om kongressens ratt att tolka konstitutionen.

    Gabriel

    1. I heard that, too.
      Maybe they should just reinstate him as president – and simultaneously arrest him to face justice, but without deposing him until he is convicted.

    2. Hon har nu recenserat din analys: http://hondurascoup2009.blogspot.com/2009/10/why-holding-opinion-poll-in-honduras.html

      Se sarskilt:

      “A ‘push poll’ forces certain kinds of answers through the wording of questions. The author of the critique of this poll takes issue with the first question, asking people’s opinion of the coup d’etat on June 28 that removed Zelaya from power. If I wanted to criticize that question, I would say that using the term “coup d’etat” (although in my view, and that of the world governments, accurate) could dissuade some people (who do not accept that characterization) from responding.

      But that is not the argument made in the critique. Instead, the author suggests that people in Honduras may have been confused and thought the reference was to a supposed coup d’etat planned by Zelaya for June 28; so he suggests the anti-coup respondents could include anti-Zelaya respondents.

      This is zany nonsense. By now, everyone in Honduras knows what actually took place June 28, and the question is about those events.”

      Att du klart och tydligt tar stallning, i din forsta poang samt nast sista paragrafen, mot att anvanda ordet “statskupp” forbigar henne totalt – att du skriver resten (“furthermore”) for att forstarka din poang verkar hon inte ta till sig.

      Man bor i allmanhet vara ytterst forsiktig med opinionsundersokningar. Men att pasta att fraga folk om de var for eller emot “statskuppen mot Zelaya”, ar inte direkt ett opartiskt satt att vetenskapligt forsoka fa klarhet i situationen.

      Det kanske ar ett hederligt misstag av professorn – sjalv tycker jag att det star ganska klart vad du anser om att fraga manniskor vad de tycker om en “statskupp”.

    3. She has a point in that for the same respondent, both objections I made cannot be true at the same time, as they contradict each other. The point she agrees with is the one I believe is the vastly more probable, though.

    4. Of course. But it seems pretty obvious that you are proposing several alternatives to why this poll might have been flawed.

      The first one, and the one I agree with, is that using the word ” is biased, and the second one is about people being confused about who carried out the coup (which I personally does not think is plausible – the professor is probably right here).

      I just wonder why she writes, “But that is not the argument made in the critique. Instead…” when it’s clear you take issue with using the word “coup”.

  2. Excellent analysis.

    I have some other questions:

    1) If this was an official poll, then why was it not officially issued by the polling company to the world as a .pdf or Word document instead of surfacing on the web as a scanned .pdf document, supplied by an un-named source to Giordano.

    2) Who sponsored the poll?

    3) Why did it take almost 6 weeks between the completion of the poll and its surface on the web? Entering 1500 forms into a PC, interpolating the results and producing a report can’t take that long, surely. Could it have anything to do with the fact that the OAS delegation was due to arrive a couple of days after the poll’s “appearance”?

    4) Why is there no date or signature on the report? And why attach copies of TSE authentication to the report? Perhaps the authors thought that nobody would believe them unless they provided some sort of proof of “legality”?

    5) Lack of conformity in the method of graph display? It looks like the report was compiled by a high-school student trying to show off his ability to use every graph display available in the word-processor package. Not very professional at all!

    5) Is there any proof that this was actually compiled or issued by Coimer&OP? Heading a document with their logo doesn’t prove anything.

    1. This is an interesting blog because it shows how a university-educated (OK, college at least) person can support a traitor like Zelaya. There are prima facie evidence for his high treason, everybody in Honduras knows about it. The difference is that those who support Zelaya consider that the president should be a King, entirely Above the Law, whereas those who oppose him consider – like we do in Scandinavia and USA – that Land is Built with Law, or as they say in the Young World, “this is a land of law and not of men”.

      It is people who argue like that blog’s author who make democratic-leaders-turned-dictators like Adolf Hitler possible. Look up tyrant in Wikipedia and you will see that the classical tyrant came to power carried by a mob. What Chávez and Zelaya are doing is the same thing: portraying mob rule as democracy.

      The person who does not obey the law can not live in society. In ancient Scandinavia his punishment was to be declared an outlaw, meaning anybody could do whatever with him, including murder him, with impunity. The law was seen as a contract, and those who broke the contract were no longer protected by it.

      It is that strong sense of law that is the basic foundation for democracy. The blog’s author apparently lacks that instinct. (I assume everyone know what I am talking about, but just in case, the background is that Zelaya violated direct court orders, and openly disobeyed the other branches of government, using force and mob rule.)

Comments are closed.