Today's measurement from Ekot/Indicator (11/9) gives the Liberals 5.4 percent. A week ago, the same company gave the party 2.2 percent (4/9).
The TV4/Novus survey has moved the same way: The Liberals got 5.8 percent on Wednesday (9/9), while they got 1.9 percent on August 19. The differences determine whether the party remains in the Riksdag - and thus which government Sweden gets.
I mean that the survey companies make it too difficult for voters, journalists and researchers to understand what such a difference means, and that we need to be critical even of well-conducted surveys.
Some of the spread may be a real voter movement, some may be about method. None of us can decide which, and that's the problem. Indikator's survey was in the field August 18–September 4, Novus' most recent September 6–8, and public opinion may have moved in between. They measure different weeks and are presented as the same thing. Indicator drew a new random sample from the population register: 5,618 respondents, 2,228 responses, a response rate of 40 percent. Novus drew its sample from a standing web panel, recruited randomly earlier. There, 2,292 people responded, but it is not clear how many were asked – now or when the panel was built. Moreover, a change within one and the same company only says that it is unlikely given the company's own model, not that the panel follows the population.
The point is not that any company made a mistake, but that the accounting is neither open nor uniform enough. The information is sometimes found in the companies' method descriptions, almost never in the news reporting. The reader cannot therefore judge what the oscillations mean.
In the media's reporting, on social media and by the parties themselves, the surveys are nevertheless regarded as reflections of reality. Of course that's not true. In the 2022 election, 84.2 percent of the almost 7.8 million eligible voters voted. In an opinion poll, a sample of a few thousand people is asked. Only a portion of them respond, and far from all companies report how large that portion is. The problem is that the people who answer are not the same people as the people who vote. Older, highly educated and politically interested people answer more often than others. And even when a group is adjusted up by weighting, the estimate rests on the few of the group who actually responded.
The experts who analyze the numbers are usually serious and well-read. They take into account margins of error and differences between companies, and refer to multiple measurements. But the best surveys are still only attempts to reflect reality - they are not reality.
This is where greater openness would have done the most good. The common man cannot be expected to familiarize himself with selection methods; it is up to the media commentators to explain what the numbers mean, and they will only be as precise as the basis allows.
A company can end up right by pure luck, or because two mistakes cancel each other out
The 2022 election showed how difficult it is. The survey that came closest to the election results was Demoskops, which is based on a panel where the participants themselves signed up, instead of being selected randomly. It is the method that has the weakest basis for saying something about the population. Still, it hit the best of 40 final measurements in 20 years. A company can end up right by pure luck, or because two mistakes cancel each other out. The fact that a measurement was right once says almost nothing about the method. You only know that after many elections.
Historically, however, there are systematic patterns. The Left Party was overestimated in all the last polls in 2018 and 2022, while the Social Democrats and Moderates were underestimated. The polls don't get any better the closer we get to election day.
When Swedish newspapers explain why a party has gone up or down in their own poll, the change in about half of the cases is so small that it might just as well be due to chance. When chance is provided with causes, a story emerges about an opinion movement that may never have taken place.
When Swedish newspapers explain why a party has gone up or down in their own poll, in about half of the cases the change is so small that it might just as well be due to chance
All this attention can influence the choice. If you hear that it's done for party X, you might ignore voting for them. If you hear that X is close to 4 percent, you might vote in support. There is research to support this reasoning: a panel study of nearly 1,900 voters ahead of the 2018 election shows that those who perceive a party to have gained in the polls rate the party more highly, and in several cases are more likely to vote for it. Furthermore, the support vote has increased from 6 percent in 1968 to 19 percent in 2018, and a third decide in the last week.
And the votes are already being cast. When early voting opened on August 26, Novus gave the Liberals 1.9 percent. On Tuesday, over 2 million voters had voted, 32 percent more than at the same time in 2022. The same company today gives the party 5.8. You can of course vote again, but that only helps those who follow the measurements and have the courage to go one more time.
This year, the Liberals' 4 percent can decide which side gets power. Whose responsibility is it? The survey companies have the information on how the measurements were made, the editors decide which place the numbers get. None of them have to answer for it.
Does this mean we should stop the opinion polls? No, I don't think so. Whether their net effect on democracy is positive or negative, we know too little about, but as long as they are published on this scale, they must stand up to scrutiny.
Two things need to happen.
1. Survey companies need to use the most appropriate methods and be more open: report sample, field period, number of respondents, adjustments and response rate - every time, next to the number. Such requirements already exist in other countries. British companies must publish full tables within two working days, French submit a methodology report to an independent commission. Sweden has no requirements. The industry should introduce them themselves, before someone else does it for them.
British companies must publish full tables within two working days, French submit a methodology report to an independent commission. Sweden has no requirements
2. The editors must stop publishing measurements where the data is missing and stop inflating small changes in the measurements. It is equally important to have a critical attitude even towards well-conducted investigations. Even an openly reported survey is based on those who had the time, desire and willingness to answer when they were asked the question. Openness makes uncertainty easier to assess. It doesn't remove it.
The difference between 1.9 and 5.8 percent can determine who governs Sweden for the next four years. The reader should be told why the numbers differ - and that neither necessarily reflects reality.
Read more articles from DN Debatt:
Simona Mohamsson (L): "The Tidö government is the most liberal in modern times"
