MEDIA RELEASE: 11 November 2020: The Association of Market and Social Research Organisations (AMSRO) today announced that the Inquiry into the Performance of the Opinion Polls at the 2019 Australian Federal Election (AMSRO Polling Inquiry) has released its final report, including recommendations for how to improve political polling in Australia.
The report follows AMSRO’s decision, along with the Statistical Society of Australia, to instigate an independent and impartial review of election polling in Australia to determine why all the published polls incorrectly called the outcome at 2019’s May Federal Election and how polling might be improved in the future.
As evidenced by the Inquiry, from 2007 until 2016, Australian pollsters had, in aggregate, a 96% success rate (that is, 25 out of 26 final week polls called the right result) across four Federal elections. The 2019 May election, however, was a notable anomaly and AMSRO and the Statistical Society of Australia, on behalf of the wider industry, wanted to know what went wrong. AMSRO invited pollsters, media organisations and others who commission election and political polling to contribute to the inquiry.
The independent inquiry report concluded that the collective performance of the published national election polls in 2019 constituted a ‘polling failure’ rather than simply a ‘polling miss’, as the polls erred in their estimate of the vote in a manner that was statistically significant and erred in the same direction and at a similar level; and found that the source of errors lay in the polls themselves and was not a result of a last-minute shift in preferences among voters.
The AMSRO Polling Inquiry Chair, Darren Pennay (Founder and former CEO of The Social Research Centre) said: “The first preference votes were either underestimated (LNP) or overestimated (ALP) because of inadequately adjusted, unrepresentative samples. It is very likely the polls were skewed towards over-representing more politically engaged and better educated voters and this bias was not corrected. As a result, the polls over-represented Labor voters.
“As a consequence, pollsters should seek to better understand the biases in their samples and to develop more effective sample balancing and/or weighting strategies to improve representativeness, by looking at education or other variables.”
The Inquiry Panel also found some evidence that the reporting of the polls failed to consistently meet the basic disclosure guidelines for editors and journalists set out by the Australian Press Council.
The Inquiry report has made 10 key recommendations aimed at improving polling methods and their disclosure. Recommendations include calling for an enforceable Code of Conduct for Election Polling; consistent, enforceable disclosure standards; and recommendations to improve polling methods and better educate the media and public on polling.
The full report and recommendations can be found here: https://www.amsro.com.au/amsro-polling-inquiry/.
AMSRO Board member (and AMSRO President when the Inquiry was launched), Craig Young, said: “Election and political polling are consequential and influential and play an important role in the democratic process. AMSRO welcomes this final report from the Inquiry Panel and is exceedingly grateful to all those who have volunteered their time and expertise to support this important Inquiry.
“We also welcome the recent formation of the Australian Polling Council and recognise that their stated objectives are aligned with the recommendations of the Inquiry report regarding the need for greater transparency, and the creation of a Code of Conduct and polling guidelines, as well as better education of the media and public about polling. We look forward to supporting Council members as they develop their Code of Conduct. A code that delivers greater transparency and disclosure regarding how the published polls are conducted, is critical to re-establishing public confidence in polling. As the research industry association, AMSRO supports extending such disclosure standards to cover all published market research, not just political polling.
“We hope that this report ultimately provides the polling companies with useful information, ideas and further lines of inquiry to assist them in their efforts to improve the accuracy – and public perceptions of the accuracy – of election and political polling in Australia. AMSRO looks forward to working with the polling companies and other important stakeholders to support the creation of appropriate standards to bring about these much-needed changes.”
The final report is released today and further information on the AMSRO Polling Inquiry can be found at: https://www.amsro.com.au/amsro-polling-inquiry/.