7 June 2019: The Association of Market and Social Research Organisations (AMSRO) today announced the draft Terms of Reference for its Inquiry into the Performance of the Opinion Polls at the 2019 Australian Federal Election (AMSRO Polling Inquiry). Members of an Advisory Board and the Inquiry Panel were also announced.
“Significant uncertainty and disquiet has emerged about the performance of the major opinion polls in the lead up to the 2019 Federal Election. An impartial and comprehensive industry-wide inquiry process is required to examine the performance of the polls and restore public confidence in the credibility of polling. While each of the polling companies is likely undertaking internal analysis of their own polling practices and performance, we anticipate they will agree to share all relevant data and practices with the Inquiry Panel, once they see the robust process established,” said Craig Young, AMSRO President.
To support the Inquiry Panel, AMSRO has established an independent Advisory Board with diverse expertise in relevant areas, for the Inquiry Panel to draw on for broad strategic guidance and input. The Advisory Board will also review and comment on the Draft Terms of Reference and the Inquiry Panel’s Report and will provide guidance and additional suggestions for lines of inquiry.
The Advisory Board comprises:
• Dennis Trewin AO – Dr Trewin was Australian Statistician (Head of the Australian Bureau of Statistics) from 2000 to 2007 and is a survey statistician of international renown.
• Ian McAllister – Distinguished Professor of Political Science, Australian National University (ANU). Professor McAllister is recognised as an international expert on opinion polling.
• Travyn Rhall – After training as a statistician, Mr Rhall spent a decade in statistical consulting at the Australian Bureau of Statistics, before embarking on a career in applied (commercial) market research. He was a Director of Newspoll for several years. His last full-time executive role was
Global Chief Executive of Kantar Insights, which operated in 70+ countries with revenues of over A$3.5 billion.
• Kerry O’Brien – Journalist and author. Foundation editor and host of the ABC’s national 7.30 Report, foundation host of Lateline, host of Four Corners and anchor of the ABC’s federal and state election night coverage for more than 20 years.
The Inquiry Panel will be headed by Darren Pennay, the founder and immediate past-CEO of the Social Research Centre (SRC), one of Australia’s leading social research organisations, and part of the Australian National University.
“Mr Pennay has agreed to Chair the Polling Inquiry. He is not a pollster but has a lifetime of experience in all of the methods employed in public opinion research and will be able to draw upon the wider resources of the Social Research Centre and the Australian National University in conducting the review,” Young said.
Pennay said: “The focus of the review will be to objectively assess the quality of the polling data by looking at the techniques and processes employed across the different polling organisations. The approach needs to be broad and comprehensive, without any a priori assumptions about what, if anything went wrong. We need to adopt a purely evidence-base approach. We also need to examine the role of the pollsters and the media in communicating polling findings to the public.
“It’s important to note that the Inquiry will not be focused on assessing the relative performance of individual polling companies, but rather on practices and processes in use across the industry. We will aim to mask or de-identify the polling companies’ data, and Panel members will sign nondisclosure agreements. The Inquiry aims to deliver an evidence-based approach to find ways to improve the conditions for polling and the processes and methods used, so all companies can operate with a full appreciation of contemporary best practice.”
Other members of the Inquiry Panel include:
• Professor Murray Goot – Emeritus Professor of Politics at Macquarie University, a leading expert in public opinion polling, voter behaviour and politics.
• Dr Phil Hughes – Asia-Pacific Head of Statistical Consulting at Engine and one of Australia’s most experienced and respected applied statisticians.
• Mr John Stirton – independent and widely respected polling expert, who ran the Fairfax Poll between 1997 and 2014.
• Dr Paul Lavrakas – internationally renowned expert in survey methodology and survey error, specifically in relation to public opinion research and election polling.
• Dr Jill Sheppard – lecturer in the School of Politics and International Relations, ANU, with special research expertise in political behaviour (voting, participation, and attitudes).
• Dr Dina Neiger – Chief Statistician, the Social Research Centre and Accredited Statistician, Statistical Society of Australia.
• Patrick Moynihan – Associate Director of international research methods at Pew Research Center and previously senior polling analyst for ABC News.
• Professor Patrick Sturgis – The London School of Economics, who headed a polling inquiry after the 2015 UK General Election.
• Associate Professor Kristen Olson – Associate Professor and Vice Chair of the Department of Sociology at the University of Nebraska-Lincoln. She is a Fellow at the American Association for Public Opinion Research (AAPOR) and worked on the 2016 US Polling Inquiry.
“I am grateful to all those who have volunteered their time to assist with the Inquiry. I’m especially grateful for the international assistance offered by Professor Sturgis, Associate Professor Olson and Patrick Moynihan in agreeing to contribute advice and expertise from their own international experiences with polling analytics and similar reviews overseas. They will have an important role in helping to advise and review our work. I expect that other technical specialists and experts may be accessed as the Inquiry gets underway,” said Pennay.
AMSRO’s Young said: “It is vital that the Inquiry Panel operates in an independent and objective way. The Inquiry Panel, not AMSRO, will be responsible for what ends up in the Inquiry’s report and recommendations and great care has been taken to appoint a high calibre team of Australia and international experts to assist with the Polling Inquiry.
“We hope that all the polling companies will work with the Inquiry to achieve better outcomes for the entire polling industry. No currently active polling organisations are, or will be, represented on the Inquiry Panel –polling can be quite competitive, so it’s essential that there is complete objectivity in the approach and no scope for competitive concerns to arise. We want everyone to have confidence in the process and outcomes.”
The Inquiry Panel will operate at arm’s length from the market research industry body AMSRO. AMSRO’s role will be to establish the Inquiry’s Terms of Reference, support and resource the Panel’s work, respond to media enquiries and publish, promote and help implement the Inquiry findings, in consultation with relevant stakeholders – especially the polling companies.
“It’s pleasing that since announcing the Inquiry, some polling companies have already proactively come forward to volunteer to share their detailed processes, methods and data, so that the Inquiry can examine and assess the evidence and develop sound recommendations to improve, where necessary, polling practices. Once the Inquiry Panel gets underway, one of its first tasks will be to engage with the polling organisations directly and canvass their assistance. Their participation will be key, and they have the most to gain from a successful Inquiry,” Young said.
Full draft Terms of Reference will be available on the AMSRO website but include:
• assessing the accuracy of the polls at the 2019 Federal Election (both national and sub-national);
• evaluating whether there is any previous pattern of inaccuracies and the reasons and causes for any inaccuracies;
• to consult and seek evidence and insights from polling companies and other relevant stakeholders, including those who commission and publish polls;
• to assess the adequacy of information provided to, and by, the media and other commentators about how polls were conducted and what the results mean;
• to submit a preliminary report to AMSRO as soon as practicable, with a view to publication of a final report by AMSRO as soon as possible thereafter (timing to be determined by the Inquiry Panel).
For more information please contact:
Sarah Campbell, Executive Director, AMSRO, (02) 8017 6717, firstname.lastname@example.org