Transparency, methodological gold standards and survey weighting: Those were three of many topics covered during two webinars The Journalist’s Resource recently hosted featuring Roper iPoll, from The Roper Center for Public Opinion Research at Cornell University.
Roper iPoll is a comprehensive opinion data research platform that offers access to nearly a million survey and poll questions from 1935 to today. Eligible small media organizations and independent journalists can apply for a one-year membership to Roper iPoll.
Kathleen Weldon, director of data operations at the Roper Center, discussed how journalists can use Roper iPoll to access both up-to-date and historical opinion polls.
With the 2026 U.S. primaries underway, keep reading for three insights that will help you accurately cover opinion polls. These takeaways are from both webinars, held on Feb. 26 and March 4. And check out our past work on covering surveys and polls and understanding question order bias.
1. Good polls will be transparent about their methodologies.
Methodological transparency is the foundation of reliable polling data.
“You can’t say that any particular poll is great just because it’s transparent,” Weldon said. “There can be bad polls that are transparent. However, it is very uncommon for good polls to not be transparent.”
The code of ethics of the Association for Public Opinion Research, a major professional organization for public opinion survey professionals, requires that member organizations commit to transparency in how they design, conduct, analyze and report their surveys and findings.
“What we push over and over again is transparency, transparency, transparency,” Weldon said. “It’s the only way that you can ensure that the people who are doing the research are acting in good faith — that they’re willing to share their information, and to allow people to interrogate the data.”
2. There’s no single gold standard for survey methodology.
Phone surveys with live interviewers and random digit dialing has long been considered the gold standard of public opinion polling. But response rates have fallen over recent decades — people don’t answer their phones like they used to — and new polling methods have emerged.
“There’s really no certainty that there is one method that is appropriate in all situations — that can be said to be a perfect gold standard,” Weldon said. “There is a method that does represent most of what currently comes into our archive. And that is online probability polling.”
In probability-based sampling, pollsters randomly select participants. This helps reduce bias in results. If everyone in a population being sampled has an equal chance of being selected, there should be an equal chance that all potential answers to questions will be represented.
3. Take a close look at survey weighting.
Non-probability online samples are another common methodology that journalists may encounter when reporting on polling data.
They may include “opt-in” surveys, where people choose to participate. And because these online samples are not random, they may introduce bias.
Some pollsters use sophisticated weighting methods to try to overcome potential bias and ensure their results more closely represent the population being studied. For example, if the target population is 50% men and 50% women, but 40% of respondents were women and 60% were men, the responses from women would count more than those from men in the final results.
“Definitely pay attention to weighting,” Weldon said. “Is there something they’re not weighting to that seems like it should be there? Almost all of them are weighting to sex and age. People have been waiting to sex and age since the beginning of polling. But some of the other things — like education and income, or even access to the internet — those types of things can be really valuable weights.”
Expert Commentary
Facts Only
The Roper Center for Public Opinion Research at Cornell University hosted two webinars via The Journalist’s Resource on February 26 and March 4.
The webinars featured discussions on transparency, survey methodology, and weighting in opinion polling.
Kathleen Weldon, director of data operations at the Roper Center, was a speaker at the webinars.
Roper iPoll is a data research platform offering access to nearly a million survey and poll questions from 1935 to the present.
Eligible small media organizations and independent journalists can apply for a one-year membership to Roper iPoll.
The Association for Public Opinion Research requires member organizations to commit to transparency in survey design, conduct, analysis, and reporting.
Phone surveys with live interviewers and random digit dialing were historically considered the gold standard for polling.
Response rates for phone surveys have declined in recent decades.
Online probability polling is now a common methodology in the Roper Center’s archive.
Non-probability online samples, such as opt-in surveys, may introduce bias due to lack of randomness.
Some pollsters use weighting methods to adjust for biases in non-probability samples.
Weighting often includes adjustments for sex and age, but factors like education, income, and internet access may also be considered.
Executive Summary
Full Take
The strongest version of this narrative is its emphasis on transparency as a foundational principle for trustworthy polling. The Roper Center’s focus on methodological rigor and the evolving standards in survey research is commendable, particularly in an era where misinformation and poorly conducted polls can distort public discourse. The acknowledgment that no single methodology is universally superior—whether phone surveys or online probability polling—reflects a nuanced understanding of the challenges in opinion research. The discussion of weighting as a tool to mitigate bias in non-probability samples is especially relevant, as it highlights the ongoing efforts to refine polling techniques in response to changing technological and social landscapes.
However, the narrative could benefit from deeper exploration of the limitations of weighting itself. While weighting can correct for some biases, it assumes that the variables used for adjustment (e.g., age, sex, education) capture all relevant differences between the sample and the population. This assumption may not always hold, particularly in polarized or rapidly changing social contexts. Additionally, the reliance on self-reported data in opt-in surveys introduces another layer of potential distortion, as respondents may not be representative of the broader population in ways that are difficult to measure or adjust for.
The broader implication here is the tension between the ideal of objective, representative polling and the practical realities of declining response rates and the rise of digital media. Who benefits from this shift? Polling organizations that adapt to online methodologies may gain efficiency and reach, but at the potential cost of accuracy if biases are not adequately addressed. Journalists and the public bear the cost of misinterpreted or misleading data, which can shape narratives and influence policy decisions.
Root cause: The narrative assumes that transparency and methodological refinement alone can ensure the integrity of polling data. Yet, it does not fully grapple with the systemic pressures that might incentivize pollsters to prioritize speed or sensationalism over accuracy. Historical patterns, such as the overreliance on landline surveys in the past, show that methodological "gold standards" are often reactive rather than proactive.
Bridge questions: How might the incentives of media organizations influence the way polling data is presented or weighted? What unseen biases might persist even after weighting adjustments, and how can they be identified? Would a more decentralized approach to polling, such as citizen-led data collection, introduce new forms of bias or correct existing ones?
Counterstrike scan: If this narrative were part of a coordinated influence campaign, the playbook might involve overemphasizing the reliability of weighted online polls to justify their use in pushing specific agendas, while downplaying their limitations. However, the content here does not align with such a pattern. Instead, it presents a balanced view of the challenges and advancements in polling methodology, without overtly favoring any particular approach or outcome.
Patterns detected: none