An AI startup recently carried out a poll on maternal health policy, which on the surface seemed routine. However, all the feedback came from computer simulations rather than actual humans, a method referred to as “silicon sampling.”
The issue was brought to light when Axios published an article discussing maternal health policy that referenced a study showing most people trust their healthcare providers. Initially, it wasn’t made clear that the results were generated by AI instead of real participants. A deeper investigation revealed that the data was produced by Aaru, an artificial intelligence startup, without any input from actual people. Following this revelation, Axios added an editor’s note clarifying the situation.
Silicon sampling is gaining popularity within the polling sector, as pointed out by two professors. The premise is straightforward and appealing: large-scale language models can produce responses that mimic human replies, allowing pollsters to conduct surveys more quickly and cheaply than traditional methods.
This method’s allure is evident, especially as traditional polling faces growing difficulties. Over the years, executing phone polls has become significantly harder, and online voting raises concerns about the quality and representativeness of samples. Silicon sampling seems to provide a way out of the costly and often cumbersome process of gathering actual opinions.
However, critics argue that this technique undermines the essence of polling. Professors Leif Weatherby and Benjamin Recht emphasize that public opinion data is crucial for informing policy and social research, stating its true value lies in accurately reflecting the views of real people.
They argue that relying on AI-generated opinions deepens the issues within the current information landscape and fosters distrust. The notion of using artificial representations to understand real-world sentiments is seen as misguided.
Historically, journalist Walter Lippmann highlighted in his 1922 book, “Public Opinion,” that people form “mental images” of their societies, which he termed “fiction” or “pseudo-environments.” He believed that while surveys have inherent limitations, they are essential for grasping the genuine sentiments of the populace.
The emergence of silicon sampling prompts essential questions about the relevance and integrity of public opinion polls today. If polling data merely reflects AI predictions rather than actual human sentiment, the fundamental purpose of polling itself is called into question.
Moreover, this issue goes beyond methodological concerns. When organizations and policymakers depend on polling data, they generally assume it captures true public feelings. If this assumption fails, as might happen with AI-generated data, it could lead to policies that don’t align with the actual needs of the community.
Wynton Hall, social media director at Breitbart News, cautioned that substituting AI for human input could be perilous due to inherent biases in these models. During a discussion on “Stinchfield Tonight,” Hall noted that many AI models carry a built-in tendency towards left-leaning perspectives.
He recounted his research, which revealed that even those within left-leaning academia recognize a bias in large-scale language models, often reflecting content from sources like Wikipedia and Reddit. Hall highlights the importance of equipping younger generations with critical thinking skills, so they can discern the neutrality of the information they encounter.
