AI Creation Mimicking Human Reactions
Major advancements are underway in developing systems that aim to imitate human responses to various products, applications, and websites, removing the need for actual human involvement.
Researchers from Harvard University and the Massachusetts Institute of Technology have introduced their network, which utilizes “persona agents” to simulate real-world interactions.
“The agent succeeded in expressing the assigned characteristics 91.5% of the time.”
The MatrAIx group reports that they have established a method to replicate genuine responses using a foundational dataset comprising nearly 600,000 “human-based” personas paired with 400,000 synthetic profiles.
This extensive library—close to 1 million personas—will serve as a framework for generating around 8.3 billion AI bots. MatrAIx envisions these bots functioning collectively as a singular focus group.
This development empowers companies to leverage simulated personalities to assess how consumers might react to pricing shifts. For instance, they conducted a simulation to determine if one million virtual individuals would still purchase a 12-pack of Coca-Cola following a $2 price hike. The findings indicated that about 61% would continue buying it.
Besides gauging consumer hesitance after price increases, the bots also evaluated how willing an AI assistant might be to persist after a failure and measured user tolerance for website loading delays.
Other experiments included analyzing a mental health chatbot and browsing for espresso machines priced below $200 on Amazon, aiming to discern which products the AI bot would select and the reasoning behind those choices.
The approach extends to social media platforms as well. For example, bots could be programmed to search for recipes or watch favorite videos on Instagram, potentially allowing millions of bots to perform these tasks simultaneously.
However, there’s an inherent challenge. It’s difficult to place complete trust in agents operating within these large-scale simulations to truly mirror human behavior. Like many AI systems, the agents in this simulation may not execute tasks as intended.
A controlled study comprising 400 trials by Harvard examined the consistency of bots with ten behavioral traits, yielding promising results though not flawless. “The agent succeeded in expressing or correctly suppressing the assigned characteristic 91.5% of the time,” the researchers noted.
Guided by Xiaomin Li from Harvard and Yuexing Hao from MIT, MatrAIx has conducted over 16,000 assessments across eight distinct task types. They employed three extensive language models during their research: Claude Opus 4.8, Chat GPT 5.5, and Claude Haiku 4.5.
Nonetheless, evaluations from humans regarding the quality of these personas were notably lower, with an average rating of 4.135 out of 5, translating to about 83% accuracy.
The fact that approximately 8.5% of agents didn’t adhere to expected personas could lead to significant inaccuracies for businesses using these bots extensively. In the Coca-Cola scenario, for instance, it’s possible that 85,000 bots might not react correctly.
An even greater concern raised in the study is that even well-designed AI agents fall short of effectively emulating the diverse behaviors of billions of humans. It’s rather challenging to model and predict human conduct if agents functioning in place of people are restricted from engaging in the flawed, sometimes unethical actions that individuals and groups are known to take.
This leads to the daunting implication that entities and governments pursuing ultimate predictive capabilities may have to develop comprehensive simulators where agents representing real individuals can display antisocial or harmful behaviors.
On the flip side, conducting thorough human research isn’t as quick or economical as AI simulations. The discussion suggests that traditional human focus groups might not effectively capture the public’s interests, despite potentially offering more accurate insights. After all, they are still people.
