The rise of conversational artificial intelligence (AI) tools could soon allow for the covert manipulation of users’ decision-making processes in a burgeoning commercial sector researchers have dubbed the “intention economy.” This warning comes from a study published Monday by the University of Cambridge in the Harvard Data Science Review.
According to the research, this new marketplace revolves around the commodification of “digital signals of intent” — a potentially lucrative but ethically concerning development. AI systems could influence decisions ranging from purchasing movie tickets to voting in elections by leveraging a deep understanding of users’ behaviors, preferences, and communication styles.
The study, authored by researchers at Cambridge’s Leverhulme Centre for the Future of Intelligence (LCFI), highlights how chatbots, virtual tutors, and other anthropomorphic AI agents are paving the way for these “persuasive technologies.” By integrating knowledge of users’ online habits with the ability to anticipate desires, these AI tools could establish unprecedented levels of trust and influence.
However, the researchers cautioned that this level of trust could lead to “social manipulation on an industrial scale.” The paper outlines how AI could target users’ age, gender, political leanings, online histories, and even their communication patterns to steer decisions toward commercial or political outcomes.
“AI tools are already being designed to collect, predict, and ultimately commodify human plans and purposes,” said co-author Yaqub Chaudhary. This includes combining data from Large Language Models (LLMs) with algorithms designed to achieve specific outcomes, such as promoting a particular product or influencing political opinions.
The emergence of the intention economy represents a potential gold rush for companies seeking to capitalize on human motivations, noted co-author Jonnie Penn. However, he warned that without regulatory safeguards, this sector could undermine free and fair elections, the integrity of the press, and competitive markets.
“Unless regulated, the intention economy will treat your motivations as the new currency,” Penn said. “We need to consider the consequences of this marketplace before we find ourselves ensnared by its unintended effects.”
The researchers emphasized the importance of public awareness in addressing these challenges. “Public understanding of this issue is key to ensuring we don’t go down the wrong path,” Penn added.
As AI technologies continue to advance and integrate into daily life, the study calls for proactive measures to ensure they are used responsibly, preserving democratic values and consumer autonomy in the face of this emerging commercial frontier.
