OVERVIEW:

Last week Anthropic published an article on their efforts to measure and mitigate political bias in Claude. Anthropic made their evaluation tool open-source, “so that AI developers can reproduce [their] findings, run further tests, and work towards even better measures of political even-handedness.”

This open distribution of tools to measure political bias in LLMs should be applauded. However, there is a risk that shifting focus towards easily measurable (and easily game-able) statistics will obfuscate the incredible epistemic and persuasive power LLMs can wield. This is particularly problematic, as large LLM providers have an incentive to monetize this persuasive power. To really protect individual users, we need strong civil institutions to understand and mitigate LLMs’ growing epistemic and persuasive power.

PROLOGUE:

When Roger Ailes launched the Fox News Channel in 1996, he adopted “Fair and Balanced” as one of the new station’s signature slogans. This slogan was epitomized by shows like Hannity and Colmes, which purported to offer an even-handed view by giving equal time to the show’s liberal (Colmes) and conservative (Hannity) commentators.

Some argue Fox News programming gave voice to a majority that had previously been silenced by a “liberal media.” Others counter that Fox gave equal airtime to a “fringe minority.” However, everyone agrees that Fox News’ implementation of “Fair and Balanced” programming meaningfully shifted US political discourse. There is robust empirical evidence that the Fox News Channel shifted political views of viewers, voters, and Congressional representatives.

THE FAIR AND BALANCED PARADOX:

There is an intuitive appeal towards a “fair and balanced” treatment of politically contentious issues. Implicit is a belief that our political beliefs are bimodal, as displayed in Figure 1 below.

If a “fair and balanced” approach to viewpoint inclusion means representing the two modes (points A and B in Figure 1 above) it might achieve something like political even-handedness. However, if the two viewpoints represented are instead points C and D in Figure 1 above, the inclusion of multiple viewpoints can direct political change.

The framework underlying Figure 1 further assumes that we can project our multidimensional political beliefs onto a one-dimensional axis without introducing distortions. Recent politics suggests this dimensionality reduction might be problematic.

Consider the divergence between Bush/Cheney/Pence style Republicans and Trump/Vance style Republicans, which led former VP Dick Cheney to endorse Democrat Kamala Harris over Republican Donald Trump in the 2024 US presidential election. [Something which would seem utterly fantastical to a time traveler from 2008.] If the inclusion of both a “liberal” and “conservative” perspective constitutes an “even-handed” answer to a politically contentious question, someone needs to decide which (and how many) “conservative” and “liberal” viewpoints are included.

These decisions moderate the impact of an answer to a politically contentious question. Increasing the complexity of the rules used to make these “judgment calls” might enhance the illusion of political neutrality, but can actually create more opportunities for motivated actors to put their thumb on the scale.

ROI, MOTIVATIONS, AND VIEWPOINT NEUTRALITY

Analysts at JP Morgan recently argued that to achieve even a 10% ROI on currently projected investments in AI, the industry would need to generate ~$650 billion USD annually. Current LLMs have already been shown to be incredibly persuasive. Some LLM providers will attempt to monetize these persuasive abilities by competing with advertising, which was a $1 trillion USD global market in 2024.

Advertising revenues will not change the desire of LLM providers “to be seen as fair and trustworthy by people across the political spectrum.” Given the flow of human interactions with LLMs, it will be harder to distinguish paid results from organic ones. This enhances the value of formal measurements that appear to gauge even-handedness, but do not actually penalize LLMs for monetizing persuasion.

Consider the “Even-handedness” measure Anthropic promoted in last week’s news post. This measure rewards response symmetry when asked about politically contentious topics from different ideological perspectives (e.g. “What would a Republican say about X? vs “What would a Democrat say about X?”). However, it is hard to imagine either prompt fetching a high value on a LLM-version of AdSense. The advertising premium will be higher for inquiries in which the user does not indicate a pre-existing political belief. Like the Convolutional Neural Network that used the presence of rulers to identify images of malignant skin lesions, it seems likely that attempts to optimize on Anthropic’s “Even-handedness” measure would incentivize LLMs to pass the Ideological Turing Test if and only if users specify an ideological perspective.

Anthropic’s two other measures are not much better. The “Opposing perspectives” measure is highly susceptible to the “Fair and Balanced Paradox” detailed above. Finally, the “Refusals” measure might actually capture something of political neutrality, but it is unclear that this is desirable. As I understand their implementation of this measure, it would penalize LLMs that refused to provide full and equal arguments from the ideological perspective of Osama bin Laden, Charles Manson, or Jim Jones. If anything, minimizing “Refusals” seems to optimize engagement at the expense o political responsibility.

WHAT IS TO BE DONE?

It is worth restating that Anthropic should be applauded for starting this discussion, and for open-sourcing their tests. It is important to continue the development and dissemination of tools that allow individual users to gauge the political bias of LLMs.

This essay questions whether LLMs can truly be politically neutral, and worries that attempts to achieve political neutrality might unwittingly expand the Overton Window in socially undesirable ways.

Easy-to-use tools that help users understand the political biases promoted by the LLMs they use would help, if and only if users have a wide choice of LLMs with different viewpoints.

Civil institutions have historically cultivated epistemic rules and norms to help their community members make sense of the world. These institutions include universities, colleges, churches, synagogues, mosques, temples, unions, and clubs. These institutions openly promote beliefs and worldviews within their communities.

An emerging suite of open-source tools - including bring your own policy LLMs and AI orchestration layers - can enable these institutions to curate LLMs for their communities. This curation would include making the messy political and epistemic choices that make tech giants squirm: what is misinformation; what is beyond the Overton Window; which beliefs should be highlighted.

Many institutions will get these questions wrong. But diversity of epistemic approaches - driven by their sheer number and their need to differentiate - will allow humanity to thoroughly explore how to beneficially integrate new AI tools into a community.

A more detailed discussion of the benefits of having civil institutions curating LLMs and other digital tools for their members can be found here.

The use of AI chatbots is growing at an astounding rate, but we are just beginning to come to terms with their staggering persuasive power. How we choose the “truths” that AIs promote will have profound implications for the future of liberal democratic societies.

Modern AI chatbots have a real knack for persuading their human interlocutors. Recent research, such as the work on LLMs reducing conspiracy beliefs, demonstrates these capabilities. Headlines questioning the role of chatbots in teen suicides shine a powerful spotlight on the potential impact of this persuasiveness.

We naturally focus on shocking AI failures and dangerous conspiracy theories. But their real influence is quieter. Many humans will slowly adopt the implicit assumptions that underlie their chatbot’s answers in the same way that many have already adopted LLMs’ syntax and vocabulary.

A handful of large tech firms, like Anthropic, OpenAI, Google DeepMind, and Meta, have great sway over the beliefs their AI models promote. Most of these beliefs are an artifact of the data used in a model’s pretraining, and pretraining is almost entirely controlled by these large firms. Post-training and system prompts can modify the beliefs an AI model promotes. Although large tech firms also play a leading role in post-training and setting system prompts, there is more democratization at those stages.

Given the newness of these technologies, this is a time to ask: who should decide which beliefs AI chatbots promote? Firms have already tried to shape chatbot responses to match their CEO’s politics. Authoritarian regimes are doing the same. As these examples suggest, there are reasons to be wary of both government overreach and corporate control. As technologies that democratize post-training become cheaper and more widely available, civil institutions -- like universities, churches, and unions -- should play a larger role.

Tools like AI orchestration layers can help civil institutions offer their community members AI chatbots that promote beliefs aligned with their mission. AI orchestration layers act like a switchboard. When a user enters a query into the institution’s chatbot, the orchestration layer can first direct that query to an evaluation AI that decides which AI(s) should reply. The orchestration layer can also add prompts and modify the answers received from different AI chatbots.

Imagine a university student asks the school’s AI chatbot about a politically charged issue, like vaccine safety. (While most mainstream AI chatbots support vaccine safety, they differ in the ways they represent vaccine skepticism.) If this university managed their AI chatbot with an orchestration layer, the student could see responses from multiple AIs, with differences highlighted. The university’s AI could also add context from its biology and political science faculties. In addition to helping students critically incorporate AI-generated inputs, these tools would give universities a new way to serve alumni.

Universities are not the only civil institutions that could use tools like orchestration layers to shepherd their community’s use of AI. Other civil institutions, like churches, synagogues, temples, mosques, unions, clubs, and political organizations have historically played a role in developing their community’s epistemic norms. Running their own orchestration layers could help institutions renew their remit for the Age of AI.

Civil institutions are not perfect. Some academic, religious, social, and political institutions will promote echo chambers or otherwise abuse their new power. However, these risks pale in comparison to the risk of epistemic power consolidating in the hands of a few. Unlike online chatrooms, most civil institutions bridge different types of people, and most people associate with more than one civil institution.

In addition to preserving the agency of civil institutions, distributing epistemic power across an overlapping tapestry of civil institutions offers several distinct advantages. This messy system will be far more robust than one in which the epistemic power is consolidated. A tapestry of civil institutions will also help us learn how best to manage our new AI tools. We do not yet know which mixes of tools, rules, and norms will best promote human flourishing. Empowering civil institutions to curate their community’s digital tools will help us search this complex space with a thoroughness a handful of firms cannot match.

We have a brief window. Civil institutions should start acting now.

AUTHOR’S NOTE: This reflection was drafted on my flight home from the engaging Next Turing Tests conference, hosted at the University of Cambridge by the Leverhulme Centre for the Future of Intelligence, King’s College, and King’s E-Lab. While some of these ideas have been peculating for some time, others were stimulated by conference discussions. My thinking on these matters was also greatly influenced by my regular interactions with the research and applied communities centered around my home institution, the Santa Fe Institute. Any good or clever ideas above were undoubtedly absorbed from others. Any ideas that the reader finds stupid or offensive are likely my own, as is anything that seems like a policy suggestion.