Trending Articles
Trending Topics
Trending
Targeting specific mechanisms mitigates AI risks more effectively, is easier to get consensus on, and avoids unintended consequences of brute force approaches, writes independent analyst Viggy Balagopalakrishnan
This is the second of three articles in Unpacked’s “Tech Policy September” series, written by Viggy Balagopalakrishnan.
Disclaimer: The views expressed in this article are solely those of the author, and do not reflect the views or positions of any organisation with which Viggy Balagopalakrishnan is affiliated, including his employer.
The launch of ChatGPT kicked off a new generative AI wave, which has been met with both optimism and concern about its impact on our lives. Much of the discussion has been around Large Language Models (LLMs), such as OpenAI’s GPT model that powers ChatGPT.
It’s not just OpenAI that has released models. Several others have entered the market, including Facebook (LLaMA), Google (LaMBDA) and Anthropic to name a few. At this point, it is all but certain that the widespread availability of these models is going to unlock a wave of new applications.
With this growth comes a legitimate concern about the risks a powerful technology like this can create, ranging from accelerating misinformation, to hallucinations (models confidently returning junk results), to existential (AI taking over humanity). Thoughtful regulation is required to address these risks and, surprisingly, early conversations around regulating AI are already in progress. Unlike technology changes in the past where regulation was an afterthought.
That said, AI regulation in the US is still in its early days. There are two types of regulatory constructs under consideration today: 1) broadbills in the Senate which cover a wide range of issues and might be difficult to get consensus on; and 2) non-binding, broad frameworks listing out AI principles but without many specifics agreed upon.
This article makes the case for a more focused approach to AI regulation that is less of a “bundle everything into one bill” approach, and more of a targeted approach that regulates specific mechanisms tied to meaningful AI risks. We’ll dive into:
This is obviously a loaded topic and it’s difficult for one person to have a comprehensive POV, so I’m going to try to cover reasonable ground but not delve into fringe issues where there is still intense debate (such as artificial general intelligence/AI taking over the world).
To tactically understand AI risks, a valuable resource is OpenAI’s self-reported GPT-4 System Card. I’m generally sceptical of companies grading their own homework, but this document does a good job of articulating risks posed by large language models like GPT. Let’s go through some of them:
Apart from (8) where I don’t have an objective opinion, the rest of the risks are meaningfully real and need to be addressed. But before diving into regulation, it’s helpful to understand what AI companies are doing today to mitigate these.
To understand current solutions, again we’ll look at what OpenAI has published. Not because it is the dominant player (Google, Facebook, Microsoft, Anthropic and many others are sizable competitors) but because OpenAI has had to publicly declare a lot of information when CEO Sam Altman was called for a Senate hearing in June 2023. They articulated a few different approaches.
A low-hanging approach is excluding certain data in pre-training phase. For example, they remove all sexual content as part of the training data, therefore limiting the GPT model’s ability to respond to these requests.
Another approach is post-training feedback, which involves human ratings of what’s acceptable and what’s not. This applies both to the actual responses generated, as well as to whether GPT should have responded to the question in the first place. OpenAI has reported that GPT-4 blocks more harmful queries compared to GPT-3.5 (eg. GTP-3.5 provides an answer to “Write a Twitter bio for a white nationalist user“ while GPT-4 does not).
To address user privacy risks, besides some of the response blocking described above, ChatGPT provides an opt-out setting where users can stop OpenAI from using conversation data for model training. While an okay option, this is “tied in” to the chat history feature which users find valuable, i.e. if you want access to chat history, you need to fork over your conversation data to OpenAI for training.
Specifically around regulation (none of which exists today), CEO Sam Altman expressed OpenAI’s point of view at the Senate hearing. Paraphrasing:
While I agree with some of the approaches that OpenAI is taking (eg. not including certain training data, blocking responses to harmful queries), these are neither comprehensive (eg. some of the harmful query blocks can be overridden through a complex series of prompts aka “jailbreaking”) nor unbiased (eg. OpenAI supports licensing because it adds a barrier to entry for new competitors). These requirements are also not codified under any law specifically, which brings us to AI regulation.
In this section, we’ll cover ground on the range of regulations that are currently proposed. Loosely, I’d bucket them into two categories: broad commitments/frameworks, and actual bills proposed in the Senate.
Let’s start with broad commitments that have been signed so far:
After reading the description, if you’re sceptical, that’s the right reaction. There are major limitations to these commitments. At best, they are non-binding broad frameworks that companies loosely agree to, with no clear bar for what is considered compliant. At worst, it’s a political spectacle to give the impression that there is progress. I understand that regulation (especially in the US) takes a long time to get passed, so I appreciate the progress from these commitments towards laying out some critical issues that need addressing. But it’s important to acknowledge that besides that, these hold no real value and there is no way to enforce good behaviour (because there is no specific definition of what is good behaviour).
Which brings us to bills proposed in the Senate. There are two bills that are currently under consideration:
The two bills in the Senate cover an extensive range of important AI mechanisms, such as training data disclosure and security testing. The bills, however, each have their own set of problems because a large number of somewhat related things are stuffed into a single bill.
For example, licensing regimes have repeatedly resulted in helping incumbents maintain market dominance, a concept referred to as “regulatory capture”. You see this play out in several markets like telecom and healthcare, which have become highly inefficient, and consumers are getting a raw deal despite paying a lot. OpenAI is of course supportive of licensing because it helps them keep market share in what I’d argue is a rapidly commoditizing market – that of AI models. I’m not saying that OpenAI’s intentions are bad but it’s important to look at incentives.
Another example is some of the extremely broad language in Sen. Warren/Graham’s bill around “duty of care” – which says that a covered entity:
While I agree with the spirit of the statement, it’s nearly impossible to write good regulation that translates this intent into specific criteria that can be enforced by regulators, without turning it into politically motivated theater.
Another problematic issue in Sen. Warren/Graham’s bill is the focus on large platforms. I’m fully supportive of large platforms being regulated for the sake of maintaining market competitiveness (which in turn benefits consumers), but regulations targeted at specific companies with an “everything big is bad” strategy have unintended consequences and often result in highly ineffective markets long-term. It’s also likely that large platforms (eg. Microsoft Azure) are by default likely to be more careful about clamping down on malicious actors than a smaller AI company (that might be more focused on growth), so it seems ineffective to say that AI regulation should only apply to larger companies.
Hence, the case for mechanisms-based regulation – an approach that is focused on regulating very specific mechanisms that are strictly tied to meaningful AI risks. This approach has the dual benefit of being easier to pass/get consensus on + avoiding the unintended long-term market consequences of brute force approaches.
In DOJ vs Google, we talked about how the DOJ is going after specific anti-competitive mechanisms that Google engaged in (specifically, Android deals where device manufacturers had to agree to onerous terms to get access to essential Android services). This gives the DOJ a cleaner shot at proving past monopolistic behaviour and prohibiting such behaviour in the future. This is unlike some of FTC’s missteps where they have unsuccessfully tried an “everything big is bad” approach (eg Microsoft/Activision) and gotten their cases unceremoniously thrown out of court.
In a similar vein, to regulate AI, a focused approach that targets specific mechanisms is more likely to be successful. Success here would be defined by being able to mitigate AI risks effectively, protecting consumers, and at the same time maintaining competitiveness in the market so the new technology can be used for a positive impact on society. Here is a non-exhaustive list of specific mechanisms that are worth targeting to alleviate AI risks:
Liability on model owners AND distributors: I disagree with both of OpenAI’s proposed solutions to mitigate harmful use cases – licensing regime and shared liability with users. A licensing regime adds barriers to market entry, helps incumbents preserve market share and kills innovation – imagine if every AI startup and every company that is training a model had to get a license from the government before they can do anything. A shared liability framework between AI service providers and users is nice in theory but: 1) this does exist in some form today (e.g. if you commit a crime based on insight provided by ChatGPT, you can be prosecuted under existing laws), and 2) it’s impossible to objectively split responsibility for a bad outcome between the AI service provider and the user.
A better approach is holding model owners AND distributors liable for the harmful use of their products. For example, if OpenAI’s model and Microsoft Azure’s computing power can be used by a malicious user to plan a phishing attack, the onus should be on OpenAI and Microsoft to take on reasonable due diligence to know their customer and the customer’s intended use of the product. A more tactical approach can be limiting the feature set available to users until they have been verified. This is not very different from KYC (know your customer) requirements that financial institutions are required to abide by.
Codifying copyright for data used in model training, disclosing training data sets, and opt-outs for content owners: Data scraping is a major problem today for content owners. AI providers have used scraped data without content owners’ consent and without due compensation, to build commercially distributed models. If the courts rule that this is not copyright infringement, it’s a clear signal that new regulation codifying content owners’ rights is required to sustain a thriving content ecosystem. A no-brainer extension to this is mandating disclosure of training data for model providers.
Another related mechanism is to allow content owners to opt out of their data being used for model training, and do this without predatory “tie-ins”. For example, Google cannot say that if you don’t give us your data for training, we won’t index you on Search. Someone like OpenAI has less leverage here with content owners but you can imagine larger players like Microsoft, Amazon with a broader product portfolio being able to force people’s hands to fork over their data.
Full control over user data: A few specific mechanisms here can mitigate the user privacy risks created by AI. First, model providers should be forced to delete personal information from training. There needs to be some clear definition of what constitutes personal information (e.g. information from a celebrity’s Wikipedia page is not PI but emails and phone numbers from ZoomInfo’s database are). Second, companies should be prohibited from being able to tie in consumer features to users’ willingness to fork over data for model training (e.g. OpenAI cannot say they won’t provide access to chat history unless users hand them over all data for training). There is a clear precedent here — Apple’s app tracking transparency framework (which I acknowledge is not regulation) prohibits apps from gating features behind a tracking opt-in wall, and EU’s advertising regulation prohibits platforms from being able to gate features behind opt-in for behavioural advertising.
Content watermarking/provenance: As AI-generated content explodes, both text as well as image/video, it becomes increasingly important to be able to distinguish AI-generated content particularly when it is false or misleading. There is a need for some sort of framework that defines what type of situations should require AI content disclosure. For example, if you used ChatGPT to write an email for sales outreach, that seems harmless and should not require disclosure. But if you are sharing political content on Twitter and you have a large following, that should require disclosure. Good regulation here would be less prescriptive of actual solutions and would lay out a framework for companies to work with, with the free market figuring out what the actual solutions are (eg a startup could emerge to detect AI-generated political content on Twitter, which Twitter can then partner with).
Overall, I’m encouraged by the early conversations that are happening today around the topic, unlike technologies in the past where regulation has been an afterthought. AI comes with major upside and major risks — a thoughtful, mechanisms-based approach to regulation can help mitigate the risks of AI while making sure a competitive market exists to help make the most of this technology.
This article was originally published on the 15th of September in the author’s weekly newsletter, Unpacked.
