How AI text watermarking works

To comply with the EU AI Act — which calls for greater transparency in AI use — Anthropic announced that it will begin watermarking AI generated text. The new watermarking feature is based on a method called SynthID-Text. Initially developed by Google DeepMind, SynthID-Text is already used in Google Gemini and will likely be adopted by other AI companies to satisfy the transparency requirements in Article 50 of the EU Act. This page breaks down how watermarking works into a series of interactive steps.

The visualizations on this page use precomputed outputs from Gemma-2-2b, an open model developed by Google. Each section builds on the previous. The bar at the top keeps track of the output as it is put together. The buttons allow you to add or remove tokens with the watermarking feature turned on or off.

1The problem

Following the precipitous rise of AI generated text in key fields including education, government, social media, and academic research among others, the need to distinguish between human and machine generated text has become more pressing. Common stylistic markers of AI generated text — the repetitive use of keywords and phrases such as “delve” and “not X but Y”, the overreliance on the emdash, hedging phrases, and generalizations without substantive backing — are often only weak and inconclusive signals of AI generated provenance. Tools trained to classify texts as human or AI based on such stylistic cues are notoriously faulty.

Watermarking promises an alternative. Rather than relying on heuristics to guess if a text is AI generated, watermarking leaves a subtle but detectable trace in the text itself. Large Language Models (LLMs) produce text by choosing possible words and characters (called tokens) from a list of plausible candidates. Given a sequence of text, the model calculates the odds of every word that could follow and selects one from amongst the most likely. If these choices can be systematically nudged ever so slightly — for example by picking the third option instead of the fourth — the accumulation of these constrained choices forms a pattern that can be detected with confidence.

A well designed watermark should meet at least four criteria. First, a watermark should be imperceptible to a reader: a person should not be able to tell if text is watermarked or not. Second, and relatedly, the watermark should not degrade the quality of the text, else the benefits of watermarking would be outweighed by the costs. Third, it should be difficult to remove. Rearranging sentences and paraphrasing words should not wholly erase the trace. Last, it should be quickly and readily measurable.

While several proposed watermarking systems meet some of these criteria, balancing all four is challenging. Hidden characters, while invisible to the average reader, are far too easy to remove. Watermarks hidden in secret patterns of words (like acrostics) are too brittle and easily broken by swapping one word for a synonym. More robust alternatives to SynthID-Text are too computationally intensive to measure at the scale and volume anticipated by AI companies. SynthID-Text, proposed by Google DeepMind, offers a technique that balances these constraints.

2How watermarking works

In this section we follow the procedure outlined by Google’s paper on SynthID-Text. The watermark is built up from five parts: 1) the numeric IDs assigned to words, 2) the list of likely next words language models can draw from, 3) a secret key, 4) mathematically determined scores assigned to each candidate, and 5) a small tournament that uses those scores to pick the next word from the set of likely candidates. Together these steps produce a largely imperceptible watermark that nonetheless can be used to identify AI generated text.

2.1Text as numbers

Language models do not see words as words. Language models process text as sequences of numbers. First, text is broken down into building blocks consisting of characters, words, and pieces of words called tokens. The set of all the tokens a model can use is called its vocabulary. Gemma’s vocabulary consists of roughly 256,000 tokens. Each of these tokens is assigned a number called the Token ID.

Pick a prompt. Flip the token to see the IDs that the model processes.

These token IDs are important. By processing text as numbers, watermarking can insert mathematical patterns in the sequence of tokens — as we will shortly see.

2.2The next-word distribution

Language models work with probabilities. For any sequence of tokens, the model predicts a plausible next token. To do so, it calculates the likelihood for every token in its vocabulary. For a vocabulary of 256,000 that means assigning each of 256,000 tokens a likelihood. The vast majority will have a near-zero likelihood of following the previous token. However, for any word there is a set of most likely candidates.

In the following visualization we depict a truncated version of this distribution. Here are the top eight most likely candidates for the current sequence. Click a token to append it to the running sequence and observe how that choice affects the candidates available for the next position in the sentence. For illustration, these options are precomputed using a real language model three steps deep.

The temperature dial flattens or sharpens the shape of the possible candidates by over- or under-weighting them. At low temperature the model is significantly more likely to pick the top candidate, while at high temperature every token becomes just as likely as any other. In practice, companies set the temperature to 1.00 to balance the need for confidence against the value of multiple options.

2.3The key

In cryptography, a key is a secret that can be used to scramble and unscramble data. Such keys are important in watermarking because they allow LLM providers to create patterns that appear random to anyone without the key. The provider creates a list of random numbers — often around 30 — and keeps them hidden. Each number is a key that helps determine the score candidate words are given during the tournament in section 2.7. This same list is used by the provider to imprint text with the watermark and to determine if a watermark was used to generate a given sequence of text.

Generate a set of keys and take note that they are now pinned to the top of the page. After exploring how watermarking works, try regenerating the keys to observe how different keys affect which tokens carry the watermark.

2.4Scoring the list of plausible tokens

To impart a watermark on the text, SynthID-Text nudges certain tokens in the list of plausible candidates higher than others. To achieve this, the technique assigns an arbitrary score to every token: a 1 or a 0. This score is akin to a coin toss. If heads, assign the token a 1; if tails, assign it a 0. However, to make the score measurable, the coin toss is rigged. The documentation for SynthID-Text explains that an LLM provider must keep a long list of 0s and 1s, in a random order. To assign a token a score of 1 or 0, SynthID-Text uses a mathematical formula to associate tokens with a position in this long list.

This formula takes six numbers: the token IDs of the four tokens within the sliding window, the candidate token’s own ID, and a key from the list of secret keys. Combine these numbers together mathematically by adding each to the previous and, every time, multiplying the result by a very large number, before dividing the final sum by 65,536 and taking the remainder.

The number produced by this process has interesting properties. Though it depends on the four tokens in the window, the resulting number is so shuffled and scrambled that, mathematically, the relationship this number has to the input would appear to an outside observer to be completely arbitrary. This number is used to select the score from the long list of 1s and 0s. By following this procedure, every token in any context can be given a seemingly random score that can only be computed by those with the secret key. In other words, to any observer the score appears to be a fair coin flip when in reality the outcome is completely rigged.

Press ▶ to see it worked out.

Rerun the process with different keys or at different positions in the sentence. Notice how, given the same key and same position, the outcome is the same every time. With a different key, with a different window of previous tokens, or with a different candidate token, the final outcome can change. Given a token you cannot consistently guess its score, but if you have the key you can always calculate it.

2.5A score for every candidate

Apply the same procedure to all eight candidate tokens at once. In unwatermarked text, the only thing that determines which token will be selected is the probability the language model associated with it. In watermarked text, the score of 0 or 1 nudges certain plausible candidates higher and others lower. To find the winning candidate, the watermark passes each plausible candidate token through a bracket.

2.6Drawing candidates

First, the language model draws a small handful of plausible candidates. This step is done before consulting the score: all plausible tokens are considered for the tournament. The process draws two candidates for every matchup in the tournament, so 2^m in all — eight slots when m = 3, sixteen when m = 4. Each draw is independent, meaning it is possible (and often likely) that the same token will be drawn several times. For illustration we only depict the top 8 plausible candidates. In practice any token in the model's vocabulary could be considered. However, a token with high likelihood of being next will fill the majority of the slots in the tournament, while an unlikely one will usually fill none but occasionally it might snag one.

This step is crucial. It means that the watermark is applied only to tokens that the model would have considered plausible to begin with. The process does not force the model to pick unlikely tokens.

2.7The tournament

The next word is finally chosen after it survives a tournament against the other candidates. Each candidate is matched with another. The tournament considers the arbitrary score assigned to each token — that 0 or 1 from before. If the token is assigned a 1, it survives that round. If the scores are equal, the winner is picked at random. The next round uses another key, so the score assigned to each surviving token is subject to change. This process is repeated until a single token remains.

Observe how winners advance along the blue line to the next layer until one token remains. That final token is selected as the model's output.

The tournament decides a winner from among the candidate tokens. In the chart, the winning candidate is coloured blue, the losing candidates are coloured orange, and the remaining candidates which were not selected for the tournament are coloured grey.

3Detection

If the text is generated by a watermarked language model, then every token in a document has a chance to hold a small trace of the mechanism described above. Watermark detection works by calculating whether the tokens in a text will score a 1 or a 0 under the constraints described above. On average, unwatermarked text will score right in the middle: 0.5. Documents written by human hand and text generated by an unwatermarked machine lack the ability to systematically select tokens associated with 1s. Remember, these scores appear to be completely random to anyone without the key. Watermarked text, in contrast, systematically nudges tokens associated with scores of 1 higher than tokens with scores of 0. The result is that watermarked text will have an average score sitting just above 0.5.

Three things to take from the chart.

• More tokens, more evidence. Scores of 1 or 0 on a single token could be explained by pure chance. However, the longer the document, the more evidence the detector has to work with. A sentence’s worth of evidence is inconclusive. A paragraph might have faint traces.
• False positives are unavoidable. For highly predictable text, like letterhead, boilerplate, templates, and formulaic code, SynthID-Text is unable to imprint a watermark, because watermarking requires choosing between alternatives. If no alternatives are available, there is no watermark. Furthermore, although it is unlikely, it is possible for a human to unknowingly pick the same tokens an AI would.
• Detection needs the key. To calculate the appropriate values, a detector needs access to the key that produced the watermark. Under the current watermarking system described here, LLM providers will be unwilling to share the key publicly because doing so would enable malicious actors to create systems that counteract the watermark. Until another method is developed, this means that only LLM providers or trusted third parties will have the ability to detect watermarks.

Here is the detector on real text. Four public-domain passages of different predictability — a speech, a novel, a statute, some code — sit next to Gemma-2-2b writing in the same four registers, once with the watermark off and once with it on under a fixed demo key that is shipped with this page. Switch to your own key from 2.3 to see what a detector without the right key sees.

In practice, the detectors that are used in production are more sophisticated than a simple average. More robust methods implement a Bayesian detector trained on watermarked and unwatermarked data in order to more accurately classify uncertain cases or flag them as such.

4Does watermarking degrade quality?

One worry users might have is: if the watermark nudges less likely tokens to the top, does that suggest watermarked text is more likely to contain errors? A central claim of SynthID-Text is that watermarking does not degrade the quality of text.

The first and most important guard that watermarking has against distortion is that it does not affect what a large language model samples. As discussed earlier, language models sample tokens from a distribution. Watermarked text samples from the very same distribution. Therefore, the tokens used to watermark text are tokens that a language model could have sampled anyway.

In principle, this means that watermarking cannot introduce any errors that a language model was not capable of making in the first place.

A shrewd reader might still be worried. Language models are capable of making errors. They hallucinate. They make mistakes. Does watermarking then affect the rate at which errors are made? Can watermarking turn a rare, sporadic error into a systematic one?

The capital of Australia is Canberra. This fact is reflected in the top result for the prompt “The capital of Australia is”. The model associates “Canberra” with a probability of 32.2%. Other plausible candidates for the next token in the sentence include “the”, “located”, “known” and “one”. However, in addition to “Canberra”, the model also predicts “Sydney” with a probability of 10.5%. This error perhaps reflects the model’s training data, which may retain human mistakes.

Watermarking does not add “Sydney” to the list of plausible candidates, but does watermarking change how often “Sydney” is selected? The answer to this question depends on a number of factors. First, for “Sydney” to enter into the tournament, it must be drawn. The likelihood of being selected rises with the number of layers the tournament has, because the more slots available, the more chances there are to select the token. Second, whether or not “Sydney” wins the tournament is arbitrary — it depends entirely on the random set of keys. Under some keys “Sydney” will be more likely to be given a score of 1, while under others it will receive a 0.

The effect a key has on what tokens are sampled is measurable but arbitrary. In the chart above you observe how, under your current key, the rate at which “Sydney” is selected will change. With certain sets of keys “Sydney” will become far more likely. With others, it will become even less likely than in the unwatermarked case. If we were to average across several sets of keys, the effect would vanish.

To see this, compare the two rows for “under the current keys” against “averaged over 300 keys”. As you can observe, the average equals the unwatermarked rate. This equivalence is what SynthID-Text promises.

In practice an LLM provider must use a small set of fixed keys. To understand the implication of using fixed keys, it is helpful to observe which tokens are selected under different keys.

Given the current position in the sequence, different sets of keys tend to select different tokens. This is shown in the barcharts below. For each, the tournament runs 600 times. Each time, chance enters into the picture when the language model samples from the distribution and when tokens with the same score need a random tie breaker. Nevertheless, the different keys tend to make some tokens more or less likely than what the language model would predict. In the plots below the coloured bars aggregate the number of times the token was selected by the watermark and the grey outlines are the model's own predicted probabilities.

Try different prompts and observe the outcomes. Each key promotes certain tokens more than others. With some prompts, where the options are relatively fixed, this effect is small. In others, where there are many plausible candidates to choose from, the effect is more pronounced. What is crucial to keep in mind is that even in cases where top tokens are suppressed and alternatives are elevated, the choice is completely arbitrary. In the long run, this means that there is no fixed bias that favours certain terms over others.