Smaug-Humanizer: An open-weight 12B humanizer

Today we are releasing Smaug Humanizer, a 12B model that rewrites AI-generated text so it reads like a person wrote it. It keeps the facts, the numbers and the layout of the input, and it is Apache 2.0.

This post is about how we trained it: where the data came from, and our training methodology.

Humanizer

Most people reach for a humanizer for an ordinary reason: they drafted an email or a report with a chatbot, and it sounds like a chatbot. They want it to sound like them.

That asks two things of the rewrite, and they pull in opposite directions:

1. Sound human. The rewrite has to lose the even rhythm, the stock transitions and the over-polished word choice that make AI text easy to spot.
2. Say the same thing. Every number, name, date and claim has to survive. A rewrite that reads well but changes the price in paragraph two is worse than no rewrite at all.

Stage 1: SFT

The idea behind the training data is simple. Take text that a person wrote, ask an LLM to rewrite it in its own style, and train the model to go backwards: from the AI version x to the human original h. We only used human text from 2021 or earlier, so we can be sure a person wrote it, and we spread the rewriting across nine different chatbots so the model sees many flavours of AI style.

The first model we trained on this data made only light edits. The cause was in the pairs. When asked to rewrite a paragraph, an LLM often keeps much of it, and about a third of the sentences in our AI drafts were near copies of the human ones. A model trained on such pairs learns to change a few words, and the result still reads as AI.

So we added filters, and kept a pair only if it passed all of them:

• A real rewrite. A deterministic check drops pairs where too many of the human sentences survive in the AI version, so the model has to learn a real rewrite to undo it.
• The same layout. A second deterministic check requires the same paragraph and line breaks in x and h. Without it, the model learned to add and remove line breaks at random, and users notice that immediately.
• The same facts. An LLM judge checks that the AI version keeps every fact, number, name and quote of the original and adds nothing. If the target says something the input does not, the model learns to make things up.

We ended up with 3,676 pairs. This stage gave most of the gain in the final model: cleaner pairs helped more than more pairs and RL, (below) adds a smaller gain on top. We scored each checkpoint with Pangram a commercial AI-text detector and with an LLM judge for faithfulness to the input, on a held-out set of AI-written texts that the model never saw in training.

Stage 2: RL

After SFT we wanted reinforcement learning, and RL needs a reward. We did not want a commercial AI detector in the loop, as they are expensive.

Instead we looked for a signal in a text embedding space. Before we built the reward, we checked whether the maximum mean discrepancy (MMD), a standard measure of how far apart two sets of samples are, can tell human text from AI text in that space. Two disjoint samples of human text were indistinguishable under MMD², and so were two disjoint samples of AI text from the same model. AI text against human text, on the other hand, showed a clear gap. In our early models, outputs that a detector labelled human more often also tended to sit closer to human text under MMD², though the relation was loose. The catch is that MMD² compares two sets of texts, while RL needs a score for every single sample. The rest of this section turns one into the other.

With a kernel k,

Call the model’s output distribution π, human text H and AI text A. We want π close to H and far from A, so we minimize the difference:

Write out both terms with the definition above:

The output-to-output term 𝔼 k(y, y′) is the same in both lines, so it cancels in the difference, and we never have to compare the model’s samples with each other. The human-to-human and AI-to-AI terms do not involve the model, so they are constants. What is left is

which is an expectation over single outputs. Minimizing the objective is the same as maximizing the expected reward of one sample y, where the reward is the MMD witness function: how similar y is to human text on average, minus how similar it is to AI text on average. That is exactly the shape a policy-gradient method wants.

The averages are the problem. Averaging over a whole human corpus is slow, and worse, it mostly measures topic. A rewrite about tax law looks “AI” next to a corpus of travel blogs for reasons that have nothing to do with style. But our data gives every prompt something better than a corpus: the human text h that the AI text x was made from. Same content, two styles. So we estimate both averages with that one pair, which gives the kernel similarity reward:

where e(·) is a text embedder and γ = 0.262. Topic is held fixed inside each pair, so what is left is mostly style. A rewrite scores well when it moves away from its own AI draft and toward the way a person actually wrote the same thing.

The gates

The reward alone does not protect the facts. A rewrite that leaves out a detail is also further from its AI draft, so it can score higher while saying less. In our first RL runs without a fact check, the rewrites became more human and less faithful. So each sample must also pass a set of deterministic gates, among them:

• Numbers: every number in the input is in the rewrite.
• Layout: the rewrite has the same paragraph and line breaks as the input.
• Length: the rewrite is 0.7 to 1.4 times as long as the input.
• Script: the rewrite does not switch to another alphabet.
• Spelling: the rewrite does not have more unknown or misspelled words than normal human text.

A sample that fails any gate gets the lowest raw reward in its group of 8:

The minimum is taken over all 8 samples, including the ones that failed. So a failed sample never ranks above a sample that passed, and if all 8 fail, they get the same reward and the prompt contributes no gradient.

The gates do not do this alone. RL starts from an SFT model that was trained only on fact-checked pairs, and a KL penalty keeps the policy close to that model.

GRPO

We train with GRPO. For each prompt the model samples 8 rewrites, each gets the gated reward, and the advantage is the reward normalized inside the group:

The group normalization also removes any per-prompt constant from the reward, which is one more reason the cheap one-pair estimate is enough. We add a KL penalty to the SFT model (β = 0.05), train a LoRA (r = 64) for 200 steps at a learning rate of 3e-5 on 2,000 prompts, and stop there.

Generations

Each input is a chatbot answer on a fictional or general topic, the right column is a Smaug Humanizer rewrite of it. Both columns are scored by Pangram.

It does not work every time. Pangram still flags some rewrites, more often for long texts. If a rewrite is flagged, sample again: the outputs vary, and a second try can land differently.

Long texts

The weakest spot is length. AI detectors score a long document in windows, and the document only counts as human if every window does, so a long text has to pass several times in a row. A longer window also gives the detector more evidence. For now we recommend splitting long texts at paragraph breaks into chunks of about 500 words, humanize each and join each humanized output.

Try it

The model is on Hugging Face at abacusai/Smaug-Humanizer. The Usage section of the model card has the prompt, the sampling settings and the code to run it with vLLM or Transformers.