A couple of days ago I saw a reference to an interesting piece of research from Meta AI about "Unslopping AI". After reading it, I came to the unavoidable conclusion that the blog reads a lot like AI slop. (For those who are interested, I'll leave some supporting evidence at the end of this post.) As a recently established slop warrior, I went on to decipher the blog and understand how people try to make AI outputs more readable.

This post is meant to be:

1. a less academic and hopefully a more fun-to-read version of the original blog post;
2. a place to share my own critique of the researchers' assumptions and approach.

I will dedicate most of my time and space to the methodology and not the results. Also, if you're interested in the bits and bytes of the experiments, you should definitely read the original source. You're welcome to discuss it with me on X.

The problem

First, the problem: AI slop is real. AIs are machines that choose the most probable next token based on what they've seen so far as input. They're trained on mediocre texts, and thus produce mediocre texts (I claim their generations are worse than mediocre, but that's a matter of taste). In practice (and this is a popular opinion), AIs write in a way that's exhausting to read, difficult to understand and does not resemble how humans interact.

Another problem is that good writing isn't a verifiable task: writing a Python function which multiplies two integers has a simple correctness test. But there's no test that passes/fails based on the mere quality of an input text.

If no such test exists, then we should use humans for scoring (RLHF), right? The issue with this approach is that the text will only get as good as what the grader (also called annotator) thinks is good, and graders who can discern excellent writing from mediocre aren't scalable.

Solution TL;DR

Their basic idea was to replace expert annotators with an LLM that is just as good, because the latter is scalable.

The solution at a high level:

• First, train LLM judges to be really good at judging*, i.e. grade excellent human writing higher than usual AI-generated text
• Then use these judges' scoring system - their "rubrics" - to train writer LLMs in reinforcement learning.

*"train LLM judges" isn't very accurate here, because what gets improved is their rubrics (the "checklists" with which they assess) and not the models.

Starting point: what do LLMs prefer?

The authors wanted to first find out how LLMs judge, before they're tuned against a certain rubric. They took an academic paper from which they dropped a section. Then they made an LLM writer complete the missing section. Finally they gave both - the original section and the new AI-written one - as competitors to an LLM judge.

There are two scoring approaches in this scenario:

1. Hand the two candidates over to the LLM judge side-by-side and let the judge choose.
2. Score each candidate against a rubric - a checklist of parameters designed to evaluate good writing for the specific section.

How are rubrics generated? The authors asked GPT-5.6 to create a meta-prompt: a prompt that takes one instance of a paper that's missing a section, and outputs a rubric that can be used to evaluate a candidate section.

The rubric approach is of course the more interesting one. A rubric gives you something to edit: you can't easily change a judge's taste, but you can change its checklist. Bad rubrics make bad judges, and bad judges leave you with sloppy generations - my nightmare.

Claude Opus-4.8 and GPT Sol-5.6 were used in both roles - judge and writer.

The result: AI judges preferred AI writing (see diagram).

The two-phase process

The authors' approach to getting AI to write better is divided into two phases:

1. Obtaining rubrics, i.e. learn how to generate, per test case, a rubric that will favor expert writing over model writing.
2. Training models with RL using the rubrics.

Obtaining rubrics

This phase (which the authors called eXpert Aligned Rubrics, or XAR) consists of the following steps:

1. Collect samples of well-above-average human writing - academic papers, novels of award-winning authors and Wikipedia entries. Split each sample into context (the text minus a part of it) and continuation (the omitted part).
2. For each context, create a competing pair of continuations (model-generated vs. human-original).
3. The pairs from (2) make training data. This data is used to learn to generate rubrics that maximize the gap between the candidates' scores.

Type	Context	Continuation
Paper	Original paper with one section omitted	The omitted section
Novel	The story up to a cut-off point	What follows the cut-off
Wiki page	Title and section heading	The section itself

Let's double click on (3).

• We start with a prompt** that says something like "here's a paper with a missing part P. create a list of parameters that estimate how well P is written".
• A rubric is created in response to this prompt.
• An LLM judge then uses the rubric to score each of the two candidates. The "human-model gap" is simply score(human)-score(model).
• A "refinement" loop runs several times, each time modifying the prompt, aiming to maximize the human-model gap (that is, make the judge strongly favor the human-written candidate).

**They call it a meta-prompt, which I take as "a prompt general enough to create rubrics for multiple cases".

To make a long story short, this method seems to work: the learnt rubrics push the judge from the AI side to the human side.

Training models to write well

This part (which the authors call Reinforcement Learning with eXpert Aligned Rubrics, or RL-XAR) performs the following steps iteratively:

1. Learn optimal rubrics (optimal meaning "highly favoring expert-quality writing").
2. Perform RL using the learnt rubrics.
3. Repeat until (1) can no longer find a quality gap.

This process is done for all three categories - computer science papers, novels from award-winning authors and high-quality Wiki pages.

I'm omitting the details about how many iterations they ran, how many were used for training vs. evaluation and how they eventually scored the trained writer model.

For the academic papers experiment, they show that a Qwen3.5-27B trained with RL-XAR performs better than frontier models (Opus 5.5, GPT-5.6). In the end, they let the original paper's author decide between the continuation of the baseline model and that of the RL-XAR-trained one. The latter won 16:2.

Putting model use in order

Models in the process take the following roles:

• Rubric generator: a model that takes a meta-prompt and outputs a rubric for a certain context (i.e. missing text)
• Meta-optimizer: a model that edits a meta-prompt to drive it to prefer expert-quality writing
• Judge: a model which,
  • during rubric-learning, compares AI and human continuations
  • during reinforcement learning, rewards a writer based on a rubric
• Writer

What optimal rubrics valued

Initial rubrics favored new paragraphs which covered substantial parts of the whole paper, repeated claims, methods, and results. The learnt rubrics (optimized ones) valued conciseness, focus on current topic and relevance.

Now take a look at this quote from the research blog and tell me a human wrote it (I mean, 99% of the texts that contain "prose" are AI slop):

Over the iterations the optimizer rewrites the guidance that mis-scores expert prose: docking the human for principled omissions, crediting a compressed summary as fluency, and mistaking surface polish for craft. The learned rubrics thus also tend to reward sectional ownership (i.e., not a miniature of the whole paper), disciplined selection, economy, and precise on-scope detail rather than breadth of coverage.

Definitely comprehensible. If you read it 28 times.

My personal thoughts

Human graders can be just humans

One of the claims undermining the use of RLHF is that "excellent annotators don't scale", namely that it's impossible (or costly) to get many human graders who can truly distinguish expert writing from mediocre writing. I claim we don't need experts. We need... AI users. Every one of them can recognize AI slop from a distance. Don't ask them "which answer is better?" This brought us to where we are today. Ask them "which answer reads like slop?"

Penalizing slop won't make AI write like a Nobel-winning writer. But it might make it write like a normal person, and honestly, that's all I'm asking for.

Academic papers don't necessarily represent "good writing"

We don't need AI to write as well as a CS professor. Personally, I don't even perceive academic papers as state-of-the-art writing. I even sometimes feel that model responses resemble the style of a tough academic paper. Perhaps what we need is the language found in beginner-friendly writeups, recipe blogs, or other resources which are inherently easy and flowy to read.

A blog post about unslopping AI should unslop itself

Two more examples, apart from the one I highlighted earlier:

1. A model trained on one rubric can learn to satisfy that rubric alone, so we evaluate every writer against all three sets of meta-optimized rubrics and rank by the minimum of the three human-normalized scores—the worst rubric, allowing no cherry-picking.

   This may be understandable to some readers, but I bet there are a thousand ways to make the claims much easier to consume.

2. Because the meta-prompt is length-bounded, the criteria settle into a small, stable, relatively paper-independent set that tends to sharpen what it rewards instead of accumulating requirements.

   Come on. You need a GPU instead of a brain for this stuff.