Research

Smaug AI Image Detector: State-of-the-Art Open-Source Detection of AI-Generated Images

Bar chart: AI images caught on real-world images at a 5% false-alarm rate. Smaug AI Image Detector 86.7%, OpenFake-SwinV2 77.4%, detectra-openfake 67.0%, Thermostatic-Frontier 60.8%.

For most of the history of photography, a picture was evidence. That is no longer true. An image produced by a generator is now indistinguishable from a photograph to the human eye, and it travels the way a photograph does: re-saved, resized, recompressed and shared until nobody remembers where it came from. Fabricated images are already used for fraud, for impersonation, to manufacture evidence and to cast doubt on real events. This is not a future risk. It is happening at scale today, and an ordinary person has no reliable way to tell.

We build AI at Abacus.AI, and we believe that carries an obligation: the problems AI creates are problems we should help solve, with AI. Telling a real photograph from a generated one should not be a closed model or a paid service. It should be a capability anyone can run on their own machine, against their own images, without trusting a third party.

So today we are releasing Smaug AI Image Detector: an open-source model that tells you whether an image was made by an AI image generator. It is the strongest open-source detector on real-world images, and it is small enough to run on a laptop.

State of the art on real-world images

We evaluated Smaug AI Image Detector against ten open-source AI image detectors, including the most downloaded models on Hugging Face and every recent release with public weights, on the same 26,070 images from three public benchmarks.

Bar chart: AI images caught on real-world images at a 5% false-alarm rate. Smaug AI Image Detector 86.7%, OpenFake-SwinV2 77.4%, detectra-openfake 67.0%, Thermostatic-Frontier 60.8%.
AI images caught on real-world (Reddit) images, holding every detector to a 5% false-alarm rate.
Smaug AI Image DetectorNext best open-source detector
OpenFake Reddit (real-world images), AUC0.9730.947
AI images caught at 5% false alarms (Reddit)86.7%77.4%
OpenFake Core, AUC0.9910.989
Average AUC across all three benchmarks0.9820.962

The gap is largest where it matters most: images as they are actually posted online. Held to the same 5% false-alarm rate, Smaug AI Image Detector catches 86.7% of AI-generated images on Reddit, against 77.4% for the next best detector. It is also precise: at its default threshold it flags only 0.4–3.0% of real photographs as AI.

It covers the generators people use today, including GPT-Image 2, Nano Banana Pro, Midjourney 7, Seedream 5, FLUX.2, Z-Image, Recraft, Ideogram, ERNIE-Image, and frames from Sora 2 and Veo 3.

A dedicated vision model, 41% smaller than its base

Smaug AI Image Detector is a fine-tune of Google’s EmbeddingGemma-2. We streamlined it into a vision-only architecture by removing its 305M-parameter audio tower, which brings the model from 744M to 439M parameters with no loss in accuracy.

That size makes it practical to run anywhere: on a single consumer GPU with about 1 GB of memory, on Apple Silicon Macs via Metal, or on a plain CPU. Images can be checked locally, without sending them to a third-party service.

Try it

from huggingface_hub import snapshot_download
import sys
path = snapshot_download("abacusai/Smaug-AI-Image-Detector")
sys.path.insert(0, path)
from smaug_image_detect import SmaugImageDetect
detector = SmaugImageDetect.from_pretrained(path)
detector("photo.jpg") # -> 0.97, the probability the image is AI-generated

The model is available now on Hugging Face under the Apache 2.0 license, and joins the Smaug family on our research page.

Related posts
Research

Abacus.AI at NeurIPS 2023

ResearchTech

Closing the Gap to Closed Source LLMs - 70B Giraffe 32k

Research

Abacus.AI at NeurIPS 2022

ResearchTutorial

Debiasing Facial Prediction Models with Adversarial Fine-Tuning

Leave a Reply

Skip to content

Discover more from The Abacus.AI Blog

Subscribe now to keep reading and get access to the full archive.

Continue reading