# Can a 7B Model Beat Frontier LLMs at Auditing Move? Inside SuiAudit Pure1

Date: 2026-02-25
Source: https://kxhuang.dev/2026/02/25/SuiAudit_paper/
Description: How SuiAudit Pure1, a 7B model, reached 31.4% accuracy auditing Sui Move — ahead of Claude Opus 4.5 and GPT-5.2 — by curating hard negatives.
Categories: blog
Tags: Sui, Blockchain, AI, Security, Research
Author: Kai Xiu Huang (https://kxhuang.dev)

---
## The Question Behind the Paper

Our team at SuiAudit Lab has published a research paper, [Can Small Models Beat Frontier LLMs? Introducing SuiAudit Pure1: Specialized Auditing via Extreme Low-Resource Fine-Tuning](https://www.techrxiv.org/doi/full/10.36227/techrxiv.177204934.48017152/v1), on TechRxiv (DOI: 10.36227/techrxiv.177204934.48017152/v1). I was a key contributor alongside Chia-Lin Chen and Fang-Yu Lien; the resulting product, [SuiAudit](/2026/03/03/SuiAudit/), is live at [suiaudit.wal.app](https://suiaudit.wal.app/).

The question in the title is the honest version of what we were testing. If a general-purpose frontier model already writes and reads code well, what is left for a 7B-parameter model specialized on one language? Our answer is that the gap frontier models leave on Sui Move is not a gap in capability. It is a gap in what they were trained to expect.

## Solidity Bias Is a Data Problem, Not a Scale Problem

General-purpose LLMs auditing Sui Move are plagued by what we call "Solidity Bias": they mistakenly apply Ethereum vulnerabilities to Sui's object-centric architecture.

My reading is that this is a wrong prior rather than a shortfall of capability, and that framing decides the whole approach: the fix is data that contradicts the prior.

So the dataset is the center of the work. It blends positive vulnerability samples, categorized A-F, with adversarial "hard negative" examples: safe code patterns engineered to trip up broader LLMs. The hard negatives are the part that matters. Training only on real vulnerabilities teaches a model to find something; training against code that *looks* vulnerable under Solidity assumptions but is safe under Sui's teaches it when to say nothing.

## The Pipeline, and How the Numbers Were Obtained

Pure1 is a 7B-parameter model built through a streamlined fine-tuning pipeline: Supervised Fine-Tuning (SFT) for domain mastery, followed by Direct Preference Optimization (DPO) enhanced with Chain-of-Thought reasoning.

Evaluation is on our own Sui-specific benchmark, where Pure1 reached **31.4% accuracy**, ahead of Claude Opus 4.5 (26.6%) and GPT-5.2 (17.2%) — a margin of +4.7% over the strongest generalist model.

Pure1 is built on Mistral-7B-Instruct-v0.2 and quantized to 4-bit NF4, which is what lets it fit on a single NVIDIA T4 (16GB) or RTX 3060 (12GB). It runs fully offline, preserving developer privacy while delivering real-time insights via VS Code and Chrome extensions. We have open-sourced the core infrastructure on GitHub and shared the weights on Kaggle.

## Where It Has Been Used

Pure1 is deployed in production.

The work also took 2nd Place at the 2025 Sui Hackathon Mandarin and 3rd at the Sui Taipei Dev Hackathon.

## What I Take Away

The result I care about is narrow and, I think, generalizable: on a task where the dominant failure is a transferred prior rather than missing capability, curating the negatives beat scaling the model. That is a cheap lever, and it is available to anyone working in a language the frontier labs have little data for.

If you're building on Sui or interested in AI-driven security, the full paper and the live tool are both linked above.