TrustSight: automated AUR PKGBUILD review, seeking technical criticism

aegis_eclipse · 2026-07-24 19:05:31

A while ago, I got really tired of reading each PKGBUILD every time I had to update AUR packages, so i created a local solution: a set of rules that check for 80% of AUR attack vectors. For a couple of months, it stayed on my (and some friends) computer, with basic rules and manual checking. While talking in discord, a rando told he also got tired of reading the AUR, and told me to package it, so i did.

I’ve built a CLI tool that automates the initial screening of AUR PKGBUILD updates. The core idea: 80% of updates are simple version bumps, but manually reading every diff across dozens of packages is error‑prone. TrustSight flags the structurally suspicious ones so human review can focus where it matters.

Key properties:
- Works entirely offline, no execution of PKGBUILD
- Deterministic risk score (0–100) from 39 regex‑based rules covering remote execution, checksum integrity, privilege escalation, unicode homoglyphs, etc. Variable expansion catches obfuscated payloads.
- Pre‑seeded novelty database eliminates cold‑start “unknown domain” noise.
- Optional LLM only translates the final score into a short verdict; it cannot alter the score.

Corpus measurement (3322 diffs from 172 packages):
- 80.9% of benign updates scored 0.
- p95 score on benign = 35, with 2.8% scoring above 50.
- 100% recall on the critical attack patterns in the test suite.

I’m posting to invite technical criticism before freezing the scoring model. In particular I’m interested in:
- What genuine AUR attacks would this approach miss? (e.g., malicious patches applied in prepare(), tampered tarballs from a clean PKGBUILD).
- If you run it on your own installed AUR packages, do you encounter false positives that undermine trust? (Reports with package names and scores appreciated.)
- The LLM integration: is it a distraction, or does a human‑readable verdict add value? It does not affect the deterministic scoring.
- The seed database is bundled as a 13MB compressed SQLite file; it auto‑imports on first run (~10 seconds). Is that acceptable for an AUR tool?
- The scoring uses weighted evidence tiers (structural, domain classification, novelty, verification). Are the weights calibrated appropriately? I’m open to suggestions based on real‑world experience.

The project is MIT‑licensed, Python 3.12+, and installable via pip install trustsight. The repository is at https://github.com/emiliano-go/trustsight.

I welcome blunt, technical feedback. My goal is to make this useful for the community, not to defend a pet project. If the approach is fundamentally flawed, I want to hear that now. Thanks in advance!

Lone_Wolf · 2026-07-24 19:51:06

Sorry, I'm not going to install pipx/pip (or any other python package manager) to install this.

It looks like you are working on a PKGBUILD for it .

# Maintainer: Emiliano Gandini Outeda <emiliano.gandini@protonmail.com>
# Contributor: Emiliano Gandini Outeda <emiliano.gandini@protonmail.com>

It doesn't make sense that you're maintainer AND contributor .

pkgdesc=''A CLI tool to vet AUR package updates before install'

I use git clone / git pull to retrieve the latest PKGBUILD and related files directly from AUR repos, then build with makepkg or clean chroot tools.
Is this tool usable for me ?

NidoBr · 2026-07-24 20:44:38

Hello, sorry if this is a naive question, but the attack surface of a PKGBUILD is, in practice, the same as that of any Bash script executed by the user. Wouldn't a simpler tool that only checks for suspicious patterns be enough to cover most cases?
I have an example if you'd like to take a look:https://codeberg.org/NidoBr/aurcheck

Additionally, I believe the main attack vector is not the PKGBUILD itself, but the AUR software supply chain. Verifying the authenticity of commits and remote repositories, and ensuring that legitimate source code has not been compromised, seems like a much bigger problem and not an easy one to solve.

Even a seemingly "clean" PKGBUILD may simply be downloading code from a repository that has already been compromised.

aegis_eclipse · 2026-07-24 21:59:26

NidoBr wrote:

Wouldn't a simpler tool that only checks for suspicious patterns be enough to cover most cases? I have an example if you'd like to take a look:https://codeberg.org/NidoBr/aurcheck

I’ve looked at it and there’s definitely overlap in spirit. But there are a few places where TrustSight tries to go a bit further than pattern‑matching alone:
- Variable expansion: a raw grep for curl … | bash won’t catch curl $url | $shell or a source=("$(curl …)") hidden behind an indirect assignment. TrustSight resolves shell variables (multiple passes) before matching, so it catches obfuscated versions.
- Checksum integrity it’s not just looking for SKIP. It detects a checksum that changed without a corresponding version bump (which could indicate a silently replaced tarball), or a checksum that’s only partially emptied while sources remain.
- Novelty awareness: the tool knows when a domain or maintainer appears for the first time globally across AUR history. That turns a harmless source=("https://random‑host.xyz/…") from “no pattern matched” into a high‑visibility warning, even if the line looks innocent.
- And, most important: Deterministic scoring. I wanted a single number, traceable to every piece of evidence, that a CI hook or a non‑expert could use. A list of matched patterns is useful; a calibrated score that combines structural risk, domain trust, novelty, and verification signals is a different kind of output.

That said, you’re right that a “clean” PKGBUILD fetching from a compromised upstream is a huge problem, and TrustSight does not solve that. It can’t audit the source code inside a tarball, and if the PKGBUILD uses the correct checksum for a tampered file, the tool will see no anomaly. I’ve tried to be painfully clear about that limitation in the docs. It’s a PKGBUILD‑level triage tool, not a supply‑chain checker.

Lone_Wolf wrote:

It doesn't make sense that you're maintainer AND contributor .

The # Maintainer / # Contributor duplication in the draft is a copy‑paste slip, thanks for the reminder

Lone_Wolf wrote:

I use git clone / git pull to retrieve the latest PKGBUILD and related files directly from AUR repos, then build with makepkg or clean chroot tools.
Is this tool usable for me ?

Yes! TrustSight doesn’t care how you build AUR packages. It only cares that they’re installed and recorded as foreign packages (pacman -Qm).

TrustSight does its own bare git clones (stored in ~/.cache/trustsight/repos/) purely to compute diffs between the last commit you reviewed and the latest HEAD on the AUR. It never touches your build directories, never runs any PKGBUILD code, and never interferes with your manual clones.

So yes, the tool is completely usable for you. The only thing you’d need to do is run trustsight review before you decide which packages to rebuild. It will list all outdated AUR packages on your system (based on pacman -Qm and the AUR’s RPC), then show you a risk‑scored summary. You can then manually go and update only the ones that need a closer look.

aegis_eclipse · 2026-07-24 22:03:37

Also:

NidoBR wrote:

Additionally, I believe the main attack vector is not the PKGBUILD itself, but the AUR software supply chain.

Yes, that's true, and I'm working on some new rules, based on the latest supply chain attacks, that may allow to better detect them, though it's never gonna be perfect. The xz case involved a trusted maintainer who slowly modified the build system to extract malicious test data. The PKGBUILD itself never changed suspiciously; only the upstream tarball (signed and checksummed) was malicious. TrustSight cannot detect that. No static PKGBUILD analysis can, because the PKGBUILD is completely honest.

seth · 2026-07-25 06:02:54

"automated"
"tired of reading each PKGBUILD every time I had to update AUR packages, so i created a local solution fallacy"
"Optional LLM only translates the final score into a short verdict"

https://wiki.archlinux.org/title/Arch_U … e_PKGBUILD

Warning
… A few tools, such as traur and ks-aur-scanner, are available to assist users in scanning PKGBUILD content; however, they are not substitutes for careful manual verification.

It must be understood that these tools must be considered input, not verdict.
They only provide an autopilot in the most trivial cases (version + checksum change) and in that case looking at the git diff is hardly more effort than running a review tool and its result.

Don't mis-advertise your tool.

NidoBr · 2026-07-25 11:27:50

This is why Seth mentioned that I was talking about a simpler alternative. I believe this is a better approach.
In the example I provided, I made it clear in the README that it is intended only to assist the user and is not meant to replace manual verification.

Lone_Wolf · 2026-07-25 12:23:36

aegis_eclipse wrote:

recorded as foreign packages (pacman -Qm).

They are not foreign but stored in my own custom local repository .

Check AUR helpers > Comparison_tables and you'll see many aur helpers use such a repo also.

If you want TrustSight to be useful for users of those also you need to add support for checking based on a specific repo. pacman -Sl <repo-name> can help with getting the packages in that repo .

aegis_eclipse · 2026-07-25 13:39:13

seth wrote:

"automated"
"tired of reading each PKGBUILD every time I had to update AUR packages, so i created a local solution fallacy"
"Optional LLM only translates the final score into a short verdict"
It must be understood that these tools must be considered input, not verdict.
Don't mis-advertise your tool.

You're right, and I'll be more careful with wording. The wiki warning is the standard I want TrustSight to be measured against. The score is input, not verdict it's a triage number that says "this diff has structural signals worth a human look," not "this package is safe/unsafe." I should have led with that, and I'll update the post and the README to make that distinction more clear.

On the trivial case: yes, a single version‑bump diff is quick to read. But when 25 of 30 outdated packages are trivial version bumps, the value isn't in reading one diff faster, it's in not having to open and scan 25 diffs at all. TrustSight can dismiss those 25 in a single table row each, and then you manually review the five that scored >0. That's the workflow I built it for.

The "automated" language was sloppy on my part.

aegis_eclipse · 2026-07-25 13:42:23

NidoBr wrote:

This is why Seth mentioned that I was talking about a simpler alternative. I believe this is a better approach.
In the example I provided, I made it clear in the README that it is intended only to assist the user and is not meant to replace manual verification.

Your aurcheck makes that distinction clearly in its README. TrustSight tries to do the same the inspect command gives a full evidence breakdown with the explicit note that the score is a risk indicator, not a safety guarantee. But if the initial post or the CLI output gave the wrong impression, that's on me, and I'll update it (english is not my first language).

The difference I'm aiming for isn't "TrustSight is a replacement for manual review," it's "TrustSight gives you a structured, reproducible reason to look at this package rather than that one." The variable expansion, checksum‑change detection, and novelty awareness are there to catch things a quick skim might miss. But the final decision is always the user's. It's a triage tool.

aegis_eclipse · 2026-07-25 13:45:10

Lone_Wolf wrote:

They are not foreign but stored in my own custom local repository.
Check AUR helpers > Comparison_tables and you'll see many aur helpers use such a repo also.
If you want TrustSight to be useful for users of those also you need to add support for checking based on a specific repo. pacman -Sl <repo-name> can help with getting the packages in that repo.

I hadn't considered the local‑repo workflow. TrustSight would miss them completely because once packages are moved into a custom repo and installed from it, they no longer appear in pacman -Qm.
Adding a --repo <name> flag that uses pacman -Sl <repo> for discovery is a easy thing to do, and it would also help users of AUR helpers that manage their own local repos. I'll add that feature in next release. Ty!

aegis_eclipse · 2026-07-26 16:42:36

Fixed some issues in the latest release.

Since v0.4.0: --repo/--foreign/--all-repos flags and config-driven discovery; --json per command; CLI migrated from argparse to typer; 49 new rules across code, dependency, naming, temporal, install, build, and maintainer categories (R039–R075, D001–D004); 6 security fixes including message-prefix bypass, line-continuation bypass, variable-resolution dead inside functions, function-scoping escape, .. cache-root deletion, and unescaped RPC injection; 3 crash-bug fixes in the analysis pipeline; corpus made reproducible with a lock file and deduplicated strata (3332→3246); drift-detection and mirror-integrity CI; auto-release workflow for AUR PKGBUILD; seed expanded to 209,909 dependency names; Python floor lowered to 3.10 with CI matrix 3.10–3.14; python -m trustsight support; fire rates documented for all rules; tool now at 702 tests and 60 rules (D001–D004 the only experimental ones).

Working on some rules based on package managers rn.

Last edited by aegis_eclipse (2026-07-26 16:43:06)

Trilby · 2026-07-26 20:43:25

I love the idea, but I'm confused by the LLM option. An LLM could (theoretically) provide a summary of the risks or questionable items in an AUR package in a prose form. But if your statement is correct that the LLM just takes the score (0-100) and turns that into some prose description of risk level, then why on earth would you use an LLM for that? That's just an array of text strings that you print out the n'th element from.

aegis_eclipse · 2026-07-26 21:54:27

Trilby wrote:

I love the idea, but I'm confused by the LLM option. An LLM could (theoretically) provide a summary of the risks or questionable items in an AUR package in a prose form. But if your statement is correct that the LLM just takes the score (0-100) and turns that into some prose description of risk level, then why on earth would you use an LLM for that? That's just an array of text strings that you print out the n'th element from.

The LLM doesn't just paraphrase the score, it doesn't even receive the raw number. Only the risk label ("Low"/"Medium"/"High"/"Critical") appears in the prompt. What it actually gets is the full PackageFact: diff summary, triggered rules with their severity/weight/reason, source URL list, novelty context, and maintainer change flag.The LLM is asked to write exactly two sentences: what technically changed in the build process, and why that leads to the given risk level.

So for a package where R001 fired, it might produce:

A curl | bash invocation was added to the build function. This is a critical remote-execution signal because the payload runs unverified shell code from an external server.

The fallback (when the LLM is disabled, unreachable, or its output fails sanity checks) is exactly the array-of-strings approach you described: a template that concatenates rule names and severities.

It works, but it's brittle: "Version bump. Signals: R001 (CRITICAL), R004 (INFO)", and doesn't synthesize the why. The LLM simply turns that same structured evidence into a short, human-readable summary that connects the dots. It's optional (TrustSight works fully offline without it), and it cannot change the deterministic score in any way.

The trust decision comes from the score breakdown and the rule system, the LLM is just commentary.

aminepro · 2026-07-26 21:54:29

Trilby wrote:

I love the idea, but I'm confused by the LLM option. An LLM could (theoretically) provide a summary of the risks or questionable items in an AUR package in a prose form. But if your statement is correct that the LLM just takes the score (0-100) and turns that into some prose description of risk level, then why on earth would you use an LLM for that? That's just an array of text strings that you print out the n'th element from.

Yeah, if the tool is intended for ppl who would read the PKGBUILD anyway the LLM is absurd.
but maybe the LLM itself is a feature that you may not be able to easily implement yourself, and can be used for aesthetics?

Last edited by aminepro (2026-07-26 21:58:09)

aegis_eclipse · 2026-07-26 21:58:36

aminepro wrote:

Yeah, if the tool is intended for ppl who would read the PKGBUILD anyway the LLM is absurd.
but maybe the LLM itself is a feature that you may not be able to easily implement yourself, and can be used for aesethetics?

if someone already reads every PKGBUILD line‑by‑line and knows exactly what “R039 fired with CRITICAL” means, the LLM verdict adds nothing. For that user, it’s just a decorative sentence they can safely ignore (or disable entirely).

But TrustSight’s triage output is also meant for two other groups:

- Someone who maintains 40+ AUR packages and wants a two‑sentence summary of why a package scored 65 before they decide whether to open the diff. The LLM translates “R001 (CRITICAL) + D001 (HIGH) + novel domain” into plain English so the decision to investigate is faster.
- Less experienced users who know they should read PKGBUILDs but aren’t yet fluent in all the attack patterns. For them, a sentence like “A hidden network fetch was added inside the build function” is immediately actionable, even if they don’t recognise the rule IDs.

The deterministic scoring and the evidence breakdown remain the authority. The LLM is just an optional readability layer on top. Think of it as the difference between git log --oneline and git log, both give the same truth, but one is faster to scan when you’re in a hurry.

aminepro · 2026-07-26 22:02:58

Your replies made me understand how it works better

Last edited by aminepro (2026-07-26 22:04:51)

aminepro · 2026-07-26 22:50:26

I got a little problem while installing the program
I aborted the installation when pacman was installing.
When I tried again these errors show up https://paste.c-net.org/BrooksKiddo.
I tried removing the packages in PKGBUILD (those that weren't already},clearing pacman cache and redownloading chttps://github.com/emiliano-go/trustsight.git for a clean installation but it still fails.

Last edited by aminepro (2026-07-26 23:17:59)

aegis_eclipse · 2026-07-26 23:15:37

That's a fuckup on my end, wrong pkgbuild setup.

Quick workaround: makepkg -si --nocheck

I'll fix it asap, I'm afk

aegis_eclipse · 2026-07-26 23:18:36

Changing the check to this should fix it though

check() {
  cd "$_pkgname-$pkgver"
  python -m venv --system-site-packages test-env
  test-env/bin/python -m installer dist/*.whl
  test-env/bin/python -m pytest
}

aminepro · 2026-07-26 23:19:24

aegis_eclipse wrote:

That's a fuckup on my end, wrong pkgbuild setup.
Quick workaround: makepkg -si --nocheck
I'll fix it asap, I'm afk

lol, it worked

aegis_eclipse · 2026-07-26 23:24:41

I have a src-layout project (src/trustsight/). During check(), the CWD is the extracted source dir, so src/ isn't on sys.path and the wheel hasn't been installed anywhere yet. package() is what installs it, and that runs after check(), and check() fails.

aegis_eclipse · 2026-07-26 23:35:56

Any bug/issue/feature reports are welcome

seth · Yesterday 06:42:03

why a package scored 65 before they decide whether to open the diff

seth wrote:

It must be understood that these tools must be considered input, not verdict.

Anybody deciding to not open the diff (or PKGBUILD) is doing it wrong.

aegis_eclipse · Yesterday 13:18:08

seth wrote:

Anybody deciding to not open the diff (or PKGBUILD) is doing it wrong.

I don’t disagree with the ideal. If everyone who used the AUR had the time, skill, and discipline to manually review every PKGBUILD line‑by‑line, TrustSight would be unnecessary. But that’s not the AUR’s user base anymore.

With the rise of Arch‑based distros aimed at gamers and newcomers (CachyOS, SteamOS, Garuda, EndeavourOS) a large and growing number of AUR users are not developers. They’re not fluent in Bash. They don’t know what a PKGBUILD is, let alone how to audit one. And yet they are using the AUR. Telling them “just read the PKGBUILD” is like telling a driver to rebuild their engine before turning the key. Correct in theory, but it won’t happen at scale. TrustSight doesn’t replace manual review. It’s a harm‑reduction tool for a user base that is already exposed. For the expert, it saves time. For the non‑expert, it provides a structured, auditable first line of defence that says “this package has three CRITICAL signals, do not install/update until you understand them.” That’s a lot better than blind trust. The wiki warning is right: tools like this are input, not verdict. But for someone who can’t distinguish a normal PKGBUILD from a malicious one, even a partial input, with a clear risk label and a pointer to the offending line, is an improvement. I’d rather they see a red “Critical” badge and pause than run yay -Syu and hope for the best, as lots do.

So yes, the gold standard is manual review. But TrustSight is built for the real‑world AUR, where millions of users will never reach that standard, and something is better than nothing.

Arch Linux

#1 2026-07-24 19:05:31

TrustSight: automated AUR PKGBUILD review, seeking technical criticism

#2 2026-07-24 19:51:06

Re: TrustSight: automated AUR PKGBUILD review, seeking technical criticism

#3 2026-07-24 20:44:38

Re: TrustSight: automated AUR PKGBUILD review, seeking technical criticism

#4 2026-07-24 21:59:26

Re: TrustSight: automated AUR PKGBUILD review, seeking technical criticism

#5 2026-07-24 22:03:37

Re: TrustSight: automated AUR PKGBUILD review, seeking technical criticism

#6 2026-07-25 06:02:54

Re: TrustSight: automated AUR PKGBUILD review, seeking technical criticism

Warning

#7 2026-07-25 11:27:50

Re: TrustSight: automated AUR PKGBUILD review, seeking technical criticism

#8 2026-07-25 12:23:36

Re: TrustSight: automated AUR PKGBUILD review, seeking technical criticism

#9 2026-07-25 13:39:13

Re: TrustSight: automated AUR PKGBUILD review, seeking technical criticism

#10 2026-07-25 13:42:23

Re: TrustSight: automated AUR PKGBUILD review, seeking technical criticism

#11 2026-07-25 13:45:10

Re: TrustSight: automated AUR PKGBUILD review, seeking technical criticism

#12 2026-07-26 16:42:36

Re: TrustSight: automated AUR PKGBUILD review, seeking technical criticism

#13 2026-07-26 20:43:25

Re: TrustSight: automated AUR PKGBUILD review, seeking technical criticism

#14 2026-07-26 21:54:27

Re: TrustSight: automated AUR PKGBUILD review, seeking technical criticism

#15 2026-07-26 21:54:29

Re: TrustSight: automated AUR PKGBUILD review, seeking technical criticism

#16 2026-07-26 21:58:36

Re: TrustSight: automated AUR PKGBUILD review, seeking technical criticism

#17 2026-07-26 22:02:58

Re: TrustSight: automated AUR PKGBUILD review, seeking technical criticism

#18 2026-07-26 22:50:26

Re: TrustSight: automated AUR PKGBUILD review, seeking technical criticism

#19 2026-07-26 23:15:37

Re: TrustSight: automated AUR PKGBUILD review, seeking technical criticism

#20 2026-07-26 23:18:36

Re: TrustSight: automated AUR PKGBUILD review, seeking technical criticism

#21 2026-07-26 23:19:24

Re: TrustSight: automated AUR PKGBUILD review, seeking technical criticism

#22 2026-07-26 23:24:41

Re: TrustSight: automated AUR PKGBUILD review, seeking technical criticism

#23 2026-07-26 23:35:56

Re: TrustSight: automated AUR PKGBUILD review, seeking technical criticism

#24 Yesterday 06:42:03

Re: TrustSight: automated AUR PKGBUILD review, seeking technical criticism

#25 Yesterday 13:18:08

Re: TrustSight: automated AUR PKGBUILD review, seeking technical criticism

Board footer