Arch Linux

aegis_eclipse

Member

Registered: 2025-10-26

Posts: 21

Re: TrustSight: automated AUR PKGBUILD review, seeking technical criticism

I’ll add one more thing, because I think seth’s caution is exactly right and I want to be clear that I’m not trying to wave it away.

The warning in the wiki is there for good reason: no tool can replace reading the PKGBUILD, and TrustSight will never claim otherwise. The tool itself prints a note to that effect, and the LLM verdict doesn’t hide the score or the evidence. If someone chooses to ignore the diff entirely, that’s a mistake, but it’s a mistake that’s already happening at scale, with or without TrustSight. What I’m hoping is that for the user who won’t read the PKGBUILD, a loud “Critical: remote execution detected” warning is better than the silence they’re currently getting. And for those who do read, it’s a fast triage step that points them to the right file in seconds. I’m not asking the Arch old guard to lower their standards (and they shouldn't) I’m just trying to build a safety net that catches people before they hit the ground. Imperfect, but better than nothing.

seth

Member

From: Won't reply 2 private help req

Registered: 2012-09-03

Posts: 76,983

Re: TrustSight: automated AUR PKGBUILD review, seeking technical criticism

Yes, in reality people will misbehave and reap the consequences of that and then whine about it.
But you're using the anticipated misbehavior as reason for the feature, ie. the feature is meant to provoke and support the misbehavior.
Don't.

A good tool will do one of three things here:
1. "the update only changes the version and checksum"
2. "line 18 will download a known virus, please report the package"
3. "line 12,13 & 15 look shady because [reason] and you need to look into that"

The idea that "oh, it's only 30% likely malicious so it's gonna be fine" is nuts.

---

How to upload text · How to boot w/o GUI · Disable Windows Fast-Start! · Fix your xinitrc

aegis_eclipse

Member

Registered: 2025-10-26

Posts: 21

Re: TrustSight: automated AUR PKGBUILD review, seeking technical criticism

A good tool will do one of three things here:

   1. "the update only changes the version and checksum"

   2. "line 18 will download a known virus, please report the package"

   3. "line 12,13 & 15 look shady because [reason] and you need to look into that"

The idea that "oh, it's only 30% likely malicious so it's gonna be fine" is nuts.

I think we're closer than it looks, because TrustSight already does exactly those three things:

1. "Version bump, no structural changes": 81% of benign updates score 0 and produce exactly that verdict.

2. "Line 18 will download a known virus": When a CRITICAL rule fires (R001: curl | bash, R012: prompt injection, R013: bidi attack), the score goes to 80–100 and the output says something like "Remote script execution detected on line 18." That's case 2.

3. "Lines 12, 13, 15 look shady because [reason]": When a mix of lower-severity rules or novelty signals fire, the inspect view shows exactly which lines, which rules, and why. The score is the weighted sum of those reasons, not a probability. A score of 35 means "there are unusual patterns here, look at lines 12 and 15," not "35% chance of malice." I can see how calling it a "risk score" might suggest a probability, and I'll tighten the language in the docs to make that distinction clearer.

On the misbehaviour point: I'm not building a tool to encourage blind trust. I'm building a tool for the reality that millions of people already install from the AUR without reading a single line. That's not something I can fix, but I can give them a red flag that says "CRITICAL: remote execution on line 18: do not install until you understand this." That's better than the silence they get now. It's harm reduction, not endorsement. I do take your warning seriously. The tool already prints a note that manual review is required, and I'll make sure the output and docs are crystal clear that the score is input, not verdict. If there's a specific place you think the messaging is still ambiguous, I'd genuinely like to fix it.

I want to be clear about who TrustSight is for. The Arch wiki's advice to read every PKGBUILD is correct, and for the experienced users who built this community, it's both sufficient and non‑negotiable. No tool replaces that. TrustSight isn't trying to replace manual review for those users; it's a triage step that saves them time by dismissing the trivial updates so they can focus on the suspicious ones.

But the AUR's user base has changed. Today, a large portion of AUR users come through Arch‑based distros aimed at gamers and newcomers (CachyOS, SteamOS, EndeavourOS, or even newbie Arch users), and many of them aren't developers. They don't know what a PKGBUILD is, and they're not going to learn it overnight. For that audience, the choice isn't between “use TrustSight” and “read the PKGBUILD”; it's between “see a loud red flag when something is obviously wrong” and “run yay -Syu completely blind.” TrustSight is built for that second group: to be the warning they otherwise wouldn't get. It's harm reduction, not a replacement for expertise. The tool does not, and will never, claim to be a substitute for reading. It just tries to put a safety barrier where there's currently nothing at all.

carlosplanchon

Member

From: Dolores, Uruguay

Registered: 2020-09-23

Posts: 1

Website

Re: TrustSight: automated AUR PKGBUILD review, seeking technical criticism

Hi! I think this is a really interesting design problem to explore.

I think Seth's model is that the analyzer should report concrete findings:

- only the version and checksum changed
- a known malicious payload was found
- specific lines look suspicious for specific reasons

The user, or another tool, then decides what those findings mean.

Aegis's concern is different: many users will not inspect the PKGBUILD at all, so some user-facing layer should turn those findings into an obvious warning.

Those goals do not necessarily conflict, but they may belong in separate layers:

For example: a policy-neutral core that emits structured findings and evidence,
and an optional triage layer that applies a scoring or warning policy for less experienced users.

That would allow experienced users, distributions and third-party tools to consume the same findings without having to accept one particular scoring model.
Other frontends could implement stricter or different policies on top of the same core.

The architectural question then is not whether inexperienced users deserve a warning (they clearly do) but whether evidence collection and the final risk judgement should be owned by the same component.

Nice work!

---

Any sufficiently advanced technology is indistinguishable from biology.