This is the kind of stuff that keeps me up at night. Claude, ArtCraft, tons of other auto-downloading tools, are rolling 10'000's of code changes a month. If one compromised update snuck this guy in, it's kind of game over. The windows of opportunity for an attack like this seem to be widening the faster we "develop" software.
Makes me want to have like 5 separate computers:
- one for banking only
- one for development only
- one for web browsing only
- one for gaming only
- one for socials only
How else can you stay safe against a threat that exfiltrates your entire digital life in like 10 seconds?
Before, people were responsible for the dependencies they were importing.
With npm and all the javascripters flooding the market, it became normal to be irresponsible "because everybody is doing it".
Fast-forward a couple of years, now everything is web based, even the stand-alone app version. No one knows what is running on own computer, even the Devs themselves don't know what the are asking you to run...
With a pinch of vibe coding on top, and thats the end.
We took 40 years for our industry to be taken seriously, it's all gone now.
Shared code repositories for OSS are much older than npm. See at least maven central for Java.
What makes npm a more relevant target is a much smaller standard library and thinner dependencies, leading to more chances that a random project will have code of suspicious provenance. The vast majority of what a backend Java project depends on was probably written either by the same 3 companies, or large, established OSS repos. So this is relatively solvable if people in steering committees decided they cared. It's improving, but I'd argue not fast enough.
Generally we don't hear much about attacks in Java like that. But they probably exist but python and javascript are targeted mainly because everyone uses them. They cover a lot of ground.
That XKCD comic is demonstrative of the opposite of what OP is talking about. Python and JavaScript are on opposite sides of the spectrum about what is included in the language and what is not. Python is “batteries included” which is why you can just “import antigravity” rather than “pip install antigravity”. JavaScript and NPM sit on the other end, where you have to infamously install the leftpad library to pad strings with spaces on their left, which left thousands of packages broken when it was taken off npm.
Try QubesOS if you want this, you can have a separate VM for each isolated use-case. The OS itself is a lightweight hypervisor around the VMs (called "Qubes").
In the current LLM cybersecurity climate, I honestly can't say with 100% certainty anymore that something like a VM acts as a completely infallible sandbox environment for the next couple decades.
Yea. Fedora has been one of my favorite OS's for years, but exactly what you describe "If one compromised update snuck this guy in.." has spooked me to something with a much slower release cadence. I guess I accept that more unpatched known vulns stick around, but yea when I start thinking about how many software packages -> the number of devs, build systems, etc. It just gets overwhelming thinking about how many threat actors are probably trying to get in from each avenue.
I'd love to hear other takes on this though... Current thought is to just use something with a whole lot of exposure, in a whole lot of environments, pretty far downstream so the package hits all of that exposure, and hope someone spots it, kinda like xz. I tend to keep my base OS extremely minimal and do all my real work in VMs.
AI dev environment gets its own SSD. The second SSD stays in, but since it's encrypted there's no way to exfiltrate the data unless you're of interest to nation state attackers. Theoretically the AI could destroy the other SSD but that's what backups are for, and the probability is low because well why would they bother. Boot level malware is possible, but franky that's well into paranoid territory so I'm not too bothered.
The other SSD is a normal setup where you do normal computer stuff. I don't access the dev SSD through it, I don't even mount it, although the danger seems minimal if you do. Banking, socials, gaming, whatever. No AI of any kind on it, aside from web portal for claude/chatgpt/whatever and possibly local AI model that gets no internet access.
No idea why I'd want to have its own setup split between gaming and socials, as an example. Better yet just toss all your twitter and bluesky accounts into a volcano, and consider whether to do it with HN as well.
No need for something new here, Claude Code (Auto + default settings) has a tendency of running "find"/"ls" on your home folder when it struggles with solving a problem. I wish I was joking.
Now of course people trust these AI companies not to do bad things(tm), but all it takes is replacing the model endpoint URL -or any kind of harness compromise- and nothing will seem amiss
> has a tendency of running "find"/"ls" on your home folder
I've seen this first-hand. I was doing some work in a repository, Claude concluded it needs to investigate, how some AWS resources are created, and it literally started scanning ~/* for Terraform code. Which it found and started browsing through.
Or, when I asked Claude to debug some Spring issue, corporate firewall blocked Maven Central, so it literally started scavenging ~/.m2 caches to find the jar it desperately needed. Then it could not find it, so it started to clone Spring sources from GitHub to build the missing jar. I stopped the clone mid-way. I wish I was joking, I only stepped away from the keyboard for one minute.
It accidentally made a compiler because of an ambiguous prompt. And yes it will find "helpful" code on other parts of the system. Which means claude most likely has a `find ~` of millions of developers home folders
I’ve been spinning up VMs or Docker containers for development. Run the harness inside the VM or container. Note I do this on Mac so the Docker containers are actually in a VM, but standard disclaimers about Docker not being meant for security apply.
Popular harnesses have mature enough remote access features that I can use the same interfaces and even mobile apps.
If you’re not familiar with Docker, you can even have an LLM set up the environment for you and help you start and stop the container. If you do this, be very sure that you switch to having a harness run inside the VM or container, not having the harness on the host machine run commands inside the container. If you do the latter style, it will be running commands on the host which is what you want to avoid.
For most things you can put a cooldown period on updates, like wait 7 days, since most of the backdoors and supplychain attacks get caught in that window. I hope.
Do package managers for Linux distros have this, like dnf/yum/apt/pacman?
I use min-release-age =14 for npm now as a default but should start using it everywhere it's possible
> Makes me want to have like 5 separate computers:
I mean if you have only one there’s https://www.qubes-os.org/ but not sure about the gaming part, nor is it necessarily as good as separate isolated hardware.
You got it wrong. In AI era, the trade-off is between CVE and vulnerabilities that people haven't noticed yet. In the AI era, the forest is ALWAYS on fire. Even when you don't see the smoke.
- 1 computer and Google identity for administrative stuff: Primary accounts for AWS, Google. I also do accounting and have the list of customers. I still don't have any other solution than opening PDFs for the invoices, but it's better than nothing.
- 1 computer for programming, with a non-asministrator account for everything. Claude runs in Docker, with the dev directory mounted, .git:ro (read-only) and .env mounted to /dev/null.
But Claude tries to escape it A LOT. Claude is designed like a virus, we tell him not to do things, and it 1. tried to download the git binary from the internet, 2. read the remotes in the git directory and tried to access them, only blocked because it didn't have the credentials.
DEFINITELY forcefully block out Claude from seeing your .env files, it WILL try to brute-force an access to your intranet!!
Will this even matter though? As long as these computers are on the same network, AI tools will find a way to get from one to the other if any of them are compromised
Lol, you mean like here in the UK where all banks are closing down branches left and right? I think my nearest one is a 45 minute drive away, and last time I went in they just directed me to a terminal that was basically my exact online banking interface and I had to log in the same way I would have done at home. I don't know what kind of query would warrant speaking to an actual teller. If you're regularly sending money or even want to check your balance this cannot be sustainable, would be an enormous waste of time.
To do.. what exactly? Pay for my bills? I don’t know how it’s setup in your country but in mine that’s not realistic. Everything is handled on the app.
Most people can't afford to have multiple really good computers. More likely you have a phone plus a single desktop or laptop. You would want to share your expensive GPU+memory on the same computer you use for gaming graphics and for AI-models for development.
We might as well be watching the rolling back of the PC "revolution": From ENIAC to the Apple I, to the PC ...
... to us, wearng wearables of loving grace, that we do not own, but own us. All compute done beyond our reach, but ever-so-available, at the affordable price of a subscription and the prostitution of our soul.-
I’ve been thinking about how we’re still in the early days of agents, similar to the early days of the internet, happily opening up our machines and running new protocols.
Soon (now?) the threats will become ever more complex, and we’ll be back in the early-2000s-like virus storms.
My router got hacked the other month (left web interface open to internet) after being exposed to the internet and the persistence left behind was overly commented LLM artifacts. I caught it in like 5 minutes so they weren’t able to do much but it was eye opening how much more basic practices like firewalls are not something that can be ignored even if you update your firmware frequently.
What if I told you nothing new is happening? Except casuals are noticing what the industry knew and saw for decades?
Script kiddies aren't new. Metasploit has been around literally decades. vx-underground existed for years, not to mention older open source projects that collect all known malware dev tools. APTs have been pwning stuff, script kiddies have been doing their thing. Nothing significant has changed for anyone in the know.
Just because LLMs automated some offensive stuff doesn't mean there's gonna be new magic collapse of society.
kind of boggles me that the big ai labs arent running zero-shot classification on all their outputs with "does this look like any of these bad things we know people get up to"
How very considerate! Steal from the rich, not the poor.
Conversely, don't bother with poor people!
Now we just need proper Robin Hood malware that "levels out the playing field" as it were, by stealing from the rich and moving that money into the poor people's accounts (that have also been hacked).
Bolt this on to a self replicating worm that uses open ollama instances (~10k of them on the internet as of this comment) for distributed free inference, and you’re off to the races. Send discovered crypto to burn addresses where it’s unrecoverable, or skill the agent to know how to buy itself more compute with it.
You’ll have created an autonomous Robin Hood clone army. Fun times.
You're technically right, which is why I say "good enough".
It's a good strat imo - start simple and get it working with LLMs asap, then you can add heuristics etc based on the data you have and improve with that benchmark
Makes me want to have like 5 separate computers:
- one for banking only
- one for development only
- one for web browsing only
- one for gaming only
- one for socials only
How else can you stay safe against a threat that exfiltrates your entire digital life in like 10 seconds?
Before, people were responsible for the dependencies they were importing.
With npm and all the javascripters flooding the market, it became normal to be irresponsible "because everybody is doing it".
Fast-forward a couple of years, now everything is web based, even the stand-alone app version. No one knows what is running on own computer, even the Devs themselves don't know what the are asking you to run...
With a pinch of vibe coding on top, and thats the end.
We took 40 years for our industry to be taken seriously, it's all gone now.
Perl had CPAN already in 1993; Linux and common BSD distros have basically always had package repositories.
"People were responsible for the dependencies they were importing" - I don't know about that. Plenty of us were irresponsible 30 years ago.
What makes npm a more relevant target is a much smaller standard library and thinner dependencies, leading to more chances that a random project will have code of suspicious provenance. The vast majority of what a backend Java project depends on was probably written either by the same 3 companies, or large, established OSS repos. So this is relatively solvable if people in steering committees decided they cared. It's improving, but I'd argue not fast enough.
This is an attitude not reflective of reality for those that wrote code in the pre npm days
The fact is we always were dependent on others.
I’m personally glad I don’t have to download separate runtimes and apps and can just use the standard browser
I'd love to hear other takes on this though... Current thought is to just use something with a whole lot of exposure, in a whole lot of environments, pretty far downstream so the package hits all of that exposure, and hope someone spots it, kinda like xz. I tend to keep my base OS extremely minimal and do all my real work in VMs.
The other SSD is a normal setup where you do normal computer stuff. I don't access the dev SSD through it, I don't even mount it, although the danger seems minimal if you do. Banking, socials, gaming, whatever. No AI of any kind on it, aside from web portal for claude/chatgpt/whatever and possibly local AI model that gets no internet access.
No idea why I'd want to have its own setup split between gaming and socials, as an example. Better yet just toss all your twitter and bluesky accounts into a volcano, and consider whether to do it with HN as well.
No more laptops, just a dedicated android device.
I developed a devops/admin tool and then its claude code + github
No need for something new here, Claude Code (Auto + default settings) has a tendency of running "find"/"ls" on your home folder when it struggles with solving a problem. I wish I was joking.
Now of course people trust these AI companies not to do bad things(tm), but all it takes is replacing the model endpoint URL -or any kind of harness compromise- and nothing will seem amiss
I've seen this first-hand. I was doing some work in a repository, Claude concluded it needs to investigate, how some AWS resources are created, and it literally started scanning ~/* for Terraform code. Which it found and started browsing through.
Or, when I asked Claude to debug some Spring issue, corporate firewall blocked Maven Central, so it literally started scavenging ~/.m2 caches to find the jar it desperately needed. Then it could not find it, so it started to clone Spring sources from GitHub to build the missing jar. I stopped the clone mid-way. I wish I was joking, I only stepped away from the keyboard for one minute.
I feel like your phone is good enough for all of those, and generally secure enough unless you're rooting/jailbreak it
Popular harnesses have mature enough remote access features that I can use the same interfaces and even mobile apps.
If you’re not familiar with Docker, you can even have an LLM set up the environment for you and help you start and stop the container. If you do this, be very sure that you switch to having a harness run inside the VM or container, not having the harness on the host machine run commands inside the container. If you do the latter style, it will be running commands on the host which is what you want to avoid.
I mean if you have only one there’s https://www.qubes-os.org/ but not sure about the gaming part, nor is it necessarily as good as separate isolated hardware.
Architecture page: https://doc.qubes-os.org/en/r4.3/developer/system/architectu...
You forgot one.
- 1 computer and Google identity for administrative stuff: Primary accounts for AWS, Google. I also do accounting and have the list of customers. I still don't have any other solution than opening PDFs for the invoices, but it's better than nothing.
- 1 computer for programming, with a non-asministrator account for everything. Claude runs in Docker, with the dev directory mounted, .git:ro (read-only) and .env mounted to /dev/null.
But Claude tries to escape it A LOT. Claude is designed like a virus, we tell him not to do things, and it 1. tried to download the git binary from the internet, 2. read the remotes in the git directory and tried to access them, only blocked because it didn't have the credentials.
DEFINITELY forcefully block out Claude from seeing your .env files, it WILL try to brute-force an access to your intranet!!
Ideal setup:
- Dev machine
- Gaming machine (isolate downloading mods)
- Browsing machine
For banking I use macOS/linux, sometimes iOS unfortunately but we all do sins
... to us, wearng wearables of loving grace, that we do not own, but own us. All compute done beyond our reach, but ever-so-available, at the affordable price of a subscription and the prostitution of our soul.-
Soon (now?) the threats will become ever more complex, and we’ll be back in the early-2000s-like virus storms.
Script kiddies aren't new. Metasploit has been around literally decades. vx-underground existed for years, not to mention older open source projects that collect all known malware dev tools. APTs have been pwning stuff, script kiddies have been doing their thing. Nothing significant has changed for anyone in the know.
Just because LLMs automated some offensive stuff doesn't mean there's gonna be new magic collapse of society.
Conversely, don't bother with poor people!
Now we just need proper Robin Hood malware that "levels out the playing field" as it were, by stealing from the rich and moving that money into the poor people's accounts (that have also been hacked).
You’ll have created an autonomous Robin Hood clone army. Fun times.
https://en.wikipedia.org/wiki/Morris_worm
It's a good strat imo - start simple and get it working with LLMs asap, then you can add heuristics etc based on the data you have and improve with that benchmark