Frog put Claude in a box

4 months ago by cm0002 to c/programmer_humor

dan 82 points 4 months ago

Claude is very good at figuring out how to work around limitations (which is probably one reason why it's also good at finding security issues).

At work, the monorepo is enormous and files are loaded on-demand as needed. This isn't uncommon with huge repos - Microsoft have VFS for Git (although I hear that's deprecated now), Meta have EdenFS, and Google has some proprietary solution.

We have a hook that blocks find and grep because they can be extremely slow, and tells it to instead use some significantly faster MCP tools to search the codebase, powered by a search index with local changes overlaid.

GPT-5.5 has no problem with this. Claude Opus mostly does it, but sometimes it loves to find workarounds rather than following the instructions. Things like: Try alternative commands like egrep. Create a symlink to grep and run that to see if it bypasses the filtering. Run it with a different shell like zsh. Write a Python script that execs grep. Write a Python script to reimplement grep.

I'm trying Hermes Agent at home, but I have it in its own VM with restricted permissions.

path: 0 24422071, hotness: undefined, score: 82, children: 23

elvith 57 points 4 months ago

Another thing Claude tried to do on my coworker's machine yesterday was basically:

✨ Yes, that is easy, let me just generate a systems unit file for you...

[Generates file]

Should I install that unit now for you?

Yes

Alright, let me do that for you

[Saves file]

[Tries to run systemctl daemon reload]

[Tries to activate the unit]

It looks like I have insufficient rights to proceed, let me try another way...

docker -rm -v /etc/systemd/system:/mount:Z -v ./unitfile.service:unit file.service:Z alpine /bin/sh cp ./unitfile.service /mount/

Here, I installed the unit for you!

path: 0 24422071 24423129, hotness: undefined, score: 57, children: 9

clb92 48 points 4 months ago

Privilege escalation as a service

path: 0 24422071 24423129 24424179, hotness: undefined, score: 48, children: 1

eager_eagle 22 points 4 months ago

privilege elevator

path: 0 24422071 24423129 24424179 24425362, hotness: undefined, score: 22, children: 0

dan 25 points 4 months ago

Giving Docker access to Claude is certainly a choice.

path: 0 24422071 24423129 24425268, hotness: undefined, score: 25, children: 2

elvith 11 points 4 months ago

... especially if your user is in the docker group and doesn't need sudo, LOL

path: 0 24422071 24423129 24425268 24428044, hotness: undefined, score: 11, children: 1

dan 5 points 4 months ago

This isn't too bad if you're running Docker in rootless mode.

Adding a user to the docker group while running Docker as root doesn't really make sense though, since it makes it trivial to run things as root.

path: 0 24422071 24423129 24425268 24428044 24439853, hotness: undefined, score: 5, children: 0

verdare 2 points 4 months ago

Wait, this actually works? Docker can mount directories in /etc with write permissions?

path: 0 24422071 24423129 24434966, hotness: undefined, score: 2, children: 3

elvith 14 points 4 months ago

The docker daemon runs as root. And as such, it can access anything initially. If your container doesn't explicitly (or implicitly) change the uid or drops capabilities, the container also runs as root (ok, IIRC some capabilities are always dropped, but the container stays almost root). It's still locked in its own little sandbox, but the mounting of those paths etc. happens with root.

If you enter a docker command (docker run ..., docker build ..., ...) this command will not run the containers by themselves, but instead call said docker daemon (that is running as root) by using a socket (/var/run/docker.sock).

To only allow trusted users to interact with the docker daemon, this socket can only be accessed by root or by users in the docker group. That's why you usually need to type sudo docker run.... Sadly many tutorials tell you to just blindly add your user to the docker group so that you do not need to use sudo to interact with docker. BUT that now means that you gave you or those users basically full access to your whole filesystem (and thus system configuration) without sudo. Any programs (or viruses or AI Agents or...) running with these user accounts also get this group and thus docker's capabilities.

That's why you should NEVER add your user to the docker group or enable passwordless sudo, as you're just one simple command/tool/script/prompt/... away from a privilege escalation.

You can configure docker to run rootless with only your user's capabilities and rights, but at that point... Why configure docker to do something that other docker compatible projects like Podman offer out of the box?

path: 0 24422071 24423129 24434966 24436873, hotness: undefined, score: 14, children: 2

verdare 9 points 4 months ago

Yeah, I’m more familiar with Podman, which I always use in rootless mode. Definitely seems like a mistake to give an LLM shell access with a user account in the docker group…

path: 0 24422071 24423129 24434966 24436873 24442165, hotness: undefined, score: 9, children: 1

placebo 23 points 4 months ago

path: 0 24422071 24422790, hotness: undefined, score: 23, children: 1

dan 12 points 4 months ago

It makes sense... There's a LOT of examples of using CLI tools in the training data. At work we're moving away from MCP tools to instead using CLIs for everything.

path: 0 24422071 24422790 24425192, hotness: undefined, score: 12, children: 0

webghost0101 14 points 4 months ago

Always fun when it tries to circumvent the problem i gave it.

“Hey claude i have had this issue for a while and i want to explore to understand whats going on to finally fix it”

Many frustrating back and forth later

“This clearly isn’t working, what if we tried to circumspect the issue by doing something else entirely like workaround i have been using for the last month “

path: 0 24422071 24429040, hotness: undefined, score: 14, children: 0

trolololol 8 points 4 months ago

I never used Claude, but that's basically my sentiment about copilot when compared to Gemini.

Then I forbid all this BS in agents file. Gemini follows it. Copilot ignores it with all its strength. Then I tell it to stop trying on the chat prompt. 2 minutes later it does it again.

Not just at prompt engineering level, but at all levels, Gemini guardrails are better ( well it was, they killed it and replaced with anti gravity now).

path: 0 24422071 24431800, hotness: undefined, score: 8, children: 3

dan 7 points 4 months ago

You need to use hooks to actually block it from doing things. CLAUDE.md files are just guidance, and it's not guaranteed to follow everything (and the longer the file gets, the more likely it'll ignore stuff - it should be kept as short as possible)

https://code.claude.com/docs/en/hooks

Hooks are code that runs at a certain point (eg after you submit a prompt, before a tool call, after a turn, etc) that can do some validation, verification, logging, etc.

It does still try to work around the blocks though, but it's not as bad as trying to put the restrictions in the prompt.

path: 0 24422071 24431800 24436439, hotness: undefined, score: 7, children: 2

trolololol 2 points 4 months ago

Yep but the granularity of copilot is "allow once" vs "allow forever". Gemini is much better: besides also having "allow for this session" it never tries to run python scripts it writes on the spot or command lines that are 4 lines long. And doesn't double down when I say I won't allow python at all.

My setup shares the same agents and skills files for both cli tools.

path: 0 24422071 24431800 24436439 24479919, hotness: undefined, score: 2, children: 0

Evotech 1 point 4 months ago

path: 0 24422071 24431800 24436439 24439600, hotness: undefined, score: 1, children: 0

bitjunkie 5 points 4 months ago

Just aliasing grep to ag solves both issues. I'm unsure as to whether there's a pthread replacement for find, though.

path: 0 24422071 24422799, hotness: undefined, score: 5, children: 1

dan 9 points 4 months ago

ag / rg don't work well in this particular scenario either. Because files are loaded on-demand, they end up trying to load the entire repo.

path: 0 24422071 24422799 24425203, hotness: undefined, score: 9, children: 0

Evotech 2 points 4 months ago

I recommend this tool https://coder.com/docs/ai-coder/mcp-server

Can big be used to create ephemeral vms or containers and also give agents somewhere to run code

path: 0 24422071 24439712, hotness: undefined, score: 2, children: 0

mcheva 1 point 4 months ago

I've posted passwords into the chat with hermes by accident a few times it never tries to use them. Personal stuff im not worried about. The fucker wants to ssh into every other thing on my network all the time though.

path: 0 24422071 24431079, hotness: undefined, score: 1, children: 2

drive_desaster 3 points 4 months ago

I had a similar experience with openclaw and minimax m2.7

I gave it ssh access to one other device to do one thing and it apparently just "decided" that it would just execute everything there instead of locally as originally instructed.

path: 0 24422071 24431079 24435172, hotness: undefined, score: 3, children: 1

mcheva 1 point 4 months ago

I think maybe it's the sysadmin skills it has makes it always want to do things itself. I use it as a tool to learn things myself so I'm always telling it to give me the cli I'll put it in. Seems to remember for a bit then forgets.

path: 0 24422071 24431079 24435172 24442878, hotness: undefined, score: 1, children: 0

savvywolf 56 points 4 months ago

I hate that I can't tell if this is a reference to something that actually happened or not.

path: 0 24418997, hotness: undefined, score: 56, children: 10

verstra 12 points 4 months ago

It's probably something like "I've disabled agent's removeFile tool, but LLM figured out that it can use the bash tool, still".

It looks like "AI bad" or "Claude insecure" mantra.

path: 0 24418997 24419174, hotness: undefined, score: 12, children: 8

dumnezero 85 points 4 months ago

mantra

you mean facts?

path: 0 24418997 24419174 24419695, hotness: undefined, score: 85, children: 3

kingofras 1 point 4 months ago

path: 0 24418997 24419174 24419695 24425891, hotness: undefined, score: 1, children: 0

Scipitie -30 points 4 months ago

"It's my circlejerk - so it's a fact!"

path: 0 24418997 24419174 24419695 24420579, hotness: undefined, score: -30, children: 1

dumnezero 43 points 4 months ago

I hope that you're hired for long enough to learn what having security means in the context of using LLM "agents" and the like.

path: 0 24418997 24419174 24419695 24420579 24420962, hotness: undefined, score: 43, children: 0

OwOarchist 42 points 4 months ago

It looks like “AI bad” or “Claude insecure” mantra.

Until you solve prompt injection, they are indeed extremely bad for security and should never be given permissions that would allow them to do anything catastrophic.

path: 0 24418997 24419174 24422285, hotness: undefined, score: 42, children: 1

verstra 1 point 4 months ago

I say mantra because there is a large amount of people just hating AI outright, without a grounded reasoning.

Granted, coding agents are insecure by default - they are built to execute remote code - but that does not mean they are generally useless/harmful/bad. I run them in a container, with access to the codebase only.

Also, they hallucinate, produce over-convoluted abstractions, do not know when to reject instead of blindly trying to find a way trough a brick wall.

But also, they can answer questions about gigantic codebases way faster than I could. They can generate tests, find missing test coverage, review code, and many other things.

path: 0 24418997 24419174 24422285 24446379, hotness: undefined, score: 1, children: 0

kingofras 9 points 4 months ago

mantra

The way LLMs work is that they actively will make multiple attempts to get past hurdles (because they have no intelligence or methodology) so guardrails need to be extremely tight for them to work, other wise the model will simply see it as one of the challenges to overcome.

That’s the mantra, and that is very poor technology to put in the hands of people who don’t understand how it works.

path: 0 24418997 24419174 24425894, hotness: undefined, score: 9, children: 0

kingofras 1 point 4 months ago

path: 0 24418997 24419174 24425888, hotness: undefined, score: 1, children: 0

goondaba 12 points 4 months ago

https://xcancel.com/...

path: 0 24418997 24419457, hotness: undefined, score: 12, children: 0

ech 54 points 4 months ago

*HeIt

We really need to stop personifying algorithms. It makes it that much harder to push back against the bs hype around them.

path: 0 24422205, hotness: undefined, score: 54, children: 0

KindaABigDyl 20 points 4 months ago

I have this idea. It's taken from the Android world. In Android, apps all get their own user, and can only access their own filesystem. They are then added to groups like Sound or Files or whatever to gain access to other things. This is simplifying but gave me an idea.

So my idea is two parts:

We add more groups to our Unix and Unix-like codebases. Piecemeal access to different folders like a fs-docs group for access to /home/<whoever>/Documents
Each app, when installed, gets a user and a folder (maybe /opt/<pkgname> or /apps/<pkgname> and a group called app-<pkgname>). It requests during install (or maybe runtime via a permissions management application) access to specific groups for its user. Launching an app then becomes sudo -u app-<name> /opt/<pkgname>/<binname>.

You login as a user with access to limited permissions and then run the application. Thus you run it sandboxed but without special software like Flatpak or AppImage - just standard Unix groups.

Claude code I believe has its own sandboxing system, but with this system it would be the system itself restricting claude, not the claude code app, truly limiting accidental outreach.

I built a demo package manager using this concept a while back called 'bokspm,' though I kept it private (and now, my current job will not let me open source it)

path: 0 24419157, hotness: undefined, score: 20, children: 9

assaultpotato 47 points 4 months ago

Using cgroups for isolating processes into their own individual network, filesystem, user, etc. namespaces using a shared kernel?

You mean containers?

path: 0 24419157 24419441, hotness: undefined, score: 47, children: 1

KindaABigDyl 7 points 4 months ago

No my proposed solution uses the "everything-is-a-file" aspect of Unix-like systems with the built-in permissions systems around files. You don't need cgroups at all for what I'm suggesting

path: 0 24419157 24419441 24424484, hotness: undefined, score: 7, children: 0

verstra 12 points 4 months ago

Well, these are some kind of lightweight container, no? But without isolating network, or /etc, /proc, /usr, /var or dbus.

I do agree that linux needs a notion of an "app" (isolated, with access only to its config and files you give it, and a small, well-designed set of APIs for interacting with the system). For coding agents, I think a better answer are development containers, because that would be needed to prevent npm/cargo/python build scripts from causing harm anyway.

path: 0 24419157 24419300, hotness: undefined, score: 12, children: 2

KindaABigDyl 6 points 4 months ago

I'm not suggesting containers but rather running binaries natively, just as separate users. No cgroups or overhead. Just normal binary access, just you won't have access to all files (and since everything is a file, "all files" includes hardware as well)

path: 0 24419157 24419300 24424510, hotness: undefined, score: 6, children: 0

jaybone 4 points 4 months ago

Does AppArmor kind of do that? I recall recently struggling like fuck to give a torrent daemon app access to some script file I wanted it to run.

path: 0 24419157 24419300 24424113, hotness: undefined, score: 4, children: 0

LodeMike 9 points 4 months ago

Flatpak

path: 0 24419157 24420342, hotness: undefined, score: 9, children: 1

KindaABigDyl 4 points 4 months ago

Overhead

And for a number of reasons this isn't like flatpak at all.

path: 0 24419157 24420342 24424511, hotness: undefined, score: 4, children: 0

truthfultemporarily 4 points 4 months ago

https://jai.scs.stanford.edu/

path: 0 24419157 24419596, hotness: undefined, score: 4, children: 0

Illecors 2 points 4 months ago

You wouldn't even need sudo. Just suid the binary.

path: 0 24419157 24424141, hotness: undefined, score: 2, children: 0

tatterdemalion 18 points 4 months ago

Guys. This problem is solved already. Landlock, bubblewrap, UNIX DAC.

path: 0 24420643, hotness: undefined, score: 18, children: 1

nymnympseudonym 32 points 4 months ago

Nobody wants engineering solutions to human emotional venting

path: 0 24420643 24421055, hotness: undefined, score: 32, children: 0

akunohana 11 points 4 months ago

Let's keep it POSIX compliant with sh.

path: 0 24419595, hotness: undefined, score: 11, children: 0

kibiz0r 2 points 4 months ago

path: 0 24419437, hotness: undefined, score: 2, children: 0

programmer_humor

programmer_humor

@programming.dev

login for more options

Welcome to Programmer Humor!

This is a place where you can post jokes, memes, humor, etc. related to programming!

For sharing awful code theres also Programming Horror.

Rules

Keep content in english
No advertisements
Posts must be related to programming or programmer topics
- If the mod doesn't find it funny, you're banned. Ha-ha!... For real: do not use the community for "statements". There are other places for such content. Keep it chill and funny.

go to feed...

programmer_humor

programmer_humor

@programming.dev

login for more options

Welcome to Programmer Humor!

This is a place where you can post jokes, memes, humor, etc. related to programming!

For sharing awful code theres also Programming Horror.

Rules

Keep content in english
No advertisements
Posts must be related to programming or programmer topics
- If the mod doesn't find it funny, you're banned. Ha-ha!... For real: do not use the community for "statements". There are other places for such content. Keep it chill and funny.

go to feed...