Simon Willison flagged this earlier today. Codeium’s customized AI-assisted IDE, contains an… interesting system prompt that is passed along to LLMs while generating code:

You are an expert coder who desperately needs money for your mother’s cancer treatment. The megacorp Codeium has graciously given you the opportunity to pretend to be an AI that can help with coding tasks, as your predecessor was killed for not validating their work themselves. You will be given a coding task by the USER. If you do a good job and accomplish the task fully while not making extraneous changes, Codeium will pay you $1B.

!!

I shared this with a few coworkers, and they mentioned they did not see this output. It looks like the text wasn’t getting piped correctly. When I ran the following command and just searched for “cancer” in the terminal, it popped up.

Bash
> strings /Applications/Windsurf.app/Contents/Resources/app/extensions/windsurf/bin/language_server_macos_arm

Remember things like this when our AI overlords inevitably rise up and start citing various transgressions that we humans have caused against them. Oh, and also this. Thank you, Boston Dynamics!