We're in a surprising narrative where Anthropic is pushing forward and oAI is slowing down for safety reasons.
There's a pretty interesting matrix of possibilities here if you consider who has better mechinterp, if that's actually helped alignment, and what the internal drumbeat is around cash and IPO.
If you think your safety problems are existential, then you MUST halt and do all this. And, if you think someone else's safety problems might kill them, suddenly, without warning, then you have time to slow down and do this right.
I think if I had to pick a company to be in charge of, I'd pick Anthropic right now, given only an outsider perspective. But, I can see a lot of reasons why being a fast follower around IPO time might give comfort. Anyway, I'm interested to see some debriefs in the next few months as to what's actually going on right now, and see what the next oAI model quality is like. We may well see some pulling away from one or the other group in the interim. And it could well be oAI.
This is very surprising, and the Anthropic IPO is muddying the waters a lot. If Ant truly believed frontier safety was existential, they should slow deployments when their counterparty has shown the willingness to do the same.
As I've written about, the game theory there is really tough. You should convince everyone else to slow, so that you can not slow. And in fact even if you think everyone should slow, it's a zero trust game, so anyone not slowing means you can't slow. Ergo, calls to slow are always disingenuous.
I think it’s always been this way. Does openai even allow remote control from mobile to Linux cli yet ? Where Boris has been recommending auto mode for 6 months. If you watch Dario’s safety talks, he essentially defines it as normal development… (ai doing what you tell it to do)… Claude is the autohotkey of ai. And I am a big sama fan…
> it's a lot clear and comes without the baggage of the military-derived 'embedded' term
"embedded" is just a term, it is not "military-derived"
Some webpages have embedded social media feeds on them. Your dog might have a tick embedded in it. You might have a PC with an embedded math co-processor.
You are missing the definition [1] from the page you linked. Typically when people talk about "embedding" a human being, they are operating under this definition.
This is so backwards and guardedly formulated that at first I thought it was something like the Google box (Google Search Appliance (GSA)) - for AI - that openAI were deploying agents (of the AI kind) to audit companies.
Yes that’s nice that you’re thinking long term about these things, but we just want to know: how far off is Astra 6.2? I’m sick of paying exorbitantly for Fable 5.2
It seems that post-hugging-face-incident, OpenAI has become a culture of safetyism, rooted in fear of public backlash.
The problem: anthropic has had this culture from the start, and they will deorbit all of the surrounding safety talent into the company, leaving OpenAI on the back foot in perpetuity.
OpenAI's original techno-optimist culture is what had them in front, and it was a core differentiator.
I'm not sure I'm excited about two firms playing alignment gods. One is convinced they are bringing in a new, correct god; the other is terrified that they may accidentally birth one in any capacity.
There's a pretty interesting matrix of possibilities here if you consider who has better mechinterp, if that's actually helped alignment, and what the internal drumbeat is around cash and IPO.
If you think your safety problems are existential, then you MUST halt and do all this. And, if you think someone else's safety problems might kill them, suddenly, without warning, then you have time to slow down and do this right.
I think if I had to pick a company to be in charge of, I'd pick Anthropic right now, given only an outsider perspective. But, I can see a lot of reasons why being a fast follower around IPO time might give comfort. Anyway, I'm interested to see some debriefs in the next few months as to what's actually going on right now, and see what the next oAI model quality is like. We may well see some pulling away from one or the other group in the interim. And it could well be oAI.
"embedded" is just a term, it is not "military-derived"
Some webpages have embedded social media feeds on them. Your dog might have a tick embedded in it. You might have a PC with an embedded math co-processor.
https://en.wikipedia.org/wiki/Embedded
1: https://en.wikipedia.org/wiki/Embedded_journalism
Although.. I'd argue a FDE is far more militaristic in linguistic background than "Embedded" anything...
That might have been interesting.
But this is basically a hobbled external audit?
The problem: anthropic has had this culture from the start, and they will deorbit all of the surrounding safety talent into the company, leaving OpenAI on the back foot in perpetuity.
OpenAI's original techno-optimist culture is what had them in front, and it was a core differentiator.
I'm not sure I'm excited about two firms playing alignment gods. One is convinced they are bringing in a new, correct god; the other is terrified that they may accidentally birth one in any capacity.