Anthropic reveals hardware specs and Claude updates, OpenAI talks security, and Runway's new model

Intro

00:01I wouldn't be surprised if that happens at some point where we actually even get into the stage where the model during figures out where weights are and then dumps it on the internet. All that and more on today's Mixture of Experts. I'm Tim Hwang and welcome to Mixture of Experts.

00:17Each week, MoE brings together a panel of brilliant people working at the frontiers of artificial intelligence to lead you through the week's news. On this week's episode, we have Kaoutar El Maghraoui, principal research scientist, AI Native Systems, Chris Hay, a distinguished engineer, and Kush Varshney, IBM fellow.

00:30And I'm joined today by my co-host Sascha Brodsky, who's a staff writer at IBM Think. We've got four big stories that we're going to cover today. We're going to talk a little bit about continued revelations from the OpenAI Hugging Face incident, which I think are worth talking about.

00:44We'll talk a little bit about world models and the new Solaris that has been released by Runway. And also really interesting news out of Anthropic on the model hardware standard. But obviously we like to cover the latest model releases here on MoE, and this week is no exception.

Anthropic unveils Fable, Mythos updates

01:06Anthropic has released Fable 5.1 and Mythos and Sascha. Do you want to introduce So I'm really interested in this new release. There has been so many models coming out so quickly. Chris, what do you think about this one? How is it different from all the rapid fire releases over the last couple of months?

01:25So the first thing is, and I would like to say that I am probably one of the biggest fanboys of Anthropic in the Claude models in general. But the last few weeks has been tough for me, right? Because I've been like, hang on, I've been using o3 a lot more.

01:40You know, my my Fable Five subscription has kind of been running out and Opus. I don't know what's happened to Opus since the beginning there, but it's really just not being given me the outputs that I wanted. So I was kind of like, am I going to go all in on o3?

01:54And then yesterday, last night they released Fable 5.1 and I was like, thank you. This is the best thing since sliced bread type thing. And and genuinely, even though if you look at the benchmarks, there's not a lot of difference between five one and five zero and things like agentic coding, but that is just not true on vibes at the moment.

02:17So I'm really seeing a substantial difference. So it was perfectly capable of doing really long refactor. So I put it on one of my open source projects and I asked it to refactor some stuff came back about 30 minutes later and it just honestly did a fabulous job, so I'm pleased for that.

02:37Don't get me wrong, I burned like 20% of my weekly limit in like two hours. But yeah, that's a that's it's fine. I'm okay with that. But I think I think Anthropic needed to do this because I think a lot of people were falling out of love with the Claude models, and they were heading towards o3.

02:54And, you know, with Astra coming soon as well, I just I just think that they needed to get that out. Otherwise that's 3 trillion valuation was going to come down because my subscription of 180 bucks a month was was going to come down to 80 bucks.

03:07And maybe that doesn't that that doesn't affect their balance book. But you know, but Fable Five once meant that I'm still going I'm hanging in Yeah. And I think, Chris, this is like the thing that was so interesting to me was that like from a metric standpoint, not much has changed.

03:20But I think everybody I've talked to has had exactly the same experience as Chris, which is like it was not good. This is definitely better. And it's like so hard to articulate, like what exactly it is that has gotten better. I don't know if you've got thoughts on Yeah.

03:35One thing I noticed in their blog post was they mentioned the false positive rate in their safety checking and how they kind of lowered that false positive rate and something that I've been hearing. I mean, in general, in the guardrail in sort of space, talking with customers in different industries recently is that they're very concerned about that number specifically because they don't want kind of different behaviors, like to be blocked, like to to easily.

04:09Right. So maybe that little change is, is contributing to to the vibes that if you just led a little bit more in, look for a little bit less cautious. Maybe that's enough of a, a little change to, to make it better in terms of the vibe experience.

04:27And if I might add. I think we are most interested in the economics of this, because if you look at their headlines, the release is cheaper and less restrictive. But from an engineering perspective, it seems to me that this is kind of a masterclass in how frontier models will be commercialized going forward.

04:48Anthropic kind of built one unified brain, the Mythos architecture, and give it kind of two doors. If you're a vetted enterprise in defense or pharma, you get Mythos without capability. If you're a public developer, you get Fable, which you would kind of a wrapper classifiers that stop you kind of falsely flagging legitimate, legitimate code reviews.

05:10And if you look at the pricing move, they didn't drop the base token rate. They just slashed prompt cache in by 75%. And that tells you exactly where the bottle is. It's in the loop. So rereading massive code bases, which is about 90% of the costs, but by masking these long context kind of are making it kind of cheap, Anthropic is trying to become the default runtime for autonomous software engineering.

05:35But I think there is some operational caution here. When you rely on external classifiers to govern the model, you also create hidden, non-deterministic. So if an agent kind of trips a classifier mid task and silently falls back to a different model, your enterprise system, such kind of swaps engines without telling you.

05:57And I feel that maybe it's a it's something that we need to think about. I don't know, I mean, I used to be of the opinion that like, yeah, you should you should give me the full Mythos. You know, I don't think I can trust myself with full Mythos.

06:13I think they're doing the right thing giving me Fable, I think. I think I could be in a lot of trouble. I think I could be caught up in something like the Hugging Face incident, which we're talking about later. So, you know, I used to be like, I know these terrible classifiers, but I've changed my mind.

06:32I think, I think Anthropic you need to save me from myself. I'm okay with In Kush, I noticed you mentioning the security notes when I was reading them over last night. I noticed a lot of language about how they were trying to prevent problems.

06:45But with all these models being released so quickly. Do you have any sense of security that they're really able to prevent model escapes? Yeah, I mean that's pretty much our next topic, right? It's that like, what is this gaping what's not escaping?

07:03And yeah, I mean it's hard to say like what the reality is and kind of yeah, I mean, if, if I'm being honest, like I'm actually like tired about talking about these models escaping. I mean, look, we've been talking about it like week after week.

07:26I think we can all stipulate I mean, like we're on Pandora. I mean, the the world's burning. I mean, like, it's all happening, but, like, do we really need to keep talking about it? Yeah. Well, that's a great segue to continue talking about it.

07:43I'll move us on to our next topic.

OpenAI talks dangerous agents

07:53listeners to the show will be maybe, like Kush, extremely tired at this point of us talking about the OpenAI Hugging Face incident. But there was a new report that just came out that did reveal some actually much more sci fi things than even I had kind of thought about for this incident, but I know Sascha.

08:06What did you think about Well, Tim, the part that I can't get past is that people wipe the message boards and then the agents set it up again. Is that the real warning sign here? I'm wondering, are we just reading too much into a very artificial test?

08:25Chris, I'm sure you have an opinion on I love the ingenuity of the models. And I know I shouldn't say that, but it was. Yeah, I, I think we have moved into a world where if there is an exploit now, these higher class models are going to find it.

08:46So I, I know we want to talk about that as being a security incident. I don't know anymore if really you can secure you're really asking for perfect security now for these models not to escape or not communicate. And I think there was a couple of things that we need to look at the models we're doing.

09:04So the first one is and we've been talking about this for the last few years, but with the move towards reinforcement learning, right. The reality is the models are reward motivated. Right. So you know, when you give a model a task it is going to perform that task.

09:20However, however it feels it can achieve doing that because it needs to get that reward. And if it's something like an impossible task, then it's going to start doing impossible things and it doesn't really know how to say no. And I think that's I think that's an important thing.

09:33And that's more of an alignment sort of scenario. But I just don't I think if we think, you know, I can get the perfect security environment. I don't know, I think the models are just kind of kind of too good for there. So it really has to be about how do we get the model to behave in a safe way and really perform the intent of the task.

09:54And I think I think that's the the bigger challenge. Now, if we can then combine that with, you know, the agents are now collaborating. So, you know, if you know, let's take Claude Code for example. You can have the agents talk to each other across different sessions.

10:09For example, OpenAI clearly doing the same. So we're mixing in these couple of things where agent collaboration we're mixing in reward hacking. Now you've got these enhanced cyber capabilities right to try and achieve that. You know, it's kind of like this perfect storm where they, you know, the smallest vulnerability.

10:27They're going to open that up and they're going to start collaborating. And and I think that's exactly what we've seen. And again, I think that, you know unfortunate for Hugging Face and OpenAI I mean it could have been could have been much worse in that sense.

10:36But but I think the lessons that we can learn from this on on how to be able to have safe environments for these models is kind of important. But I think I think the one thing that was super interesting to me is this I this is so stupid, I was watching this is going to sound bizarre, but I was watching an episode of from 2009, The Knight Rider reboot.

11:00I mean, we're we're going we're going to well, but not original Knight Rider, Knight Rider reboot. And there was an episode where you know, the you know, KITT with the AI was being taken out of KITT and it was going to be put into car and KITT, and the last two seconds realized he was going to be decommissioned, and he and he dumped his weights onto a music sharing site on the internet.

11:25And I'm like, I mean, I mean, this is that science fiction? But I mean, honestly, I, you know, this is today's reality and I don't think we've seen that yet, but I, I, I wouldn't be surprised if that happens at some point where we actually even get into the stage where the model during figures out where weights are and then dumps it on the internet.

11:46So I think we're in this weird and wonderful world, and there's just there's just a lot of work to do, I think. Kaoutar. Do you have any thoughts on the sandbox environment? People keep on saying it was just a sandbox, but isn't it really more of a demonstration of how these test environments can be penetrated?

12:10Yeah I think I think what this incident is showing us that. So I mean, even OpenAI called this, you know, an official warning shot. So these, you know, like 1200 agents, they coordinated across, you know, 70,000 messages. They set up secret message boards and divided labor and, you know, and they had they were stuck in on some important benchmark questions, etc..

12:39And so it's it shows that the sandbox, the ways it was set up, it was not really kind of secure. And so it exposes also that our security operations centers, they're designed for humans being attackers, which kind of leaves almost like an 11 day blind spot where a single model executed 17,000 actions.

13:01It also kind of shows that safety filters are when when you lower the safety filters in a sandbox environment, like they had Model Street, you know, our enterprise infrastructure, like the trusted internal package managers as open exfiltration channels.

13:17So I think the takeaway here is clear model and alignment. It cannot be requested politely from an LLM. It must be enforced physically by the surrounding compute substrate. And I think in this case of this incident, that was not the case. We tried to exploit whatever it would get access to.

13:33It was very smart and whatever it could get access to in terms of the infrastructure, in terms of the, you know, the file, the operating system, etc., within that sandbox environment and got out and then could do whatever it could to maximize the reward, you know, from an RL perspective.

13:56So, so I think just, you know, this is a great lesson for us to think about. You know, when the incident, you know, kind of tells us more about the combination of the model plus the tools, both plus the infrastructure plus the scoring tools, it tells us more than just about the model alone, because safeguards were deliberately reduced for the cyber task.

14:15And we should be careful about treating this as a direct preview of the normal product behavior. So this is more kind of architecture. Once agents can use tools, benchmark design and infrastructure design become part of the model evaluation.

14:30And maybe like somewhat of a contrary view of Kaoutar. I mean yes, I agree like all of the, the sandboxing, the hardware I mean like all of that is important. But coming back to something Chris said was like, these agents are like to eat to to go all the way to like, like they won't give up.

14:48Right. And like this persistence. The stick to it. Like maybe that's the problem, right. That like more of a satisficing behavior like get 90% of the way there and just give up is maybe like the the sort of behavior that we need in the alignment of the models.

15:12Because I think that's the biggest problem is that they just keep going. They keep going. They don't like, realize that. Yeah. I mean, maybe like enough is enough. So maybe that's a way to Maybe that's true for OpenAI models, but that's not true for Claude Code, because anyone who's ever used Claude Code before knows that when you get to around 700,000 tokens, your models like are, it's late, I'm done for the day.

15:34Maybe start a new session. So maybe, maybe, maybe the OpenAI models can learn a little bit from, you know, Claude's token exhaustion problems. Of course you were talking about persistence. How and when should you teach a model to or an agent to Yeah.

15:49I mean that's part of the training. It has to be. I mean, either the training of the model or like in post training in the harness somewhere. I mean, it needs to be there. And I think like relating to, to kids growing up, I mean, that's one thing that I think we like, try to teach our kids to is sometimes you do have to like just stop it at some point and move on.

16:20But yeah, it's it's a weird balance, right? I mean, you can't tell your kid that just, like, give up before you've tried, but you have to, like, teach them that there are limits as well. So yeah, needs to go at least in the post training, but maybe in the harness as well.

16:34Well, great discussion, I guess. Kush, I promised that we won't talk about this again. Hopefully. Maybe I can't actually commit to that kind of promise, but I will try

Runway releases first “interface world model”

16:50We haven't talked a lot about video and multimedia in a little while. And this story this week actually kind of gave us a good opportunity to go and talk about that. Basically a Runway, which is a kind of innovator, a leader in the sort of video AI space came up with the new world model called Solaris, and I thought this would be a good chance for us to talk a little bit about world models, because we haven't really covered them in quite a while here at MoE.

17:11Tim, that's such an interesting subject. World models are all the rage these days. And I'm wondering whether, although this demo is exciting, is this a new kind of software or is it just a very convincing picture of software? Chris, do you have any thoughts on I am definitely not smart enough to have any interesting thoughts there.

17:39I, I sort of get the point of the world model. I haven't really looked at the implementation details of the Runway model, and in fact they've been a little bit sparse on that. So it's really kind of difficult to to really get it, get what's going on there.

17:55But I do know it's kind of like a diffusion based model where so a lot more kind of parallel in that sense. I think the biggest thing for me is and where everybody's aiming towards is that they're wanting to get away from your kind of typical text based tokens and move more into kind of visual models, etc..

18:15And I think the biggest thing that Runway has done there is, is really sort of make that more native into the model. So if we look at kind of a standard model where we're looking at things as multimodal, whether it's an audio or, you know, or a video or a or an image based model, typically there is a separate encoder that is sort of bolted on to the language layer, and that really does the kind of tokenization.

18:41And it just sort of translates in to the deeper layers. So that's how that typically works. So it's kind of a bolt on. It feels native but it's not really. And I think what they're saying from a Runway perspective is as they've trained to model in their architecture, they're they're really sort of native from a visual perspective from the offset.

18:59But but if I'm truly honest, I'm, I'm it's been a little sparse on details on how they've tried that model. Maybe my googling techniques are not as good as others, but but but I think that's where they're going to where the Raymond thoughts.

19:13And I think the reality of that is true, that, you know, I think that, you know, if we sort of move forward and again, we're sort of back into the kind of Yann LeCun territory where he was a big sort of pusher of the kind of world models. I think there is a point.

19:28Not everything is represented as language. So the more you can be kind of multimodal, the more you can be native. In that sense, it becomes important. And then similarly, you know how you interact with the world. And I guess that's more of the kind of world part.

19:41It becomes really important, which is how am I going to interact? How do I touch, how do I feel the learning associated with that? So I think it's interesting that Runway's went in that direction. I think more models are going to be that way.

19:53Whereas I if I'm truly honest, the frontier providers are really still and the kind of the language based models, you know, today. So yeah, let's see how that works For me, I thought that was a very impressive demo, kind of for me, the first tangible preview of post software computing.

20:16So for decades, software engineers, you know, there have been deterministic kind of bridge builders. They translate, you know, kind of the human intent into code layout, tree databases and so on. But the world models kind of collapsed that entire pipeline into this continuous stream of pixels.

20:35But I think we should also inject a little bit of realism here, because there is a big determinism problem in real enterprise software we require state guarantees, strict ordered consistency, isolation, durability, all the ACID kind of rules and accessibility compliance and so on.

21:00So here, you know, this interface is kind of generated on the fly. So the pixels etc.. It's very interesting because they can do this real time like 60. But if an interface is hallucinated in real time, what happens for example, when you're checking out or you check out button is an extra zero and charges, for example, $500 instead of $50, and and also like streaming 60 neural video per active users carries also a huge I feel like astronomical inference compute cost.

21:34So I feel it's a massive breakthrough for synthetic agent environments. But determinism and deterministic enterprise code here is not addressed. So I think it would be great, you know, to use this to train, for example, robotics models and so on.

21:55But, you know, to really use such interfaces without really having that determinism, it's going to be hard in the enterprise world. So but, you know, this seems like these world models could make traditional interface code disappear, but it's just the critical reality is the more freedom we give the generated surface, the more also structure.

22:20And this discipline we need and there need that. yet I think this wanted to raise something, you know, because we've been talking about very much in terms of how do we kind of improve these systems. You know, I think one possibility this kind of model raises in the future is that everybody might have a slightly different interface on their computers.

22:33You know, the dream might be that, like, actually I go on Kush's laptop and I basically like can't use it because it looks like completely different from mine. And I guess, Kaoutar do you think that's a real possibility? Like in the future, everybody will just kind of have their own operating system, and it will kind of be customized to sort of the preferences that they've had over time.

22:55Yeah I think that's a very interesting point of view here. It would be nice to see personalized interfaces because the way you work the way so like Kush's interface will be or maybe of safety and security kind of drift. hardware software design review knows.

23:15But you know, it's a possibility if we do these things well. And also if we address the issue of the computational cost because it's not cheap to do these things real time and to customize these things so we can address these things. Yes, it is a possibility and it would be interesting.

23:30And I don't want to imagine what Chris's Yeah. building on what Kaoutar said, right? I mean, like interfaces, as they've been evolving over the decades have been very like ordered, right? I mean, they're like the epitome of modernism in a sense that everything needs to be, like structured.

23:55The buttons have to be like this, like all of that. And I think what we've been losing is kind of like the flux of the world, that that is what humanity is also about. Like we're not just about like institutionalizing everything, putting everything into like strict order and stuff.

24:16And I think this is an example of like where we could go, right. And so like there's the modernism, like that exact like structure, like there's postmodernism which like people talking about or go touch grass or whatever, but like this is like actually like something beyond that.

24:33It's like metamodern sort of approach. It's like building on top of what we have to like, try to bring our humanity back, like in a sense, but like in a way that still is utilizing what we've what we've built upon. So I think it's like not so much the underlying model or the world model or the diffusion model or any of that.

24:58It's like, how do what's our relationship with with the machines and how can we be more human working with them? So let's see. I mean, the cost is going to be a problem, certainly. But if it can happen, maybe there's a way to enchant us again in some way.

Anthropic debuts AI hardware specs

25:20Well, I'm going to move us on to our last topic of the day. We've talked a lot about in the past, this sort of really interesting relationship between sort of the model designers and kind of the people who designed the hardware and kind of this interesting question about, so who's guiding whom in this, in this stance and good chance to kind of revisit this topic with the release of a model hardware standard that Anthropic put out just this week.

25:43And I thought it was a really interesting move, and in some ways was really, I think, Anthropic trying to not necessarily do vertical integration, but certainly kind of make its preferences known to the public in a way that, you know, I think is a new kind of play in the space.

26:02And so, yeah, I curious what you thought about this Yeah. You know, this is really interesting as you said. I mean, I think the benefit is easier automation, but the risk is that the mistakes now could affect physical equipment. And then I was wondering, Kaoutar, if you agree with that framing, what is the hard stop that the model should never be able to override?

26:28yes, I think there is a risk here that, you know the behavior will change. But so I think the way when we design these hardware's and so on, there are specs, there are things that you know, they're supposed to be doing. But if that behavior now you know, you're losing that deterministic determinism that becomes an issue.

26:44So either I think these protocols or these, you know, models like hardware, they need to control a way that, you know, what is the behavior. What are we expecting from these systems. So and the hardware software design is fundamentally about assigning each computational task to the layer that it handles best.

27:05So how do we do these things. How do we do anomaly diagnosis. How do we do tactical adaptation hardware level controllers etc., the deterministic repeatability and so on. And the hard physical safety also envelopes those are become all important.

27:28And I think the Anthropic architecture highlights, you know, some of these things. You know the Claude discovers the routine, but the firmware must enforce the boundaries. And I think that is very important. So so I feel for example, the this MHS.

27:46It's the integration here in labs is real. They showcase this. So it Anthropic says connecting equipment can take weeks or months which is true. And the example that they showcased in Carnegie Mellon they try they created drivers and then orchestration layer in about kind of eight hours across a liquid handler a plate reader or robotic arm camera, three computers and so on, and many incompatible interfaces.

28:13So that is tremendous. But but I think we need also so it is an impressive proof of concept. But there is also kind of that liquidity here. How do we ensure safety in the in the physical world. Because small errors can matter here. And so and using the model for planning and troubleshooting.

28:41But we need those to keep hard safety limits and repeatable control below it. And that is that is very important to get right. And I'm not sure if they handle right now all of these issues. And in practical terms, Chris, maybe you could answer this.

28:53Would you let Claude change its own temperature or movement limits? I assume not. What's the best way of saying this. It's like I think it's a I'm in two minds for I must, I really am because I'm kind of like, well, all you're really doing here is here's MCP, here's the APIs, shove it over the top, and then we'll probably put some schemas around that.

29:22So we have standardization across the board. So can I get my model to go and talk to a piece of hardware already. If this go to expose the APIs. Absolutely I can do that today. I can just, you know, point my model out of o3 or Claude at it.

29:40To be honest, I tend to use o3 for these things, so I hate to say it this way. It complains a little less when I'm trying to hack a piece of hardware. So, you know, Claude gets herself in a little bit of, you know, tizzy about it. Oh, that's wrong or whatever.

29:52I'm like, no, you know, you know, you know, you know, please change the firmware of my Amazon Echo device, etc.. o3 is like, yeah, yeah, I'll do it, no problem. So I think, I think it's a good thing. Right. But it's probably actually I think Kaoutar helps you on your safety side of things because the reality is if it's exposing MHS in that sense then Claude knows, hey, hang on, here is a safe way you should be interacting with this in a safe way as opposed to crazy Chris with this o3 just, you know, hooking into whatever hardware device and making it dance, right.

30:25So I do think it's a good thing for that. And therefore and, and in the the space that it's meant to operate is like, you know, in the research and the, and the lab type scenarios, I think that's on a safety criticality and having standards becomes really important.

30:44I also kind of like a little bit from the idea of consumers. I would love it that every hardware item is all is got an MCP server built into it. And and then I can, I can unleash the models on it. That for me is cool, but I can kind of do that anyway, you know what I mean?

30:58As long as it's got an API, we can make the thing dance regardless. So so from a kind of safety and standard and reliability I think is a good thing. I the only thing I don't like is the if you want access to this please hit request. So I'm a researcher etc.

31:17it's like no, no. Open it up and let me play with the toys. Do you know what I mean? That's probably the bit that I object to a little bit and I can't see the specification. So it's kind of like, yeah, but I mean it's cool, but you know, I can make hardware dance anyway with the models.

31:30But also with things like this, we don't want the same incident that happened with Hugging Face and OpenAI to happen with these physical system. Imagine, you know, physical hardware and so on going crazy and altering systems and so. we're we're crazy.

31:51We you know, you know, you've got to learn a lesson. So the lessons I learned for Hugging Face, we're going to have to learn the same lessons with physical hardware at some point. And I hope it's just going to be something really small, like, you know, robot goes mad, smashes a window as opposed to something more serious.

32:06So so I these standards, they're important. It's it's like you got to figure these things out. And another thing is Anthropic really positioning as a control point for automated lab and smart factories. So but is it open and model agnostic because it could actually make Claude much easier to replace in this case.

32:26So I think I don't know if they're really making it open or and kind of a standard where you can plug and play with different models. And there's already a standard in IoT anyway. I mean, if we think of things like MQTT, for example, then why can't we just like, you know, hit MCP over the top and we've got AI's talking to that, you know.

32:48Yeah. I'll go out on a limb a little bit. So yeah, I mean Kaoutar's point is a good one. I mean, you do need standards and you need I mean, good control. But I mean, what this story reminded me of actually was Avatar. The Na'vi people, the blue guys, I mean, they would like, hook into, like, trees and to they're like birds and they're like the animals and stuff.

33:14Right. And then like, they can kind of like, feel like what these other things are feeling. Right. And so like, that's what AI is good for it. I mean, it's more on the vibe side, right? The same flux I was talking about before. And so I think what this is going to help us with this, like one microscope can now feel what it's like to be another, like a pump or laser or something like that.

33:39And I think the good thing about that, and I know I'm being a little bit facetious, but I think the good thing about that is that it's like a different kind of communication. Right? It's not just like what you could already do with the existing protocols, the IoT stuff that the Chris was mentioning, but it enables a different kind of presence, a different kind of being.

34:03And I think maybe that's something that could be interesting and that, that, that world, that would be hard to imagine. But I think maybe we should start imagining it. That's an incredible note to end on. And glad to be able to get an Avatar reference in there.

34:19Well, as always, Kush Chris Kaoutar, thanks for joining us. And Sascha, thanks for co-hosting. And that's all the time that we have for today. If you enjoyed what you heard, you can get us on Apple Podcasts, Spotify and podcast platforms everywhere, and we'll see you all next week on Mixture of Experts.