Why OpenAI is calling for a ‘cyber defense surge.’ Plus: Find Evil! winners and TeamPCP losers

Intro

00:00A new letter from OpenAI calls for a global surge in cyber defense in the face of escalating AI attacks. Panelists, What's the first move you'd like to see people make as part of that surge? Embrace it. I mean, we need to take advantage of the tools that are made available to us.

00:17We all don't have to stub our feet for each one of us to learn what goes wrong. I'd like to see the organizations that haven't had an opportunity to sign to, of course, go out, read the open letter, and then sign so that we can all be a part of this collectively.

00:36Hello and welcome to Security Intelligence, IBM's weekly cybersecurity podcast, where our expert panelists turn the biggest industry news stories into practical takeaways that you can use. I'm your host, Matt Kosinski, and joining me this week we've got Michelle Alvarez, manager IBM X-Force Threat Intelligence.

00:53We've got Nick Bradley, manager, X-Force Threat Intelligence and one of the hosts of the Not the Situation Room podcast. And J.R. Rao, IBM Fellow and CTO, Security Research. We're going to be talking about the winners of the SANS Institute's Find Evil hackathon, and the losers of Flare's recent unmasking of notorious hacking gang, TeamPCP.

01:13But first, a little bit more about that call for collective action on cyber defense.

OpenAI’s open letter

01:22So recently, a hundred organizations, including IBM, have signed on to an open letter from OpenAI urging a, quote, global surge in cyber defense to get ahead of AI-enabled attacks. So the letter reads, and this is a really good quote. So I want to just read it real quick.

01:38In the coming months, AI-enabled cyber attacks will become far more widespread and sophisticated as models around the world become increasingly capable. And we've kind of seen this already, right? With the HuggingFace hack that some of OpenAI's agents did, with Anthropic finding out that its agents were also hacking folks.

01:54So the letter calls on public and private organizations to make cyber defense an immediate priority. Second only to critical business operations, they say. The letter also includes three key principles: go beyond status quo security, empower more defenders with cyber-capable AI, and mobilize a collective response.

02:14J.R., I want to return to what you said in the opening first about how we don't all have to stub our toes to learn a lesson. I really like that. It's a nice encapsulation of what's going on here. I was hoping you could expand a little bit more.

02:25What's this letter got you thinking about for us? I think the letter is very timely because actually what it does is it shifts the some of the discussion we've heard about restricting powerful AI to one in which we say that, listen, we need to be able to get powerful defensive capabilities into the hands of trusted defenders early enough to change the whole security baseline.

02:50And, you know, I think fundamentally, for me, there are two things that matter here. One is we've spoken for decades now about sharing threat intelligence, sharing vulnerabilities and indicators of compromise. We now really have to move to the to to the next phase, which is about sharing remediation strategies and effectively sharing patches.

03:16Right. And this is the place where this is what I think about when I say we don't have to stub our toes for all of us to learn. And we have to increasingly realize that the critical metric in the industry now is it's going to be time for remediation.

03:35And it's really how quickly can you remediate and how proactively can you do that. And I think anything that helps towards that particular goal, collective defense, is an excellent idea, we should go for it. I like this idea of kind of moving beyond just sharing threat intelligence to sharing the remediation, sharing the patch, because I think it gets at another question that I wanted to raise about this letter, which was basically what is it calling on us to do differently?

04:01And Michelle, I'm going to throw this one to you. You get the lucky, you're the lucky one to get this question, right, because the letter highlights some things that it wants cybersecurity companies to do. And it says things like integrate AI into your process, bring it to your customers, share threat intel and tested playbooks, which is similar in a lot of ways to what we already do.

04:20So my question kind of is, is this a matter of just doing it more intently, doing it more intensely, or is there a qualitative change here? What do you think, Michelle? Yeah, I think, and I'd like to kind of jump on what J.R. said where he stressed about sharing.

04:33I think the word share came up multiple times in the letter. So I think we can be doing that more broadly, more extensively between private and public sectors especially. Also you know, there's also mention of leveling up, right, going beyond the security status quo.

04:51So and when I think about this, I think about, you know, you have to assess what you're currently doing in your organization and where are the gaps. And so for some, some organizations may not even be leveling the current status quo, where many of these organizations that have signed the open letter are already implementing the recommendations that are within the letter.

05:12So I think this is really a call for more and more widespread sort of implementation of these recommendations, which really is a lot of the baseline security that many enterprises and governments are already implementing. That makes perfect sense.

05:29And yeah, I think again, it really does highlight this, this idea that J.R. has brought up about, you know, we don't all have to suffer the attacks to learn the lessons, and we can't really afford to do that right now. I think with AI, especially how fast we've seen some of it move and what it can do.

05:44And you brought up this kind of point that it brings about how we can't really rely on status quo cybersecurity anymore. And I want to drill into that a little bit more. And I'm going to throw this one to Nick, because I think your answer in the beginning, you kind of touched on this already, talking about how we've got to embrace what's new but not throw the baby out with the bathwater, Basically.

06:00Tell me a little bit, what are your thoughts in terms of especially this point on going beyond the status quo security? Any thoughts there for us? Well, I think that was a much, much more politically correct way of saying it. So thank you. Thank you for doing that.

06:13That's why I'm the host, folks. Go on. It all leads back to what, and I've said this on many other podcasts. It's what's old is new again, right? And we use the word sharing over and over again. And it is. I've kind of built my career in the threat intelligence field on sharing, sharing information, because the more we know, the more we we grow.

06:33The more we know, the more we grow. No more rhyming, right? But it really is accurate because every week I bring people together to talk over the latest threats and go over what we've all been seeing, what we've been studying, what we're researching.

06:47And it never fails that you have people come across, Hey, I'm looking at that. Oh, hey, I'm looking into that. What do you know? What do you know? And so it's the information sharing aspect of this that is key because, you know, we all, we keep seeing new technology right now.

07:01You know AI is is the new shiny. But what's behind it is still the same, right. It's still people. It's still people that need to share information. It's still people that need to be empowered. It's still people that have to make it work. And so that's where I also want to go back to where it mentioned specifically empowering defenders.

07:20We have to empower defenders to be able to do almost all the same things that the bad guys can do, right. So, you know, we put these guardrails in place and restrictions in place and try to lock down AI so that it can't break bad on us. But then, you know, the threat actors get ahold of it and they're the first thing to do is take those off like, no, we're just going to make it do what we want to do.

07:40So they they change it to where it can be used for whatever they want. Meanwhile, the the people trying to defend and do the right thing are still being constrained by those restrictions. And so you have inherently hobbled the good guys. You know, it reminds me of something that Jeff Crume has said on the show multiple times, right?

07:59Which is that like the bad guys don't follow the rules. That's why they're the bad guys, right? Like, you can put all kinds of rules around how these things work for defenders. But yeah, there is a little bit of tension there. And I do feel like increasingly, defenders are getting a little miffed about that.

08:15Like I remember having EvilMog on to talk about GLM-5.2 and he was like, I think everybody's going to jump to open weight models because they don't have the same restrictions on them, and I can do a heck of a lot more with that than I can with some of these other things.

08:27So I think that's a real concern here, especially for talking about like a collective effort, is what are we empowering people to actually do? Like you said, Nick. And there's another angle here, a potentially cynical angle that I've seen going around that kind of compares this letter to—not compares, but basically says this letter is kind of a publicity stunt in the wake of the HuggingFace hack, right?

08:49Basically, OpenAI capitalizing on that to maybe make themselves look a little bit better. That's the cynical take. J.R., I wanted to ask you about that. Do you think there's anything to that, or is that overly, you know, cynical? I think we may be getting overly cynical here.

09:04Honestly. I think in the security business in the last few weeks, I think we've seen a lot of attention being paid to agents breaking out of their harnesses to cross enterprise trust boundaries and to hack into other enterprises. And we've also seen companies like NVIDIA come forth with things like the Open Secure AI Alliance, which would enable defenders to collectively to to use open weight models to defend themselves and not rely upon closed weight models alone.

09:44And I think in that sense, some of the work that OpenAI is doing is, I see, is putting its shoulder to the wheel to push that endeavor further. It should not be judged in isolation, but it's it should be judged as recognition of the fact that the game has changed with the power that AI models have and and that we really need to do something differently.

10:11To close out this segment, though, you know, we talked a little bit in the intro about kind of what we want to see folks do next, but I'd like to kind of close out actually on a slightly different question, which is, are there any pitfalls you're worried about people falling into as we go forward here?

10:26Michelle, I'll ask you, any any pitfalls you think we need to watch out for as we try to ramp up this surge in cyber defense? Yeah. So I think, you know, in terms of possible pitfalls is not putting a plan in place, right? So just sort of rushing to do maybe you know what you've seen your competitor do.

10:44You know, what you've read about, but maybe not have done like a self-assessment, like a risk assessment for your environment and sort of, you know, rushing to the next tool or solution that may not be the right fit just to do something. So I still think there's time to take a minute, take pause and think about things sort of in a more pragmatic way.

11:08Yeah, that makes perfect sense to me. Nick, how about you? Any pitfalls you think people should avoid? I think I actually stated my pitfall in the opening question, right? You did, you did. Yeah. Let's go. Let's go ahead and embrace this. But the pitfall is don't, you know, don't get rid of your skilled labor that needs to know how to do things just because you think AI is going to replace them.

11:27Remember, it's augmented. It's augmentation, not a replacement. Absolutely. And J.R. round it out for us any pitfalls you think we should watch out for. Yeah I think maybe Michelle referred to this as well. Collective defense is a great idea but inherently there will be an asymmetry.

11:45We know the banks and the technology companies will get really sophisticated AI defenses. Let's not forget the hospitals, the municipalities, the water systems— ring a bell?—and small critical infrastructure operators, right? So we need to worry about them as well.

12:02Folks watching on YouTube. The comments are open. Remember, if you've got thoughts about this open letter, about what's going on, about a surge in cyber defense and what you'd like to see happen, drop them. I do read, I do respond. I'd love to hear from you.

12:13But I've got to move us along to our second story for this week. SANS announces the Find Evil winners.

Find Evil! winners

12:24So after the OpenClaw debacle earlier this year, which I'm sure you all remember, the SANS Institute called for a hackathon to develop AI agents for good. Find Evil, as they called it, asked people to develop harnesses for autonomous incident response agents primarily focused on investigation and forensics.

12:43The results are in, with five winners named, and all of them are available on the SANS SIFT workstation. Michelle, I want to start with you and just get initial reactions on this kind of hackathon approach to developing AI security, especially, we just talked about the need for a collaborative effort.

13:00This kind of feels like one of those. How are you feeling about this? It does feel like the two stories sort of marry each other right here. We're calling for more collaboration, more sharing, and this hackathon certainly did that. And as to, you know, agents and doing forensic analysis.

13:16So I think one of the things that stood out to me just from the results of the winners, that the ones that won didn't necessarily create the fastest harness. It had way more to do with, like the smarter. Like is it questioning itself? Things that we would want these agents to do in an actual incident?

13:38So I think that was interesting in and of itself that we're not necessarily looking for, of course, looking for speed. Right. We want to close in on that defender window that is closing in on us. But and we want to speed up our investigation process, but we also want to make sure that we're asking the right questions, much like the human would, right, in the loop.

13:58So I thought that was of interest. I'm glad you brought that up, because that was the thing that really stuck out to me too, was like, this was a hallmark of like, I think every single one of the winners, it had a kind of self-questioning like, you know, ability.

14:10And it would push back on its own findings. It would even, there was one that even, they found it pushing back on its own briefing when it found information that didn't, you know, align with the actual briefing itself. And that was really nifty to me because like you said, Michelle, it was like, yeah, speed's important.

14:23But there's this recognition that like the real skill of investigation and incident response is knowing when to ask questions, even about the questions you're asking. Right. It's like a second order kind of thing. Nick, how about you? And especially did that self-critical faculty in the agents stick out to you too?

14:41Did other things stick out to you? How are you feeling about this one? It did. And so what I, what I really liked about this is, you know, what's what's even better than sharing and collaboration? Making it, incentivizing it. Right. So at this point we're talking about information sharing, collaborating on solutions.

14:58But now you put some incentive behind it. That's a golden opportunity right there. And so I think more of this. Now I will tell you the part about this, that kind of, you know, chapped my hide a little bit, but I have to I have to eat crow in this case.

15:13And that's because all five of the winning harnesses were open source and free. And, you know, I'm not the biggest fan of open source, but I will say it has its place. And in this case, it seemed to it seemed to really have its place. So my hat's off to that.

15:28Yeah. And I think, you know, I know that you're not the world's biggest open source fan, but I do think, especially as we talk about the need for collaborative tooling, I think we're just going to see more of that. You know, man, I think that the open source is going to be.

15:37It has to, you know. It's going to be a big part of it. J.R., how about you, looking at the Find Evil hackathon? Anything sticking out to you? What caught your attention? Where are you landing here? You know what what sticks out to me is that, you know, finding evil and acting on evil are fundamentally different activities.

15:58I think what Find Evil actually showed is that, you know, autonomous cyber investigation is becoming very credible, that, you know, this is where we can go. But what it didn't prove to me is that we should hand over unrestricted incident response authority to the agents.

16:14And so we have to be able to distinguish between investigation and response. Yeah, absolutely. Use the agents, you know, to correlate evidence and reconstruct timelines and do threat hunts across machines, generate hypotheses and so on. But the action part, they are shutting down servers, revoking identities, changing firewall policies, stuff like this.

16:36Terminating processes. I think you do need human in the loop, still, given some of the uncertainties around agent actions. That's really interesting to me. I hadn't thought about that, that that kind of clear delineation. But you're right and none of these models do kind of do the actual response.

16:54Right. They do the investigation—or not models but the harnesses, you know what I mean? And I think you're right. I think it's worth drawing that distinction because the stakes are a little bit higher if you let the agent respond rather than just investigate.

17:06Right. Like you got to have a person in there making that call. That's fascinating. And it kind of also got to where I wanted to go with this conversation, which was, what does the future of autonomous agents in cybersecurity look like for you folks?

17:20And you know, J.R., you're bringing up this idea that we have to think about how we use it for actual response and whether humans are still there. Michelle, any thoughts on your end in terms of where do you think this autonomous agent stuff's going to go for cybersecurity?

17:32How do you feel like the future's looking? Yeah, I mean, I think we can each individually think about how this would impact in our current role and how we're actually using it. I know Nick can speak to it quite a bit in terms of how he and his team are leveraging it, but I do see where we can do things that are autonomous in nature, where it's not going to have an impact necessarily in production.

17:58Right. So we can leverage the agent in a way that we can do Analysis, we can corroborate and corral the data and look at it in a way that we can make an assessment, but not necessarily just have it autonomously push out to publication. Right.

18:20So we're writing a report. We're able to leverage these tools in a way that allows us to speed up analysis, but still have that authority over whether or not we push it to production. Absolutely. Again, like leveraging the importance of a human being, being in that seat, especially because humans are going to be accountable for that, whatever gets put out there anyway, right?

18:40Like so humans should be making that decision. Nick, anything you wanted to add there? Michelle talked about your team using stuff like this. I don't know if there's anything on that front you want to talk about, but anything to add on the future of agents?

18:49Autonomous agents in cybersecurity? Sure. So one of the things that really got my attention here, other other than what I brought up already, was also the self-testing, like the self-questioning, the am I sure I'm right about this? And that's something that AI doesn't really do very well.

19:06So to see that happening now is a good thing, because it is all it is all too often that I have used this, as Michelle said, we use AI to to help as a force multiplier, but sometimes it will just make assumptions. And you look at it and you know it's wrong and you tell it.

19:21Can you please reassess this part of the investigation that looks inaccurate? Oh, you are right. That is inaccurate. Let me double check that. It's like, well, wait, if you knew it was inaccurate, then why did you tell me that in the first place?

19:31Right. And so those are the those are the parts of this that that's really why you have to have that human in the loop. Right. Because it does help us to write reports faster. I'm able to grab, you know, a little bit of this from here, a little bit of that from there and put it together into a report.

19:48But then I still have to make sure it's accurate. I have to still make sure it got the point of what I was trying to get across, and then have another person proof it and publish it afterwards. So there's still that human element, but the fact that we got to the proof and review stage so fast is where the AI helped.

20:06Instead of having to take a whole day or two days or longer to actually write the report first. So that's the point. It helps. It's a force multiplier, gets you, it gets you closer to the finish line, but don't let it take you across the finish line.

20:20I'm glad you brought up that hallucination issue, because it's a it's a very important reminder that even the investigation can't really be totally turned over, right? Like even there, you've still got to have a person looking. And even if these things are good at self-questioning, they hallucinate.

20:34It's not outside the realm of possibility that they could hallucinate about their own self-questioning, right? So like, there's got to be someone looking at this stuff. J.R., close us out here. Any final thoughts on on the Find Evil winners or on autonomous agents and cybersecurity in general?

20:45What are you thinking? I think there's basically a gradient of autonomy that goes across the lifecycle of threats, by which I mean the whole process of, you know, identify, protect, detect, monitor, respond, recover. And I think the the initial phases, like investigation, recommendations, are becoming, you know, that's absolutely the place where AI can play a great role.

21:13And we've seen that with with the programs we've seen, like Glasswing and Daybreak. I think getting into remediation and getting automation and AI involved with that is where the game is at. How we do this in a more of a bounded way, maybe for low-blast actions, or low-blast-radius actions, and then for high consequence actions, having a human involved is probably the way to go.

21:41Absolutely. And I think that that circles back very nicely to what we talked about in that last segment. You're talking about sharing the patching and the remediation as well. It feels like getting AI involved in that is how we start to share those patches and those remediations.

21:54Right. We're going to move on here though to our final story for the week here folks. This is unmasking TeamPCP.

TeamPCP takedown

22:06So we've covered TeamPCP on the show before, attached to the LiteLLM breach. To recap for those who don't remember, TeamPCP stole a service account token for security scanner Trivy, published a malicious release, LiteLLM ran the malicious release, and bang!

22:18TeamPCP had a backdoor into LiteLLM and other programs running it. As you might have heard, a couple of alleged TeamPCP leaders were arrested in Australia last week after researchers at Flare identified them. They published a report about how they identified them.

22:38And it's kind of hilarious to me because it all came down to very bad password hygiene and reused names across platforms, right? Basically, they saw the name deadCatX3 associated with TeamPCP. They found some other accounts using that same name.

22:54They were all attributed to the same person. They found the passwords, searched for those, they were the same passwords. So the bad guys basically got caught for having bad credential hygiene, which is very funny to me. And Nick, I know you folks have covered TeamPCP shenanigans on Not the Situation Room, so I'd love to open up with your reaction first.

23:10What are you thinking about here? How are you feeling? I love seeing them get their just desserts and allegedly so it just goes to prove that they are no different than the rest of us. I mean, they make the same mistakes. They they take the same shortcuts.

23:27They do the same silly things that get us compromised sometimes. Right. And I think the funniest part to me about it is it also was a Steam account. So it was somebody's video game account and it was it's just that I guess that I'm untouchable feeling where I can just use my same hacker name as my gamer name.

23:48No one will ever figure it out. That's brilliant. Please keep keep doing those things. It is a very nice reminder that these folks aren't necessarily supervillains, you know what I mean? Like, it is just regular people doing this kind of stuff.

24:02And like you said, they make the same mistakes we make. You know, I just that, that ultimate irony of like, Trivy getting unfortunately caught because the security token wasn't, you know, rotated. It's very funny to me that TeamPCP got caught because a username was used on the same, on a Steam Account, on a video game account.

24:19You know. J.R., how about you? I see you laughing. I see you nodding along to what Nick's talking about. What are you thinking about here with the TeamPCP unmasking? What I really like is the symmetry, right? Exactly as Nick was saying this, right?

24:31I mean, there are two stories to this. There's a story about how the attackers moved from, TeamPCP moved from attacking cloud environments to attacking the supply chain, and sort of exploiting the whole software trust graph. And then replaying the same thing, we see how investigators are learning, learned how to traverse the identity graph, if you will, and nail the attackers at their own game.

24:56But to me, and so I like that symmetry, that symmetry is very pleasing to me. But the thing I, I think that always brings out a chuckle in me is how in cybercrime, ego, right, pure ego has created telemetry for us. And so that's awesome. I mean, I think that's, you know, we were able to nail them just out of ego.

25:17Yeah, that symmetry is really nice. And I like the way you put it that it's almost like this investigation is kind of like the defender's version of like a supply chain attack, quote unquote. Right. It's like it's going through that kind of funnel, these related pieces, putting it all together and getting like maximum, you know, blast radius, if you will, for an investigation.

25:36Right. That's really that seems like something useful to keep in our back pockets. Michelle, how about you. Any thoughts here? Anything about the Flare investigation, about TeamPCP? Where you landing here? Yeah. So it makes me think that based on this story.

25:50Right. Because everything like we just talked about earlier is, you know, awareness, right. Anything that is a news story that everyone is looking at, right? Both good and bad guys. So how many groups out there are now like resetting their passwords, right.

26:06They're now making sure they're not going to have the same operational mistakes that TeamPCP has made. So it's kind of you know, it's one of those things, right? Like they put out this report and of course we want to, you know, we do that as well, right.

26:21Like look what we did. We were part of Operation Endgame, that was recently announced right where we took down some of StealC infrastructure. So that's huge right. And we want to keep doing those things. But as we do those things we're also letting the bad guys know, hey look.

26:36Look how we found you. And then I'm imagining they take that back and and then try to clean up their infrastructure. Michelle, I'm hearing a business opportunity here. I think what we need to do is put together, you know, a security hygiene training class for threat actors.

26:56That's going to. That's going to be cut from this program. I know, you see, all you do is you just you ask, you say, hey, give us your passwords or we'll tell you how to make them stronger. Wink, wink, you know, and then you've got all the passwords.

27:10The joke slash irony behind what I just said is it doesn't need to be a special training class for threat actors. Just use a standard security hygiene training class because it's the same for all of us, that that's the overall point. It was the same security, hygiene, bad practices that we get busted for.

27:27Well, we being the good guys that bust the bad guys. Nick. You're not going to get them to sign up for your threat actors class, because they're not in the business of taking classes. What you should get them to do is to come teach the class, because pander to their ego and ask them to teach the class on security hygiene, and that's how you get them.

27:46Good call. Appeal to their ego. Appeal to the ego, yeah. I'm imagining a whole new kind of honeypot where you get them to teach a class that's filled with just chatbot students, you know, posing as students, and then. But anyway. No, but I do.

27:59What's. The resetting password thing is very interesting to me, because I'm sure that some of them will take this to heart, but I'm sure that a lot of them won't. And I'm also sure that even the ones who take it to heart might still miss something.

28:11Because I keep thinking about how Trivy got breached and they did the right thing. They rotated everything. It was just one little token, didn't get rotated, it didn't get caught, and that was enough. And so it's like, look, man, if the hackers are out there using the same name on every account, maybe they forgot about an old one on a forum somewhere, you know what I mean?

28:29And so it's like that too, is a lesson here that it's tough for us to do our own credential hygiene. It's tough for them to do it too. I have to close out the episode for us here, folks. That does it for this week. Thank you so much to our panelists Nick and Michelle and J.R.

28:42Thank you to the viewers and the listeners. Thank you to the producers. Subscribe to Security Intelligence wherever podcasts are found so that you never miss an episode. Stay safe out there. And remember, if you're one of the good guys, change your password often.

28:54If you're one of the hackers, just leave it, man. It's fine. You don't have to worry about it.