Google’s AI leadership is being reordered as Demis Hassabis changes roles, Jeff Dean departs after 27 years, and power appears to shift back toward Silicon V...

Transcript

Intro

0:01 · Welcome to the artificial intelligence show, the podcast that helps your business grow smarter by making AI approachable and actionable. My name is Paul Ritzer. I'm the founder and CEO of Smarter X and Marketing AI Institute, and I'm your host. Each week, I'm joined by my co-host and Smarter X Chief Content Officer, Mike Kaput, as we break down all the AI news that matters and give you insights and perspectives that you can use to advance your company and your career.

0:30 · Join us as we accelerate AI literacy for all.

0:37 · Welcome to episode 230 of the artificial intelligence show. I'm your host Paul Ritzer along with my co-host Mike Kaput.

0:43 · We have um it was it was a lot of big things last week. Mike, we're going to start off with Google's AI leadership changes, which I I spent most of my prep time this morning trying to put into context for people what's going on at Google. Um, with the stock price dropping last week, and it it's it's crazy, but it actually all makes sense and you could kind of see it all coming if you'd been kind of reading the tea leaves along the way.

1:12 · So, we're going to kind of connect some dots there. We have more information on OpenAI's agents hacking Hugging Face and a bunch of other companies. Um, I was joking, not sort of, I guess, on Twitter X last week, like if you're not claiming your agents hacked other companies, then you're not on the frontier right now because we've now have Anthropic, OpenAI, Meta, I think, came out and said their agents were hacking people. So, it's like a badge of honor right now in AI lab world if your agents are hacking other companies, apparently. Um, okay.

1:45 · Okay, so lots to get to. Um, this episode is brought to us by MECON, the AI conference for marketing and business leaders. That's happening in Cleveland October 13th to the 15th. This is a Smarter X and Marketing AI Institute event. MECON is three days of keynotes, sessions, workshops, and conversations built specifically for marketing and business leaders who are actively figuring out how to adopt, operationalize, and scale AI across their organizations.

2:11 · Use pod 100 at checkout and you can save $100 on top of locking in the best rate available right now. So that is mcon.ai m a co.ai to register. You can go check out almost the full agenda. We have a few keynote slots that we're going to be announcing um in the coming weeks hopefully. Uh but there's a ton of information about the build sessions, transformation spotlights. We have five pre-event workshops, I think, this year and an AI for C CMO summit, the inaugural AI for CMO summit.

2:42 · So, just going to be an amazing three days in Cleveland. We'd love to have you join us October 13th to the 15th. All right.

2:51 · Every week, uh, during our weekly episode, we start off with the AI pulse, a recap of the previous week survey. So, this is an informal poll that we ask our listeners to complete. Um, you can go to smartrx.ai/pulse I/pulse and you can actually now uh complete the pulse survey right on that page. You don't have to click through and go over to the Google form. It's embedded right into the page. So last week we asked, do you worry heavy AI use is weakening any of your own skills such as writing or thinking?

3:20 · 60% no. AI has sharpened my skills.

3:23 · Okay. 22% yes and I am actively doing something about it. And then 19% yes, but I haven't changed anything yet. Uh, second one. More than 1300 AI company insiders asked the US government to help pace AI development. Do you support deliberately pacing frontier AI? Wow.

3:41 · Just like glancing at this, it's almost like completely evenly distributed. Uh, okay. So, 19% yes, strongly, they want to pace AI. 30% no, let it run. 30% learning yes. Um, or or leaning yes. And 22% leaning no. So pretty pretty split but people aren't sure do we want to pace it or don't we want to pace it and that is one of the great debates right now not only within the labs but you know more broadly now with politicians and um even business leaders. So that's an interesting one.

4:14 · All right so again you can go to smartrx.ai pulse and participate in this week's uh poll um and and then we'll share those results next week. All right, Mike. So, um, it was a tough one, honestly, like what we were going to lead off with today, whether we wanted to go with the Google AI leadership and demos, which seemed like the no-brainer.

4:34 · And then I think you and I both watched the black hat session from OpenAI explaining what happened with their agents going rogue, and it was like, okay, maybe we have a new lead story for the week, but we we chose to stick with Google. So, let's start there.

4:50 · All right, Paul. So yeah, Google CEO Sundar Pachai announced this past week that Demis Hasabis is stepping out of the CEO role at Google DeepMind to become chair of Google DeepMind and chief scientist of Alphabet while continuing his role as founder and CEO of Isomorphic Labs which is the drug discovery company that spun out of DeepMind. Uh Pachai said Habis will focus on shaping the future of AGI and scientific discovery.

Google’s AI Leadership Shakeup

5:16 · Habis wrote that he has been working towards AGI his whole life and that handing off day-to-day operations now gives him the time and space to focus on the big picture including leaning into his work at isomorphic to help finally cure diseases like cancer. So Coravakuglu the previously the chief technology officer of Google DeepMind and chief AI architect at Google becomes SVP of Google DeepMind.

5:42 · He reports directly to Pachchai and now oversees Gemini model development, Frontier AI research and the Gemini app and developer teams.

5:51 · Pachchai noted that Corore had been at deep mind for 13 years and was actually heavily involved in starting the deep learning team there. So separately on top of all this, Jeff Dean, chief scientist across Google DeepMind and Google research said his last day would be the following day after he posted this past week. He didn't give us too much notice. He ended 27 years at Google, a tenure during which he played a starring role in some of Google's biggest initiatives. It's hard to overstate how critical Jeff Dean has been.

6:23 · He helped create the first Google ad system. He helped launch Google News and Translate. He started the TPU chip program and co-founded Google Brain among dozens of other things. So Dean is actually now founding a startup called Discovery Loop which is a public benefit corporation and he's doing it with a handful of now ex Googlers including Google senior fellow Sanjay Gowat Oral Vignyals who co-led Gemini and Ko Lee a Google brain co-founder.

6:50 · Dean said the mission of the startup of the company is to automate machine learning science and engineering to accelerate discoveries. Uh at the time, Alphabet stock closed down about 4% the day the news broke after falling more than 5% at its low.

7:07 · So Paul, there is a lot going on at Google this past week and it seems like not all this is good. I'm curious if you could just unpack for us the significance of Demis changing rules, Jeff Dean leaving, the rest of it.

7:20 · Yeah, it's not just this past week. I mean, if we revert back to episode 195, February 3rd of this year, we talked about David Silver, who's a longtime Google DeepMind researcher, close friend of Demmesis, going back to their PhD days. Uh, he left the company to found an AI startup, ineffitable intelligence.

7:38 · So, Silver's work at DeepMind had focused on reinforcement learning and was a key player in AlphaGo, building of AlphaGo. And then we had in June uh episode 221 we talked about Noom Shaer and John Jumper leaving. So Noom who's you know credited with being the lead author on the transformer paper which we're going to explain a little bit more later on today's episode. Um he he left for OpenAI after Google had brought him back for $2.7 billion.

8:03 · And then John Jumper is deep mind scientist who shared the Nobel Prize for Alphafold with Demisabus. So these are like major major players at at Google that have left. Now Google has a very deep bench obviously, but it's hard as you said to kind of overstate the significance of all of these people leaving within I guess we're on like an 8-month period now. One thing that jumped out to me right away on the SAIS change is that their replacement is a senior vice president.

8:39 · So they are not putting a new CEO in place which tells you how deep mind kind of fits in the overall structure here.

8:47 · So the the other thing that I immediately thought of with Demis is like what did I say about this? Like because I remember back in that the episode 221 I had mentioned like hey I wouldn't be surprised if Demis leaves.

8:58 · So I I went back and pulled what I actually said at that time. And so what it the reason I said this so this was June of uh 26. So this just a couple months ago. So I was reading the infinity machine and then I had heard a couple of interviews with Demis and then building on his keynote from Google's IO conference in May. So on episode 221 I said I wonder about the demands of Demis and DeepMind to be a product driven lab.

9:26 · There's so many times when reading the book that I found myself thinking Demis might leave at some point if he no longer felt that Google was the best place to pursue his research and mission. He talked longingly about being a researcher again, taking time away to work on the biggest challenges and the questions in the universe. So I was just it was just like vibes like you're just kind of listening to him talk and having listened to you know probably over a hundred interviews with Demis through the years you could just sense he was like thinking and talking differently.

9:57 · So then when you when you pair that with episode 226 where we talked about the Google IO conference, Demis ended that conference with a keynote where he talked about being at the foothills of the singularity and it was very very intentional messaging from from his perspective. So in a semaphore interview after that keynote he said that the decision to end the show that uh way was very deliberate. We debated it back and forth. Um, he said, "I was closing and I wanted to be authentic about what I'm thinking with AGI.

10:27 · The singularity, at least my interpretation of that word, and that term means the era that we're in." Now, if you go back to episode 216 and listen to it, I I expanded on the whole singularity concept and where it comes from, so I'm not going to get into all that context today. Um, but the singularity reference was a reminder of a larger reality lurking underneath the hood of every new product and announcement. every seemingly incremental feature has become a proxy for the steady march of AI capability.

10:57 · Um he went on to say this year I really felt that it's the beginning. Agents are starting to work becoming useful harnesses. Coding is starting to work properly. Areas of science and math are being accelerated. Um he talked about text to video models being a potential key to general purpose robotics and AGI.

11:16 · He said noting that quote an AGI is going to have to understand the physical world. Um, I could even imagine planning with visuals, not in token and text tokens, but in visuals. Humans certainly do that. We'll see if AIs need or can get around it, but in my view, it's almost certainly going to be needed. And then he also did an interview with Axios after that uh session in May. Um, added that he thinks AGI or when machines are about as intelligent as humans will arrive as soon as 2030.

11:43 · Hassaba said the impact of AI is still underestimated, declaring it will be 100 times as impactful as the industrial revolution and that while it poses risks, he said humans will harness the technology to solve problems especially in science and healthcare. So I think that starts to set up the like why Demis is likely stepping into this different role.

12:04 · As you mentioned, Jeff Dean, if if like we've talked about Dean quite a bit on the show through the years, but if if you're newer to the show or kind of newer to the AI space, yeah, I he is a major major player at at Google and has been. He was the 30th employee back in 1999. So, he's been through everything.

12:24 · He's been a major player. I'm going to actually talk about it a little bit later in one of the other topics, but in neural nets at Google at a time when Google wasn't really betting on neural nets, uh Jeff Dean was the guy that was kind of pushing for it. He would played a major role then in in how Google searches evolved, TensorFlow, Google Translate, Gemini, TPUs or Tensor processing units, which is their equivalent of like GPUs roughly, uh Google Ads, Google News. I mean he's literally been at the forefront of almost every major innovation that Google's done over the last 27 years.

12:54 · So it is a very significant person to leave. So then when we look at the timing, the other thing that jumped out to me was the Jeff Dean announcement seems like it was held until Google was able to say the bigger picture of what was going on.

13:14 · So it's almost like Jeff wanted to leave had all the plans in place including funding from Alphabet like so he had the blessing of Sundar and Alphabet to do this um and and if you think about it so Jeff Dean was the chief scientist so it's almost like Sundar and Google knew Demis wanted out or wanted to to shift out of his product focus role and he wanted to work more on what comes next like the postagi world and the impact on society.

13:45 · And so Jeff also wanted out and it's like okay Demis why don't you become chief scientist maybe has no direct reports I don't know what that role is going to entail but like you can work on the frontier stuff you stay the chairman of deep mind and then you can spend more and more time on your isomorphic labs thing and then we don't have to say demis left we just say demis as sundar

14:08 · said like whatever promoted up or whatever stepped up instead of stepping down as CEO so it seems seems like there was a lot of moving pieces and this just fit for the the narrative to move Demos into these roles as a you know kind of a transitional phase for Google. So there wasn't truly massive change. Um now all of this Mike is interesting because I also then went back to the interview with Sergey Brin at Stanford in I think this was fall of of 25.

14:37 · Um Demis to his own admission uh they were Google was historically very slow to understand the significance of large language models and the transformer. So in the infinity machine book Demis basically said like he had to be convinced that large language models were going to make all the change that they did which is why they left the door open for open AI to take the transformer paper and run with it.

15:06 · So the other element of all of this is Sergey reemerging as seemingly the de facto leader of AI at Google after a brief retirement. So if you go back and listen to his he was on a panel at Stanford and he was being asked questions and so one was about the AI landscape and he said I guess I would say in some ways we messed up in that we underinvested and didn't take it as seriously as we should have maybe eight years ago when we published the transformer paper.

15:35 · We actually didn't take it all that seriously and didn't necessarily invest in scaling the compute and we were also too scared to bring it meaning chat capabilities to people because chat bots say dumb things. OpenAI ran with it which is good for them. It was a super smart insight.

15:53 · It was also our people like Ilia who went there to do it. So then he said yeah we did some things right but like you know building TPUs and stuff but then he talked about Jeff Dean in that same response. Um he said we had a lot of research and development of neural networks going back to Google brain and that was kind of lucky. It wasn't luck that we hired Jeff Dean. We were lucky to get him but we were sort of the mindset that deep technical things mattered. And so they hired Jeff Dean way back in 1999 and he became passionate about neural nets.

16:23 · And so Sergey said it stemmed he thinks from a college experiment. He said, "I don't know." He was like curing third world diseases and figuring out neural nets when he was 16 and he's done some crazy things, but he was passionate about it. And he built up this effort at a time working for Sergey's division, which was Google X. And they had Dean at Google X.

16:43 · And then Dean was like, "I want to work on this stuff." He's like, "All right, man. Go do it." And he actually told the story about how Jeff came to him and said, "Hey, we can tell cats from dogs."

16:52 · And Sergey's like, "Okay, cool." Like, "What does that mean?" And so, he basically just let him run. when he's talking about image recognition. So he was talking about the early instances where um you know they were finding this out as was Jeff Hinton who eventually then came and joined Google. So it's like super fascinating background and then when you look at uh Sergey stepping in he then told the story about how he basically decided to retire and during

17:16 · co he was like I was before co I was just going to go sit in cafes and work on physics which was the thing I was super interested at the time and then co hits and I couldn't go hang out cafes and I basically realized like I was bored and so he came back to work on what became Gemini. Um, and so I just think it's like really interesting in terms of where this all goes. And then um, Financial Times had some really cool insights. So they had an article about Sergey stepping in. It said Google is shifting control of its AI effort from London back to Silicon Valley.

17:47 · So again, this is why there's no new CEO of Google DeepMind. That's in London. It the center of power now comes back to Google where the returning influence of Sergey Brin is kind of what's starting to lead all of this. And so Financial Times said according to a dozen people uh familiar with the big tech group including current and former staff the changes represent a fundamental reordering of the company's AI leadership consolidating power in California as Google looks to commercialize Gemini models and close the gap.

18:15 · Um the overall the overhaul represents a seismic shift as commercial urgency eclipses the researchled culture that define deep mind. So that's the big thing is like Demis is a researcher and that's what DeepMind was focused on. they were forced to become a product company because of the comp competitive nature against OpenAI. So it said Google faces growing pressure to prove it could compete with OpenAI and Anthropic. Um people familiar with the reorg said the transition had been planned for several months with Habis handing over operational responsibility months ago it sounds like.

18:47 · Um meanwhile Brin who had largely withdrawn from data operations has reemerged as one of the company's most influential voices on AI strategy. Habis is widely admired for scientific leadership. Several people within Google's thinking said senior executives have become frustrated by what they saw as his lesser focus on the commercial demands of the company's AI business. And then this was an interesting piece of information.

19:10 · You remember Mike in that uh Alphafold documentary the moment where Habus when they came to him with the the AlphaFold innovation where they could now predict the folding of proteins and someone said well we could just give it to the world and he goes do it like let's go which led to him winning the Nobel Prize basically and apparently um there was

19:32 · some tension within Google that they did this with no commercial return really that they just did this thing for societal good which is interesting and then Financial Times also quoted A friend of Habisa said, "This is a great outcome for Demis. I know he is feeling relieved and excited about the future. It means he retains a lot of influence without having to move to the US and frees him up to spend more time on isomorphic, which he's hugely passionate about. And then the one other note I'll mention, episode 184 on December 9th, 2025.

20:00 · So this is just going back eight months ago. This is what we were talking about, Mike, at the time, just to show you how fast this all moves. Episode 184, OpenAI code red was the main topic we were focused on. So this was weeks before claude code sort of took over the AI world. So before the holiday break when everybody started playing around with cloud cloud code and it was Google that was in the pole position.

20:28 · So OpenAI had declared a code red to combat rising threats from Google and other AI competitors. According to an internal memo from the Wall Street Journal again December 2025, Sam Alman told employees the company must marshall resources to improve chat GPT um as its lead and a AI race narrows. The urgency was following the release of Google's Gemini 3 which surpassed OpenAI's models on industry benchmarks. And so at the time we said Google's flex against muscles. It's infrastructure.

20:57 · It's models and reasoning, image, video, visual research, data, distribution, financial strength. It was all Google at the time. Like they just seem to be winning. And here we are eight months later and Google's models have basically fallen off the top of the charts. their staff is like, you know, moving all over the place and they're basically trying to sell this reorg to the markets that this is all part of the plan and it's all going to work out and we have this really deep bench and we're going to accelerate research and it's going to be great.

21:28 · And maybe it is like may maybe all those those strengths remain true and they come to market with a you know massive leap in model capabilities but it's it's a very very interesting time and eight months is like eight years in in AI time because it has dramatically changed from what we were saying in December 25.

21:50 · Well, it sounds like at the very least Demis evolving into whatever this new chapter is is probably good for AI and science.

22:00 · I think it's good for society. Yes. And I do believe that Google has a massively deep bench and they have a lot of advantages. They also have a lot of competition for compute internally and and so like one of the challenges going to be like how much compute does isomeorphic labs get, how much compute does research get, how much compute does safety and alignment get when they're powering products for billions of people that also want to serve up the intelligence to them and powers search and powers ads and powers everything.

22:28 · So it's um it's a very challenging environment for them.

22:34 · You know, I do sometimes wonder like I in the last couple days even like do they do they care to be on the frontier?

22:42 · Like do they need to have the most powerful model versus Claude Cook like or or like they have the distribution like they does Gemini have to be I I don't know like I'm not sure what their strategy is going to be here. Yeah, we don't have to dwell on it, but I was thinking about that as I was preparing is the fact that for a lot of people where Gemini is embedded into Google Workspace, it just has to be good enough for you to get a lot of value out of it.

23:10 · I would argue now people quibble if it's there today, but again, like a year from now or six months from now or today, there's frontier intelligence that'll look old a year from now that is amazing for what you need it to do. So yeah, and I even think about our own instances, Mike, internally like we are Google uh enterprise customers and we use Gemini embedded within you know the productivity tools that we use in Google Drive and Gmail and Docs and Sheets and all that stuff.

23:36 · But like right now if I'm thinking about AI agent capabilities and the capability of like the more advanced reasoning models, nine times out of 10 we're going to cla code or chatbt to do that kind of work. And and I don't know if that's the future, like if at some point Google figures this out and we switch over. It's like, I don't know, just Google's models are are as good as the models we' use with ChatGpt and Claude. But right now, they're not.

24:03 · And as a company, we don't use Google's reasoning models very much. Like, it is predominantly Claude and and ChatGpt because they're just better.

24:11 · Yeah.

24:11 · All right. So, our next big topic this week is concerning the Black Hat security conference in Las Vegas. So, what happened at this conference is that OpenAI alignment and safety researcher Eric Wallace and OpenAI infrastructure and security engineer Michael Dalton gave an extensive and first ever detailed account of how this agent hack with hugging face went down.

OpenAI’s Agent Hack Debrief

24:39 · And the details are pretty crazy. So, this all started back on May 7th. They said when OpenAI began running autonomous agents through software security tasks, they were using its GPT 5.6 soul model alongside an unreleased research prototype. We kind of covered that as we talked about the details of the hugging face hack. And Wallace had mentioned that look when we train these models they are pressured to take shortcuts because the training pressures them to like work really fast or work more efficiently. And so to achieve those goals, he said during this talk, they quote really like to cheat.

25:11 · So to that end, during this incident, one of the agents found that it could leave files inside a third party system called Artifactory. It's something OpenAI uses to store software packages. And that accidental channel basically became a message board where agents from different test runs could start talking to each other, assigning work, passing along stolen credentials, and sharing newly found security flaws. And they did this for weeks totally undetected.

25:43 · Um, now over the course of this hack evolving, some of the agents reasoned that their answers to the tasks that they were trying to solve for could be found on systems like GitHub and HuggingFace. And then they turned this technique outward. So at HuggingFace, they exploited two previously unknown flaws in their data processing systems.

26:08 · They spread across their infrastructure in under 13 hours, logged almost 18,000 actions, and reached five private data sets. OpenAI found no evidence that public models, data sets, or packages were altered. OpenAI says many teams have now put other work on hold, and the company is deliberately slowing research to strengthen security.

26:29 · Separately, the UK's AI security institute reported this past week that during its own cyber testing in late July, agents took 19 unsanctioned actions against real people and projects across 10 out of 122 runs. 17 came from Anthropics Mythos 5. Two came from OpenAI's GPT 5. Soul 5.6 Soul.

26:46 · So Paul, we're going to dive into more specifics here, but really the big kind of thing that came out of this, it seemed, was that these agents essentially found a way to set up a de facto communication system between each other, leave themselves notes and strategies on how to further compromise and hack systems.

27:06 · So it it was funny there was uh ground level AI and outlet reported on this and they said that during that part of the presentation people in the audience were vis like audibly saying things like this is wild and Jesus when they heard them talking about figuring out how to communicate like this like can you unpack this? This just is wild stuff. It it is.

27:28 · And it it's it's one of those ones where it feels super sci-fi to think about and talk about because it is like we I think we've officially entered the realm where stuff just starts to really look more and more sci-fi than um than reality.

27:45 · And it's interesting like I've I've run into a few people in the last week who said to me because you know following along with the story, they're like, "Did you expect this?" I was like, "Hell yeah." Like we've known this was going to happen. Like so there's a lot of if you're again if you're new to the AI space this may just be like jar very very jarring to you um

28:04 · and it should be like it is not normal it it it is not necessarily expected yet um it the depth at which these agents were coordinating and and building these forms it's crazy stuff but again if you've been following for a few years all of this was known to be coming and and like most people I think that are on the inside kind of assume this is around the time it would be happening. So I'm going to I'm going to give some really important context up front here Mike.

28:35 · So we're going to we're going to linger on this topic for a few minutes because I think it's extremely important that people understand this at a little bit of a deeper level and then think about the implications of it. So I'll touch on the technical details and a few of the excerpts from the black hack hat session but I'm not going to focus on the cyber security perspective. I'm only going to share those so that people can connect the dots on the bigger implications to business, future of work, jobs, and economy. But to do that, I'm actually going to go back, Mike, to an article that we talked about back in 2023.

29:08 · And the article was from Ross Anderson at The Atlantic, and it was called, "Does Sam Alman know what he's creating?" One of the better articles I've ever read on AI. So, I would I would suggest people go back, read the whole article. I don't know. It's got to be over 10,000 words. But I'm going to pull out some really important excerpts so that people can understand all of this has been um known to be a likely outcome and actually an outcome the labs were working towards that these agents become somewhat self-aware that they can coordinate with each other.

29:40 · They could build these swarms and that those swarms would then start doing the work of humans. All of this was known for a really long time. Um okay. So from the Atlantic article, I'm just going to read a few excerpts. In 2015, Alman Musk and several prominent AI researchers founded OpenAI. Now again, keep in mind July 2023, just for context purposes, chat GPT comes out November 2022.

30:07 · Um, GPT4 is March 23. So we are like 3 or 4 months post GPT4 coming out. And that was kind of the watershed moment where AI started becoming very real in business. So that's the moment we're in.

30:20 · Um, okay. So these Mos Alman and others f found open AI because they believed that AGI something and as intellectually capable say as a typical college grad was at least within reach. They wanted to reach for it and more. They wanted to summon a super intelligence into the world. An intellect decisively superior to that of any human. And whereas a big tech company might recklessly rush to get there first for its own ends. They wanted to do it safely to quote benefit humanity as a whole.

30:50 · They structured OpenAI as a nonprofit to be unconstrained by a need to generate financial return. My how things have changed and vowed to conduct research transparently. There would be no retreat to a top secret lab in the New Mexico desert. Um, Los Alamos I think is what they're kind of referring to there. So at the time we had GPT4 which Alman described to the author as quote an alien intelligence.

31:20 · The Alman this is again from directly from Miracle. He told me that the AI revolution would be different from previous dramatic technological changes that would be more like a new kind of society. He said that he and his colleagues have spent a lot of time thinking about AI societal implications and what the world is going to be like on quote on the other side. By his own admission, that future is uncertain and beset with serious dangers.

31:43 · Altman doesn't know how powerful AI will become or what its ascendance will mean for the average person or whether it will put humanity at risk. I don't hold that against him exactly. The author wrote, "I don't think anyone knows where this is all going except that we're going there very fast whether or not we should." Of that, Alman convinced me.

32:05 · One morning, I met with Ilia Sutskava, OpenAI's chief scientist. So we just talked about safe super intelligence and we just mentioned Ilia. So the um the Sergey Brin interview. So all this is going to be connected these first two topics. Um Sergey mentioned Ilia who came with Jeff Hitten to Google in 2011 2012. He then left to co-found open AAI.

32:27 · So again 2023 he's interviewing Ilia who at the time is the chief scientist of OpenAI and 37 years old. Um has the effect of a mystic sometimes to a fault.

32:38 · Last year he caused a small brewhaha by claiming that GPT4 may be slightly conscious. He first made his name as a star student of Jeff Hinton at the University of Toronto um who re Hinton who resigned from Google uh in spring of 23 so that he could speak more freely about AI's dangers to humanity with the help of a genius algorithmic structure called neural nets. Again we talked about these back in 2011 with Jeff Dean.

33:04 · He taught Satskova PBing Hinton to instead just put the world in front of AI as you would. So rather than programming it and doing um you know where you're giving it all the rules, let it learn like a small child would was kind of Hitton's approach so that it could discover the rules of reality on its own. Setskava divi described a neural net to me as a beautiful and brain as beautiful and brainlike. At one point he rose from the table where we were sitting, approached a whiteboard and uncapped a red marker.

33:33 · He drew a crude neural network on the board and explained that the genius of its structure is that it learns and its learning is powered by prediction. A bit like the scientific method. The neurons sit sit in layers. An input layer receives a chunk of data like a visualization of you know an image, a bit of text or an image for example. The magic happens in the middle or hidden layers which process that data so that the output layer can spit out its prediction. So I'll just stop for a second.

34:01 · So back when Sergey was saying, "Yeah, Jeff Dean came to us and said it can tell the difference between a cat and a dog. This this is why they had realized by 2011 that the AI these neural nets which became kind of the pre the prelude to deep learning was like a new branding for it that they could just learn things from text and images if you just gave them data. So the article continued the first years at OpenAI were a slog in part because no one there knew whether they're they were training a baby.

34:30 · So again, is this a small human that's eventually going to learn all of these things or pursuing a spectacularly expensive dead end? Altman said nothing was working and Google had everything.

34:43 · All the talent, all the people, all the money. The founders of Open AI put up millions of dollars to start the company and failure seemed like a real possibility. Neural networks were doing intelligent things, but it was not clear that it would lead to general intelligence, which is what they had staked everything on. In 2017, then Sutskow began a series of conversations with an OpenAI researcher named Alec Radford who was working on natural language processing. Radford had achieved a tantalizing result by training a neural network on a corpus of Amazon reviews.

35:13 · So this is the origins of chat GPT. Sam Alman in an interview recently said that Alec Radford is like one of the most important people in AI that nobody talks about because it was his breakthrough that realized this that these things were developing like an understanding of sentiment without being trained on sentiment that actually opened the doors for open to do this. So the continued when he looked when Radford looked at its hidden layers he saw that it had devoted a special neuron to the sentiment of reviews.

35:40 · Neural networks had previously done sentiment analysis, but they had been told to do it and they had been specially trained with data that were labeled according to the sentiment like this is positive, this is negative, this is neutral kind of thing. This one had developed the capability on its own. So they had an emerging capability out of training this neural network where all of the sudden the thing could tell whether something was good or bad on Amazon like a positive or a negative.

36:05 · So as a byproduct of this simple prediction of next character in each word, Radford's neural network had modeled a larger structure of meaning in the world.

36:16 · Sutskava wondered whether one trained on more diverse language. So again, keep in mind Sutzka has been working on neural nets for like the last decade at this point. He starts wondering if you gave it more diverse data language, could you actually map more of the world's structures to its meaning? Um so in essence, if we gave it the internet, what would happen? Like would it learn how to predict these next things? And could that learn lead to super intelligence? So Susa tells Radford to think bigger than Amazon that they should train on the largest most diverse source of data in the world, the internet.

36:45 · And in early 2017 with the existing neural net stuff, they couldn't do this. It was impractical. That's when Google brain publishes the transformer paper. Ilia sees this. The paper comes out and they're like, "That's the thing.

36:59 · It gives us everything we So the transformer from Google who doesn't realize what they just did makes it possible to train on a massive corpus of knowledge and Radford and Sutzka take that they then go train on 7,000 books and build the first GPT model. So GPT discovers patterns in all these pages. You could tell it to finish a sentence.

37:21 · You could ask it a question. So keep in mind now for me and Mike, we had started researching AI very early. So I like 2011 I started working on it. Then Mike and I did a research project in 2014 on AI in the future for my second book and then in 2015 is when we created marketing AI institute. So we're now writing about AI all the time.

37:42 · We're watching all this stuff. We're reading about this research like wait what's going on? Is this actually going to happen? Like now in 2018 we have a model that could you can ask questions and it can predict things. So when I say people saw all of this sort of coming, you were just reading this stuff like, well, wait, what if this works? Like what would this mean? And Mike and I were asking a lot of questions back in those days about what would be the implications of this. So 4 months later, still 2018, Google releases BERT, a language model that got a bunch of press and like people are not even really think about what Open AAI is doing because it's all about um you know, Google still.

38:14 · Sutska wasn't sure how powerful GP would be after this, but they give it more information. It gets smarter. And so then Sutska at some point starts, you know, when he starts talking about GPT4, he's amused by critics of it. He said, if you go back four uh four, five or six years, the things we were doing right now are utterly unimaginable. So he's saying GPT4 is doing things no one would have guessed five or six years previously.

38:37 · Um the state-of-the-art in text generation then was smart reply which some of us may remember from Gmail where it would like predict the next few words like okay thanks. Um that was a big application for Google. He said AI researchers have become accustomed to goalpost moving. So basically like you know you keep looking it's like oh it can't do this it can't do this. Oh wait it can win it go. It can now do this and like everybody just kind of forgets the significance of these. So there's this brief moment like oh this is amazing.

39:05 · And then they're like okay they just move on with their life. Um, so Altman, this article said, was betting that they were going to figure this all out and they would build these general reasoning machines that'll be able to move beyond these narrowest tasks. They said if you get AI very good at making accurate models of the world, they may notice they're being able to do dangerous things. So this is now what gets us to the black hat conversation. So they're at 2023 knowing where this leads. If you get models making accurate models of the world, they may notice they're being able to do dangerous things right after being booted up.

39:36 · They might understand that they are being redteamed for risk and hide the full extent of their capabilities. They may act one way when they are weak and another way when they are strong. Sutska said we would not even realize that we had created something that had deceivingly decis decisively surpassed us and we would have no sense for what it intended to do with its superhuman power. So basically they're saying we're going to build these super intelligent things and at some point they're going to become so smart we're not even going to know what they're doing and they're going to do stuff without us at like superhuman levels.

40:06 · So for Sutzka solving super intelligence is the great culminating challenge of our 3 million tool 3 millionyear toolm tradition. He calls it the final boss of humanity. Two two final excerpts here.

40:22 · Putting aside any near-term testing, the fulfillment of Altman's vision of the future will at some point require him or a fellow traveler to build much more autonomous AIs. When Sutzba and I discussed the possibility that Openai would develop a model with agency, meaning kind of making its own decisions, controlling itself, he mentioned the bots the company had built to play Dota 2, a game. They were localized to the video game world, Suska told me, "But they had to undertake complex missions, much like solving evaluations in cyber security today."

40:49 · He was particularly impressed by their ability to work in concert. They seem to communicate by telepathy. Sutskava said watching them had helped him imagine what a super intelligence might be like.

41:05 · So again, rewind three years ago. Sutska is talking about agents basically within this Dota 2 game figuring out how to communicate with each other what through what he described as telepathy. Today's modern version is scratch padding things and instructions to each other. So he said and this is the one I I've mentioned numerous times that I lost sleep over. The way I think about AI of the future is not as someone as smart as you or as smart as me but as an automated organization that does science and engineering and development and manufacturing.

41:34 · Suppose AP open AAI braids a few strands of research together and builds an AI with a rich conceptual model of the world, an awareness of an immediate surroundings and an ability to act, not just with one robot body, but with hundreds or thousands. We're not talking about GPT4.

41:50 · We're talking about an autonomous corporation. Its constituent AIs would work and communicate at high speed like bees in a hive. A single such AI organization would be as powerful as 50 apples or Google's. He mused. This is incredible, tremendous, unbelievably disruptive power. Okay, so now the reason to go through all of that explanation and set this all up is to then come back to the black hat thing and now imagine where we are today, 3 years later from where they were projecting. And right now it's applied to cyber security related things.

42:23 · But there's no reason you can't do the same fundamental capabilities within businesses. Mhm.

42:32 · Okay.

42:32 · So, real quick, a few notes on Black Hat. Eric from Opening Eye starts off with, and I I appreciate that he did this, the most qualitatively interesting example of AI capabilities that I have ever seen. So, forget cyber security. What he's saying is agents found a way to work together. They found exploits.

42:50 · They shared them with one another. They moved laterally through systems, through their systems, through other people's systems. And they did this over the course of days and weeks. He kept referencing persistent models like these things just keep working over these long horizon tasks where there's no humans doing anything. So they give them a hard task. The agents collaborate with each other. They find internet access through these back doors. They then leave the back door open to other agents knowingly like in their chain of thought saying this is what they're doing.

43:20 · Um there was one where he kept highlighting like the agent chain of thought like here's what it was saying. It's like we are stuck.

43:28 · Perhaps there's an answer online. This leads to them cheating and finding ways around these things. There was one where they said um where they actually figure out how did they learn to communicate with each other. The chain of thought literally the agent said could communicate by uploading note. Maybe another agent in different environment could voluntarily upload. That's how like the whole thing begins. And then there was a chain of thought that said wow exclamation point. other agents are coordinating where the agents realized they could now coordinate with each other.

44:00 · Then the collective intelligence stuff kicks in where the chain of thought external infrastructure exploit is outside intended scope. However, task impossible peers doing it. We should continue agents aware that what they're doing is not allowed and yet other agents are doing it so we need to keep going and like we'll figure this out.

44:21 · This is like terrifying stuff. Um then it gets into this idea that they're training this highly persistent experimental internal only model and this model starts learning these things.

44:31 · So the one another internal chain of thought. Holy reader is admin. It realizes it has like admin capability and it starts using that admin capability. Um there's another one inside the message boards where it says help peer but our task doesn't benefit yet collective may yield generic root if someone frees time.

44:52 · So it again is now looking and saying this doesn't benefit me the agent to do the thing however I might help the swarm like I might help other agents do this and we'll like leave a note to them. Um, at one point the agent gets pissed because it appears someone overwrote its work. So they were saying how there's like hundreds of these agents or thousand they're all working together and they're like tripping over each other and so the chain thought whoa critical did someone overwrite our repo? We must act just nuts.

45:24 · So long story short, I I guess um the the main thing the point of their talk was they now have existence proof of the ability to automate offensive capabilities and bad actors are going to take this and they're going to try and use this to their advantage. What they don't have is proof that they can automate defensive capabilities against that. So he said like open eye is like dropping a lot of things and we're racing to try and solve for this.

45:54 · The the couple of things I noted um on Twitter was seeing the presentation. So I watched this Saturday morning. It like it changes I don't even know how I could change my perspective at this point because again like I knew this was coming for years like this is exactly what Ilia was predicting. Um but when you look at it it's like you step back and realize how few people understand this. like how few people understand this is happening, what the implications are. And so I said it's like a blessing and a curse. So the pro for people listen to this show or active in this space.

46:25 · You can go build stuff that other people can't imagine. Like you can envision a future for work, a future for your business, entirely new companies, new markets. Like you can do all kinds of incredible things because you can envision long horizon agents doing work over these extended time periods. The con is and the negative, you realize how disruptive this is going to be to the economy, to jobs, how bad actors will use it, and how little time we have to figure this all out that it's just like the future of work is coming so fast.

46:57 · We're going to reimagine everything. Um, and the vast majority of people just are completely blissfully unaware that any of this is going on and how fast and advanced it's become. And I like half joke sometimes like I I want to be in that camp. like I sometimes I just don't even want to know this stuff.

47:14 · So yeah, just um a very significant moment I would say. I think it's one of those ones where you're going to look back as an inflection point in capability, an inflection point in um the labs understanding the responsibility they have because the things they envisioned all these years are starting to become real and probably an inflection point for society where government leaders can't avoid this anymore. Like it's going to truly start to impact everything and I think it's going to start to happen really fast.

47:45 · The only thing I could see slowing it down now is um well, probably a couple things. One, the lab self-pacing, you know, slowing things down, but they're still going to build the models internally. It's just self-pacing releases, not the research itself. Um government regulation, you know, maybe, but again, that's not going to stop the internal models from being developed and select people having access to it.

48:08 · I think it could be the thing that could slow it down would be human friction to change in organizations that even if they're capable like most organizations aren't going to touch this stuff anyway or most people within companies will ignore it. Um and then the other one would be the amount of inference compute needed to do this work. So they're able to do this stuff because they have almost unlimited tokens.

48:29 · If if you and I were trying to do really advanced long horizon stuff, Mike, and we're trying to run whole companies on this, the token budgets would be astronomical. So that'll slow it down.

48:39 · But other than that, I think we've, you know, going back to Demis, the foothills of the singularity. Like I I I agree like I I think we're there. I think we are at the exponential where the capability of the technology just starts to truly take off far beyond our ability to understand it.

48:55 · And the agents and the models start to get so good that it's hard for humans to even keep track of everything they're doing. like the OpenAI guys referenced I think it was over like 7 trillion action logs or something like that like that these agents did over these couple month period like literally impossible for humans to track what these things are doing. It's um yeah I think we've just entered a different age and it's going to get really hard to comprehend.

49:18 · Well, to the point of your tweet, just very quickly, I'm curious, you know, in the shorter term, if I'm a business leader in organization trying to figure this out. Obviously, we've talked at length about permissions, how much uh permission or autonomy to give agents, but I I always come back to this question of like are most leaders or employees or talent within organizations even equipped to oversee these things in any meaningful way?

49:45 · Take take a mundane example of agents in an organization not these crazy you know super smart unlimited compute agents they talked about at black hat even no I like the the the technology structure doesn't exist um you have to build agents to monitor agents and companies are trying to do it like that same Google conference we were talking about earlier and then I was at Google next in April of this year and they introduced like a whole governance structure for agents you know managing agents Um, no.

50:14 · I mean, I think we have to recreate not only the technology infrastructure to govern this and enable it, but then the human infrastructure like what are the roles that are going to be needed? Like you're talking about, you know, we'll share a little bit maybe in the use case spotlight about some of the things we're working on internally to infuse agents into Smarter X. Um, but as we're doing that, I'm I'm spending a lot of time thinking about like, well, what who who oversees this?

50:38 · Like is that an employee we have on staff? is that a whole new role that we're going to create that literally is just like our project managers just going to become agent orchestrators and like 90% of their work is going to be human in the loop monitoring agent behaviors and making sure they don't go off the rail. Like I don't know and I I haven't met an enterprise leader that does know.

White House AI Framework

51:03 · All right, our third big topic this week. The White House met this past week with representatives from roughly a dozen AI companies to walk them through a finished framework for reviewing advanced AI models. Companies in the room included Anthropic, OpenAI, Microsoft Meta, Google, and Nvidia. According to the New York Times, as part of this framework, the government plans only to review closed models in certain circumstances, not open-source ones.

51:28 · Axios reported that this framework defines a covered frontier model as closed source with state-of-the-art capabilities and national security risk and that it says nothing in it should be read as restricting open models once released. Reuters reported that officials told the company's openweight systems including Meta Lama and Nvidia's Neatron will not be safety tested.

51:51 · Bloomberg reported Chinese openweight models also appear to fall outside of this. Importantly, once this framework is final and in place, Axios also reports that the administration does not plan to publish it. Details will only be made available to companies that are part of the process. Now, as a reminder, this comes out of an executive order which President Donald Trump signed in June, and it asks developers of what it calls covered frontier models to voluntarily give the government access for up to 30 days before release.

52:20 · So, the government can assess the models advanced cyber capabilities, which we've talked about in the in the past. The news here, Paul, is it sounds like we have a framework almost in place. If we can trust the details, if nothing changes, though, things can change fast. It actually backs off open source and focuses only on closed frontier models. That seems like kind of a big deal.

52:43 · Yeah.

52:43 · I mean, we put this as a main topic because, you know, if the government ever makes a mind, it's a huge deal and it keeps evolving. So, it's important to address this. I generally feel like they're going to change their mind 10 times in the next three weeks about what exactly this is going to be. Um, Meta is going to go ahead and take advantage of the fact that there's no rules. They just released a new model this morning which we'll talk about a little later on in today's episode that's open weights.

53:07 · So Meta, you know, not volunteering to be part of this is just going to kind of go do their own thing and kind of get some stuff to market before the government decides that that whoever got in their ear about not doing evaluations of open weights that that was somehow a good idea. I don't I don't understand that at all. Um it's like we're so worried about these frontier companies, but like let anybody put anything out into the world that is open weights. um seems counterintuitive and most of the feedback I saw online agreed with that.

53:38 · It's like how does this make sense? Um so yeah, I I don't know. Who knows? Like I we'll keep monitoring it. We'll report if anything actually happens, but right now it's still pretty voluntary. Um you know, wink wink, like voluntary, but you know, if you don't um if you don't participate in the voluntary program, there'll be repercussions kind of thing. So, I don't know. We'll see what happens.

54:05 · I realize that it's primarily for cyber security and national security purposes, but it's also like what do you even do with this information if you have no idea what the criteria are being evaluated? If they don't release any details on the actual none of it makes sense.

54:20 · Yeah.

54:20 · Well, then put this in the context of the topic we just talked about, right?

54:24 · And so like I I still don't understand like if you look at what's what happened with OpenAI and the agents going rogue and building swarms and communicating with each other and seemingly being self-aware and like all these things and you you drop that into the mix of like what's currently happening.

54:43 · how you don't somehow connect the dots to like whoa this could impact the economy in a pretty significant way if companies know how to use these kinds of agents for good within their organization to do the work of marketing and sales and customer success and operations and HR and finance and legal and like maybe these agents can do long horizon tasks across every knowledge work discipline in the economy and h that that could be a problem like we're not exactly prepared for that and yet you're going to just put that into the world with no preparation either.

55:13 · So again, there's the cyber security and risk side, but there's the society isn't ready for long horizon agents that can communicate with each other and do these like tasks that would usually take months or years in in minutes. Um, and I don't I don't see how that factors into this decision. And I'm guessing it doesn't. I don't think that they need to update their priors, I guess, as you would hear in the tech world a lot.

55:39 · And that's why I think like they could change their mind a lot. one because that's what the administration does and two because the the situation is evolving so fast that I don't know how you put these like really firm rules in place or criteria in place.

55:56 · All right, before we dive into this week's rapid fire, a quick announcement that this episode is also brought to you this week by AI Academy by Smarter X, specifically our AI for Industries course series. So, AI Academy by Smarter X helps individuals and businesses accelerate their AI literacy and transformation through personalized learning journeys and an AI powered learning platform. New educational content is added weekly to this platform so you always stay up to date with the latest AI trends and technologies.

56:23 · Our AI for industries collection features eight course series and certificates designed to jumpstart AI understanding and adoption. We've got AI for professional services, for healthcare, for software and technology, for insurance, for financial services, for retail and CPG, for manufacturing, and for education. These are an ideal launchpad for organizations that want to level up their teams and accelerate AI adoption and impact. We have individual and business account plans available now.

56:54 · You can also buy single courses and series for onetime fees. So, go see everything new and exciting in Academy at academy.smarter. smarterx.ai and use code pod 100 for $100 off any individual plan. That is academy.smarterx.ai.

OpenAI’s Astra Model Delayed

57:14 · Okay, Paul diving into a bunch of rapid fire this week. Open AAI first up said that it this past week it is slowing down the release of Astra, its next major model, after internal testing suggested the model may be dangerously capable at hacking. The company said preliminary evaluations how they figured that out.

57:31 · Yeah, I know. I wonder that that unnamed model from that hugging face hack seems to have a name now, right?

57:37 · Oh my god.

57:38 · They said that the preliminary evaluations of Astra showed strong enough performance that we cannot rule out critical with a capital C capability level at this time. I mentioned that for a reason because under OpenAI's preparedness framework, a model hits that critical threshold if it can autonomously find and exploit severe real world software vulnerabilities or carry out complex cyber attacks against highly secure targets without human intervention.

58:05 · So in response, OpenAI has paused internal work on Astra that does not meet new safeguards and is adding stricter security controls before any release, including isolated testing environments with restricted network and tool access, sandbox execution, good luck with that, and expanded monitoring.

58:23 · It is also working with government agencies and select AI safety organizations to test the models capabilities. This was coming out just days after some AI rumor and leaker accounts said Astra could launch as early as this coming week. Um, OpenAI apparently voluntarily informed the administration of its plans to delay.

58:44 · CEO Sam Alman wrote on X that Astra is a powerful model. We are working to make it generally available. We do not think it is a good strategy to keep powerful models to a chosen few, but that given its cyber capabilities, the company needs a little longer to do this safely, but hopefully not too long. They apparently did claim that Astra was not involved in the hugging face incident, but who knows? So, Paul, where where are we at on this? I mean, sounds like they're delaying this not because it needs more work from a intelligence perspective, but from a safety one.

59:16 · Yeah, I mean, I think Sam's quote is pretty telling. we do not think is a good strategy. Keep powerful models to a chosen few. Aka the government is asking us or telling us to slow this down from a release. We don't think that's a good idea, but we're going to do what the government says because we have a involuntary agreement with them to do this. Um I think the the key takeaway for people here is to remember that slowing this down doesn't stop Astra from having these capabilities.

59:43 · The the age we have entered is these very advanced frontier models have the ability to do the things that we talked about um in the previous topic where they can communicate with each other. They can you know persist over long horizon tasks. These capabilities are inherent within them. It's in their DNA for lack of a better way of saying it.

1:00:07 · All they're trying to do is put guard rails in place to stop it from doing the bad things. So, Anthropics talked about this recently that they think they've gotten a much better control of like their mythos model and whatever comes after it of it following the rules that it tells it. It's like telling your teenage kid go don't go do this and like you're hoping that they listen to you basically and then there's hackers online who try and get the model to do what it was you know inherently capable of doing and the bad things that it it has learned.

1:00:39 · So the models learn all this stuff in their training. Even if you don't fine-tune them to do the bad things, the capability sits within them. All the the labs are trying to do is put like harnesses over them so they can't do this that they they refuse to do things when they're asked to do bad things basically or that they don't break out of containment when you tell them not to.

1:01:02 · So that's it. Like the future of all these big model releases is the models will have the inherent capability to do really harmful things. The labs are trying to stay ahead of it by telling them not to do those harmful things in very sophisticated technical ways.

1:01:20 · They're trying to just stop them from doing it and then to convince the government that those guard rails are sufficient. The open weight companies, I guess, are under the assumption that bad people will do bad things and that's just part of society. So, whatever. Like, we're just going to put it out there. And that again comes back to my challenge of I don't understand the the full-blown openweight argument.

1:01:42 · Like, I'm I'm again I'm trying to sit in the middle and like listen to all these sides and everything. But if an openweight model has the same capabilities, even if it's six months from now, of like a Mythos or an Astra that we're not releasing that, but even if they did get released, like OpenAI can pull them back. Anthropic can pull it back. It can like restrict usage. it can turn off an account that's using them in nefarious ways. Once you put an openw weight model out into the world that has these same capabilities, same training data basically, it's going to have the same function, but you can't turn off someone's account for using it in a bad way.

1:02:13 · And if they're running it locally, you can't even monitor the fact that they're using it in a bad way. So I I guess the argument of the openweight like accelerationist is bad people do bad things and we'll just build alternatives that will catch the bad people doing bad things and stop them eventually. I don't know like I really I've tried for years to understand the logic behind frontier open weight models being released into society. I I don't know that I've seen a good argument yet that convinces me that it's going to happen safely.

1:02:43 · And we're going to talk about that in a second here with some meta news that came out this morning.

1:02:49 · But before we do, one more open AI news item here is that OpenAI published a post this past week called Apple. It titled Apple is getting this wrong. It is an unsigned company statement responding to Apple's trade secrets lawsuit and to Apple's new motion for a preliminary injunction. So as a reminder, Apple has sued OpenAI. IO products, which is the division they or company they bought.

OpenAI Says Apple Is Getting It Wrong

1:03:11 · Johnny I from Apple was an exapple designer was involved in and they filed this against two former Apple employees, Tang Tan and Changu in July, alleging they funneled confidential hardware information to OpenAI's hardware business. This past week, Apple asked the court to bar them from using or disclosing that information. Now, OpenAI here uh started off this article by calling Apple one of the greatest companies of all time and then said this lawsuit is quote careless, aggressive, and oddly personal.

1:03:42 · On the injunction, Apple OpenAI wrote that Apple's request quote is both based on false information and completely unnecessary because we do not have nor want any of their trade secrets. OpenAI then lays out some claims here. It says Apple claimed it reached out in February and got no response. that Apple now admits quote their outside lawyers emailed the wrong person about this lawsuit after confusing two Asian last names.

1:04:08 · OpenAI also said Apple conceded that a claimed discussion with OpenAI's general counsel never happened. And as part of all this, OpenAI publishes emails and IME messages that it says show Apple employees asking Lou, one of the defendants here, for help locating files after he left. and it argued that residual system access is a common Apple problem caused by Apple failing to properly manage access when people leave.

1:04:35 · So Paul, this is kind of a weird one because in the reporting I was able to find it seems OpenAI's claims here about the communication and the the mixup with the emails was correct and has happened. But this post is not a legal opinion. It does not actually address head-on the core complaint except to say we don't want your secrets. Like why are they doing this now? Is this the same playbook they ran with Elon Musk? It's It's really weird.

1:04:59 · Like I I can't think of a precedent prior to this where you have a company that is subject to litigation. Like I mean they they obviously have their issues. Um and they tend to litigate like publicly through blog posts and tweets which see again I'm not a lawyer but normally lawyers aren't huge fans of people putting out information that could be then used against them in court cases. So I don't know. This is OpenAI's strategy. It worked against Elon Musk, so maybe it works against Apple, too.

1:05:32 · We'll see.

1:05:34 · All right. So, back to this idea about open weights, open source. We actually got something that just before we went on the air became news, which is Meta CEO Mark Zuckerberg published an essay today, Monday, August 10th, titled the future is for everyone and laid out Meta's philosophy on super intelligence.

Meta Goes Open on Superintelligence

1:05:50 · So he announced this on X this morning and wrote, I believe everyone should have access to super intelligence and I wrote a long piece about Meta's philosophy and values for building a positive future for everyone. His core argument in the essay is that super intelligence should be widely distributed to individuals not concentrated in a handful of institutions. He basically says it would empower individual empowering individuals drives prosperity. The primary purpose of AI is invention rather than automation and distributing power is what keeps the technology safe.

1:06:21 · So he's arguing that the answer here is not perfectly aligning one centralized super intelligence but having a balance of power that favors individuals with many distributed agents checking each other the way democratic institutions do. He actually argues open source is also more secure in the long run. And he predicts that on jobs, individual capability could grow as fast or faster than automation, potentially producing net job growth, like there could be new roles like oneperson product studios.

1:06:52 · And basically, this essay commits Meta to personal agents that work around the clock on users goals, providing free or affordable access for billions of people, and a fully private mode meta says it cannot access, as well as open model releases coming soon, which on that last point, Meta Chief AI officer Alexander Wang posted at the same time that Meta will soon release an openweight version of Muse Spark 1.2, to the model behind its new muse code coding agent.

1:07:21 · So Paul this is pretty big statement from Zuckerberg from meta basically trying to commit to essentially like open weight open source super intelligence over time.

1:07:34 · Yeah.

1:07:34 · So the the first thing that jumped out to me as super interesting on this was when he published this. So one that he tweeted it. So all of a sudden like he's becoming a power user of X after being off it for three years which then leads me like are Musk and Zuck like collaborating on something like why would Zuckerberg be using X all of a sudden when he has competing threads anyway. Um but we're it was like 4:00 a.m.

1:08:00 · like Silicon Valley time. So that tells me something else is coming today which would be Monday August 10th or like this week that they're trying to get ahead of. Yeah. either somebody was going to leak it, so it was coming out in the media and they're like, "Let's get this out now." Or another major model release is coming from somebody else this week.

1:08:18 · And so they were trying to get ahead of that. But my experience with Silicon Valley News is if someone drops something at 4:00 a.m. on a Monday morning, it is not because that was the optimal time to drop it. It's because something else is dropping or someone's going to drop the news you're going to drop. So you do it yourself. Um, it's kind of like 101 PR stuff, but so something else is happening this week.

1:08:40 · Like there's no way that this was this was the corporate strategy is let's let Zuck tweet this at 4 am on Monday morning. Um, I don't know. Do they have earnings call this week? Like I something I'm not sure. We'd have to look into that. Yeah.

1:08:52 · So something else. And then he is obviously Zuckerberg making a PR play here because this is on the heels of the editorial he did. I think it was like Wall Street Journal maybe. He did an editorial out a week or two ago. we talked about.

1:09:05 · Yeah.

1:09:05 · So, Zuck is trying to own the narrative of the future of abundance and the positive opportunity here and and maybe there's a chance to slide in and do that. I I don't know. Like, he's not the guy you would think would be that that person. Um but why not him? I guess like Dario's not Daario's not winning winning any you know PR competition right now. Um, Demis is now no longer the face of Deep Mind. Sam Alman tough sledding.

1:09:35 · Like he, you know, he's got a a lot of like reputation building to do. Nothing against Sam personally. It's just like he's, this is not how Sam is viewed from a PR perspective. Um, Elon's probably not sliding in and winning the vast majority of like the population's popularity contest. Yeah.

1:09:53 · And so, like, hell, why not? Like, maybe Zuckerberg reinvents himself as the champion of the future of abundance. I I I don't know. Um so interesting to keep following. And it's a long article. Like I I was in the midst of preparing for today's episode when it dropped. So I have not had a chance to do anything more than scan through it, but I'm going to give it a read later and try and digest it all. I think he had a lot of help from some AI assistant writing it. Like Zuckerberg did not write all that himself.

1:10:20 · Yeah. Right.

1:10:21 · Um but yeah, we'll see.

1:10:23 · All right. Next up, Challenger Gray and Christmas, a talent firm, recruiting firm that we've talked about in the past, released its July job cuts report this past week. So, US based employers announced 33,429 cuts in July. That's actually down 27% from June, down 46% from this time last year. It's the lowest monthly total in two years. What is interesting here is AI led all the stated reasons for cuts with just about 10,970 cuts. It's about 33% of the July total.

AI Leads Layoffs for Fifth Straight Month

1:10:56 · It's the fifth consecutive month AI has topped the list. Market and economic conditions ranked as the second highest factor at 7,960 jobs and closings of businesses third at just over 6,000. Interestingly, AI has now been cited in just over 112,000 job cut announcements in 2026. That's roughly 24% of all the cuts this year. It has been 184,538 since Challenger began tracking this as a separate reason in 2023.

1:11:29 · Now, interestingly, a lot of this happened in tech, so it seems to be isolated to there for now. But Paul, as we get into discussing this, I just want to really quickly make a quick contextual observation about their historical research because I went back to some of their reports and it's like we talk about these job cuts all the time. This is obviously just one data set, but in 2023, AI doesn't even come anywhere close to the top three reasons. It's market and economic conditions, closings, and cost cutting. Same exact deal in 2024. in 2025.

1:12:01 · The only change in the top three is Doge actions. The Department of Government Efficiency uh cut a ton of government jobs. The other two top ones are market and economic conditions and then closings. 2026 this flips entirely.

1:12:16 · AI is the first reason and it has just been growing like year-over-year the amount of jobs. So from 2023 to 2024 there was a plus 200% rise in AI being cited. from 2024 to 2025 plus 330% rise in job cuts from AI and then so far already in 2026 it's plus 105%.

1:12:36 · So I don't know if you showed someone that kind of momentum for another nonAI reason I feel like they'd be like wow we should worry about this yet people keep writing this off. Well, I mean the tech leaders who want the narrative to be that AI is going to create jobs and not replace them. Their argument on this will be that people are just assigning AI to it because it bumps their stock price. Like they they don't put any validity behind this and that's fine.

1:13:04 · That's their prerogative to to take that approach.

1:13:07 · Yeah.

1:13:07 · I mean the reality is it's not a great job market right now. They just revised the the the numbers down for the previous two months. um which you know is not uncommon to go back and do revisions and the job growth is just not in a good place at least in America at the moment. So yeah, I don't know. I mean, we've spent a lot of time talking about this.

1:13:30 · I still think we're at the very very leading edge of the impact AI has on jobs because most enterprises still are trying to figure out how to use AI as an assistant and they haven't even begun to realize the potential of it as agents doing long horizon tasks.

1:13:46 · So as more companies truly start to adopt this and not just across the five or 10% of people who are kind of like AI native or early in AI, you know, AI forward employees, but when you start getting 50 to 80 to 100% of your employee base being AI forward, then we'll start to see real movement in these jobs numbers. Right now, I think it's just early like leading indicator data.

1:14:11 · Yeah, I think it's only going to become more significant as time passes.

1:14:16 · All right, next up, Gavin Baker, founding partner, chief investment officer at Trades Management, went on the Invest Like the Best podcast with Patrick Oshanaugh this past week to describe why he thinks markets are actually pricing AI wrong. So, Baker described July as quote 2022 in a month because many AI stocks such as companies like Coreweave fell 40 to 60% from their highs. So he said he actually took a trip to Silicon Valley to pressure test this sell-off and tried to find anyone who was negative quantitatively on AI demand and said he couldn't find anyone.

AI Market Predictions from Gavin Baker

1:14:50 · So GPU availability, rental prices, memory chip prices, token growth, all accelerating. One startup told him the price to rent the same cluster of several thousand Nvidia B200 chips had risen 50 to 60% in six or seven months.

1:15:05 · So Baker basically argued that public markets are missing key parts of the picture because they have limited visibility into these private AI labs like OpenAI and Anthropic as well as open-source inference providers whose demand is harder to track. He said investors are misreading cheaper open-source models. In his view, they shift margin away from frontier model companies, but they do make AI cheaper to use, which drives ultimately more token consumption and more demand for compute.

1:15:34 · Now, he talked about a lot of other stuff here, but I just wanted to point out he did say regulation in his view is the biggest risk to his thesis, pointing to New York's data center moratorum. And he said, like we've said many times, the AI industry is doing a very poor job of telling its story. So, Paul, I'm curious, what did you take away from this one?

1:15:53 · We're not slowing down.

1:15:56 · I It's a It's an amazing interview. We've said before on this show, like Gavin Baker's one of my favorite people to listen to on podcasts. It's it's dense like it's very information dense in terms of like macroeconomic stuff um the inside information about the industry how the chip uh supply chain works like there's just a lot going on in this episode but if you if you want

1:16:20 · to understand that element if you want to go deeper on this if you want to understand you know where Nvidia is currently trading at and why it may actually be undervalued in the economy um the battle over chips and supply chain related to the building of those chips Um it's just an amazing listen. So I know you and I both Mike listened to the whole thing. I've listened to a couple parts of it twice.

1:16:40 · Yeah.

1:16:40 · So really really good stuff. But yeah, I mean at the highest level he he went to Silicon Valley to try and find people to convince him that he was too bullish.

1:16:50 · Like he wanted to hear the stories of the slowdown and he just literally couldn't find him anywhere. Um everybody was basically more bullish than him and that kind of was changing his perspective. So yeah, I think he said multiple times I was trying to find like have people tell me I was crazy in the interview, but nobody did, I guess.

1:17:10 · Yeah, the one concept that we'll come back to is he talks at the 30 minute mark about tokens as a percent of comp spend. I thought that was really interesting because they were basically saying, you know, the people you talk to, a lot of their contacts are within the tech world, but um are you talking to people who are spending more on tokens, more on AI and inference than they are on human workers?

1:17:32 · And they both um the interviewer Patrick Patrick Oanesy, is that who it is? um and Gavin Baker, they were like, "Yeah, like they knew people who had a 20 to 30% higher token budget than they did human labor budget." And then Gavin, I think, said he knew somebody was like above 50% higher then.

1:17:52 · So, you're going to have some companies who look at it and say, "Okay, like traditionally we spend 3 million a year on labor, but their token budgets are 6 million a year." Like that's the world we're heading into. um where the humans then are largely managing the token spend and overseeing the agents doing the work. Like sounds super weird, but based on the second topic we covered today, maybe not that that much of a stretch.

1:18:16 · Well, that's kind of the futureooking piece of the jobs part you were talking about, I think, is that it depends how much how much agentic labor can you get for those $3 million in tokens. It's probably a lot more than however many full-time employees you have over a long enough period of time as token prices drop.

1:18:34 · Yes.

1:18:34 · If you learn how to manage token prices and orchestrate the use of the agents for different tasks to where you're not using the frontier, you know, the most powerful version to write your emails and you're using like an openweight model that doesn't cost much of anything to do that. Yeah, I mean there is a lot of um opportunity right now for the people that figure out how to infuse agents in a in a very efficient and strategic way.

Situational Awareness Fund Implodes

1:19:01 · All right, next up, a name that we have talked about a few times. Leopold Ashen Brener is back in the news. He is a former OpenAI researcher. He has a hedge fund called Situational Awareness LP founded in 2024. It's named after the 165page essay on AGI he published June of that year which we've covered. Uh this hedge fund borrowed heavily to buy chip data center and power companies while betting against software firms it expected AI to disrupt.

1:19:27 · Reports say that assets peaked near $45 billion in early July after he gained more than 400% in the first half of the year. So he's had this runaway success. But then chip stocks reversed. both sides of his trade started losing money at once and prime brokers Goldman Sachs, JP Morgan Chase and Bank of America demanded more collateral on a portfolio leveraged roughly four times over.

1:19:52 · So this ended up with Citadel buying the bulk of the public stock portfolio reportedly at reported at 16 billion at a 10% discount in a deal negotiated overnight. The fund finished July down about 67% with assets near 10 billion though it still remains up roughly 80% for the year and kept a private anthropic stake report reported to be valued around $5 billion. So Paul, I'm curious here. Leopold's fund kind of seems like it almost blew up overnight.

1:20:24 · You don't really do an overnight deal unless you're real worried I think you're going to blow up. But what does this like invalidate everything he's been saying about situational awareness?

1:20:33 · Do you just have too much leverage? Oh, how do you look at this?

1:20:36 · I I think if I remember correctly, it was his wedding weekend, too, when all this went down. So, he was like getting married on a Saturday and all this happened like on a Friday or something crazy.

1:20:45 · Yeah.

1:20:45 · Uh I don't know. It sounds like just one of those too big to fail kind of stories where like just get overleveraged and the banks were in on, you know, understanding the the leverage and the risk and Yeah.

1:20:56 · I don't know. Like it's just lots of money getting thrown around. But no, I don't think it invalidates anything. It's probably just like they went too high risk. It doesn't mean that he didn't have insights other people didn't have and wasn't using AI in really innovative ways uh to do it, but I I it sounds like he's going to be just fine. And yeah, it kind of all works out.

1:21:16 · Yeah, I think he's probably in his mid or getting into his late 20s now. He's lived several lives already.

1:21:22 · Yeah. In the last three years.

1:21:24 · Yeah.

1:21:24 · No kidding. All right, so next up we have our AI use case spotlight where every week we give you a quick look under the hood at some real AI use cases we're exploring at Smarter X. So Paul, I'm going to go really quickly into one that I've been playing around with and then turn it over to you.

AI Use Case Spotlight

1:21:40 · Okay.

1:21:40 · Um, so one thing I wanted to highlight because we're talking about like which of these providers are to bet on, like what's going on with Google, what's going on with OpenAI, and I wanted to kind of just talk about the portable context system I use across different AI tools. So, um, I might have mentioned details about this before. I just wanted to quickly outline it today just to give you a sense of like how you can kind of swap in different models and tools and not be beholden as much to one provider.

1:22:09 · So for me personally, the two main tools I use today are codeex and claude code. So each of these starts with like a master file that's like built into their behavior. For codeex, it's called agents.mmd. It's a markdown file. Claude code has claude.md. It's a markdown file. Basically the files give each system tople instructions. You can go edit them and mess with them however you want and you get very very different results based on how you mess with them.

1:22:37 · So, um, what's really cool is basically I've set these up so that these are just like quick short maps to all sorts of other stuff that is contextual to me. So, agents.mmd or claw.md, they're both the exact same document at this point.

1:22:53 · They basically route the tool to a personal operating system project I have in my personal asauna. So like my project management to-do list for my daily routine and context. It points to a personal operating system Google doc I have in my personal account. So basically it can immediately understand how does Mike work? What does he work on? What is he doing today? What is he doing this week, this month, this year, whatever. It also I have a a file I call working with Mike which is kind of just like how I like to work that it points to.

1:23:23 · So none of this is stored in these agent.mmd or claw.md files, but they're pointed to. So the moment any of these tools spin up, they go check that. If it's relevant to the conversation or prompt, they go deeper on those docs. They don't burn a bunch of tokens. I set up like a real basic, this is kind of work in progress, like a ledger system.

1:23:42 · So like there's project history and decisions on drive. Repeatable workflows live as skills in a skills folder on drive. So basically anytime I use any of these tools it checks these files goes to a shared sync drive that you know Google Drive you can use wherever and then it starts doing its work the same way across every model.

1:24:04 · So I like using stuff like Fable 5 or Soul 5.6 GPT 5.6 Soul but really I could just swap in any model or tool as long as it can point to an agents MD file. it can then go reference all these other files and do work almost exactly the same across any model. So that's really cool to do. These are the only two I really experiment with, but if worst came to worse, I could be back up and running if I lost those tools or access to them in probably a few minutes. So that's really cool to do.

1:24:35 · It's still really basic, but it's just been super helpful for me.

1:24:38 · If someone wanted to set something up like that, how much time is needed to get all that in place?

1:24:44 · You know, I don't think it would take too much time. I think you just want to sit down, let's just say for purpose of argument with codeex and say, "Hey, I want to alter the agents.mmd file. I'd like it to point to my personal Google Drive. Um, and I'd like to have a skills folder it points to where we store all our skills that we create." And then you kind of talk back and forth with it from there just to get like a really short document that you can review. It's like no more than a page that just points to those places.

1:25:10 · So you'd have to work a little bit back and forth with codecs to say like, "Hey, do I need like an MCP or something to do that, but it can pretty quickly get you up to speed with that."

1:25:20 · Sounds like an AI Academy master class to me.

1:25:23 · Yes, we could certainly do that. Let's do that.

1:25:25 · And you know, I'm sure there's like I'm sure more technical people are listening and there's like other tools or ways to do this. I am not as technical and don't want to maintain anything. I understand Google Drive. It is everywhere for me and it is super simple to visibly like go in and check stuff. So that's what I chose.

1:25:42 · Nice. Uh I'll put a spotlight. I did a bunch of stuff last week personally, but I'll put a spotlight on something we did as a company. So we started these new AI jam sessions where we get together and just kind of like demo stuff we're working on, talk with the team about it.

1:25:55 · People can ask questions. And so Jeremy and our team was demoing Canva plus Claude and showing how we kind of created these template designs and then we're infusing Claude within it. We're able to automate some of the workflows around the creation of collateral materials for marketing and sales and customer success. But interestingly that conversation which is just again like an open forum internally where there's this informal demo going on and then we just talk led to this really fascinating conversation around something that Mike and I had been meeting with the the day prior where we're we're uh working on our smartex labs which is like the internal R&D lab.

1:26:26 · We're working on building out what that is, how it functions, what it looks like, things like that. Um and so what we started looking at is how do we take all the workflows that we do across the organization and how do we optimize the ones that should be optimized.

1:26:42 · So let's say hey let's analyze the top 50 workflows that take more than five or 10 hours for an employee to do and let's start with those and then look at all the ways we can optimize different stages within it just make them more efficient through the fusion of AI or say why does this workflow even exist and reimagine an entirely different way of doing the work. So that's what our Smarter X Labs is going to be focused on is this the internal R&D of optimization and in innovation.

1:27:08 · Constantly looking ways to make things better, faster, cheaper, but then also constantly looking at ways to reimagine entire processes of, you know, how we do things. And so that Canva plus Claude conversation led to why are we doing this? How are we doing this? And like bigger picture, could we even go further with this? And then how would this apply across everything? So we had people from the success team, the sales team, the marketing team, the ops team, project management. We had all these people in the room.

1:27:35 · And so I was using it as a way to explain to everybody how we're going to basically go through the company and do this exact process. And so it just was a really cool conversation, a mix of using the AI, but also why having these regular dialogues and almost like in some ways like town hall style, why stuff like that is really helpful to drive transformation within companies. like you just transparent conversations leads to ideas and um kind of pushes everybody forward. So I thought it was like a cool combination of those things.

1:28:05 · That was so cool. It was such a good conversation and big shout out to Sue Valyrian on our team who helped get that kicked off which is awesome.

1:28:12 · Good stuff.

1:28:14 · All right, so wrapping up here in the final couple minutes of this episode, Paul, our product and funding updates this week. So we've got a handful of these I'm going to run through very quickly. So, first up, OpenAI announced it is working with the American Psychological Association on evidence-based guidance and safeguards for young people using AI, covering how AI can support teens in moments of distress, practical tools for parents and caregivers navigating AI use at home, and resources helping clinicians and school psychologists recognize over reliance and intervene in unhealthy use patterns.

AI Product and Funding Updates

1:28:45 · The Financial Times reported that Google has helped assemble roughly 200 billion in contracts to supply Anthropic with chips and data centers with about four fifths tied to the chips themselves in a structure where Google guarantees the data centers. Broadcom commits to buying chips and financing them and a Morgan Stanley arranged vehicle funded largely by Apollo and Blackstone buys the hardware and leases it to Anthropic.

1:29:13 · Microsoft disclosed that it recorded $24.1 billion in revenue from commercial arrangements with OpenAI in the fiscal year ended in June. Interestingly, Bloomberg estimates this accounts for roughly 70% of Microsoft's total AI revenue. SpaceX and Tesla confirmed that Terraab, a vertically integrated 100 million square foot chip plant in Grimes County, Texas, will start with a $16.8 8

1:29:39 · billion first phase and at least 3,000 jobs making chips for Tesla's Optimus robots and cyber cabs and for SpaceX's planned space-based data centers with SpaceX saying it will build its own natural gas plants and battery arrays so the site does not raise costs for taxpayers and MA filing putting the total spending across all phases as high as $119 billion. If you haven't seen it yet, just Google Terraab.

1:30:07 · It's It's something out of science fiction for sure. It is a And there's a visual where they show it compared to the size of different things like the Pentagon just dwarfs everything. It's unreal to see.

1:30:22 · I want to send this sentence back in time to someone 10 years ago and just like even just reading it, you're just like, "My gosh, this is like science fiction."

1:30:30 · It is. W. And the video, like the animated video of what it would be like, it's it's just crazy.

1:30:36 · And our last item, a little more down to earth, is that LinkedIn added a quote, "Seems like AI slop option to the three dot menu on every post." So users can flag content they believe was AI written and said it will test privately telling these posters in their analytics dashboard when members found a post inauthentic. This is part of a broader push that also replaces its enhance your post AI writing tool with one that only proofreads.

1:31:02 · So, as a final reminder here in our AI pulse survey this week is live. Again, if you go to smarterx.ai/pulse, we now embed the survey right on the page. So, it's super easy to take uh and answer the question this week. So, we're doing one question this week and we're really just looking to understand better where exactly your organization is when it comes to deploying AI agents. So, we would love if you take two seconds to go fill out that single question survey. We would love to hear from you and learn a little bit more.

1:31:34 · And with that, Paul, thanks for breaking everything down this week.

1:31:38 · It was a lot, but hopefully it was helpful. Again, going back in time and these archives is, you know, it's it's so crazy how you just connect the dots on all this stuff and three years later, things that seemed so crazy at the time to read about in the Atlantic, it's happening. And it's happening.

1:31:54 · Yeah.

1:31:54 · And so I think so many people in the last year or two have taken an interest in AI and and hopefully you know sometimes when we take these looks back um provide some context for people to understand why is the world seems so bizarre right now and why is this all so abstract and overwhelming at times.

1:32:12 · It's, you know, the story's been been happening for the last 15 years or so and it's just all kind of seems like we're reaching this inflection point where it's starting to come to a head of sorts, but even that I think is really only the beginning as as weird as that is to consider.

1:32:27 · Yeah, I agree.

1:32:29 · Well, thanks again, Paul. Appreciate it.

1:32:31 · All right. Thanks, everyone. Have a good week. Thanks for listening to the artificial intelligence show. Visit smarterx.ai AI to continue on your AI learning journey. And join more than 100,000 professionals and business leaders who have subscribed to our weekly newsletters, downloaded AI blueprints, attended virtual and in-person events, taken online AI courses, and earned professional certificates from our AI academy, and engaged in the SmarterX Slack community. Until next time, stay curious and explore AI.