Ep. 62: Is AI Going to Kill Us All?

[00:00:05] Welcome to Prompting Curiosity, a podcast for the AI curious. No coding background required. I'm your host, Dr. Shantae Cofield, also known as the Maestro, and I created this show to explore what these AI tools actually are. Really, though, are the files in the computer, how to use them, and what they might mean for how we think, work, create, and move through life. Whether you're skeptical, intrigued, or already experimenting, you're in the right place. All that I ask is that you stay curious. All right, let's get into it. [00:00:38] Hello, hello, hello. M. My curious people, and welcome to episode 62. 62 of, uh, prompting Curiosity. I am your grateful host, the Maestro, and today we are talking about. Get ready for it, folks, whether or not AI Is going to kill us all. So perhaps you are, like, what in the actual Are you talking about, Maestro? That's ridiculous. And if that's the case, awesome. [00:01:02] If you do know what I'm talking about, and perhaps you've seen, read, heard all the chatter on the interwebs, or perhaps you heard about last week's episode. Uh, this is me making good on my promise to devote an entire episode to sharing my thoughts on the matter. So right off the bat, let me directly answer the question. No. No part of me believes that AI Will kill us all. [00:01:26] I am not sure if any of you believe that. Um, and it's fine if you do, right? I'm not here. Like, that's so dumb that you believe that. No. [00:01:33] Uh, what I do know is that all the media noise last week did have folks stirred up, and, you know, by the time this comes out, maybe it was, like, two weeks ago. Um, which is interesting, right? In and of itself, because I thought about actually holding off on this episode and seeing if this stuff was even an issue. Right. Uh, this. The news cycle goes so quickly as it relates to AI that if you don't talk about something suddenly, it's literally. It's literally dead. It's old news. No one knows about it. So I was like, you know, I could give this two weeks and see if it's even a thing. Um, but I did get some questions, and it was really cool to get some questions around it and, you know, kind of be the local AI person. So that's awesome. Uh, but, you know, and having people reach out to me and ask for my thoughts, I was like, yeah, let me just. Let me make this episode so. [00:02:28] And also, I always need things to talk about. I. I do an episode every week, and AI is not changing that fast. Like, it is making exponential leaps. [00:02:38] Uh, and making continued leaps, but in one single domain, which what I talked about last week, that it's a. Continues to be this agentic workflow and these agentic capabilities that continues to get better. But just like in terms of new things to talk about, it is not changing that fast. Um, which we will. I will talk about loosely in the episode. Um, but I needed an episode to talk about today, so here we are. Uh, I'm not gonna crown myself an official AI expert by any means. Um, but y' all know I spend a ton of time learning this stuff. I'm in it. I have been in it since I started this podcast last year. Uh, since before then, and then really getting into it when I started the podcast last year. Um, but I have also been learning about it and continue to learn about it from all angles, right? Financial angles, political, political angles, not just the tech side of things. Um, and I absolutely have thoughts. Right. So again, to state it plainly and answer the question right off the bat, I do not think that AI is going to kill us all. All right, a little bit of background on that statement for those of you who have no idea what the I'm talking about. On September 8, former Anthropic employee Jacob Cox, and he only been there for a few months, actually. He was at, um, OpenAI before. Um, but he resigned and he posted a seven part thread on X. I will link that in the show notes. Um, and he shared why he resigned and he shared his concerns about AI in one part of that, the. The threads that he wrote. Um, the tweet, I don't know what the you call it, but like, I'm gonna call it a tweet, right? And one part of it, he wrote the people building AI earnest that it could kill us all by the end of the decade. [00:04:13] That tweet, it now has over 170 million views. the time that I wrote this outline and that I'm recording this episode, things got even more interesting when Evan, I think he pronounces his name Huinger, I don't know, but his name is Evan. And you see, if you're reading, you'll see this name come up. But, um, he's the head of alignment stress testing at Anthropic Translation. He runs the team that tries to break Anthropic's safety measures. And you know, he's doing so on purpose. Um, but he tweeted in response to Jacob's statement, right? Evan wrote, jacob is correct here. We really do earnestly believe AI could kill us all. Could kill all humans. [00:04:54] Exclamation point. I personally think it is greater than 10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for super intelligence and are not clearly on track to. To. He followed that up by writing. To be clear, as we say in our latest risk report, I think the risk from present models is low. What I am worried about is super intelligence arising from recursive self improvement, as we have said, is happening faster than we thought. Ah. Uh, in that case, I feel so much better. Like, let's take a. Let's take a moment to step back from the dramatic predictions and let's first go over some terms that I think we should know. And as these terms are the new kids on the block. So the first one is. Or the. Rather the two terms are super Intelligence and recursive self improvement. Rsi. I briefly mentioned these in the last episode, um, but I want to take a moment to kind of define them again, because you will continue to see them. So superintelligence is AI that substantially, substantially exceeds human ability across essentially that word is doing a lot of heavy lifting, by the way. Essentially all cognitive tasks, not just faster or bigger, but smarter than the best humans at everything, including the task of improving AI itself. Which leads us into the next term, recursive self improvement, or rsi. [00:06:21] An AI improving its own intelligence or capabilities, which lets it. Which lets it improve itself. Again, a feedback loop where each iteration makes the next one faster or better with no human in that loop. [00:06:36] So again, I spoke about this last both of these terms last episode, but I want to once again highlight that that researchers have moved the AI accomplishment goal posts from. Initially it was AGI, artificial General Intelligence, which is AI that can understand, learn and perform any intellectual task that a human can across different domains, not just the thing that it was trained on. So it's as good as humans. Right. They moved that goal post to now super intelligence with rsi. Right. So we went from AI that can just do what humans can do to AI that is substantially better than humans across, you know, essentially all cognitive tasks. That that word's doing a lot of heavy lifting there. [00:07:19] Uh, and it has rsi meaning it has the ability to improve itself. [00:07:23] Right. Ultimately, I. It's not so much that I don't think that these things could happen. [00:07:29] It is the way that these concepts are talked about that makes it difficult for me to, uh, to take these for foreboding claims seriously. [00:07:39] Right. [00:07:41] I will not deny that AI is absolutely Incredible technology. Y' all know I love it. And I'm like, wow, it's crazy what this thing can do. [00:07:50] All right. And it has advanced rapidly since it came on the site seen. [00:07:55] However, like I discussed in the previous episode, these advances have continued to focus on agent capabilities. Right? The very narrow scope of things with improvements most noticeable on things like benchmark testing and for folks using it for things like coding, research and these like really long agentic flows. For something that is supposedly headed towards super intelligence, AKA it should exceed human ability across quote unquote, essentially all cognitive tasks, you'd expect all users to notice an improvement in capability. [00:08:25] I'm not sure the last time you had AI try to write something for you, but it seems pretty far off from super intelligence in that domain, right? Yes, the videos, that stuff. That's crazy. [00:08:37] Agreed. Uh, agreed. I. So yes, it is, it is very good in certain domains, but not everywhere. [00:08:45] And I think they kind of built in that little fail safe when in their. [00:08:49] Their definition, they doesn't say the definition I gave you says essentially, um, but the actual definition, they like that. The. I don't see the actual definition. The statement that one of the men, one of the people put out something about like practically all domains. Either way, they are hedging with the way they describe it. And to me, if, um, I'm going to like be like, yo, this thing is like the. This is Skynet. Skynet's good at everything, right? It is. If again, Skynet is from Terminator and I, I feel zero kind of way bringing up a fictional movie because I think a lot of these folks, especially in some of the like, AI doomerism spaces and things like that, like, that's where they draw their thoughts, that's where they draw their, you know, their, their brain goes to these, to these fiction. [00:09:38] And so I have no problem bringing that up and being like, this is not Skynet. It can't write it up so much. Is it moving towards it? Yes. And you know, very much like, um, uh, what is the man's name? Hubinger said, Evan said, like it's not the current models, it's what it could be. And to me it's just like there's so many things that you could say this for, right? What it could be. [00:10:05] Humans could just extinct ourselves in the next five years. What we could do. Yes, maybe we're not worried about presently, but who knows what happens in 10 years. That's where I'm just like, I take the claims, I want to say that I take the claims Seriously about how good it is and what's happening. But I'm also just like, this is a bit of a fantastical statement and I think we need a little bit more discussion. And I'm not going to just be scared by this statement, right? And this is why I am not scared by that statement. I don't just take it and be like, yeah, it's definitely gonna kill us all. Like, what? [00:10:39] No, I don't think that. Um, but circling back to its, its coding capabilities. Right. I have spent the past few episodes speaking about the fact that I am most concerned about AI's hacking capabilities. Right. That concern still stands. I read a really good article, a really good piece by Jake Handy. Um, I would link it, but it's behind a paywall as part of his paid, uh, substack that I, that I subscribe to. And it spoke about his prediction about our future with AI. Right. And I very much agree with it. Here's what, here's just one section of what he wrote. [00:11:08] What I do think is that it's very likely something big will break within five years. There are clear gaps and vulnerabilities in these lab based systems that are starting to show. Inevitably this will lead to some kind of expensive crisis. Financial plumbing, a clearing or settlement system, a major cloud dependency, a ah, payments network, a uh, utilities, billing or scheduling layer. [00:11:30] One or more of these things can and likely will get wrecked by an AI system soon. And hopefully it's a continuation of the wake up call Coxin has kick started. [00:11:42] This is a prediction I can absolutely get behind and using COXSwain's, you know, seven part tweet, I can get behind. I think it was kind of irresponsible how he shared it. Um, but is it helpful in general to kind of open people's eyes to like, yo, this is getting very good. Yes. Do I think the next statement is that could kill us all? That seems like a bit of a jump. And you know, Jake talks about it in his, in his newsletter and he's like, you know, if, if that's what it takes to like open people's eyes to the fact that this thing is getting really good, then so be it. And I'm kind of like, yes. And I don't know if like scare tactics is the best way to get people to actually like change something as opposed to just be afraid of it. So again, I am not saying that, you know, I'm not going to say that the AI killing us talk is propaganda, but I will say it's irresponsible like, what do you mean you think there is a greater than 10% chance that AI kills us all within the next decade and you're just going to post about it on X? That's how we're releasing this, this discussion. Give us a plan. [00:12:49] Ask AI for the plan. Right. Uh, give us some transparency. That's probably the bigger thing. Give us some transparency and actually explain the dange. Bring the receipt. The receipts. Show the receipts. Right now all we have are tweets and feelings. [00:13:04] Again, I do believe in how fast this is, uh, improving in specific domains. And I do believe that one, it's already in the wrong hands. But like, yes, could this end up very bad? Yes, but give us receipts. If you want us to actually do something, you want us to understand it and not just be scared of it, give us an action plan. [00:13:23] Right now. It does feel like a lot of lip service and it is very easy to think that, hey, this is partially a propaganda stunt. This is partially being done to help with, you know, bring attention to this of like, basically to bring attention to how capable this is so that as it we move forward, people will continue to invest money in it. Right? We have the IPOs of the companies being pushed. I think it was, it's, it's OpenAI, um, that push theirs back because it's not like a good time. You can't be like, hey, it's gonna kill us all and then like, oh, here's our ipo. Uh, ipo. Initial public offering. Right. [00:14:00] Not a good time. But does it perhaps still land as net positive to have so much discussion about how capable these models are from m. People that don't give a and they just want to make money. Yes, I do think that. So you know, where this leaves me is, is what I've said before, right. I believe in the capability of AI. I respect the fact that it could be dangerous. And then it needs regulations. I've been saying that, but I find it hard to just believe and take at face value and be scared of the words, uh, you know, coming from the intermittently coming from whistleblowers, maybe I'll call them intermittent whistleblowers along with the folks who are in charge because they continue to be vague. [00:14:45] Right. And as it relates to regulations, this is a really good point that was made by a guy named Ed Zitron. If you're in the AI space, you know, that name Edron is like a very popular, like, AI antagonist, pundit, if you will. Um, and I, I, I used to listen to a lot more Stuff. His stuff. When I was first getting into AI and just to like, round out my. [00:15:06] What I was learning and who I was hearing from. Um, and I recently listened to a podcast he was on. He's actually on, ah, a Define. A fintech podcast that I listen to, that I actually stopped listening to because I was kind of just getting depressed. Um, Prof. G. Podcast with, uh, Ed. Ed Elson and Scott Galloway. And I stopped listening to it because it was just like, the worst. People are doing the worst things and having zero repercussions. On to the next story. And I was like, I can't. I just, like, it stresses me out. Um, but Ed Zitron was on. Ed was speaking with Ed Elson. And I will link that episode in the. The show notes. It's like the first 20 minutes of the. Of the episode. But basically he more or less said that if they wanted to, they would. Right? If the government wanted regulations, they would have us have them. And he didn't say this next part. I'm saying this, which is that Mom Dani has shown us this. It is possible to get done. [00:15:56] Our government doesn't want to regulate this technology. They want to own it. They don't want to regulate it. [00:16:02] Right. Uh, our government wants to profit from this technology. [00:16:07] If you haven't already, prepare yourself for the incessant it's coming. The onslaught of China propaganda and messaging them. We have to beat them in the AI race. That is the next thing you're going to see a ton about. Whatever that means. Beat them in the race. Uh, whatever that means. Honestly, I'm gonna say it. [00:16:24] Beating them in the race, being first. [00:16:27] We don't win anything because they will just train off of our models. It's already happening, right? Like, it's actually pretty smart. You're like, let the people go first. And it is illegal to. To train off of someone else's model. But, like, look at our president. Look at our government. We ain't. We're not even doing anything legally over here, so shut the up. Um, but Trump has even said. And it's annoying. You'll hear it if you listen to that podcast episode that I, That I shared, um, with Ed Zitron. They put in a clip where Trump is talking to. [00:17:01] What, um, is his name? [00:17:02] Uh, the Nvidia owner. Um, there was like a big conference and Jensen Huang. And Trump literally says that data centers are the oil of the next 20 to 25 years. It is all about money. He's not trying to regulate because regulation means decreased revenue. [00:17:24] As for the the heads of AI, right. Dario Zario's at, at Anthropic. Sam Altman, you know, he's at OpenAI. [00:17:33] They have made statements and even Elon chimed in, fuck that guy. Uh, they've made statements calling for and encouraging the slowing of the pace of AI research and advancements. And only Dario, um, CEO again of Anthropic, he has set forth a plan. [00:17:49] Real talk, he wrote a long ass blog post about this. Um, and to me it feels a bit like lip service and wishful thinking. If, um, you got some time, give it a read. I will link that in the show notes as well and you can be the judge yourself. Um, but in one part of the, the blog post Dario wrote, I believe all Frontier Labs should partner with government to formalize the idea of permanent embedded evaluators to better prevent and document internal alignment accidents. Excuse me, internal alignment incidents like those that have occurred in the last few months. And to implement regulation focused on keeping capabilities in balance with safety. Dude, my guy Trump is out here likening data centers to oil, right? And saying we've got to beat China in the Aras, literally saying that I would not be holding my breath about any kind of significant government partnership focused on regulation. [00:18:43] So it's great that you wrote that, but like these, you're so smart that you can be the, the, the CEO and the, you know, the leaders of, of AI and of these companies and you think that this is actually going to happen, that the government's going to actually do this. You know, better. [00:18:59] You know, you're literally built a whole business on the how fuck it smart you are, and now you're going to be like, yes, and the government should do. They ain't going to fucking do this. That plan is not good. What else you got? What else do you got? So hopefully I am not coming off all doom and gloom, because I don't really feel doom or gloom about AI. Right. I just simply consider myself a realist in all domains, but definitely an AI realistic. [00:19:25] And that is how I continue to approach things. [00:19:28] I do want to say one last thing before I wrap this up. And it's something that I think about quite often, and that is that change does not occur in a vacuum. Yes, AI will be making advances over the next decade, but so will humans in general. [00:19:45] Perhaps you find this silly, that's fine. But to me, things like YouTube and Reddit, they continue to give me faith in humanity as so many folks, they just upload tutorials and they type out multi step solutions to super obscure problems just to be helpful because they are curious. And then they're like, oh, this helped me. I'm just going to put it up here. And it makes me feel good. [00:20:08] Open source anything that gives me hope. These people that are doing this, people like them, they will be showing up for the next decade as well. The good guy and girl. The good girl and guy. And they. Them, hackers, they exist. These people. People are out there. They're just so smart and they don't work for these labs. [00:20:27] They will also be around making advances. [00:20:30] A, uh, change doesn't happen in a vacuum. And I'm not saying like, oh, it's just like up to them to save us. No, I'm simply again, saying change doesn't happen in a vacuum. It is not like we stay exactly as we are right now and AI gets to improve for the next decade. [00:20:47] I all this to say that, yes, I absolutely take things like the OpenAI hugging face breach very seriously. And I fully agree that we need AI regulation. [00:20:58] But no, I don't think that AI is going to kill us all. And as per always, my preferred choice of preparation is. Wait for it. Drum roll. [00:21:09] Education. [00:21:11] Getting curious and continuing to learn the things. [00:21:16] All right, let's head into the last part of the episode, How I Use AI this week. If you're new here, welcome. Each week I share a quick example of how I use AI. That's that past week. So this, this time, um, last week, I was swapping out the thermostat on my Jeep Wrangler, and I went to install the new thermostat housing. And when I took it out of the box, it was making a rattling sound. And I was like, what the. [00:21:40] Because the thermostat housing that I took off of the Jeep did not make that rattling sound. [00:21:44] And so I asked Claude, I was like, is this normal that it's making a rattling sound? And it said, no, it's probably broken. [00:21:52] Spoiler alert. It was not broken and the noise was, in fact normal. [00:21:59] So I wound up driving to the dealership. I called them to see if they had more thermostat housings, and they did. So I drove to the dealership. It's like, literally, I don't know, three miles away from the house. [00:22:08] Uh, I love where I live. Uh, and I went there to buy another thermostat housing. And the woman brought the part out, and the part she brought out was also making the rattling sound. And I was like, is it supposed to do that? And she was like, oh, my God, I don't know. So she went to the back and she had a whole entire box of thermostat housings. She went through all of them, and they were all jingling. And so she brought out a mechanic and was like, it's supposed to do that. She's like, I went through all of them first and then found a mechanic. And the guy came out and he taught me that woman. And then the other man that was like, working the front desk because the other guy was like, I want to learn. And he showed me that the. The noise, the rattling was coming from one, like a very, very small ball bearing that helps regulate pressure. I think that on the older thermostat, it's not that it was broken, it's that, like, it's coated in thermos in the coolant. And so it just doesn't rattle because it just has liquid around it. [00:23:01] Um, but it's supposed to do that. [00:23:04] So the point here, right, all that to say, as we know, AI can absolutely be wrong and it will do so very confidently. So we could have that tie into, you know, today's. My thoughts on, is AI going to kill all of us? Um, but more importantly, I don't know, it's funny to say more importantly, uh, but, uh, more specifically, as it relates to my example, always check your work. When you're using the AI, Always check your work. All right, that, friends, is all for today. Hopefully you found this episode helpful if you did consider sharing this one with someone who you know is curious about AI and perhaps a bit concerned about AI. Um, don't forget I have a companion newsletter and blog, the Curious Companion, that drops every Thursday and is basically. By basically, I mean exactly the podcast episode in text format. So if you prefer to read or you just want a written record, join the newsletter, check out the blog. You can head to prompting curiosity.com newsletter or forward slog. Or keep it simple and just check out the link in the show notes. [00:24:11] As always, endlessly, endlessly, endlessly appreciative for every single one of you. This is a fun episode. Thank you for giving me the space to riff on this. Would love to hear from you. [00:24:22] Until we chat again next Thursday, stay curious.