The Limitless Podcast

Ejaaz:
A bunch of AI researchers from China just released a brand new AI model called

Ejaaz:
Kimi K2, which is not only as good as any other top model like Claude,

Ejaaz:
but it is also 100% open source, which means it's free to take,

Ejaaz:
customize and create into your own brand new AI model.

Ejaaz:
This thing is amazing at coding, it beats any other model at creative writing,

Ejaaz:
and it also has a pretty insane voice mode.

Ejaaz:
Oh, and I should probably mention that it is one trillion parameters in size,

Ejaaz:
which makes it one of the biggest and largest models to ever be created.

Ejaaz:
Josh, we were winding down on a Friday night and this news broke that this team

Ejaaz:
had released this model.

Ejaaz:
Absolutely crazy bomb, especially with like OpenAI rumored to release their

Ejaaz:
open source model this week.

Ejaaz:
You've been jumping into this. What's your take?

Josh:
Yeah. So last week we crowned Grok 4 as the new leading private model, closed source model.

Josh:
This week we got to give the crown to Kimi K2 we got another crown

Josh:
going for the open source team they are winning I mean this is

Josh:
better than DeepSeek and DeepSeek R2 this is basically DeepSeek R3

Josh:
I would imagine um and if you remember back a couple months DeepSeek really

Josh:
flipped the world on its head because of how efficient it was and the algorithmic

Josh:
upgrades it made and I think what we see with Kimi K2 is a lot of the same thing

Josh:
it's it's these novel breakthroughs that come as a downstream effect of their

Josh:
needing to be resourceful

Josh:
China, they don't have the mega GPU clusters we have, they don't have all the

Josh:
cutting edge hardware, but they do have the software prowess to find these efficiencies.

Josh:
I think that's what makes this model so special. And that's what we're going

Josh:
to get into here is specifically what they did to make this model so special.

Ejaaz:
Yeah, I mean, look at these stats here, Josh, like 1 trillion parameters in total.

Ejaaz:
It's 32 billion active mixture of expert models. So what this means is,

Ejaaz:
although it's really large in size, typically these AI models can become pretty

Ejaaz:
inefficient if it's large in size, it uses this technique called mixture of

Ejaaz:
experts, which means that whenever someone queries a model,

Ejaaz:
it only uses or activates a number of parameters that are relevant for the query itself.

Ejaaz:
So it's more smarter, it's much more efficient, and it doesn't use or consume

Ejaaz:
as much energy as you would if you wanted to run it locally at home or whatever

Ejaaz:
that might be. It's also super cheap.

Ejaaz:
I think I saw somewhere that this was 20% the cost of clawed,

Ejaaz:
josh which uh we love that insane uh

Ejaaz:
for all the nerds that kind of want to run you know

Ejaaz:
really long tasks or you know just set and

Ejaaz:
forget the ai to to run on like your coding log or whatever that might mean

Ejaaz:
you can now do it at a much more affordable rate at one-fifth the cost uh than

Ejaaz:
some of the top models that are out there and it is as good as those models

Ejaaz:
so just insane kinds of things josh i know there's a bunch of things that you

Ejaaz:
wanted to point out here on benchmarks um And what do you want to get into?

Josh:
Yeah, it's really amazing. So they took 15 and a half trillion tokens and they

Josh:
condensed those down into a one trillion parameter model.

Josh:
And then what's amazing is when you use this model, like she said,

Josh:
it uses a thing called mixture of experts.

Josh:
So it has, I believe, 384 experts.

Josh:
And each expert is good at a specific thing. So let's say in the case you want

Josh:
to do a math problem, it will take a 32 billion parameter subset of the one

Josh:
trillion total parameters, and it will choose eight of these different

Josh:
Experts in a specific thing. So in the case of math, it'll find an expert that

Josh:
has the calculator tool.

Josh:
It'll find an expert that has a fact, like a fact checking tool or a proof tool

Josh:
to make sure that the math is accurate.

Josh:
It'll have just a series of tools to help itself. And that's kind of how it

Josh:
works so efficiently is instead of using a trillion parameters at once,

Josh:
it uses just 32 billion and it uses the eight best specialists out of the 384

Josh:
that it has available to it. It's really impressive.

Josh:
And what we see here is the benchmarks that we're showing on screen.

Josh:
And the benchmarks are really good.

Josh:
It's up there in line with just about any other top model, except with the exception

Josh:
that this is open source.

Josh:
And there was another breakthrough that we had, which was the actual way that

Josh:
they handled the training of this.

Josh:
And yeah, this is the loss curve. So what you're looking at on screen for the

Josh:
people who are listening, it's this really pretty smooth curve that kind of

Josh:
starts at the top and it trends down in a very predictable and smooth way.

Josh:
And most curves don't look like this. And if they do look like this,

Josh:
it's because the company has spent tons and tons of money on error correction

Josh:
to make sure this curve is so smooth.

Josh:
So basically what you're seeing is the training run of the model.

Josh:
And a lot of times what happens is you get these very sharp spikes and it starts

Josh:
to defer away from the normal training run.

Josh:
And it takes a lot of compute to kind of recalibrate and push that back into the right way.

Josh:
What they've managed to do is really make it very smooth.

Josh:
And they've done this by increasing these efficiencies. So if you can think

Josh:
about it, there's this analogy I was thinking of right before we hit the record button.

Josh:
And it's if you were teaching a chef how to cook, right?

Josh:
So we have Chef Ejaz here. I am teaching him how to cook. I am an expert chef.

Josh:
And instead of telling him every ingredient and every step for every single

Josh:
dish, what I tell him is like, hey, if you're making this amazing dinner recipe,

Josh:
all you need that matters is this amount of salt applied at this time,

Josh:
this amount of heat applied for this length of time, and the other stuff doesn't matter as much.

Josh:
So just put in whatever you think is appropriate, but you'll get the same answer.

Josh:
And that's what we see with this model is just an increased amount of efficiency by being

Josh:
direct by being intentional about the data that they used to train it on,

Josh:
the data that they used to fetch in order to give you high quality queries.

Josh:
And it's a really novel breakthrough. They call it the MuonClip optimizer,

Josh:
which, I mean, it's a Chinese company, maybe it means something special there,

Josh:
but it is a new type of optimizer.

Josh:
And what you're seeing in this curve is that it's working really well and it's

Josh:
working really efficient.

Josh:
And that's part of the benefit of having this open source is now we have this

Josh:
novel breakthrough and we could take this and we could use this for even more

Josh:
breakthroughs even more open source models and and that's part that's been really cool to see

Ejaaz:
I i mean this is just um time

Ejaaz:
and again from china uh so so amazing from their research team so so like just

Ejaaz:
to kind of like um pick up your comment on deep seek at the end of last year

Ejaaz:
we were utterly convinced that the only way to create a breakthrough model was

Ejaaz:
to spend billions of dollars on compute clusters.

Ejaaz:
And so therefore it was a pay-to-play game. And then DeepSeek,

Ejaaz:
a team out of China, released their model and completely open-sourced it as well.

Ejaaz:
And it was as good as OpenAI's Frontier model, which was the top model at the time.

Ejaaz:
And the revelation there was, oh, you don't actually just need to chuck a bunch of compute at this.

Ejaaz:
There are different techniques and different methods if you get creative about

Ejaaz:
how you design your model and how you run the training cluster,

Ejaaz:
the training one, which is basically what you need to do to make your model smart,

Ejaaz:
you can run it in different ways that is more efficient, consumes less energy,

Ejaaz:
and therefore less amount of money, but is as smart, if not smarter,

Ejaaz:
than the frontier models that American AI companies are making.

Ejaaz:
And this is just a repeat of that, Josh.

Ejaaz:
I mean, look at this curve. For those who are looking at this episode on video.

Ejaaz:
It is just so clean yeah it's beautiful

Ejaaz:
the craziest part about this is when deep

Ejaaz:
seek was released they pioneered something called uh reasoning

Ejaaz:
or reinforcement learning uh which are two separate

Ejaaz:
techniques that made the model super smart um with less energy and less compute

Ejaaz:
spend um with this model they didn't even implement that technique at all so

Ejaaz:
theoretically this model can get so much more smarter than it already is um

Ejaaz:
and they just kind of leveraged a new method to make it as smart as it already is right now.

Ejaaz:
So just such a fascinating kind of like progress in research from China.

Ejaaz:
And it just keeps on coming out. It's so impressive.

Josh:
Yeah, this is this was the exciting part to me is that we're seeing so many

Josh:
algorithms or exponential improvements in so many different categories.

Josh:
So this was considered a breakthrough by all means. And this wasn't even the

Josh:
same type of breakthrough that DeepSeek had.

Josh:
So we get this now compounding effect where we have this new training breakthrough

Josh:
and then we have DeepSeek who has the reinforcement learning and that hasn't

Josh:
even yet been applied to this new model.

Josh:
So we get the exponential growth on one end, the exponential growth on the reasoning end,

Josh:
those come together and then you get the exponential growth on the hardware

Josh:
stack where the GPUs are getting much faster and there's all of these different

Josh:
subsets of AI that are compounding on each other and growing and accelerating

Josh:
quicker and quicker and what you get is this unbelievable rate of progress and

Josh:
that's what we're seeing. So

Josh:
reasoning isn't even here yet and we're going to see it soon because it is open

Josh:
source so people can apply their own reasoning on top of it i'm sure the moonshot

Josh:
team is going to be doing their own reasoning version of this model and i'm

Josh:
sure we're going to be getting even more impressive results soon i see you have

Josh:
a post up here um about the testing and overall performance can you please share yeah

Ejaaz:
Yeah so um this is a tweet that summarizes really well how this model performs

Ejaaz:
in relation to other Frontier models.

Ejaaz:
And the popular comparison that's taken for Kimi K2 is against Claude.

Ejaaz:
So Claude has a bunch of models out.

Ejaaz:
Claude 3.5 is its earlier model, and then Claude 4 is its latest.

Ejaaz:
And the general take is that this model is just better than those models,

Ejaaz:
which is just insane to say, because for so long, Josh, we've said that Claude

Ejaaz:
was the best coding model.

Ejaaz:
And indeed it was. And then within the span of, what is it, five days?

Ejaaz:
Grok 4 released and it just completely blew Claude 4 out of the water in terms of coding.

Ejaaz:
Now Kimi K2, an open source model out of China who doesn't even have access

Ejaaz:
to the research and kind of proprietary knowledge that a lot of American AI

Ejaaz:
companies have also beat it as well, right?

Ejaaz:
So it kind of beats Claude at its own game, but it's also cheaper.

Ejaaz:
It's 20% the cost of Claude 3.5, which is just an insane thing to say,

Ejaaz:
which means that if you are a developer out there that

Ejaaz:
wants to try your hand at kind of like vibe coding

Ejaaz:
a bunch of things or actually seriously coding something you

Ejaaz:
know that's quite novel but you don't have the hands on deck to do that you

Ejaaz:
can now spin up a Kimi K2 AI agent actually multiple of them for a very cost-efficient

Ejaaz:
reasonable you know salary you don't have to pay like hundreds of thousands

Ejaaz:
of dollars or you know hundreds of millions of dollars which is what Meta is

Ejaaz:
doing to kind of buy a bunch of these software engineers,

Ejaaz:
you can spend, you know, the equivalent of maybe a Netflix subscription or $500

Ejaaz:
to $1,000 a month and spin up your own app. So super, super cool.

Josh:
And also one added perk that's there is it's that even if you have a lot of

Josh:
GPUs sitting around, you can actually run this model for free.

Josh:
So that's the cost if you actually query it from the servers.

Josh:
But I'm sure there's going to be companies that have access to XS GPUs.

Josh:
They can actually just download the model because it's open source,

Josh:
open weights, and they could run it on their own.

Josh:
And that brings the cost of compute down to the cost per kilowatt of the energy

Josh:
required to run the GPUs.

Josh:
So because it's open source, you really start to see these costs decline,

Josh:
but the quality doesn't.

Josh:
And that's every time we see this, we see a huge productivity unlock in encoding

Josh:
output and amount of queries used. It's like, this is freaking awesome.

Ejaaz:
Yeah josh i saw something else come up as well so so do you remember when claude

Ejaaz:
first released um their frontier model i think it was 3.5 or maybe it was four

Ejaaz:
one of their bragging rights was it had a one million uh token context window which.

Josh:
Oh yes which was huge

Ejaaz:
Yeah which for listeners of the show is huge it's like several uh book novels

Ejaaz:
worth um of words or characters you could just bung into one single prompt.

Ejaaz:
And the reason why that was such an amazing thing was for a while,

Ejaaz:
people struggled to kind of communicate with these AIs because they couldn't set the context.

Ejaaz:
There wasn't enough bandwidth within their chat log window for them to say,

Ejaaz:
you know, and don't forget this. And then there was this.

Ejaaz:
And then, you know, this detail and that detail, there just wasn't enough space.

Ejaaz:
And models weren't performing enough to kind of consume all of this in one go.

Ejaaz:
And then Claude came out and was like, hey, we have one million context windows.

Ejaaz:
Don't worry about it chuck in all the research papers that you want chuck in

Ejaaz:
your essay chuck in reference books and we got you um i saw this tweet that

Ejaaz:
was uh deleted i think you sent this to me um.

Josh:
We got the screenshots we always come with receipts yeah i

Ejaaz:
Wonder why they deleted it but uh good catch from you um yeah let's get into this.

Josh:
What's your take on it was was first posted i think

Josh:
earlier today yeah like an hour ago and then deleted pretty shortly afterwards

Josh:
and this is from a woman name crystal crystal works with the moonshot team she

Josh:
is part of the team that that released kimmy k2 um and in this post it says

Josh:
kimmy isn't just another ai it went viral in china as the first to support

Josh:
A 2 million token context window. And then she goes on to say,

Josh:
we're an AI lab with just 200 people, which is ministerially small compared

Josh:
to a lot of the other labs they're competing with.

Josh:
And it was acknowledgement that they had a 2 million token context window.

Josh:
And for those who, just a quick refresher on the context window stuff,

Josh:
it's imagine you have like a gigantic textbook and you've read it once and you

Josh:
close it and you kind of have a fuzzy memory of all the pages.

Josh:
The context window allows you to lay all of those out in clear view

Josh:
and directly reference every single page so when

Josh:
you have two million tokens which is roughly two million words

Josh:
of context we're talking about like hundreds and hundreds

Josh:
of books and textbooks and knowledge and you could really dump a

Josh:
lot of information in this for the ai to readily access and

Josh:
that if they release that a two million token

Josh:
open source model that's huge

Josh:
deal i mean even grok 4 recently i believe

Josh:
what did we say it was it was a 256 000 uh token context window something like

Josh:
that so grok 4 is one eighth of what they supposedly have accessible right now

Josh:
which is a really really big deal um so i'm hoping it was deleted because they

Josh:
just don't want to share that not because it's not true i would like to believe

Josh:
that it's true because man that'd be pretty epic yeah

Ejaaz:
And the people are loving it josh um check out this

Ejaaz:
graph from uh open router which basically shows

Ejaaz:
uh the split of usage between everyone

Ejaaz:
on their platform that are querying different models so for context

Ejaaz:
here open router is a website that you can go to

Ejaaz:
and you can type up a prompt just like you do at chat gpt and

Ejaaz:
you can decide which model your

Ejaaz:
prompt goes to or you could let open router decide for you

Ejaaz:
and it kind of like divvies up your query so if you have a coding query it's

Ejaaz:
probably going to send it to claude or now kimmy k2 or grok4 but if you have

Ejaaz:
something that's more like to do with creative writing or something that's like

Ejaaz:
a case study it might send it to OpenAI's O3 model, right? So it kind of like decides for you.

Ejaaz:
OpenRacha released this graphic, which basically shows that KimiK2 surpassed

Ejaaz:
XAI in token market share just a few days after launching, which basically means

Ejaaz:
that XAI spent, you know,

Ejaaz:
hundreds of billions of dollars training up their Grok4 model,

Ejaaz:
which just kind of beat out the competition just last week.

Ejaaz:
Then KimiK2 gets released completely open source

Ejaaz:
and everyone starts to use that more than

Ejaaz:
grok 4 which is just an insane thing to say and

Ejaaz:
just shows how rapidly these ai models compete with each other and surpass each

Ejaaz:
other um i think part of the reason for this josh is it's open source right

Ejaaz:
which means that not only are retail users like myself and yourself using it

Ejaaz:
for our daily queries you know uh you know,

Ejaaz:
create this recipe for me or whatever, but researchers and builders all over

Ejaaz:
the world that have so far been challenged or had this obstacle of pots of money

Ejaaz:
basically to start their own AI company now have access to a frontier,

Ejaaz:
world-renowned model and can create whatever application, website,

Ejaaz:
or product that they want to make.

Ejaaz:
So I think that's part of the usage there as well. Do you have any takes on this?

Josh:
Yeah, and it's downstream of cost, right? We always see this when a model is

Josh:
cheaper and mostly equivalent, the money will always flow to the cheaper model.

Josh:
It'll always get more queries. I think it's important to note the different

Josh:
use cases of these models. So they're not directly competing head to head on the same benchmarks.

Josh:
I think what we see is like when we talk about Claude, it's generally known as the coding model.

Josh:
And I don't think like OpenAI's O3 is not really competing directly with Claude

Josh:
because it's more of a general intelligence versus a coding specific intelligence.

Josh:
K2 is probably closer to a Claude. I would assume where it's really good at

Josh:
coding because it uses this mixture of experts.

Josh:
And I think that helps it find the tools. It uses this cool new novel thing

Josh:
called like multiple tool use.

Josh:
So each one of these experts can use a tool simultaneously and they could use

Josh:
these tools and work together to get better answers.

Josh:
So in the case of coding, this is a home run.

Josh:
Like it is very cheap cost per token, very high quality outputs.

Ejaaz:
I actually think you can compete with OpenAO3, Josh. Check this out.

Ejaaz:
So Rowan, yeah, Rowan Cheng put this out yesterday And he basically goes,

Ejaaz:
I think we're at the tipping point for AI-generated writing.

Ejaaz:
It's been notoriously bad, but China's Kimi K2, an open-weight model,

Ejaaz:
is now topping creative writing benchmarks.

Ejaaz:
So just to put that into context, that's like having the top most, I don't know,

Ejaaz:
smartest or slightly autistic software engineer, at the top engineering company

Ejaaz:
working on AI models, also being the best poet or creative script and directing

Ejaaz:
the next best movie or whatever that might be,

Ejaaz:
or creating a Harry Potter novel series.

Ejaaz:
This model can basically do both. And what it's pointing out here is that compared

Ejaaz:
to 03, it tops it. Look at this. Completely beats it.

Josh:
Okay, so I take that back. Maybe it is just better at everything.

Josh:
Yeah, that's some pretty impressive results.

Ejaaz:
I think like what's worth pointing out here is, and I don't know whether any

Ejaaz:
of the American AI models do this, Josh, but mixture of experts seems to be clearly a win here.

Ejaaz:
The ability to create an incredibly smart model doesn't come without,

Ejaaz:
you know, this large storage load that is needed, right? One trillion parameters.

Ejaaz:
But then combining it with the ability to be like, Like, hey,

Ejaaz:
you don't need to query the entire thing.

Ejaaz:
We've got you. We have a smart router, which basically pulls on the best experts,

Ejaaz:
as you described earlier, for whatever relevant query you have.

Ejaaz:
So if you have a creative writing task or if you have a coding thing,

Ejaaz:
we'll send it to two different departments of this model.

Ejaaz:
That's a really huge win. Do any other American models use this?

Josh:
Well, the first thing that came to my mind when you said that is Grok4,

Josh:
which doesn't exactly use this, but uses a similar thing, where instead of using

Josh:
a mixture of experts, It uses a mixture of agents.

Josh:
So Grok4 Heavy uses a bunch of distributed agents that are basically clones of the large model.

Josh:
But that takes up a tremendous amount of compute. And that is the $300 a month plan.

Ejaaz:
That's replicating Grok4 though, right? So that's like taking the model and copy pasting it.

Ejaaz:
So let's say Grok4 was one trillion parameters just for ease of comparison.

Ejaaz:
That's like creating, if there was four agents, that's four trillion parameters,

Ejaaz:
right? So it's still pretty costly and inefficient.

Josh:
Is that what you're saying no it's the actually the opposite direction of k2

Josh:
so what they have used is just and again this is kind of similar to tracking

Josh:
sentiment between the united states and china where the united states will throw

Josh:
compute at it where china will throw like

Josh:
kind of clever resource at it so grok yeah

Josh:
when they use their mixture of agents it actually just costs a lot more

Josh:
money whereas k2 when they use their mixture of

Josh:
experts well it costs a lot less instead of using 4 trillion

Josh:
parameters in this case it uses just 32 billion and it

Josh:
kind of copies that 32 billion over and over and it's really it's a really

Josh:
elegant solution that seems to be

Josh:
yielding pretty comparable results so i think as we

Josh:
see these efficiency upgrades i'm sure they will

Josh:
eventually trickle down into the united states models and when they do that

Josh:
is going to be a huge unlock in terms of cost per token in terms of the smaller

Josh:
distilled models that we're going to be able to run on our own computers um

Josh:
but yeah i don't know of any who are also using it at this scale it might be

Josh:
novel just to k2 right now and

Ejaaz:
And i think that this is the method that probably scales the best josh like.

Josh:
Yeah it makes sense efficiency

Ejaaz:
Always wins at the end right and to see um this kind of innovation come pretty

Ejaaz:
early on in a technology's life cycle is just super impressive to see,

Ejaaz:
Another thing I saw is there's two different versions of this model, I believe.

Ejaaz:
There's something called Kimi K2 Base, which is basically the model for researchers

Ejaaz:
who want full control for fine-tuning and custom solutions, right?

Ejaaz:
So imagine this model as the entire parameter set. So you have access to one

Ejaaz:
trillion parameters, all the weight designs and everything.

Ejaaz:
And if you're a nerd that wants to nerd out you can

Ejaaz:
go crazy you know if you have like your own gpu

Ejaaz:
cluster at home or if you happen to have a convenient

Ejaaz:
warehouse full of of servers that you weirdly

Ejaaz:
have access to you can go crazy with it you can if you

Ejaaz:
think about like um the early gaming days of counter-strike and then you could

Ejaaz:
like mod it you can basically mod this uh model to your heart's desire and then

Ejaaz:
there's a second version called k2 instruct which is for drop-in general purpose

Ejaaz:
chat and AI agent experiences.

Ejaaz:
So this is kind of like at the consumer level, if you're experimenting with

Ejaaz:
these things, or if you want to run an experiment at home on a specific use

Ejaaz:
case, you can kind of like take that away and do that for yourself.

Ejaaz:
That's how I understand it, Josh. Do you have any takes on this?

Josh:
That makes sense. And I think that second version that you're describing is

Josh:
what's actually available publicly on their website, right?

Josh:
So if you go to Kimmy.com, it has a text box. It looks just like ChatGPT like you're used to.

Josh:
And that's where you can run that second tier model which

Josh:
um you described as that's the the drop in general purpose

Josh:
chat and then yeah for the the hardcore researchers there's

Josh:
a github repo and the github repo has all the weights and all the code and

Josh:
you can really download it dive in use the full thing i

Josh:
was playing around with the kimmy tool and it's it's really cool

Josh:
it's fast oh i mean it's lightning fast if you

Josh:
go from a reasoning model to an inference model like kimmy

Josh:
you get responses like this like when

Josh:
i'm using grok 4 or o3 i'm sitting there sometimes for a couple minutes it's

Josh:
waiting for an answer this you type it in and it just types back right away

Josh:
no time waiting so it's it's kind of refreshing to see that but it's also a

Josh:
testament to how impressive it is i'm getting great answers and it's just spitting

Josh:
it right out so what happens when they add the reasoning layer on top well it's

Josh:
probably going to get pretty freaking good

Ejaaz:
So the trend we're seeing, and we saw this last week with Grok4,

Ejaaz:
is typically we're expected to wait a while when we send a prompt to a breakthrough

Ejaaz:
model because it's thinking, it's trying to basically replicate what we have in our brains up here.

Ejaaz:
And now it's just getting much quicker and much smarter and much cheaper.

Ejaaz:
So the long story short is these incredibly powerful, I kind of think about

Ejaaz:
it as how we went from massive desktop computers to slick cell phones,

Ejaaz:
Josh, and then we're going to eventually have chips in our brain.

Ejaaz:
AI is just kind of like fast tracking that entire life cycle within like a couple

Ejaaz:
of years, which is just insane.

Josh:
And these efficiency improvements are really exciting because you can see how

Josh:
quickly they're shrinking and allowing eventually for those incredible models

Josh:
to just run on our phones.

Josh:
So there's totally a world a year from now in which like a

Josh:
grok 403 kimmy k2 capable model

Josh:
is small enough that it could just run inside of in our

Josh:
phone and run on a mobile device or run locally on a laptop

Josh:
or you're offline and you kind of have this portable intelligence

Josh:
that's available everywhere anytime even if

Josh:
you're not connected to the world and that seems really cool

Josh:
like we were talking a few episodes ago about apple's um local

Josh:
free ai inference running on an iphone

Josh:
but how the base models still kind of suck like they don't really do

Josh:
anything super interesting they're basically good enough to do what

Josh:
you would expect siri to do but can't do and these

Josh:
models as we get more and more breakthroughs like this that allow you to

Josh:
run much larger parameter counts

Josh:
on a much smaller device it's going to start really

Josh:
super powering these mobile devices and i can't help but think about the open

Josh:
ai hardware device i'm like wow that'd be super cool if you had like oh three

Josh:
running locally in the middle of the jungle somewhere with no service and you

Josh:
still had access to all of its capabilities like that's probably coming downstream

Josh:
of breakthroughs like this where we get really big efficiency unlocks

Ejaaz:
I mean, it's not just efficiency, though, right? It's the fact that if you can

Ejaaz:
run it locally on your device, it can have access to all your private data without

Ejaaz:
exposing all of that to the model providers themselves, right?

Ejaaz:
So one of the major concerns of not just AI models, but also with mobile phones is privacy.

Ejaaz:
I don't want to share all my kind of like private health, financial,

Ejaaz:
and social media data, because then you're just going to have everything on

Ejaaz:
me and you're going to use me.

Ejaaz:
You're going to use me as a product, right? And that's kind of like been the

Ejaaz:
quota for the last decade in tech.

Ejaaz:
And so with AI, that's a supercharged version of it. The information gets more

Ejaaz:
personal. It's not just your likes.

Ejaaz:
It's, you know, where Josh shops every day and, you know, who he's dating and

Ejaaz:
all these kinds of things, right?

Ejaaz:
And that becomes quite personal and intrusive very quickly.

Ejaaz:
So the question then becomes, how can we have the magic of an AI model without it being so obtrusive?

Ejaaz:
And that is open source locally run AI or privately run AI. and Kimi K2 is a

Ejaaz:
frontier model that can technically run on your local device if you set up the right hardware for it.

Ejaaz:
And the way that we're trending, you can basically end up having that on your

Ejaaz:
device, which is just a huge unlock.

Ejaaz:
And if you can imagine how you use OpenAI 03 right now, Josh,

Ejaaz:
right? I know you use it as much as I do.

Ejaaz:
The reason why you and I use it so much isn't just because it's so smart,

Ejaaz:
but it's because it remembers everything about us.

Ejaaz:
But I hate that Sam knows or has access to all that data.

Ejaaz:
I hate that if he chooses to switch on personalized ads, which is currently

Ejaaz:
the model where most of these tech companies make money right now,

Ejaaz:
he can, and I've got nothing to do about it because I don't want to use any

Ejaaz:
other model apart from that.

Ejaaz:
But if there was a locally run

Ejaaz:
model that had access to all the memory and context, I'd use that instead.

Josh:
And this is suspicious. I mean, this is a different conversation in total,

Josh:
but isn't it interesting how other companies haven't really leaned into memory

Josh:
when it's seemingly the most important mode that there is.

Josh:
Like Grok4 doesn't have good memory rolled out. Gemini doesn't really have memory.

Josh:
There's no, Claude doesn't have memory the way that OpenAI does.

Josh:
Yet it's the single biggest reason why we both continue to go back to ChatGPT and OpenAI.

Josh:
So that's just been an interesting thing. I mean, Kimmy is open source.

Josh:
I wouldn't expect them to lean too much into it. But for these closed source

Josh:
models, that's just, it's another interesting just observation.

Josh:
Like, hey, the most important thing isn't, doesn't seem to be prioritized by

Josh:
other companies just yet.

Ejaaz:
Why do you think that is so so my theory um at least from xai or grok force

Ejaaz:
perspective is elon's like okay i'm not going to be able to build a better chat

Ejaaz:
bot or chat messenger than openai has there's not too many features i can um.

Ejaaz:
Set Grok 4 apart, then that O3 doesn't already do, right?

Ejaaz:
But where I can beat O3 is at the app layer.

Ejaaz:
I can create a better app store than they have because I haven't really created

Ejaaz:
one that is sticky enough for users to continually use.

Ejaaz:
And I can use that data set to then unlock memory and context at that point, right?

Ejaaz:
So I just saw today that they released, they

Ejaaz:
being um xai released a new feature for grok 4

Ejaaz:
called i think it's uh companions josh um

Ejaaz:
and it's basically these yeah these animated um

Ejaaz:
avatar like um characters so they basically look like they're from an anime

Ejaaz:
show and you know how you can use voice mode in open ai and you can kind of

Ejaaz:
like talk to this uh realistic human sounding ai you now have a face and a character

Ejaaz:
on grok 4 and it's really entertaining, Josh.

Ejaaz:
Like I find myself kind of like engaged in this thing because I'm not just typing words.

Ejaaz:
It's not just this binary to and fro with this chat messenger.

Ejaaz:
It's this human, this cute, attractive human that I'm just like now speaking to.

Ejaaz:
And I think that that's the strategy that a lot of these AI companies,

Ejaaz:
if I had to guess, are taking to kind of like seed their user base before they

Ejaaz:
unlock memory. I don't know whether you have a take on that.

Josh:
Yeah, I have a fun little demo. I actually played around with it this morning

Josh:
and I was using it totally unhinged, no filter, very vulgar,

Josh:
but like kind of fun. It's like a fun little party trick.

Josh:
And yeah, I mean, that was a surprise to me this morning when I saw that rolled

Josh:
out. I was like, huh, that doesn't really seem like it makes sense.

Josh:
But I think they're just having fun with it.

Ejaaz:
Can we for a second talk about the team?

Ejaaz:
So we've mentioned just now how they've all come from China and how China's

Ejaaz:
like really advancing open source AI models, and they've completely beat out

Ejaaz:
the competition in America, Mata's Lama being the obvious one.

Ejaaz:
We've got Kwen from Alibaba.

Ejaaz:
We've got Deep Seek R1. Now we have Kimi K2. The team is basically...

Ejaaz:
The AI Avengers of China, Josh. So these three co-founders all have deep AI

Ejaaz:
ML backgrounds that hail from the top American universities,

Ejaaz:
such as Carnegie Mellon.

Ejaaz:
One of them has a PhD from Carnegie Mellon in machine learning,

Ejaaz:
which is basically, for those of you who don't know, is like God-tier degree for AI.

Ejaaz:
That means you're desirable and hireable by every other AI company after you graduate.

Ejaaz:
But it's not just that. They also have credibility and degrees from the top universities in China.

Ejaaz:
Especially this one university called Tsinghua, which seemed to be the top of their field.

Ejaaz:
I looked them up on rankings for AI universities globally, and they often come

Ejaaz:
in number three or four in the top 10 AI universities. So pretty impressive from there.

Ejaaz:
But what I found really interesting, Josh, was one of the co-founders was an

Ejaaz:
expert in training AI models on low-cost optimized hardware.

Ejaaz:
And the reason why I mentioned this is it's no secret that if you want a top

Ejaaz:
frontier AI model, you need to train it on NVIDIA's GPUs.

Ejaaz:
You need to train it on NVIDIA's hardware.

Ejaaz:
NVIDIA's market cap, I think, at the end of last week, surpassed $4 trillion.

Ejaaz:
That's $4 trillion with a T. That is more than the current GDP of the entire British economy.

Josh:
Where I hail from. And the largest in the world.

Ejaaz:
And there's never been.

Josh:
A bigger company

Ejaaz:
There's never been a bigger company it it's just

Ejaaz:
insane to grab your head around and it's not without

Ejaaz:
reason they supply basically or they have a

Ejaaz:
grasp or a monopoly on the hardware that

Ejaaz:
is needed to train top models now kimmy k2

Ejaaz:
comes along casually drops a one trillion parameter model one of the largest

Ejaaz:
models ever released um and it's trained on hardware that isn't nvidia's um

Ejaaz:
and jensen huang i i need to find this clip josh but But Jensen Huang basically

Ejaaz:
was on stage, I think it was at a private conference maybe yesterday,

Ejaaz:
but he was quoted as saying 50% of the top AI researchers are Chinese and are from China.

Ejaaz:
And what he was implicitly getting at is they're a real threat now.

Ejaaz:
I think for the last decade, we've kind of been like, ah, yeah,

Ejaaz:
China's just going to copy paste everything that comes out of America's tech sector.

Ejaaz:
But when it comes to AI, we've kind of like maintained the same mindset up until

Ejaaz:
now where they're really just competing with us.

Ejaaz:
And if they have the hardware, they have the ability to research new techniques

Ejaaz:
to train these models, like DeepSeek's reinforcement learning and reasoning,

Ejaaz:
and then Kimi K2's kind of like efficient training run, which you showed earlier.

Ejaaz:
They've come to play, Josh. And I think it's worth highlighting that China has

Ejaaz:
a very strong grasp on top AI researchers in the world and models that are coming out of it.

Josh:
Where are their $100 million offers? I haven't seen any of those coming through.

Josh:
None, dude. The most impressive thing is that they do it without the resources that we have.

Josh:
Imagine if they did have access to the clusters of these like H100s that NVIDIA is making.

Josh:
I mean, that would be, would they crush us?

Josh:
And we kind of have this timeline here where we're kind of running up against

Josh:
the edge of energy that we have available to us to train these massive models.

Josh:
Whereas China does not have that constraint. They have significantly more energy to power these.

Josh:
So in the event, the inevitable event that they do get the chips and they are

Josh:
able to train at the scale that we are, I'm not sure we're able to continue

Josh:
our rate of acceleration in terms of hardware manufacturing,

Josh:
large training as fast as they will.

Josh:
And they already have done the hard work on the software efficiency side.

Josh:
They've cranked out every single efficiency because they are doing it on constrained hardware.

Josh:
So it's going to create this really interesting effect where they're coming

Josh:
at it from the like ingenuity software approach we're coming at it from the

Josh:
brute force throw a lot of compute added approach and we'll see where both both

Josh:
sides end up um but it's clear that china is still behind because they are the

Josh:
ones open sourcing the models and we know at this point now if you're open sourcing

Josh:
your model you're doing it because you're behind

Ejaaz:
Yeah yeah i mean one thing

Ejaaz:
that did surprise me josh was that they released a one

Ejaaz:
trillion parameter open source model i i didn't

Ejaaz:
expect them to catch up that quickly um like one

Ejaaz:
trillion is a lot um yeah another thing

Ejaaz:
i was thinking about is china has dominated

Ejaaz:
hardware for so long now so it wouldn't

Ejaaz:
really surprise me if like i don't know a

Ejaaz:
couple years from now they're producing better models

Ejaaz:
at specific things basically because they have better

Ejaaz:
hardware than america than the west um but

Ejaaz:
where i think the west will continue to dominate

Ejaaz:
is at the application layer and i don't

Ejaaz:
know if i was a betting man i would say that most of the money is eventually going

Ejaaz:
to be made on the application side of things i think grok

Ejaaz:
4 is starting to um kind of show that

Ejaaz:
with all these different kinds of novel features that they're releasing i i

Ejaaz:
don't know if you've seen some of the games that are being produced from grok

Ejaaz:
4 josh but it is ultimately insane and i haven't seen any similar examples come

Ejaaz:
out of uh asia from any of their ai models even when they have access to american

Ejaaz:
models So I still think America dominates at the app layer.

Ejaaz:
But Josh, I just came across this tweet, which you reminded me of earlier.

Ejaaz:
Tell me about OpenAI's strategy to open source model, because I got this tweet

Ejaaz:
pulled up from Sam Altman, which is kind of hilarious.

Josh:
Yeah. All right. So this week, if you remember from our episode last week,

Josh:
we were excited about talking about OpenAI's new open source model.

Josh:
OpenAI, open source model, all checks out. This was going to be the big week.

Josh:
They released their new flagship open source. Well, conveniently,

Josh:
I think the same day as K2 launched, later in the day, or perhaps the very next morning.

Josh:
Sam Altman posted a tweet. He says, Hey, we plan to launch our open weights model next week.

Josh:
We are delaying it. We need time to run additional safety tests and review high-risk

Josh:
areas. We are not yet sure how long it will take us.

Josh:
While we trust the community will build great things with this model,

Josh:
once weights are out, they can't be pulled back. This is new for us and we want to get it right.

Josh:
Sorry to be the bearer of bad news. We are working super hard.

Josh:
So there's a few points of speculation. The first, obviously,

Josh:
being, did you just get your ass handed to you and now you are going back to

Josh:
reevaluate before you push out a remodel?

Josh:
So that's one possible thing where they saw K2. They were like,

Josh:
oh, boy, this is pretty sweet.

Josh:
This is our first open source model. We probably don't want to be lower than them.

Josh:
And there is this second point of speculation, which, Ejaz, you mentioned to

Josh:
me a little earlier today, where maybe something went wrong with the training one.

Josh:
And it's not quite that they're getting beat up by a Chinese company.

Josh:
Is that like they actually made a mistake on their own accord and can you explain

Josh:
to me specifically what that might be what the speculation is at least yeah

Ejaaz:
Well i'll keep it short i think it was a little racist under

Ejaaz:
the hood and i i can't find the tweet but basically

Ejaaz:
one of these um ai researchers slash

Ejaaz:
product builders on x got access to

Ejaaz:
the model supposedly according to him and he tested it

Ejaaz:
out uh in the background and he said yeah it's it's

Ejaaz:
not really an intelligence thing it's just worse than

Ejaaz:
what uh you'd expect from an alignment and uh consumer facing approach it was

Ejaaz:
it was ill-mannered it was saying some pretty wild shit kind of the stuff that

Ejaaz:
you'd expect coming out of 4chan um and so sam altman decided to delay whilst

Ejaaz:
they kind of like figured out why um it was kind of acting out.

Josh:
Got it okay so we'll leave

Josh:
that speculation where it is there's a there's a funny post

Josh:
that i'll actually share with you if you want to throw it up which was actually from elon

Josh:
and we'll abbreviate but it was like elon was basically saying um

Josh:
it's hard to avoid the the libtard slash

Josh:
mecha hitler like approach both of them

Josh:
because they're on so polar opposite ends of the spectrum and he said he spent

Josh:
several hours trying to solve this problem with the system prompt but there's

Josh:
too much garbage coming in at the foundation model level so basically i mean

Josh:
what happens with these models is you train them based on all the human knowledge

Josh:
that exists right so everything that we've believed all the ideas that we've

Josh:
shared it's been fed into these models.

Josh:
And what happens is you can try to adjust how they interpret this data through

Josh:
the system prompt, which is basically an instruction that every single query

Josh:
gets passed through, but at some point is reliant on this swath of human data that is just

Josh:
It's too overbearing. And that's kind of what Elon shared.

Josh:
And the difference between OpenAI and Grok is that Grok will just ship the crazy

Josh:
update. And that's what they did. And they caught a lot of backlash from it.

Josh:
But what I find interesting and what I'm sure OpenAI will probably follow is

Josh:
this last paragraph where he says, our V7 foundation model should be much better.

Josh:
And we're being far more selective about training data rather than just training on the entire internet.

Josh:
So what they're planning to do to solve this problem, which is what I assume

Josh:
OpenAI probably ran into in the case that the AI training model kind of went

Josh:
off the rails and it started saying bad things about lots of people is that

Josh:
you kind of have to rebuild the foundation model with new sets of data.

Josh:
And in the case of Grok, I know one of the intentions for v7 is actually to

Josh:
generate its own database of data based on synthetic data from their models.

Josh:
And I'm assuming OpenAO will probably have to do this too if they want to calibrate.

Josh:
A lot of times people call that the temperature, which is the like variance

Josh:
of aggression in which a model uses.

Josh:
And I don't know, I think we're gonna start to see interesting approaches from

Josh:
that because as they get smarter, you really don't want them to necessarily

Josh:
have these evil traits as the default.

Josh:
And it's very hard to get around that when you train them on the data that they've been trained on so far.

Ejaaz:
It just goes to show how, I guess, cumbersome it is to train these models,

Ejaaz:
Josh. It's such a hard thing.

Josh:
Yeah. Yeah.

Ejaaz:
It's not something that you can just kind of like jump into the code and tweak a few things.

Ejaaz:
Most of the time you don't know what's wrong with the model or where it went

Ejaaz:
wrong. I mean, we've talked about this on a previous episode, but

Ejaaz:
So essentially, if you build out this model, right, you spend hundreds of millions

Ejaaz:
of dollars, and then you feed it a query.

Ejaaz:
So you put something in and then you wait to see what it spits out.

Ejaaz:
You don't really know what it's going to spit out. You can't predict it.

Ejaaz:
It's completely probabilistic. and so if you

Ejaaz:
release a model and it starts being a little racist or uh

Ejaaz:
you know um kind of crazy uh you

Ejaaz:
have to kind of like go back to the drawing board and you have

Ejaaz:
to analyze many different sectors of of this model

Ejaaz:
like was it the data that was poisoned or was it the way that we trained it

Ejaaz:
or maybe it was a particular model weight that we tweaked too much or whatever

Ejaaz:
that might be so i i think over time it's going to get a lot easier once we

Ejaaz:
understand how these models actually work but my god it must be so expensive

Ejaaz:
to just continually rerun and retrain these models.

Josh:
Yeah when you think about a coherent cluster of 200

Josh:
000 gpus the amount of energy the amount

Josh:
of resources just to to retrain a mistake is is huge so i think i mean the more

Josh:
we go into it the deeper we get the more it kind of makes sense paying so much

Josh:
money for talent to avoid these mistakes where if you pay a hundred million

Josh:
dollars for one employee who will give you a strategic advantage to avoid having

Josh:
to do another training run, that will cost you more than $100 million.

Josh:
You've already, you're already in the profit. So you kind of start to see the

Josh:
scale, the complexity, the difficulties.

Josh:
I do not envy the challenges that some of these engineers have to face.

Josh:
Although I do envy the- I envy the salary.

Ejaaz:
I envy the salary, Josh.

Josh:
I envy the salary and I envy the adventure. Like how cool must that be trying

Josh:
to build super intelligence for the world as a human for the first time in like

Josh:
the history of everything.

Josh:
So it's gotta be pretty fun. This is where we're at now with the open source

Josh:
models closed source models k2's pretty epic i think that's a home run i think

Josh:
we've crowned a new model today um do you have any closing thoughts anything

Josh:
you want to add before we wrap up here this is pretty amazing i

Ejaaz:
Think i'm most excited uh for the episode that we're probably going to release

Ejaaz:
a week from now josh when we've seen what people have built with this open source

Ejaaz:
model that's the best part about this by the way just to remind the listener that,

Ejaaz:
anyone can take this model right now you if you're listening to this can take

Ejaaz:
this model right now run it locally at home and tweak it to your preference

Ejaaz:
now yes it's going to be you know you kind of need to know how to tweak model

Ejaaz:
weights and stuff but i think we're going to see some really cool applications

Ejaaz:
get released over the next week and i'm excited to play around with them personally.

Josh:
Yeah if you're listening to this um and you can

Josh:
run this model let us know because that means you have quite a solid uh

Josh:
rig at your home yeah i'm not sure the average person is

Josh:
going to be able to run this but that is the beauty of the open weights is that anybody

Josh:
with the capability of running this can do so they

Josh:
could tweak it how they like and now they have access to the new

Josh:
best open source model in the world which i mean just a

Josh:
couple months ago from now would have been the best model in the

Josh:
world so it's moving really quickly it's really accessible and

Josh:
i'm sure as the weeks go by i mean hopefully we'll get open ai's model open

Josh:
source model soon in the next few weeks we'll be able to cover that but until

Josh:
then just lots of stuff going on this was uh another great episode so thank

Josh:
you everyone for tuning in again for rocking with us We actually plan on making this like 20 minutes,

Josh:
but we just kind of kept tailing off into more interesting things.

Josh:
There's a lot of interesting stuff to talk about. I mean, there's really,

Josh:
you could take this in a lot of places.

Josh:
So hopefully this was interesting.

Josh:
Go check out Kimmy K2. It's really, really impressive. It's really fast.

Josh:
It's really cheap. If you're a developer, give it a try.

Josh:
And yeah, that's been another episode. We'll be back again later this week with

Josh:
another topic. and just keep on chugging along as the frontier of AMI models continues to head west.

Ejaaz:
So also we'd love to hear from you guys. So if you have any suggestions on things

Ejaaz:
that you want us to talk more about, or maybe there's like some weird model

Ejaaz:
or feature that you just don't understand and maybe we can do a job at explaining it, just message us.

Ejaaz:
Our DMs are open or respond to any of our tweets and we'll be happy to oblige.

Josh:
Yeah, let us know. If there's anything cool that we're missing,

Josh:
send it our way and we'll cover it. That'd be great.

Josh:
But yeah, we're all going on the journeys together. We're learning this as we go.

Josh:
So hopefully today was interesting. And if you did enjoy it,

Josh:
please share with friends, likes, comment, subscribe, all the great things.

Josh:
And we will see you on the next episode.

Ejaaz:
Thanks for watching. See you guys. See you.