The Limitless Podcast

Josh:
The unthinkable has just happened open ai

Josh:
has released an open source model open ai

Josh:
has been closed ai since the time that i knew them

Josh:
they have been named themselves open ai they were not

Josh:
open source they have finally released an open source model and surprise surprise

Josh:
it's actually really great and i think the downstream implications of an open

Josh:
source model from a company like this that is this good are really it's a really

Josh:
big deal i think this really matters a lot just yesterday they announced the release of GPT-OSS.

Josh:
There are two models. There's a 120 billion parameter model and there's a 20

Josh:
billion parameter model. We're going to get into benchmarks.

Josh:
We're going to get into how good they are.

Josh:
But the idea is that OpenAI has actually released an open source model.

Josh:
And this can compare to the Chinese models because we recently had DeepSeek and we've had Kimi.

Josh:
And those would be very good. But this is the first really solid American-based open source model.

Josh:
So Ijaz, I know you've been kind of digging in the weeds about how this works.

Josh:
Can you explain us exactly why this is a big deal why this happened what's going on here

Ejaaz:
Yeah it's it's pretty huge so so here

Ejaaz:
are the hot highlights um as you mentioned there's two

Ejaaz:
models that came out the 20 billion parameter model which is actually small

Ejaaz:
enough to run on your mobile phone right now and they have a 120 billion parameter

Ejaaz:
model which is big but still small enough to run on a high performance laptop

Ejaaz:
so if you guys have a macbook out there jump in go for it um it's fully customizable.

Ejaaz:
So remember, open source means

Ejaaz:
that you can literally have access to the design of the entire model.

Ejaaz:
It's like OpenAI giving away their secret recipe to how their frontier models

Ejaaz:
work. And you can kind of like recreate it at home.

Ejaaz:
This means that you can customize it to any kind of use case that you want,

Ejaaz:
give it access to all your personal hard drives, tools, data,

Ejaaz:
and it can do wonderful stuff.

Ejaaz:
But Josh, here's the amazing part.

Ejaaz:
On paper, these models are as good as GPT-4 mini models, which is,

Ejaaz:
it's pretty impressive, right?

Ejaaz:
In practice and i've been playing around with it for the last few hours they're

Ejaaz:
as good in my opinion and actually quicker than

Ejaaz:
gpt-03 which is their frontier model and i

Ejaaz:
mean this across like everything so

Ejaaz:
reasoning um it spits out answers super quickly and i can see its reasoning

Ejaaz:
it happens in like a couple of seconds and i'm so used to waiting like 30 seconds

Ejaaz:
to a couple minutes on gpt-03 josh so it's pretty impressive and an insane unlock

Ejaaz:
on coding it's as good and on creativity as well.

Ejaaz:
So I'm my mind's pretty blown at all of this, right? Josh, what do you what do you think?

Josh:
Yeah, so here's why it's impressive to me is because a lot of the times I don't

Josh:
really care to use the outer bands of what a model is capable of.

Josh:
Like I am not doing deep PhD level research. I'm not solving these math Olympiad questions.

Josh:
I'm just trying to ask it a few normal questions and get some answers.

Josh:
And what these models do is an excellent job at serving that need.

Josh:
They're not going to go out and solve the world's hardest problems,

Josh:
but neither do I. I don't want to solve those problems.

Josh:
I just kind of want the information that I want, whether it be just a normal

Josh:
Google type search or whether it be asking it some miscellaneous question about

Josh:
some work that I'm doing.

Josh:
It's really good at answering that. So I think initial impressions,

Josh:
because they did allow you to test it publicly through their website,

Josh:
it's just really good at the things that I want.

Josh:
So the fact that I can run one of these models on a local device on my iPhone,

Josh:
well, it feels like we're reaching this place that AI is starting to become

Josh:
really interesting because for so long we've had compute handled fully on the

Josh:
cloud and now this is the first time where

Josh:
Compute can really happen on your computer. It could happen on your laptop.

Josh:
I could download the model and I could actually store the model,

Josh:
the 120 billion parameter model on a 56 gigabyte USB drive.

Josh:
So you can take the collective knowledge of the world and put it on a tiny little USB drive.

Josh:
And granted, it needs a bit of a bigger machine to actually run those parameters,

Josh:
but you can install all the weights. It's 56 gigabytes.

Josh:
It's this incredibly powerful package. And it probably, I don't know if this

Josh:
is true, but it's probably the most condensed knowledge base in the history of humanity.

Josh:
They've really managed to take a tremendous amount of tokens,

Josh:
smush them into this little parameter set, and then publish it for people to

Josh:
use. So for me, I'm really excited.

Josh:
I like having my own mini portable models. I am excited to download,

Josh:
try it out, run it on my MacBook.

Josh:
I'm not sure I could run the 120 billion parameter model, but at least the 20B

Josh:
and give it a shot and see how it works.

Ejaaz:
You need to get the latest MacBook, Josh. I know, I got to upgrade. We can test that out.

Ejaaz:
What I also love about it is it's fully private, right? So you can give it access

Ejaaz:
to your personal hard drive, your Apple Notes, whatever you store on your computer, basically.

Ejaaz:
And you can basically instruct the model to use those different tools.

Ejaaz:
So one review that I keep seeing from a number of people who have been testing

Ejaaz:
it so far is that it's incredibly great and intuitive at tool use.

Ejaaz:
And the reason why this is such a big deal is a lot of the Frontier models right

Ejaaz:
now, when they allow you to give access to different tools, they're kind of clunky.

Ejaaz:
The model doesn't actually know when to use a specific tool and when not to.

Ejaaz:
But these models are super intuitive, which is great. The privacy thing is also

Ejaaz:
a big thing because you kind of

Ejaaz:
don't want to be giving all your personal information away to Sam Altman.

Ejaaz:
But you want a highly personalized model.

Ejaaz:
And I think if I was to condense this entire model release in a single sentence,

Ejaaz:
Joss, I think I would say it is the epitome of privacy and personalization in an AI model so far.

Ejaaz:
It is that good. it is swift it is cheap and I'm going to replace it completely

Ejaaz:
with all my GPT-4.0 queries as you said earlier like,

Ejaaz:
Who needs to use the basic models anymore when you have access to this?

Josh:
Yeah. So it's funny you say that you're going to swap it because I don't think I'm going to swap it.

Josh:
I still am not sure I personally have a use case right now because I love the

Josh:
context. I want the memory.

Josh:
I like having it all server side where it kind of knows everything about me.

Josh:
I guess in the case that I wanted to really make it a more intimate model experience

Josh:
where you want to sync it up with like journal entries or your camera roll or

Josh:
whatever, whatever interesting like personal things, this would be a really cool use case.

Josh:
I think for the people who are curious why this matters to them,

Josh:
well, we could talk a little briefly about like the second order effects of

Josh:
having open source models as powerful, because what that allows you to do is

Josh:
to serve queries from a local machine.

Josh:
So if you are using an app or let's say you're an app developer and you're building

Josh:
an application and your app is serving millions of requests because it's a GPT wrapper.

Josh:
Well, what you could do now is instead of paying API calls to the OpenAI server,

Josh:
you can actually just run your own local server, use this model,

Josh:
and then serve all that data for the cost of the electricity.

Josh:
And that's a really big unlock for the amount of compute that's going to be

Josh:
available for not only developers, but for the cost of the users in a lot of these applications.

Josh:
So for the applications that aren't doing this crazy moon math and that are

Josh:
just kind of serving basic queries all day long, this like really significantly drops the cost.

Josh:
It increases the privacy, like you mentioned. And there's a ton of really important

Josh:
upsides to open source models that we just haven't seen up until now.

Josh:
And I'm very excited to see come forward.

Ejaaz:
Well, Josh, the thing with most of these open source models,

Ejaaz:
we spoke about actually two major Chinese open source models that were released last week.

Ejaaz:
It's not accessible to everyone. Like you and me aren't necessarily going to

Ejaaz:
go to Hugging Face, a completely separate website, download these models,

Ejaaz:
run the command line interface.

Ejaaz:
Most of the listeners on the show doesn't even know what that means.

Ejaaz:
I don't even know if I know what that means, right?

Ejaaz:
But here you have a lovely created website where you could just kind of log

Ejaaz:
on and play around with these open source models. And that's exactly what I've been doing.

Ejaaz:
I actually have a few kind of demo queries that I ran yesterday, Josh.

Josh:
Yeah, walk us through, let's see.

Ejaaz:
Okay, so there's an incredibly complex test, which a lot of these AI models,

Ejaaz:
which cost hundreds of billions of dollars to train, can't quite answer.

Ejaaz:
And that is how many R's, the letter R's are there in the word strawberry? Most say two.

Josh:
The bar's on the floor,

Ejaaz:
Huh? Yeah, if we were to go with most models, they say two. They're convinced that they are only two.

Ejaaz:
And I ran that test today, rather yesterday, with these open source models,

Ejaaz:
and it correctly guessed three, Josh. So we're one for one right now.

Josh:
We're on our way.

Ejaaz:
But then I was like, okay, we live in New York City. I love this place.

Ejaaz:
I'm feeling a little poetic today. Can you write me a sonnet?

Ejaaz:
And my goal with this wasn't to test whether it could just write a poem.

Ejaaz:
It was to test how quickly it could figure it out.

Ejaaz:
And as you see it thought for a couple of seconds on this so it literally spat

Ejaaz:
this out in two seconds um and it was structured really well you know it kind

Ejaaz:
of flowed would i be you know reciting this out loud to the public no but you

Ejaaz:
know i was pretty impressed.

Ejaaz:
And then, Josh, I was thinking, you know, what's so unique about open source models?

Ejaaz:
You just went through a really good list of why open source models work.

Ejaaz:
But I was curious as to why these specific open source models were better than

Ejaaz:
other open source models or maybe even other centralized models.

Ejaaz:
So I wrote a query. I decided to ask it. I was like, you know,

Ejaaz:
tell me some things that you could do that are the larger centralized models.

Ejaaz:
And I spat out a really good list. I'm not going to go through all of them,

Ejaaz:
but, you know, some of the things that we've highlighted so far, you can fine tune it.

Ejaaz:
It's privacy. See, I really like this point that it made, Josh,

Ejaaz:
that it just shows that AI is probably getting smarter than us,

Ejaaz:
which is you can custom inject your own data into these models.

Ejaaz:
Now, without kind of digging deeper into this, when you use a centralized model,

Ejaaz:
it's already pre-trained on a bunch of data that companies like Anthropic and

Ejaaz:
Google have already fed it.

Ejaaz:
And so it's kind of formed its own personality, right?

Ejaaz:
So you can't change the model's personality on a centralized model.

Ejaaz:
But with an open model you have full reign to do whatever you want and so if

Ejaaz:
you were feeling kind of uh adventurous you could use your own data and make

Ejaaz:
it super personal and customizable so i thought that was really cool and fun

Ejaaz:
demo josh have you been playing around with this.

Josh:
Yeah it's um it's it's smart it's fun it's smart i wouldn't say it's anything

Josh:
novel the like query results that i get are you know on par with everything

Josh:
else i don't notice the difference which is good because it means they're performing

Josh:
very well it's not like i feel like i'm getting degraded performance because

Josh:
I'm using a smaller model.

Josh:
But it's just like it's nothing too different, I would say.

Josh:
The differences, I mean, again, all this boils down to the differences of it

Josh:
being open source versus being

Ejaaz:
Run on the server. Well, let me challenge you that, right? OK,

Ejaaz:
so you're saying it's good but nothing novel.

Ejaaz:
Would you say it's as good as GPT-4.0,

Ejaaz:
minus the memory let's just put memory aside for a second would you use it if

Ejaaz:
it had memory capability.

Josh:
Actually no probably not um i still wouldn't

Josh:
because i love my desktop application too much i

Josh:
love my mobile app too much and i like that the conversations are

Josh:
shared in the cloud um so i can use them on my phone i could

Josh:
start on my laptop and go back and forth so even in

Josh:
that case i'm probably still not a user um because

Josh:
the convenience factor but there are there are a

Josh:
lot of people and a lot of industries that would be and this is actually something probably

Josh:
worth surfacing is the new industries that are now able to

Josh:
benefit from this because a lot of industries have

Josh:
a tough time using these AI models because

Josh:
of the data privacy concerns particularly I mean if you think about a

Josh:
healthcare industry people who are dealing with patients data it's

Josh:
very challenging for them to fork it over to open AI and just trust that they're

Josh:
going to keep it safe so what this does is it actually allows companies that

Josh:
are in like the healthcare industry the finance industry who's dealing with

Josh:
very like high touch personal finance the legal industry who's dealing with

Josh:
a lot of legality government and defense a lot of these industries that were

Josh:
not previously able to use these popular AI models,

Josh:
well, now they have a pretty good model that they could run locally on their machines.

Josh:
And that doesn't have any possibility of actually leaking out their customer

Josh:
data, leaking out financials or healthcare data or, or like any sort of legal documents.

Josh:
And, and that feels like a super powerful unlock. So for them,

Josh:
it feels like a no brainer, obviously get the 120 B model running on a local

Josh:
machine inside of your office, and you can load it up with all this context.

Josh:
And that seems to be who this would be most impacting, right?

Ejaaz:
But still to that point, I wonder how many of these companies can be bothered

Ejaaz:
to do that themselves and run their own internal kind of like infrastructure.

Ejaaz:
I'm thinking about OpenAI, who cracked, I think, $10 billion in annual recurring

Ejaaz:
revenue this week, which is like a major milestone.

Ejaaz:
And a good chunk of that, I think 33% of that is for enterprise customers.

Ejaaz:
And to your point, like these enterprise customers don't wanna be giving open

Ejaaz:
AI their entire data. You know, they can be used to train other AI models.

Ejaaz:
So their fix or solution right now is they use kind of like private cloud instances,

Ejaaz:
that I think are supplied by Microsoft by their Azure cloud service or something like that.

Ejaaz:
And I wonder if they chose that,

Ejaaz:
One, because there wasn't any open source models available or because they kind

Ejaaz:
of just want to offload that to Microsoft to deal with.

Ejaaz:
My gut tells me they're going to want to go with the latter,

Ejaaz:
which is like, you know, just give it to some kind of cloud provider to deal with themselves.

Ejaaz:
And they just trust Microsoft because it's a big brand name.

Ejaaz:
But yeah, I don't really know how they'll materialize. I still think,

Ejaaz:
and maybe this is because of my experience in crypto, Josh, that the open source

Ejaaz:
models are still for like people that are at the fringe that are really experimenting

Ejaaz:
with these things. but maybe don't have billions of dollars.

Josh:
Yeah, that could be right. It'll be interesting to see how it plays out on all

Josh:
scale of businesses because I mean, as a, like I think of a lot of indie devs

Josh:
that I follow on Twitter and I see them all the time

Josh:
just running local servers and they just, if they had this local model that

Josh:
they could run on their machine and it takes the cost per query down from like

Josh:
a penny to zero, that's like a big zero to one change.

Josh:
So he does this model special because there are also a number of breakthroughs

Josh:
that occurred in order to make this possible,

Josh:
in order to condense this knowledge to be so tight so here's this

Josh:
tweet from the professor talking about the cool tech tweaks in

Josh:
this new model and what open ai was able to achieve some of

Josh:
these i believe are novel some of these are seen before um if

Josh:
you look at point two mixture of experts we're familiar with mixture of experts

Josh:
we've seen other companies use that like kimmy and deep

Josh:
seek basically instead of one brain doing everything the ai

Josh:
has this team of experts that are kind of like mini brains

Josh:
and specialize in different tasks it picks the right expert for

Josh:
the job and it makes it faster so like instead of

Josh:
having the entire 120 million parameter model search for one question maybe

Josh:
you just take a couple million of those parameters that are really good at solving

Josh:
math problems and they use it and that that's what brings compute down the first

Josh:
point is this thing called the sliding window attention so if you imagine an

Josh:
ai is like reading a really long book

Josh:
It can only focus on a few pages at a time this trick

Josh:
kind of lets it slide its focus window along the text so

Josh:
when you think of a context window generally it's fixed right where you can see

Josh:
a fixed set of data this sliding window

Josh:
attention allows you to kind of move that context back and forth a

Josh:
little bit so it takes what would have normally been

Josh:
a narrow context window and extends it out a little bit to

Josh:
the side so you get a little bit more context which is great for a

Josh:
smaller model again you really want to consider that all of these are

Josh:
are optimized for this microscopic scale that

Josh:
can literally run on your phone and then the third point is this

Josh:
thing called rope with yarn which sounds like a cat toy but this

Josh:
is how the ai keeps track of the order of words so like the position

Josh:
of the words in a sentence um so rope

Josh:
you could imagine it like like the twisty math way to do

Josh:
it and yarn makes it stretch further for really long stuff

Josh:
so we have the context window that is

Josh:
sliding we have this rope with yarn that allows you

Josh:
to just kind of like stretch the words a little bit further and

Josh:
then we have attention sinks which is the last one which is

Josh:
there's a problem when ai is dealing with these endless chats that

Josh:
lets it it kind of sinks in or ignores the boring old

Josh:
info so it can pay attention to the new stuff so basically what it

Josh:
is is if you're having a long chat with it and it determines hey this stuff

Josh:
is kind of boring i don't need to remember it it'll actually just throw it away

Josh:
and it'll increase that context window a little bit so again hyper optimizing

Josh:
for for the small context window that it has and those are kind of the key four

Josh:
breakthroughs that made this special again i'm not sure any of them are particularly novel,

Josh:
But when combined together, that's what allows you to get these 04 mini results

Josh:
or even 03 results on the larger model on something that can run locally on your laptop.

Josh:
So it's a pretty interesting set of breakthroughs. I think a lot of times OpenAI,

Josh:
we talk about them because of their feature breakthroughs, not really their

Josh:
technical breakthroughs.

Josh:
I think a lot of times the technical breakthroughs are reserved for like the

Josh:
Kimi models or the DeepSeq models

Josh:
where they really kind of break open the barrier of what's possible.

Josh:
But I don't want to discredit OpenAI because these are pretty interesting things

Josh:
that they've managed to combine together into this like one cohesive,

Josh:
tiny little model, and then just gave it away.

Ejaaz:
Yeah. I mean, they actually have a history of front-running open source frontier breakthroughs.

Ejaaz:
If you remember when DeepSeek got deployed, Josh, one of their primary training

Ejaaz:
methods was reinforcement learning, which was pioneered by an open AI researcher,

Ejaaz:
which who probably like now works at Meta.

Ejaaz:
Yeah, and I was I was I was looking at the feature that you mentioned just not

Ejaaz:
the feature, but the breakthrough sliding window attention, and you mentioned

Ejaaz:
that it can basically toggle reasoning.

Ejaaz:
And I was pleasantly surprised to just notice that on the actual interface of

Ejaaz:
the models here, Josh, can you see over here?

Ejaaz:
You can toggle between reasoning levels of high, medium and low.

Ejaaz:
So depending on what your prompt or query is, if it is kind of like a low level

Ejaaz:
query where you're like hey just record this shopping or grocery list you know

Ejaaz:
that's probably like a medium or a low query so oh it's pretty cool to to see

Ejaaz:
that surface to the user like see it actively being used.

Josh:
Yeah, no, super cool. I think I like the fine tuning of it.

Josh:
And again, allowing you to kind of choose your intelligence levels,

Josh:
because I imagine a lot of average people just don't, a lot of average queries

Josh:
just don't need that much compute.

Josh:
So if you can toggle it for the low reasoning level and get your answers,

Josh:
that that's amazing. Super fast, super cheap.

Ejaaz:
Did you see that trending tweet earlier this week, Josh, which basically said

Ejaaz:
that the majority of ChatGPT users have never used a different model than ChatGPT 4.0?

Josh:
I haven't seen it, but that makes sense.

Ejaaz:
Yeah i i feel like the bulk of people i was chatting to

Ejaaz:
my sister yesterday and she was kind of

Ejaaz:
like using it for some research project at work and the

Ejaaz:
screenshot she sent me over was foro and i was like hey you know like

Ejaaz:
you could just run this on like a model that's like

Ejaaz:
five times better than this right uh we'll come

Ejaaz:
up with a much more creative set of ideas so just made me think that

Ejaaz:
like i don't know how many people like care that they are like

Ejaaz:
these brand new novel models and maybe um you know

Ejaaz:
this kind of like basic model is good enough for everyone i don't know

Ejaaz:
but um but moving on josh um there

Ejaaz:
was a big question that popped into my head as

Ejaaz:
soon as these models released which was are they as good

Ejaaz:
as the chinese open source models right i wanted

Ejaaz:
to get some opinions from people and and the reason

Ejaaz:
why this matters i'm just give the listeners some context

Ejaaz:
is china has been the number one

Ejaaz:
nation to put out the best open source

Ejaaz:
models over the last 12 months it started with deep seek

Ejaaz:
and then alibaba's quen models got involved

Ejaaz:
and then recently we had kimmy k2 and i think

Ejaaz:
there was another ai lab out of china which came out so they

Ejaaz:
have outside of america the highest density.

Ejaaz:
Of the top ai researchers they all come out of this one university

Ejaaz:
zinghua i believe they kind of like partially work

Ejaaz:
or train in the u.s as well so they've got this like kind of hybrid ai

Ejaaz:
mentality of how to build these models and they come up with a lot of these

Ejaaz:
frontier breakthroughs um kimmy k2 for context had uh one trillion parameters

Ejaaz:
in their model right comparing this to like 120 billion and 20 billion parameters

Ejaaz:
models from open air i was curious like does this beat them to the punch some people josh.

Ejaaz:
Don't think so okay this guy jason lee

Ejaaz:
he asks uh is the gpt oss stronger

Ejaaz:
than quen or kimmy or chinese open models and then

Ejaaz:
he later kind of quote tweets that tweet and says answer the model is complete

Ejaaz:
junk it's a hallucination machine overfit to reasoning benchmarks and has absolutely

Ejaaz:
zero recall ability so a few things he's mentioning here is one it hallucinates

Ejaaz:
a lot so it kind of makes up jargon terms,

Ejaaz:
ideas, or parameters that didn't really exist before.

Ejaaz:
Number two, he's saying that OpenAI designed this model purely so that it will

Ejaaz:
do well on the exams, which are the benchmarks that rate how these models compare to each other.

Ejaaz:
So they're saying that OpenAI optimized the model to kind of like do really

Ejaaz:
well at those tests, but actually fail at everything else, which is what people want to use it for.

Ejaaz:
And the final point that he makes is that it has zero recall ability,

Ejaaz:
which is something you mentioned earlier, Josh, which says it doesn't have memory

Ejaaz:
or context so you can have a conversation and then open up another conversation

Ejaaz:
and it's completely forgotten about the context that it has for you from that

Ejaaz:
initial conversation okay.

Josh:
So not not the best not to be unfair to open ai but it feels like they delayed

Josh:
this model a good bit of times oh yeah and they wanted it to look good and it

Josh:
intuitively makes sense to me that they would be kind of optimizing for benchmarks

Josh:
with this one um but nonetheless it's still impressive i'm seeing this big wall

Josh:
of text now what is what is this what is this post here

Ejaaz:
Well it's this post from uh one of these accounts i follow and they have an

Ejaaz:
interesting section here which says comparison to other open weights oh sick.

Josh:
Yeah what is this

Ejaaz:
So he goes while the larger gpt oss

Ejaaz:
120 billion parameter model does not come

Ejaaz:
in above deep seek r1 so he's saying that deep seek r1

Ejaaz:
just beats it out the park it is notable that

Ejaaz:
it is significantly smaller in both total and active

Ejaaz:
parameters than both of those models deep seek

Ejaaz:
r1 has 671 billion total parameters and

Ejaaz:
37 billion active parameters and is released natively right which makes it 10x

Ejaaz:
larger than gpt's 120 billion parameter models but what he's saying is even

Ejaaz:
though gpt's model is smaller and doesn't perform as well as deep seek it's

Ejaaz:
still mightily impressive for its size.

Josh:
Okay that's cool because that gets back to the point we made earlier in the

Josh:
show that this is probably the most densely condensed

Josh:
however you want to say it like base of

Josh:
knowledge in the world they've used a lot of efficiency gains

Josh:
to squeeze the most out of it so in this small model

Josh:
it is i guess if we're optimizing maybe we

Josh:
can make up a metric here on the show which is like um output per

Josh:
per parameter or something like that like based on the total parameter

Josh:
count of this model it gives you the best value per

Josh:
token and that seems to be where this falls

Josh:
in line where it's not going to blow any other open source model out of the

Josh:
water but in terms of its size the fact that we can

Josh:
take a phone and literally run one of these models on a phone and

Josh:
you could go anywhere in the world with no service and have access to these models running

Josh:
on a laptop or whatever mobile device that that's super

Josh:
powerful and that's not something that is easy to do with the other open source

Josh:
models so perhaps that's the advantage that open ai has it's just the density

Josh:
of intelligence and the efficiency of these parameters that they've given to

Josh:
us versus just being this like home run open source model that is going for the frontier,

Josh:
it's just a little bit of a different approach.

Ejaaz:
Yeah, we need like a small but mighty ranking on this show, Josh,

Ejaaz:
that we can kind of like run every week when these companies release a new model.

Ejaaz:
No, but it got me thinking, if we zoomed out of that question,

Ejaaz:
right, because we're talking about small models versus large models,

Ejaaz:
parameters and how effectively they use versus other models that are bigger.

Ejaaz:
What really matters in this, Josh? In my opinion, it's user experience and how

Ejaaz:
useful these models are to my daily life, right?

Ejaaz:
At the end of the day, I kind of don't really care what size that model is unless

Ejaaz:
it's useful for me, right? It could be small, it could be personal, it could be private.

Ejaaz:
It depends on, I guess, the use case at the time. And I have a feeling that

Ejaaz:
the trend of how technology typically goes, you kind of want a really high-performant

Ejaaz:
small model, eventually.

Ejaaz:
Right? I try and think about like us using computers for the first time,

Ejaaz:
you know, back in our dinosaur age.

Ejaaz:
And then, you know, it all being condensed on a tiny metal slab that we now

Ejaaz:
use every day. And we can pretty much work from remotely from wherever.

Ejaaz:
And I feel like this is where models are going to go. They're going to become

Ejaaz:
more private. They're going to become more personal.

Ejaaz:
Maybe it'll be a combination of, you know, it running locally on your device

Ejaaz:
versus cloud inference and trusting certain providers.

Ejaaz:
I don't know how it's going to fall out, but I think Like it's not a zero to

Ejaaz:
one. It's not a black or white situation.

Ejaaz:
I don't think everyone's just going to go with large centralized models that

Ejaaz:
they can inference from the cloud.

Ejaaz:
I think it'll be a mixture of both. And how that materializes,

Ejaaz:
I don't know, but it's an interesting one to ponder.

Josh:
Yeah, I think this is funny. This is going to sound very ironic,

Josh:
but Apple was the person that got this most right.

Ejaaz:
Sorry, who's Apple again?

Josh:
Yeah, right. I mean, it sounds ridiculous to say this. And granted,

Josh:
they did not execute on this at all.

Josh:
But in theory, I think they nailed the approach initially,

Josh:
which was you run local compute where all of

Josh:
your stuff is so my iphone is the device i never

Josh:
leave without it is everything about me it is all of my messages my

Josh:
contacts all the contacts you could ever want from me and then the idea was

Josh:
they would give you a local model that is integrated and embedded into that

Josh:
operating system and then if there's anything that requires more compute well

Josh:
then they'll send the query off into the cloud but most of it will get done

Josh:
on your local device because most of it isn't that complicated and i think as

Josh:
a user when i ask myself what i want from AI.

Josh:
Well, I just want it to be my ultimate assistant. I just want it to be there

Josh:
to make my life better. And so much of that is the context.

Josh:
And Apple going with that model would have been incredible.

Josh:
It would have been so great. It would have had the lightweight model that runs

Josh:
locally, it has all the context of your life, and then it offloads to the cloud.

Josh:
I still think this model is probably the correct one for optimizing the user

Josh:
experience. But unfortunately, Apple just has not done that.

Josh:
So it's up for grabs. I mean, again, Sam Altman's been posting a lot this week,

Josh:
we do have to tease what's coming because this is probably going to be a huge

Josh:
week. There's a high probability we get GPT-5.

Josh:
And then they've also been talking about their hardware device a little bit. And they're saying how

Josh:
It's like it's genuinely going to change the world. And I believe the reason

Josh:
why is because they're taking this Apple approach where they're building the

Josh:
operating system, they're gathering the context, and then they're just they're

Josh:
able to serve it now locally on device.

Josh:
They're able to go to the cloud when they need more compute.

Josh:
And it's going to create this really cool, I think, duality of AI where you

Josh:
have your your super private local one, and then you have the big brain one,

Josh:
the big brother that's off in the cloud that does all the hard computing for you.

Ejaaz:
Well, one thing is clear. There are going to be hundreds of models and it's

Ejaaz:
going to benefit the user, you and I, for so many multiple...

Ejaaz:
It's the big company's problems to figure out how these models work together

Ejaaz:
and which ones get queried. I don't care.

Ejaaz:
Just give me the good stuff and I'm going to be happy.

Ejaaz:
Folks, OpenAI has been cooking. This was the first open source models they've

Ejaaz:
released in six years, Josh.

Ejaaz:
The last one was 2019 GPT-2, which seems like the stone age and it was only like four years ago.

Ejaaz:
Thank you so much for listening. We are pumped to be talking about GPT-5,

Ejaaz:
which we hope to be released in maybe 24 hours.

Ejaaz:
Hopefully this week, fingers crossed. I don't know, we might be back on this

Ejaaz:
camera pretty soon. Stay tuned.

Ejaaz:
Please like, subscribe, and watch out for all the updates. We're going to release

Ejaaz:
a bunch of clips as well if you want to kind of like get to the juicy bits as well.

Ejaaz:
Share this with your friends and give us feedback. If you want to hear about

Ejaaz:
different things, things that we haven't covered yet or things that we've spoken

Ejaaz:
about, but you want to get more clarity on or guests that you want to join the show, let us know.

Ejaaz:
We're going full force on this and we'll see you on the next one.

Josh:
Sounds good. See you guys soon. Peace.

Music:
Music