The Limitless Podcast

Ejaaz:
What if I told you there was a single website you could go to where you can

Ejaaz:
chat to any major AI model from one single interface?

Ejaaz:
It's kind of like chat GPT, but instead every prompt gets routed to the exact

Ejaaz:
AI model that will do the best job for whatever your prompt might be.

Ejaaz:
Well, on today's episode, we're joined by Alex Atala, the founder and CEO of Open Router AI.

Ejaaz:
It's the fastest growing AI model marketplace with access to over 400 LLMs,

Ejaaz:
making it the only place that really knows how people use AI models,

Ejaaz:
and more importantly, how they might use them in the future.

Ejaaz:
It's at the intersection of every single prompt that anyone writes and every

Ejaaz:
model that they might ever be.

Ejaaz:
Alex Atala, welcome to the show. How are you, man?

Alex:
Thanks, guys. Great. Thanks so much for having me on.

Ejaaz:
So it is a Monday. How does the founder of OpenRouter spend his weekend?

Ejaaz:
Presumably you know out and about chilling relaxing not at all focused on the company oh

Alex:
I usually i love weekends with no

Alex:
meetings planned and i just go to a coffee shop and just have tons of hours

Alex:
stacked in a row uh to do things that require a lot of momentum build up so

Alex:
i did that at coffee shops on saturday and sunday and then i watched blade runner again.

Ejaaz:
Again okay um well so

Ejaaz:
when we were preparing for this episode alex

Ejaaz:
um i couldn't help but think that you've had a pretty insane decade of startup

Ejaaz:
foundership right um so open router is kind of like your second major thing

Ejaaz:
that you've done but prior to doing that you were the founder and cto of OpenSea,

Ejaaz:
the biggest NFT marketplace out there.

Ejaaz:
And now you're focused on one of the biggest AI companies out there.

Ejaaz:
So it sounds like you're at kind of like the pivot point of two of the most

Ejaaz:
important technology sectors over the last decade.

Ejaaz:
Can you just give us a bit of background as to how you ended up here?

Ejaaz:
And more importantly, where you started.

Ejaaz:
Walk us through the journey of OpenSea and how you ended up at OpenRouter AI.

Alex:
Yeah, so I co-founded OpenSea with Devin Finzer the very beginning of 2018, very end of 2017.

Alex:
It was the first NFT marketplace. And...

Alex:
It was not dissimilar to OpenRouter in that there was a really fragmented ecosystem

Alex:
of NFT metadata and media that gets attached to these tokens.

Alex:
And it was the first example of something in crypto that could be non-fungible,

Alex:
meaning it's a single thing that can be traded from person to person.

Alex:
Most things in the world are non-fungible. A chair is non-fungible. a

Alex:
currency is fungible so it was

Alex:
back back in 2018 no

Alex:
one was really thinking about crypto in terms of non-fungible goods

Alex:
and uh and the problem with the non with

Alex:
non-fungible goods is that there weren't any real standards set up

Alex:
um there was a lot of heterogeneous like

Alex:
implementations for how to get uh like

Alex:
a non-fungible item represented and tradable in a decentralized way So OpenSea

Alex:
organized this like very heterogeneous inventory and put it together in one

Alex:
place. We came up with like a metadata standard.

Alex:
We did a lot of like a lot of work to really make the experience super good for each collection.

Alex:
And you see a lot of those a lot of similarities with how AI works today,

Alex:
too, where there's also just a very heterogeneous ecosystem.

Alex:
On a lot of different APIs and different features supported by language model providers.

Alex:
And Open Router similarly does a lot of work to organize it all.

Alex:
Um i was at open sea uh until 2022 um when i was kind of feeling the itch to do something new.

Alex:
And um i'm at the very end of i left in august and then chat gpt came out a few months later.

Alex:
And uh and my biggest question around that

Alex:
time was whether it was going to be a winner take all market

Alex:
because opening i was very far ahead of

Alex:
everybody else and um you know

Alex:
we had cohere command we had a couple open source models um

Alex:
but opening i was the only really usable one i

Alex:
was doing little projects to experiment

Alex:
with the gpt3 api and uh

Alex:
and then llama came out in january um really

Alex:
exciting about a tenth the size one on a

Alex:
couple benchmarks but it wasn't really chattable yet and

Alex:
uh and it wasn't until uh a few

Alex:
months later that somebody a team at

Alex:
stanford distilled it into a new

Alex:
model called alpaca um distillation means

Alex:
you you take the model and you customize it or fine tune it

Alex:
on a set of synthetic data that they

Alex:
made using chat gpt as a research project

Alex:
and uh and that was it was

Alex:
the first successful major distillation that i'm aware

Alex:
of um and it was an actually usable model i

Alex:
was like on the airplane talking to him i was like wow this

Alex:
is if it only took six hundred dollars to make something like this then you

Alex:
don't need ten million dollars to make a model there might be like tens of thousands

Alex:
hundreds of thousands of models in the future and suddenly this started to look

Alex:
like a new like economic primitive a new building block that people that kind

Alex:
of deserve their own place on the internet.

Alex:
And there wasn't one. There wasn't a place where you could discover new language

Alex:
models and see who uses them and why.

Alex:
And that's how OpenRouter got started.

Josh:
That's amazing. So one of the things that we're obsessed with on this channel

Josh:
in particular is exploring frontiers and how to properly see these frontiers

Josh:
and analyze them and understand when they're going to happen.

Josh:
And when I was going through your history, you have this talent consistently over time.

Josh:
And even as far back as early on, I read you were hacking Wi-Fi routers in a hackathon.

Josh:
You're very early to that. You were early to the NFTs. You were early to understanding

Josh:
AI and the impact that it would have. And what I'd love for you to explain is

Josh:
the thought process and the indicators you look for when exploring these new

Josh:
frontiers, because clearly there's some sort of pattern matching going on.

Josh:
Clearly you have some sort of awareness of what will be important and why it

Josh:
will be important, and then inserting yourself into that narrative.

Josh:
So are there patterns? Are there certain things that you look for when searching

Josh:
for these new opportunities and that led you to make these decisions that you have?

Alex:
I think there's there's a lot to be said for finding enthusiast communities

Alex:
and and seeing if you're going to join it.

Alex:
Like, can you be an enthusiast with them?

Alex:
Like whenever something new comes out that has like some kind of ecosystem potential,

Alex:
there's there are going to be enthusiast communities that pop up.

Alex:
And the Internet has made it self-certain. You could just join the communities.

Alex:
Um discord i think is a incredible

Alex:
and super underrated platform because

Alex:
the communities feel kind of private you're

Alex:
like getting you don't feel like you're you know

Alex:
seeing somebody trying to get s you

Alex:
know like advertise something for seo juice there's

Alex:
no seo juice in discord um it's it's

Alex:
just people talking about what they're passionate about and and it

Alex:
goes it gets really niche um and when

Alex:
you find a like an interest group in discord that

Alex:
like has to do with some some new

Alex:
piece of technology that's just being developed right now and doesn't really

Alex:
work very well at all um you get people who are just trying to figure out what

Alex:
to do with it and how to make it better and i think that's like that's the first

Alex:
core piece of magic that jumps to mind,

Alex:
there's got to be like a willingness to be weird because if you jump into any

Alex:
of these communities at face value it's stupid.

Alex:
Like oh this is like just a game or it's like a really weird game I mean I'm

Alex:
not really a collectible game so I'm going to leave right now and yeah.

Alex:
Not only do you have to be aware, but you have to be creative.

Alex:
Like, okay, these are just cats on the blockchain, and people are just trading cats back and forth.

Alex:
You can't look at the community as simply that.

Alex:
Think about what you could do with it.

Alex:
Like, what is this unlock that wasn't achievable before?

Alex:
Um and uh

Alex:
and and i think there are there are people who

Alex:
just are good who will do this and they'll join the communities

Alex:
and and brainstorm live and you can see everybody

Alex:
brainstorming uh in real time but like

Alex:
another incredible example of this was the mid-journey discord

Alex:
you know it became the

Alex:
biggest biggest server in discord by

Alex:
far uh and you know

Alex:
why did that happened well you could it started with

Alex:
something weird silly maybe

Alex:
not super useful but you could see all the

Alex:
enthusiasts like remixing and

Alex:
brainstorming live how to turn it into something beautiful

Alex:
and how to how to make it useful and um

Alex:
and then you know just explode it like i it's the most it's the it's the most

Alex:
incredible like niche community uh i think that discord has ever seen because

Alex:
of like how useless it started and how insanely exciting it became.

Alex:
So um like i mean i i

Alex:
think i saw big sleep i was like playing around with this model

Alex:
called big sleep in 2021 that uh

Alex:
let you generate images that

Alex:
look kind of like deviant art okay and

Alex:
uh you could see you could like they're all

Alex:
animated images and they none of them really made sense but you could get some

Alex:
really cool stuff not like potentially something you'd want to make your desktop

Alex:
wallpaper and if you're really like deep in some deviant art communities you

Alex:
know you kind appreciate it and so and that that that was like oh there's like

Alex:
a kernel of something here,

Alex:
and uh it took like a like another year or two before mid-journey started to

Alex:
like pick up but that was like.

Ejaaz:
Where were you seeing all of this alex like where were you scouring just random

Ejaaz:
forums or just wherever your nose told you to go

Alex:
But basically there's this twitter account I'm trying to remember what it's

Alex:
called that posts AI research papers and and like kind of tries to show what you can do with them.

Alex:
And I discovered this Twitter account in like 2021.

Alex:
And I.

Alex:
I think it was not it was it wasn't at all like related to crypto but it was

Alex:
a way you know big sleep was like the first thing i saw that used ai to generate

Alex:
things that could potentially be nfts,

Alex:
so i started experimenting around like how how much you could direct it to make

Alex:
an nft collection that would make any sense it was very very difficult um but

Alex:
that was how uh that was like the first generative and.

Ejaaz:
This was before you were even thinking about starting open router right

Alex:
Um yeah yeah this was back this was when i was

Alex:
full-time at openc um oh is

Alex:
yeah i got the it's a colic

Alex:
this twitter account all right

Alex:
i really recommend it they basically post papers and like explainate and explore

Alex:
how this paper gets useful um they post animations uh like they make they make

Alex:
ai research like kind of fun to engage with and that was that was my first experience.

Ejaaz:
Okay, so I mean that's a massive win for X or formerly as it was known back

Ejaaz:
then, Twitter as a platform, right?

Ejaaz:
It gave birth to kind of like two of the biggest technologies crypto,

Ejaaz:
also known as crypto Twitter, and now apparently all the AI research stuff which

Ejaaz:
kind of put you on to the path that led you to OpenRatter.

Ejaaz:
So if I've got this right, Alex, you were full-time at OpenSea with a

Ejaaz:
multi-billion dollar company loads of important stuff to do there,

Ejaaz:
but you still found the time to kind of scour this fringe technology because

Ejaaz:
that's what AI was at the time.

Ejaaz:
Prior to kind of GPT-2 or GPT-3, no one really knew about this.

Ejaaz:
And you were playing around with these gen AI models, these generative AI models

Ejaaz:
that would create this magical little substance and maybe it came in the form

Ejaaz:
of a pitcher or a weird little cat.

Ejaaz:
And you kind of like jumped into these niche forums of enthusiasts,

Ejaaz:
as you say, and kind of explored that further.

Ejaaz:
And it sounds like you kind of like honed that even beyond your journey from OpenSea when you left.

Ejaaz:
I remember actually meeting you in this kind of like this abbess between you

Ejaaz:
leaving OpenSea and starting OpenRouter where you were kind of brainstorming

Ejaaz:
a bunch of these ideas. And I remember a snippet from our conversation

Ejaaz:
In like one of the WeWorks here, where you just kind of like had whiteboarded a bunch of AI stuff.

Ejaaz:
And one of those things was kind of like the whole topic of inference.

Ejaaz:
And if I'm being honest with you, I had no idea what that word even meant back then.

Ejaaz:
I was extremely focused on all the NFT stuff and all the crypto stuff,

Ejaaz:
my background's in all of that.

Ejaaz:
But I just found that fascinating that you always had your nose in some of the

Ejaaz:
early communities. And I think that's a really important lesson there.

Ejaaz:
I want to pick up on something that you actually brought up when you said you

Ejaaz:
discovered kind of like your path to open router, Alex.

Ejaaz:
And that is, you said you were playing around with these early AI models.

Ejaaz:
So not the GPTs before Claude was even created.

Ejaaz:
You're playing around with these random models that you would find either on

Ejaaz:
forums, on Twitter, or on Reddit, right? and you would experiment with them.

Ejaaz:
And I find it fascinating that back then, even when GPT became a thing,

Ejaaz:
you were convinced that there would be hundreds of thousands,

Ejaaz:
or did you say hundreds of thousands of AI models?

Ejaaz:
Back then, that wasn't a normal view.

Ejaaz:
Back then, everyone was like, you need hundreds of millions of dollars.

Ejaaz:
Maybe it was tens of millions of dollars back then. And it was going to be a rich man's game.

Alex:
Yeah, it was basically the Alpaca Project that kind of put me over the sack.

Alex:
On there being many, many, many models instead of just a very small number.

Ejaaz:
And can you explain what the Alpaca project is for the audience? Yeah.

Alex:
So the Alpaca project, after Lama came out, you really could not chat with it

Alex:
very well. It was a text completion model.

Alex:
There were a couple benchmarks where it beat GPT-3.

Alex:
And... It was about a tenth the size of what most people thought GPT-3 was sized at.

Alex:
So it was a pretty incredible achievement.

Alex:
But it wasn't really like, the user experience wasn't there.

Alex:
And the Alpaca project took ChatGPT and generated a bunch of synthetic outputs.

Alex:
And then they fine-tuned Llama on those synthetic outputs.

Alex:
And this did two things to Llama. It taught it style, and it taught it knowledge.

Alex:
It taught it, like, the style is like how to chat, which was the big user experience gap.

Alex:
And it made it smarter.

Alex:
Like, you can, fine-tuning transfers both style and knowledge.

Alex:
And the model would, like, respond to things that it had, you know,

Alex:
like, the content of the synthetic data, like, was reflected in the model's

Alex:
performance on benchmarks after that point.

Alex:
So um so if you can do

Alex:
that without revealing all

Alex:
the data that goes in um now now

Alex:
there's like a way you could sell data via api without

Alex:
like like just dumping all the data out to the world and then never being able

Alex:
to to like monetize it again so there's like a brand new business model around

Alex:
data that emerges um yet like the ability to create just like work towards open intelligence,

Alex:
and uh and build like new

Alex:
architectures test them more quickly and and and

Alex:
uh uh fine-tune them quickly basically you

Alex:
can build on top of the work of giants i mean

Alex:
you don't have to start from zero every time a lot

Alex:
of like the biggest developer experience innovations just involve like giving

Alex:
developers a higher stair to start walking up so they don't have to start at

Alex:
the bottom of the staircase every single time um and you know that was like the the the big.

Alex:
Like generous give that llama had for the community um and it wasn't you know

Alex:
that wasn't the only company doing open source models, Mastral,

Alex:
came out with 7B Instruct a few months later. It was an incredible model.

Alex:
Then they came out with the first open-weight mixture of experts a few months later.

Alex:
It felt like actual intelligence, but completely open.

Alex:
And all of these provide higher and higher stairs for other developers to kind

Alex:
of like, basically to crowdsource new ideas from the whole planet.

Alex:
Uh and and let these new ideas build on

Alex:
top of really good foundations so and

Alex:
you know when that when that like whole picture started

Alex:
to form into place um it felt like okay this is going to be like a huge inventory

Alex:
situation you kind of like nft collections were a huge inventory situation obviously

Alex:
completely different really different market dynamics really different type

Alex:
of of goal that buyers have.

Alex:
And so a lot of like my early experimentation, like I made like a Chrome extension called Window AI.

Alex:
I did like a few other things were just about learning how the ecosystem works

Alex:
and like what makes it different and how the like, like what people really want,

Alex:
what developers really want.

Josh:
So that leads us to OpenRouter itself, right? So I kind of want you to help

Josh:
explain to the listeners who aren't familiar with OpenRouter what it does.

Josh:
Because I think a lot of people, the way they interact with an AI is they send

Josh:
a prompt to their model of choice.

Josh:
They use ChatGPT or they use the Grok app or they're on Gemini and they kind

Josh:
of live in these siloed worlds.

Josh:
And then the next step up from the people are those kind of who use it professionally,

Josh:
who are developers. They're interacting with APIs.

Josh:
Maybe they're not interfacing with the actual UI, but they're calling a single model.

Josh:
And OpenRouter kind of exists on top of this, right? Can you walk us through

Josh:
how it works and why so many people love using OpenRouter?

Alex:
Open Router is an aggregator and marketplace for large language models.

Alex:
You can kind of think of it as like a Stripe meets Cloudflare for both of them.

Alex:
It's like a single pane of glass. You can orchestrate, discover,

Alex:
and optimize all of your intelligence needs in one place.

Alex:
One billing provider gets you all the models.

Alex:
Uh there's like 470 plus now uh

Alex:
like all the models like they sort of implement features

Alex:
but they do it differently and they also there's

Alex:
a lot of like intelligence brownouts as andre carpoffi calls them yeah where

Alex:
models just go down all the time even the you know even the top models like

Alex:
anthropic and gemini and and open

Alex:
ai um so what we do is you know we like developers need a lot of choice.

Alex:
CTOs need a lot of reliability.

Alex:
CFOs need predictable costs. CISOs need complex policy controls.

Alex:
All of these are inputs to what we do, which is build a single pane of glass

Alex:
that makes models more reliable, lower costs, gives you more choice, and,

Alex:
and then and helps you choose between all the options for where to source your intelligence.

Josh:
How does it work uh because i would imagine like what

Josh:
each as and i on the show we frequently talk about benchmarks right where

Josh:
a certain model is the best at coding and that infers that maybe you should

Josh:
go to that model to do all of your coding needs because it's the best at it

Josh:
but it would appear as if it's not true if you're routing through a lot of different

Josh:
providers so how do you consider which provider gets routed to when and how

Josh:
to get the best result for what you're asking

Alex:
So we've taken a different approach so

Alex:
far which is instead of like focusing on

Alex:
a production router that picks

Alex:
the model for you um we try

Alex:
to help you choose the model so we

Alex:
we build lots we create lots of analytics both on

Alex:
your account and uh and on our

Alex:
rankings page to help you browse and discover the models that

Alex:
like the power users are really using successfully on

Alex:
a certain type of workload um because we

Alex:
think like developers today primarily want to

Alex:
choose the model themselves um switching between all

Alex:
families can result in like a lot like very

Alex:
unpredictable behavior but once you've

Alex:
chosen your model um we try to

Alex:
help developers not need to think about the provider there are

Alex:
like sometimes dozens of

Alex:
providers for a given model uh all kinds

Alex:
of companies including the hyperscalers like aws google vertex and azure um

Alex:
and uh like scaling startups like together fireworks deep infra um and a long

Alex:
tail of providers that provide,

Alex:
like very unique features,

Alex:
very like exceptional performance.

Alex:
There's all kinds of differentiators for them.

Alex:
So what we do is we collect them all in one place. And if you want a feature,

Alex:
you just get the providers that support it.

Alex:
If you want performance, you get prioritized to the providers that have high performance.

Alex:
If you really are cost sensitive, you get prioritized to the providers that

Alex:
are really low cost today. and we basically create all these lanes. There's.

Alex:
Innumerable ways you could get routed but

Alex:
you're in full control of the of the overall user

Alex:
experience that you're aiming for and that's

Alex:
what that's what we found that was missing from the

Alex:
whole ecosystem was just a way of doing that and uh

Alex:
and you know we get like between on average five to ten percent uptime boosts

Alex:
over going to um providers directly just by load balancing and sending you to

Alex:
the top provider that's up and able to handle your request.

Alex:
We really focus hard on efficiency and performance.

Alex:
We only add about 20 to 25 milliseconds of latency on top of your request.

Alex:
It all gets deployed very close to your servers up the edge.

Alex:
We overall get just We stack providers.

Alex:
We figure out what you can benefit from that everybody else is doing and just

Alex:
give you the power of big data as a developer just accessing your model choice.

Josh:
So it kind of allows you to harness the collective knowledge of everybody, right?

Josh:
You get all of the data, you have all of the queries, you know which yields

Josh:
the best result, and you're able to deliver the best product for them.

Josh:
Now, in terms of actual LLMs, EJ has actually pulled this up just before, which is a leaderboard.

Josh:
And I'm interested in how you guys think about LLMs, which are the best,

Josh:
how to benchmark them, and how you route people through them.

Josh:
Is there a specific... Do you believe that benchmarks are accurate,

Josh:
and do you reflect those in the way that you route traffic through these models?

Alex:
In general, we have taken the stance that we want to be the capitalist benchmark for models.

Alex:
What is actually happening?

Alex:
And part of this is that I really think both the law of large numbers and the

Alex:
enthusiasm of power users are really, really valuable for everybody else.

Alex:
Like when you're routing to

Alex:
um like clod in

Alex:
let's say you're routing to clod 4 and you're

Alex:
based in europe um there you

Alex:
know all of a sudden there might be like a huge variance in in throughput from

Alex:
one of the providers and you're only able to detect that if like some other

Alex:
users have discovered it before you and so we route around the provider that's

Alex:
like running kind of slow in Europe and send you,

Alex:
if your data policies allow it,

Alex:
to a much faster provider somewhere else.

Alex:
And that allows you to get faster performance. So, like, um...

Alex:
That's, like, on the provider level, how, like, numbers help.

Alex:
On the, like, model selection level, like, what you see on this rankings page

Alex:
here, power users will, like, when we put up a model, like, we put up a new

Alex:
model today from a new model lab called ZAI,

Alex:
like, the power users instantly discover it.

Alex:
We have this LLM enthusiast community that dives in and really figures out what

Alex:
a model is good for along a bunch of core use cases.

Alex:
The power users figure out which workloads are interesting, and then you can

Alex:
just see in the data what they're doing. And everybody can benefit from it.

Alex:
That's why we open up our data and share it for free on the rankings page here.

Ejaaz:
I'm seeing this one consistent unit across all these rankings,

Ejaaz:
Alex, which is tokens, right?

Ejaaz:
And Josh and I have spoken about this on the show before, but I'm wondering

Ejaaz:
how, like you've chosen this specific unit to measure how good or effective

Ejaaz:
these models are or how consumed or used they are.

Ejaaz:
Can you tell us a bit more as to why you picked this particular unit and what

Ejaaz:
that tells you as like the open router platform as to how a user is using a particular model?

Alex:
Yeah, I think dollars is a good metric too.

Alex:
The reason we chose tokens is primarily because we were seeing prices come down really quickly.

Alex:
Open Router has been around since the beginning of 2023.

Alex:
And I didn't want a model to be penalized in the rankings just because the prices

Alex:
are going down really dramatically now like there's a,

Alex:
There's a paradox called Jevons paradox, which is that when prices decrease like 10x,

Alex:
users' use of some component of infrastructure increases by more than 10x.

Alex:
And so maybe they didn't get 10x at all.

Alex:
But I thought there were some other advantages to using tokens,

Alex:
too. Tokens don't have this penalty and don't rely on Jevon's Paradox,

Alex:
which can have a lot of lag.

Alex:
They also are a little bit of a proxy for time.

Alex:
A model that is generating a lot of tokens and doing so for a while across a lot of users.

Alex:
It means that a lot of people are reading those tokens and actually doing something with them.

Alex:
And same goes for input. But if I really want to send an enormous number of

Alex:
documents and the model has a really, really, really tiny prompt pricing,

Alex:
I think that's still valuable and something that we want to see.

Alex:
We want to see that this model is processing an enormous number of documents.

Alex:
That's a use case that should show up in the rankings.

Alex:
And so we decided to go with tokens. We might like add dollars in the future,

Alex:
but I think tokens are, you know, they don't have this like Jevons Paradox lag.

Alex:
And there wasn't anything else. Like nobody was doing any kind of like overall analytics.

Alex:
We didn't see any other company even do it until Google did a few months ago

Alex:
where they started publishing the total amount of tokens processed by Gemini.

Alex:
So we'll see which use cases really need dollars.

Alex:
But tokens have been holding up pretty well.

Ejaaz:
Yeah, I mean, this dashboard is awesome. And I recommend anyone that's listening

Ejaaz:
to this that can't see our screen to get on OpenRouter's website and check it out.

Ejaaz:
I've been following it for the last two weeks kind of pretty rigorously, Alex.

Ejaaz:
And what I love is you can literally see...

Ejaaz:
So two weeks ago Grok 4 got released right

Ejaaz:
and Josh and I were making a ton of videos on this we were

Ejaaz:
using it with pretty much everything that we could do and

Ejaaz:
then this other model came out of China pretty much a few days after called

Ejaaz:
Kimi K2 and I was like oh yeah whatever this is just some random Chinese model

Ejaaz:
I'm not going to focus on it and then I kept seeing it in my feed and I thought

Ejaaz:
okay maybe I'll give this a go and I kind of like went straight to open rather than just

Ejaaz:
almost gauge the interest from a wider set of AI users. And I saw that it was skyrocketing, right?

Ejaaz:
And then I saw that Quen dropped their models last week.

Ejaaz:
And again, I came to Open Router and it preceded the trend, right?

Ejaaz:
People had already started using it. So I love how you describe Open Router

Ejaaz:
as this kind of like prophetic orb,

Ejaaz:
basically, where the enthusiasts and the community itself can kind of like front

Ejaaz:
run very popular trends. And I think that's a very powerful moat.

Ejaaz:
And kind of on this path, Alex, I noticed that a lot of these major model providers

Ejaaz:
see the value in this, right?

Ejaaz:
So if I'm not mistaken, OpenAI kind of like used your platform to kind of secretly

Ejaaz:
launch their Frontier model before they officially launched it, right?

Ejaaz:
Can you walk us through, you know, how that comes about and more importantly,

Ejaaz:
why they want to do that and why they chose OpenRoddy to do that?

Alex:
Uh open ai will sometimes

Alex:
give uh early access

Alex:
to their to models to some of their customers for

Alex:
testing and we asked them if they

Alex:
wanted to try a stealth model with us which we had never done before um it involved

Alex:
like launching it as under another name and seeing how users respond to it without

Alex:
having any bias or sort of inclination for against the model at the onset.

Alex:
And it would be like a new way of testing it and a new way of...

Alex:
It was like an experiment for both us and them.

Alex:
And they generously decided to take the leap of faith and try it. And we...

Alex:
Launched gpt 4.1 with

Alex:
them at and we called it quasar alpha and

Alex:
it was a million uh

Alex:
token context length model opening us first very

Alex:
very long context model and it was also optimized

Alex:
for coding and the incredible

Alex:
there were a couple incredible things that happened first

Alex:
we have this community uh of benchmarkers

Alex:
that run open source benchmarks and we give

Alex:
a lot of them grants to help fund the benchmarks

Alex:
grants of open router tokens they'll just run the

Alex:
suite of tests against all the models and some of them are very creative like

Alex:
there's one that tests uh like the ability to generate fiction there's one that

Alex:
tests um like how like whether it can make a 3d object project in Minecraft called MCBench.

Alex:
There are a few that test different types of coding proficiency.

Alex:
There's one that just focuses on how good it is at Ruby, because Ruby is,

Alex:
turns out a lot of the models are not great at Ruby.

Alex:
There are a lot of like languages that all the models are pretty bad at.

Alex:
And so we have this like long tail of very niche benchmarks,

Alex:
And all the benchmarkers ran, you know, for free their benchmarks on Quasar

Alex:
Alpha and found pretty incredible results for most of them.

Alex:
And so the model got like, you know, OpenAI got this feedback in real time.

Alex:
We kind of like helped them find it.

Alex:
And they made another snapshot, which we launched as Optimus Alpha.

Alex:
And they could compare the feedback that they got from the two snapshots.

Alex:
Um, and, and then they, and then like two weeks later, they launched GPT 4.1 live for everybody.

Alex:
So it was like, uh, uh, was it an experiment for us?

Alex:
And, and we've done it, um, again since, uh, with, uh, another model provider

Alex:
that, uh, that's still working on it.

Alex:
Um, and it, and it's kind of like a cool way of learning of like crowdsourcing,

Alex:
uh, benchmarks that you wouldn't have expected. and also getting unbiased community sentiment.

Josh:
That's great. So now when we see a new model pop up and we want to test GPT-5,

Josh:
we know where to come to to try it early.

Josh:
We'll see because rumor is it's coming soon. So we'll be, we're on your watch list.

Josh:
But having, I do want to ask you about open source versus closed source because

Josh:
this has been an important thing for us. We talk about this a lot.

Josh:
You have a ton of data on this.

Josh:
I'm looking at the leaderboards there. There are open source models that are

Josh:
doing very well, closed source.

Josh:
What are your takes in general? How do you feel about open source versus closed

Josh:
source models, particularly around how you serve them to users?

Alex:
Both models, both types of models have supply problems, but the supply problems are very different.

Alex:
Typically, what we see with closed source models is that there's there's very

Alex:
few suppliers, usually just one or two.

Alex:
Like with Grok, for example, there's Grok Direct and there's Azure.

Alex:
Um with anthropic there's anthropic direct there's google vertex there's aws

Alex:
bedrock um and then we also like deploy it in different regions like we have

Alex:
an eu deployment um for customers who'd like only want their data like to stay in the eu,

Alex:
and uh and we do custom deployments for

Alex:
the for the closed source models too to just kind of guarantee good

Alex:
throughput high and high rate limits for people um

Alex:
but uh the

Alex:
like a tricky part is that like the the demand usually the like the closed source

Alex:
malls are doing most of the tokens on open router um it's it's dominant you

Alex:
know it's probably 80-ish 70 to 80 percent closed source tokens today.

Alex:
But the open source models have a much more fragmented supply, like cell supply.

Alex:
Side order book um and and like

Alex:
the rate limits for each provider is

Alex:
like a like less stable on average um it

Alex:
usually takes a while for the hyperscalers to serve a

Alex:
new closed source a new open source model um so we so the load balancing work

Alex:
that we do on um open source models tends to be a lot more valuable the load

Alex:
balancing work that we do for closed source models tends to be very focused

Alex:
on caching and feature awareness,

Alex:
making sure you're getting clean cache hits and only transitioning over to new

Alex:
providers when your cache is expired.

Alex:
For open source models, there's way less caching. Very, very few open source

Alex:
models implement caching.

Alex:
And so switching between providers becomes more common. and

Alex:
uh like we we also track a

Alex:
lot of quality differences between the the open

Alex:
source providers some of them will deploy at lower

Alex:
quantization levels which means like it's kind of like a way of compressing

Alex:
the model um generally doesn't have an impact on the quality of the output uh

Alex:
but and yet we still see some odd things from some of the open source providers.

Alex:
And so we run tests internally to detect those outputs. And we're building up

Alex:
a lot more muscle here soon.

Alex:
So that like, they get pulled out of the routing lane and don't affect anyone.

Josh:
So closed source accounts for 80% or something like that, a very large amount.

Josh:
Do you see that changing?

Josh:
Because that post we just had, it's at nine out of the 10 fastest growing LLMs

Josh:
last week, they were open source.

Josh:
And every time it seems like China comes out with another model,

Josh:
it was Kimmy K2 a week or two ago, it kind of really pushes the frontier of open source forward.

Josh:
And the rate of acceleration of open source seems to be as fast,

Josh:
if not faster than closed source, where it's just, it's making these improvements very quickly.

Josh:
It has the benefit of being able to compound in speed because it's open source

Josh:
and everyone can contribute.

Josh:
Do you think that starts to change where the percentage of tokens you're issuing

Josh:
are from open source models versus closed source?

Josh:
Or do you continue to see a trend where it's going to be Google,

Josh:
it's going to be OpenAI that are serving a majority of these tokens to users?

Alex:
In the short term, we're likely to see open source models continue to dominate

Alex:
the fastest growing model category on OpenRouter.

Alex:
And the reason for that is that a lot of users who come for a closed source

Alex:
model, but then decide they want to optimize later,

Alex:
either they want to save on costs or try out a new model that's supposed to

Alex:
be a little bit better in some direction that their app cares about or their use case cares about,

Alex:
then they leave the closed source model and go to an open source model.

Alex:
So open source tends to be like a last mile optimization thing,

Alex:
making a big generalization because the reverse can happen too.

Alex:
And so because it's a last mile optimization thing,

Alex:
the jump from this model is not being used at all to this model is really being

Alex:
used by a couple of people who have

Alex:
left Claude 4 and want to try some new coding use case will be bigger.

Alex:
Than the closed-source models, which start at a really high base and don't have

Alex:
growth quite as dramatic.

Alex:
So the other part of your question, though, was whether there's going to be like a flippening of.

Josh:
Close or some sort of like chipping it away at that monopoly of close source tokens.

Alex:
It's hard to predict these things because, you know,

Alex:
I think like the the biggest problem today with open source models is that the

Alex:
incentives are not as strong like the model lab and the model provider.

Alex:
Um they've you know they're they're

Alex:
sort of established incentives for how to

Alex:
grow as a company and attract good high quality um ai talent and um and giving

Alex:
the model weights away impairs those incentives now like we might see yeah this

Alex:
is where we might see like decentralized providers,

Alex:
helping in the future.

Alex:
A way for like,

Alex:
uh you know like a really good incentive scheme that

Alex:
like allows high quality talent

Alex:
to work on an open source model um

Alex:
that remains open weights at least uh like could fix this i like i you know

Alex:
i stay pretty i try to stay close to the decentralized providers um and like

Alex:
learn a lot from them there's some like cool on the provider side on like on

Alex:
running inference i I think there's some really cool incentive schemes being worked on.

Alex:
But on actually developing the models themselves, I haven't seen too much, unfortunately.

Alex:
So I think if we see one, flipping in the radar. And until we do, I personally doubt it.

Josh:
TBD, do you have personal takes on how you feel about open source versus closed source?

Josh:
Because this has been a huge topic we've been debating too. It's just the ethical

Josh:
concerns around alignment and closed source models versus open source.

Josh:
When you look at the competitors, China, generally speaking,

Josh:
is associated with open source, whereas the United States is generally associated with closed source.

Josh:
And we saw Llama and Meta release the open source models, but now they're raising

Josh:
a ton of money to pay a lot of employees a lot of money to probably develop a closed source model.

Josh:
So it seems like the trends are kind of split between US and China.

Josh:
And I'm curious if you have any personal takes, even outside of OpenRouter,

Josh:
of which you think serves better for the long term outlook on,

Josh:
I mean, the position of the United States or just the general safety and alignment

Josh:
conversation around AI?

Alex:
I mean, like a very simple fundamental difference between the two is that an

Alex:
innovation in open source models can be copied more quickly than an innovation

Alex:
in closed source models.

Alex:
So in terms of velocity and like how far ahead one is over the other,

Alex:
that is like a massive structural difference.

Alex:
That means that closed source models should be theoretically always ahead until

Alex:
a really interesting incentive scheme develops, like I mentioned before.

Alex:
Uh, I think, and I think that's, you know, I don't see like evidence that that's

Alex:
going to change in terms of China versus the U S.

Alex:
Um, it's, I think it's very interesting that China has not had like a major closed source model.

Alex:
Um and i don't really

Alex:
see a great reason why i'm

Alex:
not aware of any reasons that's not that's not going

Alex:
to be going to be the case in the future um my prediction

Alex:
is that there's going to be a closed source model from china um

Alex:
and uh you know if uh uh you know if like it's possible that DeepSeas and Moonshot

Alex:
and Gwen have built up really sticky talent pools.

Alex:
But generally with talent pools, after enough years have passed,

Alex:
people quit and go and create new companies and build new talent pools.

Alex:
And so we should see some of that. It's not the case that the AI space has NDAs

Alex:
or non-competes that the hedge fund space has.

Alex:
That might happen in the future too. But assuming that the current non-compete

Alex:
culture continues, there should be more companies that pop up in China over time.

Alex:
And I'm betting that some of them will be closed source.

Alex:
And my guess is that the two nations will start to look more similar.

Ejaaz:
Yeah, I guess that's why you have Zuck dishing out 300 mil to a billion dollar

Ejaaz:
salary offers to a bunch of these guys, right?

Ejaaz:
One more question on China versus the US. I kind of agree with you.

Ejaaz:
I didn't really expect China to be the one to lead open source anything,

Ejaaz:
let alone the most important technology of our time.

Ejaaz:
Do you think is their secret source to building these models, Alex?

Ejaaz:
And I know this might be out of the forte of

Ejaaz:
open router specifically but as someone who has studied this technology for

Ejaaz:
a while now i'm struggling to figure out you know what advantage they had you

Ejaaz:
know they're discovering all these new techniques and maybe the simple answer

Ejaaz:
is like constraints right they don't have access to all of

Ejaaz:
nvidia's chips they don't have access to infinite compute so then maybe they're

Ejaaz:
forced to kind of like figure out other ways around the same kinds of problems

Ejaaz:
that western companies are focused on But it's pretty clear that America, with all its funding,

Ejaaz:
hasn't been able to make these frontier breakthroughs.

Ejaaz:
So I'm curious whether you are aware of or know some kind of technical moat

Ejaaz:
that Chinese AI researchers or these AI teams that are featuring on Open Rata

Ejaaz:
day in and day out have over the U.S.?

Alex:
Well, I don't know.

Alex:
There are certainly some that they've come up with that like DeepSeek had a

Alex:
lot of very cool inference innovations that they published in their paper.

Alex:
But a lot of what they published in the original R1 paper were things that OpenAI

Alex:
had done independently themselves many months before.

Alex:
So uh i like

Alex:
on the inference side and on

Alex:
uh some of the model side i think like deep seek we we

Alex:
had talked to their team for years before r1 came

Alex:
out they had many models before that and

Alex:
they were always like a pretty sharp optimum like

Alex:
team for doing inference um like they

Alex:
came up with like the best user experience for caching prompts

Alex:
long before deep cpr1 came out and they had very good pricing um they uh they

Alex:
were just they were like you know by far the the strongest chinese team um that

Alex:
we were aware of uh well before that happened and so i'm guessing there was like some talent.

Alex:
Uh accumulation that they were working on in china

Alex:
for people who wanted to stay in china and yeah that's

Alex:
that's a huge advantage like american companies are obviously not

Alex:
doing that there's a duck is very on

Alex:
point that a lot of this is just based on talent

Alex:
um there are a lot of

Alex:
ai is open and out there and just like and

Alex:
very composable like a big tree of knowledge

Alex:
there's a paper that comes out and it cites like

Alex:
20 other papers and you can go and read all

Alex:
of the cited papers and then you like have kind of

Alex:
a basis for understanding the paper but you really have to

Alex:
go one level deeper and read all the cited papers two levels

Alex:
down to really understand what's going on and it's.

Alex:
Just that no very few people can do that um and

Alex:
it takes like a lot of years of experience to like actually

Alex:
apply that knowledge and learn all these

Alex:
things that have not been written in any paper at all and uh

Alex:
and there's just there's just such such it

Alex:
like a small number of people um who can

Alex:
really lead research on all the different dimensions that

Alex:
go on to making a model and uh um and

Alex:
and like the the border between china and the u.s is

Alex:
is pretty defined you have to leave china move to the u.s

Alex:
and really establish yourself here um so

Alex:
i do think there's like country arbitrage there's like

Alex:
there's you know the head the hedge fund background arbitrage there's uh there's

Alex:
there's hardware arbitrage like there's like a ton of hardware that's only available

Alex:
in china but not here vice versa that creates an opportunity um and this this

Alex:
will just continue to happen.

Ejaaz:
Yeah, I think this arbitrage is fascinating.

Ejaaz:
I read somewhere that there's probably less than 200 or 250 researchers in the

Ejaaz:
world that are worthy of working at some of these frontier AI model labs.

Ejaaz:
And I looked into some of the backgrounds of the team behind Kimi K2,

Ejaaz:
which is this recent open source model out of China, which broke all these crazy rankings.

Ejaaz:
I think it was like a trillion parameter model or something crazy like that.

Ejaaz:
And a lot of them worked at some of the top American tech companies.

Ejaaz:
And they all graduated from this one university in China.

Ejaaz:
I think it's Tsinghua, which apparently is like, you know, the Harvard of AI

Ejaaz:
in China, right? So pretty crazy.

Ejaaz:
But Alex, I wanted to shift the focus of the conversation to a point that you

Ejaaz:
brought up earlier in this episode, which is around data.

Ejaaz:
Okay, so here's the context that like Josh and I have spoken about this at length, right?

Ejaaz:
We are obsessed with this feature on OpenAI, which is memory, right?

Ejaaz:
And I know a lot of the other memory, sorry, a lot of the other AI models have memory as well.

Ejaaz:
But the reason why we love it so much is I feel like the model knows me, Alex.

Ejaaz:
I feel like it knows everything about me. It can personally curate any of my prompt.

Ejaaz:
It just gets me. It knows what I want and it just serves up to me in a platter

Ejaaz:
and off I go, you know, doing my thing.

Ejaaz:
Now, Open Router sits on top of like kind of like the query layer, right?

Ejaaz:
So you have all these people writing all these weird and wonderful prompts and

Ejaaz:
kind of routing it through on towards like different AI models.

Ejaaz:
You hold all of that data or maybe you have access to all of that data.

Ejaaz:
And I know you have something called private chat as well, where you don't have access to it.

Ejaaz:
Talk to me about like what OpenRouter and what you guys are thinking about doing

Ejaaz:
with this data, because presumably,

Ejaaz:
or in my opinion, you guys have actually the best mode, arguably better than

Ejaaz:
ChatGPT, because you have all these different types of prompts coming from all

Ejaaz:
these different types of users for all these different types of models.

Ejaaz:
So theoretically, you could spin up some of the most personal AI models for

Ejaaz:
each individual user if you wanted to.

Ejaaz:
Do I have that correct? Or am I, you know, speaking crazy?

Alex:
No, that's true. No, it's something we're thinking about.

Alex:
By default, your prompts are not logged at all.

Alex:
We don't have prompts or completions for new users by default.

Alex:
You have to toggle it on in settings.

Alex:
But the result, a lot of people do toggle it on. And as a result,

Alex:
I think we have by far the largest multi-model prompt data set.

Alex:
Uh, but what we've done today, we've barely done anything with it.

Alex:
We classify a tiny, tiny, tiny subset of it. And that's what you see in the rankings page.

Alex:
Um, but, uh, what it could be done on like a per account level is really,

Alex:
um, like three main things.

Alex:
One memory right out of the box. You can, you can get this today by like combining

Alex:
open router with like a memory as a service. We've got a couple of companies

Alex:
that do this, like Memzero and SuperMemory.

Alex:
And we can partner with one of those companies or do something similar and just

Alex:
provide a lot of distribution.

Alex:
And that basically gets you a chat GPT as a service where it feels like the

Alex:
model really knows you and the right context gets added to your prompt.

Alex:
The other things that we can do are help you select the right model more intelligently.

Alex:
There's a lot of models where there's like a super clear, like migration decision that needs to be made.

Alex:
And, and we can just see this very clearly in the data.

Alex:
But we right now we just like, you know, we have like a channel or like some

Alex:
kind of communication channel open with the customer, we can just tell them

Alex:
like, hey, and we know you're using this model a ton.

Alex:
It's been deprecated. This model is significantly better. you

Alex:
should move this kind of workload over to it or like

Alex:
this workload you'll get way better pricing if you do this um

Alex:
and and that's basically like that's the

Alex:
only sort of guidance and kind of like

Alex:
opinionated routing we've done so far and it could

Alex:
be a lot more intelligent a lot more out of the box a lot more

Alex:
built into the product um and then

Alex:
the the last thing

Alex:
we can do i mean there's there's probably tons of

Alex:
things we're not even thinking about um but

Alex:
like getting really

Alex:
really smart about how

Alex:
models and providers are responding to prompts and

Alex:
uh showing you just the really coolest

Alex:
data just like telling you

Alex:
what kinds of of prompts um are

Alex:
going to which models and how those models are replying and

Alex:
just like characterizing the reply in all kinds of interesting ways

Alex:
like did the model refuse to answer what's the refusal rate

Alex:
did the model um did the.

Alex:
Model like successfully make a tool call or did it decide to

Alex:
ignore all the tools that you passed in that's a huge one

Alex:
um did the model like pay

Alex:
attention to its context did uh you know did what did did some kind of truncation

Alex:
happening happen before you sent it to the model So there's all kinds of like

Alex:
edge cases that cause developers apps to just get dumber and they're all detectable.

Ejaaz:
I'm so happy you said that because I have this kind of like hot take,

Ejaaz:
but maybe not so hot take, which is I actually think all the Frontier models

Ejaaz:
right now are good enough to do the craziest stuff ever for each user.

Ejaaz:
But we just haven't been able to unlock it because it just doesn't have the context.

Ejaaz:
Sure, you can attach it to a bunch of different tools and stuff,

Ejaaz:
but if it doesn't know when to use the tool or how to process a certain prompt

Ejaaz:
or if the users themselves don't know how to read

Ejaaz:
the output of the AI model themselves, like you just said, we need some kind

Ejaaz:
of analytics into all of this,

Ejaaz:
then we're just kind of walking around like headless chickens almost.

Ejaaz:
So I'm really happy that you said that. One other thing that I wanted to get

Ejaaz:
your take on on the data side of things is, I just think this whole concept

Ejaaz:
or notion of AI agents is becoming such a big trend, Alex.

Ejaaz:
And I noticed a lot of Frontier Model Labs release new models that kind of spin

Ejaaz:
up several instances of their AI model.

Ejaaz:
And they're tasked with a specific role, right?

Ejaaz:
Okay, you're going to do the research. You're going to do the orchestrating.

Ejaaz:
You're going to look online via a browser, blah, blah, blah,

Ejaaz:
blah, blah. And then they coalesce together at the end of that little search

Ejaaz:
and refine their answer and then present it to someone, right?

Ejaaz:
You know, Grok4 does this, Claude does this, and a few other models.

Ejaaz:
I feel like with this data that you're describing, OpenRouter could be or could

Ejaaz:
offer that as a feature, right?

Ejaaz:
Which is essentially, you can now have super intuitive, context-rich agents

Ejaaz:
that can do a lot more than just talk to you or answer your prompts.

Ejaaz:
But they could probably do a bunch of other actions for you.

Ejaaz:
Is that a fair take, or is that something that maybe might be out of the realm of open router?

Alex:
Our strategy is to be the best inference layer for agents.

Alex:
And what I think developers want...

Alex:
Is control over how their agents work.

Alex:
And our developers at least want to use us as a single pane of glass for doing

Alex:
inference, but they want to see and control the way an agent looks.

Alex:
An agent is basically just something

Alex:
that is doing inference in a loop and controlling the direction it goes.

Alex:
So um what what

Alex:
we want to do is just like build incredible docs

Alex:
really good primitives that make that easy

Alex:
to do so that you know like i think like

Alex:
a lot of our developers are just people building agents and so

Alex:
what they want is they want the primitives to

Alex:
be solved so that they can just keep creating new

Alex:
versions and new ideas um without worrying

Alex:
about like you know re-implementing tool calling over

Alex:
and over again and um and and

Alex:
and so like at least for this is like a it's it's

Alex:
a tough problem given how many models there's like a new model or provider every

Alex:
day and uh and people actually want them and use them so uh to standardize this

Alex:
like make make these tools like really dependable um that's kind of like where

Alex:
we want to focus and uh so that like agent developers don't have to worry about it.

Josh:
As we level up towards closer and closer to getting to AGI beyond,

Josh:
I'm curious what Open Router's kind of endgame is.

Josh:
If you have one, what is the master plan where you hope to end up?

Josh:
Because the assumption is as these systems get more intelligent,

Josh:
as they're able to kind of make their own decisions and choose their own tool

Josh:
sets, what role does Open Router play in continuing to route that data through?

Josh:
Do you have a kind of master plan, a grand vision of where you see this all heading to?

Alex:
You're saying like as agents get better at choosing the tools that they use

Alex:
what what becomes our role when like the agents are really good at that yes.

Josh:
Yes and like where do you see open router fitting into the picture and what

Josh:
would be the best case scenario for this this future of open router

Alex:
Right now open routers bring your own tool,

Alex:
platform um we don't have like a

Alex:
marketplace of mcps yet uh and

Alex:
and i i do think like a lot of the i think most of the most used tools will

Alex:
be ones that developers configure themselves agents just work like they're given

Alex:
access to it like i think like a holy grail for for open router is that.

Alex:
The the ecosystem is going to like basically my

Alex:
prediction for how the ecosystem is going to evolve is that um

Alex:
all the models are going to be adding state and

Alex:
other kinds of stickiness that just make you want to stick

Alex:
with them so they're going to add server-side tool calls

Alex:
they're going to add like um you know web search that that is stateful they're

Alex:
going to add memory They're going to add all kinds of things that try to prevent

Alex:
developers from leaving and increase lock-in.

Alex:
And OpenRouter is doing the opposite.

Alex:
We want developers to not feel vendor lock-in.

Alex:
We want them to feel like they have choice and they can use the best intelligence,

Alex:
even if they didn't before.

Alex:
It's never too late to switch to a more intelligent model. That would be like,

Alex:
you know, a good always on outcome for us.

Alex:
And so what I think we'll end up doing is, is like partnering with other companies

Alex:
or building the tools ourselves if we have to, so that developers don't feel stuck.

Alex:
That's how I, you know, there's a lot of ways the ecosystem could evolve,

Alex:
but that's how I would put it in a nutshell.

Josh:
Okay, now there's another personal question that I was really curious about,

Josh:
because I was also right there with you in the crypto cycle when NFTs got absolutely

Josh:
huge, was a big user of OpenSea.

Josh:
And it was kind of this trend that went up and then went down.

Josh:
And NFTs kind of fizzled out, it wasn't as hot anymore, and AI kind of took the wind from the sails.

Josh:
And it's a completely separate audience, but a similar thing where now it's

Josh:
the hottest thing in the world.

Josh:
And i'm curious how you see the trend continuing is this a cyclical thing that

Josh:
has ups and downs or is this a one-way trajectory of more tokens every day more

Josh:
ai every day is do you see it being a cyclical thing or is this a a one-way

Josh:
trend towards up into the right nfts

Alex:
Kind of follow uh crypto in a,

Alex:
indirect way um when crypto

Alex:
has ups and downs nfts generally lag a bit

Alex:
but they they have similar ups and downs and um

Alex:
and crypto is an extremely long-term play on like building a new financial system

Alex:
and there are so many reasons that it's not going to happen overnight um and And they're like,

Alex:
it's very, very entrenched reasons.

Alex:
Whereas AI, there are some overnight business transformations going on.

Alex:
And the reason AI, I think, moves a lot, one of the reasons that AI moves a

Alex:
lot faster is it's just about making computers behave more like humans.

Alex:
So if a company already works with a bunch of humans, then there's,

Alex:
you know, there's some engineering that needs to be done.

Alex:
There's some like thinking about how

Alex:
to like scale this but

Alex:
but in general i think that it's not like

Alex:
after seeing what can be possible um inference

Alex:
will be the fastest growing operating expense for all companies

Alex:
it'll it'll be like oh we can just hire

Alex:
high-performing employees at a click of a

Alex:
and they they work 24 7 they

Alex:
scale elastically it's like you know

Alex:
it it's not that hard it's not like huge mental

Alex:
model shift it's just like a huge upgrade to the way companies work today um

Alex:
in most cases so it's just completely different from crypto there's there's

Alex:
like other than both being you know than nfts i mean other than both being new

Alex:
they're fundamentally very different changes.

Ejaaz:
You're probably one of very few people in the world right now that has crazy

Ejaaz:
insights to every single AI model.

Ejaaz:
Definitely more than the average user, right? Like I have like three or four

Ejaaz:
subscriptions right now and I think I'm a hotshot.

Ejaaz:
You get access to like 400 and what is it? 57 models right now on OpenRata.

Ejaaz:
So an obvious question that I have for you is

Ejaaz:
I'm not going to say in the next couple of years, because everything moves way

Ejaaz:
too quickly in this sector.

Ejaaz:
But over the next six months, is there anything really obvious to you that should

Ejaaz:
be focused on within the AI sector?

Ejaaz:
Maybe it's like the way that certain models should be designed,

Ejaaz:
or perhaps it's at the application layer that no one's talking about right now.

Ejaaz:
Because going on from our earlier part of the conversation, you just pick these

Ejaaz:
trends out really early. and I'm wondering if you see anything.

Ejaaz:
It doesn't have to be open-racket related. It could just be AI related.

Alex:
I've seen the models trending towards caring more about how resourceful they

Alex:
are than what knowledge they have in the bank.

Alex:
Not all of, I feel like a lot of the applications, I think the model labs maybe,

Alex:
a lot of them, I don't know how many of them really deeply believe that,

Alex:
but a couple of them uh talk about it and i don't think it's really hit the

Alex:
application space yet um because people will will ask chat gpt things and if

Alex:
the knowledge is wrong they think the model is stupid,

Alex:
and that's just kind of a bad way of evaluating a model um

Alex:
like whatever knowledge a person has whatever

Alex:
a person like where calls happen at a certain time like

Alex:
does not it's not a proxy for how smart they are um

Alex:
like the the intelligence and usefulness of a model

Alex:
is going to trend towards how good it is at using tools and

Alex:
uh and and how good it is at like paying

Alex:
attention to its context of a long long long long context and so it's like it's

Alex:
it's total memory capacity and accuracy um so i think those two things need

Alex:
to be like emphasized more um the.

Alex:
Like it might be that that models pull all

Alex:
of their knowledge from like online databases

Alex:
from like real-time uh scraped

Alex:
index indices of the web along with a

Alex:
ton of real-time updating data sources um and

Alex:
they're never they're always kind of like relying on some some sort of database

Alex:
for knowledge but relying on their reasoning process for for tool calling you

Alex:
know like we we put it We spend probably the plurality of our time every week

Alex:
on tool calling and figuring out how to make it work really well.

Alex:
Humans, the big difference between us and animals is that we're tool users and tool builders.

Alex:
And that's where human acceleration and innovation has happened.

Alex:
So how do we get models creating tools and using tools very,

Alex:
very effectively? there's very little,

Alex:
There are very few benchmarks. There's very little priority.

Alex:
There's the Tau Bench for measuring how good a model is at tool calling.

Alex:
But there's, and there's like maybe a few others.

Alex:
There's Swee Bench for measuring how good a model is at multi-turn programming tasks.

Alex:
It's very, very hard to run, though. It costs like, you know,

Alex:
for Sonnet, it could cost like $1,000 to run it.

Alex:
And it's like the user experience for kind of like evaluating the real intelligence

Alex:
of these models is not good.

Alex:
And so like I love, as much as we don't have benchmarks listed on OpenRouter

Alex:
today, I love benchmarks.

Alex:
And I think like the app ecosystem and like developer ecosystem should spend

Alex:
a lot more time making very cool and interesting ones.

Alex:
Also, we will give credit grants for all the best ones. So I highly encourage it.

Ejaaz:
Well, Alex, thank you for your time today. I think we're coming up on a close

Ejaaz:
now. That was a fascinating conversation, man.

Ejaaz:
And I think your entire journey from just non-AI stuff, so OpenSea all the way

Ejaaz:
to OpenRouter has just been a great indicator of where these technologies are

Ejaaz:
progressing and more importantly, where we're going to end up.

Ejaaz:
I'm incredibly excited to see where OpenRatter goes beyond just prompt routing.

Ejaaz:
I think some of the stuff you spoke about on the data side of things is going

Ejaaz:
to be fascinating and arguably one of your bigger features. So I'm excited for future releases.

Ejaaz:
And as Josh said earlier, if GPT-5 is releasing through your platform first,

Ejaaz:
please give us some credits. We would love to use it.

Ejaaz:
But for the listeners of this show, as you know, we're trying to bring on the

Ejaaz:
most interesting people to chat about AI and Frontier Tech. We hope you enjoyed this episode.

Ejaaz:
And as always, please like, subscribe, and share it with any of your friends

Ejaaz:
who would find this interesting. And we'll see you on the next one. Thanks, folks.