Search is the gateway to the world of information. If you can make it perfect, then that has so many downstream positive implications for the world.
You can kind of think of Google as being synonymous with search, right? It's one of the greatest technopolies of the last few decades. >> want to
go really deep into some topic, Google fails. [music] Most people want to understand the world, but
they're getting fed information that's just like, you know, misleading in some way or straight up wrong. And if everyone had like information that was
accurate, most reasonable people would be reasonable. >> We have a family open Claude, Michael Claudeberg, and we wanted to give it web access, and he
was like, "I recommend [music] Exa." >> World of agents searching is just completely different
from human searching. An agent doesn't just want 10 pieces of information. It wants everything. With Exa, like you could search something and then get
not just like 10 results or 100 results, but 1,000 results or 10,000. How have we, a [music] team that, you know, has always been below 100 people
been able to build a search engine that's better than Google in all sorts of ways? Um, well, it's
because >> Welcome, well. Thank you for being here. >> Well, excited to be here. >> So, I want to start with the origin story. Um, you've been
interested in search for a long time. In fact, you and your co-founder Jeff actually started building a mini search engine in college, which is not
what I was doing in college. Can you say more about when you started getting interested in search and why you
wanted to solve this problem? >> Yeah, yeah, sure. So, I would say it's a life mission. So, since I was a kid, I've cared about like finding the
highest quality knowledge, right? I was obsessed. Uh, and then, you know, in high school, I wanted to start a new type of news organization cuz I
thought, you know, we're a we're a civilization that got to the moon, we split the atom, and yet we can't
understand what's going on at the border or uh, like in some in science news. Like, why can't we fully understand any topic? Uh, and then in college,
like me and I you know, I was roommates with Jeff, and we were like, we could just build a better search uh, using like crowdsourcing the highest
quality links. Um, and [clears throat] we did build like a a pretty solid search. Um, but then, you
know, 5 years ago, so in 2021, that's when Transformer started to get really good, and it suddenly became possible to build like a better search
engine than Google. Uh, and that was like a really important opportunity because, you know, search is the gateway to the world of information. If you
could if you can improve search, if you can make it perfect, then that has so many downstream positive
implications for the world, like across every industry, across every part of human life. Uh and so it just felt like this huge opportunity that no one
was pursuing and I was like, I'm going to devote my life to this because it's, you know, everything I care about is about information. Uh and so yeah,
started XA and um and now it's gone, you know, you know, we actually made a lot of progress >>
[laughter] >> and um you know, we're a lot closer to that mission. There's still a huge amount of things you know, huge amount to go. Um but yeah,
it's been crazy to see like uh how far we've come to achieving that mission. Um yeah, that I've been thinking about for years. >> [snorts] >> Maybe
just to probe a little bit deeper. You know, you can kind of think of Google as being synonymous with
search, right? It's one of the greatest tech monopolies of the last few decades. Um and you know, the idea that a startup could be better at than
Google at search is quite amazing. Um but how do you define perfect search? And what do you see as the limitations of Google? And I'll throw in more
recent events because obviously at, you know, IO just took place last week and they're very focused on AI
mode and how, you know, they talked about the idea of information agents and things like that. How do you think about beating old Google, if you will?
And then there's of course new Google that's evolving. >> Yeah, yeah. I mean, Google was amazing and it and is amazing for what it's meant to do,
which is like get quick answers to consumers. Um and now it's like increasingly longer answers. But
really it's like been focusing on like what do what do most of the billions of people in the world search for, care about, and like making sure
they're really happy. And they do a great job of that. That's what they're optimized for. That's why they're optimized for human clicks. It's like
what you know, like when you type in some like you're really tired, you type in a few keywords that make no
sense and Google just magically understands what you're saying. That's magical because it has like billions of other people searching similar things.
Um I and I you know, I'm I'm excited by Google too, but like there are certain times when you want something deeper. And I was actually before
starting Exa writing a history book like just I was I just got obsessed with history and I wanted to get to
the bottom of like what did it feel like to live in every period in history going back 5,000 years. I don't know if you ever talked about it, but uh
I want to read this book. It's probably at home right now. >> I was really good. I would have finished it around now. So I >> [laughter] >> but at
some point I was like okay, let's maybe I can build a search engine and then I can automate the
building of writing of books and I think that turned out to be true. Um but anyway like in writing that book it became extremely obvious that uh if
you want to go really deep into some topic Google fails. Like Google's great at surfacing information which is great for most of the billions of
consumers, but if you want to really understand what it was like to live, you know, in the Roman Empire uh
you know, in like 100 AD it's actually quite hard and like that information is scattered. It's everywhere, but it's like you need really good search
like really deep search to understand it. And so that was like the first realization that or one of the first realizations I wait like what if you
could have like true perfect understanding of any topic. Um and so yeah, okay. So like Google has been
changing their search engine a little bit. I would say I I've seen many Google IOs now at this point at Exa. Uh every Google IO I'm like okay, they
say they're changing search and they do, but they change it more to be valuable for the consumer type use cases. Which for me it's like there are so
many different use cases that go beyond that. Um you know, there's like really deeply understanding the
Roman Empire, but there's also like finding every competitor to your company. Uh and right now Google is just not going to do that. You no matter how
many changes they make like you don't trust Google to find you literally every competitor your company whether it's in Europe or Asia. Uh you don't
trust you don't use Google for recruiting. And and this announcement doesn't change that. Like you're
not going to go and say hey Google, I'm looking for machine learning engineers in San Francisco who have a you know, background at startups because
it's it's not built for that kind of thing. So um so like there is an opportunity to build like a new type of search engine that's meant for extremely
like deep complex queries that uh you know, businesses really care about and agents really care
about. >> Yeah, absolutely. And I mean to the extent you talked about starting to build Exa 5 years ago and then it feels like in the last 5 years the
world has completely changed which is actually in my opinion a great thing to happen to you and to Exa while you're building because you can you know
bring in this new technology versus trying to either fight it or have some sort of innovators
dilemma. Maybe share a little bit more on how you decided to build or why you decided to build Exa from the ground up and what parts were the hardest
or have been the hardest and then how has it changed as the world of LLMs have changed? >> Building from the ground up basically we were like 2021 we
could build a better search engine than Google. I don't care how long it takes. I guess we were
young high energy just like ready to do anything devote our lives to this and there was a there was a thought experiment that really excited me which
was what which is that wait I could totally build a better search engine than Google right now and here's how I would do it. For every query I would
take all the trillion documents on the web and I would run GB3 over it I would say does this document
match the query is this document match the query and that would be and then I would filter it down to the top 10 documents and that would be better
than Google. The problem is that would cost like you know 10 billion dollars per query and so then it became an optimization problem but at least
there was like an existence proof that it's possible to build a better search engine than Google and that
was very inspiring for me. So then it was like okay like how do we optimize the hell out of that? And and transformers had gotten really good at time
and Google wasn't really leaning into it and like we just had this you know deep belief in the bitter lesson maybe more than Google which was that for
search which is that like if we could develop systems like neural systems where you pour more
compute more pour more data into it just gets better and better for the thing we're optimizing for then you could actually just totally be better than
Google. When we released this to the world in November 2022 like it was actually shocking and like you know Andre Karpathy retweeted it like you know
it got it got pretty popular on Twitter it was like this new way to find information it was the
first time it was I think it was it was like the first time people were like holy cow it's possible to find things beyond Google and then by the way 2
weeks later ChatGPT came out so then >> [laughter] >> people realized, okay, there's a uh you know, there's another new way of finding information
outside Google with LLMs. And that was very critical to us. So, like we released our first search
engine to the world in a year and a half after starting Exa in November 2022. Two weeks later chat GPT came out. Actually, to me, I was at NeurIPS and
I I saw the announcement. I played with it. I was like, this feels like GPT-3, but a little bit like better UI. And I went back to like looking at
research papers. [laughter] Um but to the world, I think it was the first uh it was the first like
time they met this new creature. Uh and it was just easy to use, which was a very big learning, which was like, okay, you make something easy to use
is very important, obviously. Um anyway, so then, you know, AI started really taking off. Uh and then early 2023, people started asking us for API
access to our search engine that they had used based on that Twitter announcement in November 2022. And
um and that's when we realized, oh wait, like you know, we could start serving the search engine to these not quite agents, cuz that was that wasn't a
term at the time, just these AI products, these AI workflows. Um and they're going to want comprehensiveness. They're going to want Um they're going
to want it to search in these more complex ways, all the ways that we as nerds in 2021 wanted it to
search. Agents were very similar. So, that was another interesting realization. It was like >> Yeah. >> Like we're not normal I you know, I'm not a
normal consumer. I want to get really deep into any topic, and so do agents. And so, it's cool that we were building a search engine for ourselves, it
ended up being the exact same search engine for agents or very similar. >> This is a big paradigm
shift. Even as investors, we're thinking like, hey, it's it's not humans who are deciding the dev tool that wins, it's actually agents. Um and uh
personal anecdote, I think I told you this one, but we have a family open claw, Michael Clawdberg, and we wanted to give it web access. And he was
like, I recommend Exa. Before we invest, I was like, sure, I'll go with whatever you recommend, right? And
so, it sounds like there's this nice dovetailing of how you were intending to build Exa in the first place to what agents want. But how do you think
about what agents want, right? Like that's sort of the holy grail right now of, hey, I don't care what the I'm using. My agent's going to select
Convex or Superbase. These are entire tailwinds that are making some of these companies. How do you, you
know, optimize for that, think about that? >> I've been thinking about this for a long time since there were the first agents, right? Like we were the
first search engine. Like we were we were an early search engine and they then the first AI products came to us because they were like, okay, they
could be a search API. Uh and so I've been thinking about this for a long time. Um and yeah, I think
the world of agents searching is just completely different from human searching. Uh you I guess you can make the analogy of like agents to humans is
like humans to sloths. Like I I I I I >> [laughter] >> Imagine we had a search engine for >> I'm picturing that Zootopia DVD cover. >> too. It's like
imagine we had a great search engine for sloths and then humans came around. Like they're not going
to want to use that same search engine. And so you should think of agents as these like crazy creatures that have like infinite like time that is
meaningless for them. They just want to like uh make complex queries very fast and like analyze it really fast and they want perfect output for their
human users, right? So you want to build a search engine for that. So how do you What do you What
matters for that type of uh creature? Well, lots of different things. So first of all, you need a search engine that can handle complex queries,
right? Like you do not want that creature to have to simplify its complex need for its user into simple keyword phrases cuz you're just losing
information. So you want to want something that could actually semantically handle uh complex queries, but also
handle keywords cuz sometimes you just literally want, hey, like I have this uh like complex chemical formula like I want that to be part of the
document, right? So you want to you want to a a tool that can handle both semantic queries, keyword queries, really just like expose all the
fundamental toggles to the agent. Cuz the agent by the way has the patience to like, you know, make a add a domain
filter here and a keyword filter there or like um search in this way, search in that way. So you want to like have a very controllable search engine.
You know how like [snorts] with Google like you search something and then you're like, no, wait, that's not what I want and then you try to change
some keywords and it's like it's just missing it. It's not like doesn't feel like very controllable,
togglable. You want the opposite for an agent cuz the agent is going to keep searching until it gets to its outcome. And you don't you don't want it
to have to make like a thousand like 10,000 keyword queries and still never get to its comprehensive phrase. You want it to like make a few queries
and get comprehensive information. Anyway, so complex queries toggleable, also like comprehensive
results. So this is a thing where it's like you don't an agent doesn't just want 10 results or 10 pieces of information. It wants everything cuz
imagine you're an investor. You don't have to imagine that. >> [laughter] >> If you're an investor and you're looking at biotech companies, you want
complete information cuz you're making very important monetary decisions and you don't want to miss
anything. You don't you don't want to have any a FOMO. Like any like you're missing some critical startup that exists that actually like reflects well
or badly on the current one you're thinking about. And so you want your agent to have complete information about every topic. So like with exile, like
you could search something and then get not just like 10 results or 100 results, but 1,000 results
or 10,000. And increasingly agents are wanting this. You also want like lower latency because like agents search faster than humans. But at the same
time you want higher latency because certain applications don't care about latency at all. So I think another big thing with serving agents is like
extreme customizability because like we're serving businesses, we're serving agents that are very
different. Some want super low latency, some want super high latency or latency doesn't matter to them. And so it's just a whole it's like hard to
express how different I have like a list of like 20 different ways like humans and agents are different. And when you just build it from scratch for
agents, you just make fundamentally different architectural decisions. >> So maybe just to go back to
this point you made on model intelligence improving and how that's kind of changed the game in search as well. And I want to pose this thought to you
that I'm sure you've heard before, but given the fact that model intelligence is, you know, getting better, it sort of can almost make up for or do
some of the heavy lifting in this user signal, right? Like Google has collected over 20 years for
PageRank etc. It can actually help get over that hump and do a pretty good job. And so my question to you I guess in that is how do you think about
this trade-off of compute latency, cost, right? There's all these trade-offs that have to happen in terms of what you're actually using to and you
know, to your point you said it's like a big optimization exercise. Like how do you think about what to
optimize? And I know you have different products, right? And so maybe your answer is like well, depends on the product offering you're doing, but
bring that into it as well. >> It's easier and harder to build a search engine for AI agents. So I'll explain that. >> Oh, interesting. >> [laughter]
I think that's interesting. Um okay, so why is it easier? Well, well first of all like this whole
click you know, Google has an insane amount of human click data. Just doesn't matter like that much for serving agents. I remember saying this like
years ago. People thought that was crazy. Um but it turns out to be right. Like human click data is great for humans. But if you when you want to find
results that humans click on, which is obvious, right? Like say if you get a huge amount of human
click data, you could train on that and now you know, you Google could understand what you mean even when you don't even know what you mean. However,
agents like just don't they don't benefit that much from click. I mean well, maybe a little bit in terms of like the ranking signal. It's also it's
valuable for agents to know what humans think is valuable. That's a very minor thing. Um >> [laughter]
so it's interesting that like all that click data that Google has accumulated does just doesn't really matter for agents and so it's a whole new
ball game. So there's not much of an advantage there. There are also other things like yeah, like Google probably had a you know, hundreds of people
working on re-ranking. Whereas you know, cuz that was a complex thing before LLMs, but now with LLMs
you could have a re-ranker that you just call an LLM and like you have one engineer work on it. Now obviously we have more than one engineer work on
re-rankers at this point because you don't want to just call an LLM, you want to train your own models and make them you know, faster, higher quality.
But you don't need a team of hundreds. You could do it with like a couple people. So like how are
how have we a team that you know, has always been below 100 people been able to build a search engine that's better than Google in all sorts of ways?
Um well, it's because like LLMs unlock the technology unlocks like new types of techniques. And then also like because serving agents like you don't
need all the click data and like yeah, data that Google been collecting. So those are some ways why
it's easier to build a search engine and why we've been able to build a fantastic search engine with a small team. I think it's small crack team. >>
[laughter] >> But it's also harder. Why is it harder? Well, it's because like the requirements for a search engine now are getting more and more
intense. And so like you know, I wake up in the morning and we have like you know, customers love us but
we still have customers being like wait, why can't you be perfect at this search or what about this search? And like they're constantly pushing us
towards the edge because we're serving business use cases that have like deeper and deeper needs like you know, billion dollar investments are on the
line. Like this has to be perfect. And that's great because it's pushing us towards perfect search. And
so like basically like if traditional search engines had like 99.9% quality or reliability, like these new search engines for agents need 99.99,
99.9999. And this is a great thing because it's just pushing us towards that dream of perfect search I've always been dreaming about. It's very
similar to like LLMs. Like you know, Opus 4.6 comes out, everyone's really happy this is working on 99.9% use
cases. And then when the next one version comes out, everyone wants that thing, right? Because the extra nines of quality are so important in this new
agentic economy. And so you have a similar thing for search. So it's both easier to build a search engine fast but really hard to build a perfect
search engine. >> Yeah, you know, it's interesting. When we were doing our diligence in the space which
you know, has taken place over the last few years now, some people came to us with the opinion that search is just getting commoditized. And I think
we look at that and you know, if you go out there and search for information we know is out there public etc., you can't find the answers to
everything still. So the fact that it's commoditized, you know, you'd have to kind of divide up oh, what type
of search is commoditized? So maybe I'll ask you that question. And then what is the type of search that's still really hard? And what is the key to
unlocking that? Is it data partnerships? Is it a technical breakthrough? Like how do you think about the edges as you as you mentioned in terms of you
know, perfecting or pushing forward the frontier of search? >> Yeah, I guess what will just
commoditize me means like over time will the thing will it just not matter which tool you use? Like they're all kind of the same. >> Mhm. >> argue
that the LLMs were going to get commoditized or are going to commoditize faster than searches. Uh and the reason is cuz you don't need to run like
mythos over every cell on your Excel sheet when you're trying to find competitors or something like that.
There'll be a lot like most of knowledge work does not require the smartest model. You act like you know, like just an open-source model that's big
enough. Uh you know, now the infrastructure for running them is very good. Like the it's pretty cheap. Like you can just use open-source models for
most of knowledge work. Not that the crazy smart LLMs don't they do have a super amount of value in
terms of like inventing new science and I I I and math and and and certain cases you want like the you know, find any bugs in your code or something
like that. But like increasingly like I would say like if you think of knowledge work, like all the different tasks you might do as like concentric
circles of difficulty >> Mhm. >> a huge amount of that service area is covered by like off-the-shelf
models you get right now. >> Yeah, very fair. >> right. So but like on the other hand like search like when you are trying to you know, enrich every
cell on your Excel sheet with competitors or people you're trying to recruit, then like every extra nine of quality in search really matters. It's
basically I would argue that a lot of knowledge work is actually search problem not only an intelligence
problem. >> Mhm. >> Um and so and so um Uh yeah, I mean what are some examples where search is not good? I do think company and people search is the
most like easy to see and just most valuable to people. Like every company in the world has to search over companies to sell to or almost every
company in the world has to search for companies to sell to and people to hire. >> Right. >> You could ask
yourself if like finding companies to sell to or finding people to hire is a solved problem. I think every company would say no, it's not. Uh that's
why people are constantly, you know, switching tools, like trying out new tools is because like we just don't have comprehensive information over all
the people or companies we want. So, that's a really good example. That's something Aexus is leaning
very deeply into, like go-to-market intelligence. Because we care a lot about it. It's very exciting. It's also very useful to use internally. We have
companies to sell to and we have people to hire. So, >> [laughter] >> it's been great to dogfood our own thing. It's an advantage. Um but yeah, that's
an example. I think you just want comprehensive like you just want all the all the people that are
that could be connected to you that are that are relevant. And and that By the way, it's a very beautiful thing. Like uh You didn't ask us, but I
think one beautiful thing about search is that a lot of important problems in the world are actually search problems like dressed up in a different
way. >> Say more. What's an example? >> Right. Okay, political polarization. I would argue that's a search
problem. Uh because there are people out there who want to Everyone wants to understand the world or most people want to understand the world, uh but
they're getting fed information that's just like, you know, misleading in some way or straight-up wrong. And if everyone had like information that was
accurate and like uh controllable and like comprehensive, I think all most reasonable people would
be reasonable. And I think uh because our information environment is so chaotic, it's so polarized, it's causing reasonable people to be unreasonable.
Um uh By the way, I'm part of this. Like I'm sure I have incorrect beliefs on all sorts of political things because my information is not perfect.
It's something I really want to solve. Um uh loneliness is a search problem. Weird. No one thinks it's
a lon- >> Say more. Yeah. >> Uh a lot of people are feeling lonely in in modern society. Well, it's because they're not finding Well, I not finding
people to hang out with or to be in relationships with, right? And so, like yeah, loneliness is a search problem and it's in especially in a city
life, like it's hard to find other people with similar interests or you might bond with. Um and this is
like, you know, with a perfect search engine with whatever information people are willing to share, it would help uh to find other people. So, for
example, I have a lot of crazy ideas about flying cars. I really want flying cars. I hate cars on the road. >> [laughter] >> I would love to like go
to a a group of people talking about flying cars. I'm sure I'd be great friends with those people. I
can't [laughter] just be like find me all the flying car enthusiasts in San Francisco. >> [gasps] >> I would love to attempt to go to do that. Okay,
and then and then this gets in your earlier part of your question is like what allows for this differentiation? So it's a long answer to how to search
for commodity. You can start to see that search is like a bigger thing that people don't People
think of search is like in the 1990s you see a text box, you type in a few keywords, you get a few things. Like that was search. No, like search is
way broader. Search is coordinating the human species around anything we're trying to do. How how does search become less and less commodity? Well,
it's it's always it's always about really good retrieval and really good data. So if you if you do
perfectly on both of those things, you have all the data and you have all the best possible retrieval, that that is perfect search. And so it is like
both accumulating all the web's data, you know, accumulating data that's not on the web, and then training extremely powerful models to search over
it. Those are Those have always been the two Index and retrieval have always been the two pushes at X
on the engineering side since 5 years ago. >> Can you say more about the data element and you know, obviously no need to share any secret sauce there,
but it does feel like the web is getting increasingly closed to some extent, parts of the web, right? Um but there's a lot of fear out there from data
providers on oh, we don't want to be Stack Overflowed, right? Especially if your business model is
around impressions you serve to humans visiting your site. Um I think the fear is the greatest among those business models. How do you think about
just sort of that interplay with data providers and um making sure you can get to the path of perfect search, but you know, work with these data
providers that are maybe becoming more closed? >> Yeah, so I want to get to the ideal world where you have
perfect search and then every everyone is incentivized to create amazing content even more so than before. I actually think there's an opportunity
here and I'll talk about how to create a system where like content providers are making more revenue uh because they are participating in this massive
agentic economy. Right? So, the what like I I like to think from first principles. Uh you know, the
first principle's idea here is that the agentic economy is going to be huge. Like basically like we're all going to have agents and those agents are
going to be participating, you can imagine, in this like agentic economy in cyberspace where basically they're doing like commerce, they're like
reading information, they're it's like everything we do on the internet, they're doing but like a a
thousand times bigger, right? So, there's a massive amount of value in this agentic economy meaning money. And um and so if there's so much value like
instead of like, you know, hundreds of billions of dollars going into one company, what if, you know, 50 billion dollars what a year went to one
company and the other 150 billion went to all the content providers, right? Like there's ways to
distribute the value in this new agentic economy that are um more favorable towards providers of content. Uh and like th- this is kind of how I think.
And by the way, it won't be 200 billion, it'll be a trillion dollars a year of value. So, there's if we could figure this out well, like there's an
opportunity for everyone to just like do amazingly. >> I loved all the use cases you talked about.
There's go-to-market, right? There's also the um you know, I'll put it in the loneliness bucket but finding people you can connect with. Um one that
you didn't talk about that I just want to pause on really quickly is coding. Can you share more about why web search makes coding agents more powerful
for this use case in particular um and why Exa such a perfect fit for the coding use case? >> Yeah,
yeah, for sure. So, every agent at some point are like humans. Like at any point when you're coding or when you humans used to code, you would have to
look up information, right? Because you want like the most recent uh technical documentation or you might look at a blog for inspiration. And so,
agents are very similar. Uh in particular, they really want the freshest information so that every uh
every line of code they write does not have some critical error. And you know, especially with coding agents like the stakes are so high that again
every extra line of quality matters. So, these coding agents are very intelligent right now uh but in terms of retrieval quality they've been in the
dark ages or the dark ages of like the early 2000s in terms of search quality but like search can be
way better over coding it you know we're talking technical documentation, we're talking SDKs just like as like perfect retrieval over the syncs. That
was our goal and so we're not yet perfect but we're extremely good at search over any sort of coding material and so yeah, so you know when you know a
bunch of coding agents try us like cognition for example, we've talked about >> Esther tried us
like we power Devin now and they've just found when they tested it that it just makes Devin way better way more accurate make way fewer mistakes which
really matters and this will just continue. By the way like you could also think about coding agents as just like every agent is going to want to do
everything and so it's not just searching over code it's also at some point searching over the
world's information just being up to date with the news because like if the coding agent will become your agent. It's like I think coding agents and
just general agents are going to like merge and it's an interesting trend but yeah. >> Right now I mean it's clearly Codex is not for developers right
it's sort of like the everything app to that point. I want to bring in mostly because it's in the
it's all over Twitter right now and I think it might be an interesting tie to eggs uh and that's sort of this topic of token maxing tokenomics just
the fact that you know Uber is talking about how they're spending too much service now hit their budget for the year already you know I think
Microsoft talked about pulling their cloud code licenses >> token apocalypse >> token apocalypse yeah okay
there we go yeah there and yeah exactly better word for it but we've talked about this before but in terms of how you think about search actually
making token consumption more effective and efficient. Can you share more perspective on that and to the extent you can like what are results that
you're seeing on that front? >> Yeah sure so like retrieval can help solve the token apocalypse Uh, because
like we should not be using gigantic models for every task. We should be using like and people are starting to realize this. Like you should use a
family of models of different sizes. The big model decides what to do and it dishes out commands to the small models. And those small models can be
way more accurate and reliable if they're using retrieval. So retrieval helps small models act like big
models in kind of in kind [laughter] of like in a cheap way. And so we do save our customers a huge amount of tokens because they can use smaller
models and use retrieval. Um, they can also we have all we care a lot about this. So we have like we've put a lot of research effort into how to, you
know, extract only the most relevant information from documents so that these models can just like not
have to consume too much tokens cuz like a lot of you know, any sort of input tokens can dramatically increase expense. So we get like we could save
like 20x on cost for customers compared to other providers by like being very efficient in like what information does the from the web does the agent
actually see. But yeah, in general smaller models using retrieval is much more efficient. And like
Andrej Karpathy had a tweet about that. I keep mentioning Andrej Karpathy on Twitter. I don't know why. He had >> [laughter] >> I know, we all do. >>
Yeah, he's great. He he had a tweet about this couple I think a couple years ago where it was like the trend is towards like small smaller raw
intelligence modules using tools. And that trend will like that's an important trend because like you have
you know, you have a limited like the cost of the model is determined by the number of weights. That that you know, determines the cost of the
inference. And and if those weights are if you're wasting those weights on like all sorts of information about the world like the capital France or
you know, this random blog that you read. Like you're just wasting tokens. Sorry, you're wasting a weights.
Those weights should be focused only on like intelligent processing. And you could you could probably get to models that are like 1 billion even less
than a billion parameters that are extremely hyper intelligent and completely unknowledgeable. >> Yeah. >> Like Einstein like who never saw the world.
That's kind of like the way to think about it. And then it uses tools that are very cheap and
efficient. And that that's a much more efficient world. That will help solve this like um this compute shortage that is affecting everybody. >> Do you
think um this is more of like a hot take moment, but do you think that reality will start to take place second half of 2026? Or cuz right now we're in
this phase where everyone's playing with the biggest best model that just arrived, and to your
point, probably overspending. So, when do you think this reality kind of sets in? >> I mean, the reality is definitely starting. I don't have the
exact I It's like trends are everywhere. >> Yeah, yeah. >> When does it become noticeable? Yeah, I I would say by end of 2026 it's very noticeable. >>
Wow, okay. Kind of a hot take, actually. Um So, uh you talked about um doing just the research that
you're doing. And you know, I think Exa you're sort of famously structured as more of a research lab, honestly, than you know, what people are calling
application layer companies, infrastructure companies, right? You know, this is sort of a research lab focused on search. Um and coming out of that,
one of the things that we were most excited about, frankly, is just the exciting cutting-edge work
that you guys are doing. Um one of those things was actually search as it pertains to RL. Um and there's a lot of efficiencies there, etc. Um I wanted
to just flag that because it was an interesting finding. I think you used Tinker, so shout out to Thinking Machines. Um but say more about what you
guys are finding there. And also just sort of what research directions you guys think about as being
important. >> Yeah, it's at a high level like a lot of the big ideas in training LLMs apply equally well to training search models. So, for example,
like we do pre-training of embedding models, we do post-training of embedding models. Uh we do RL on like search tools, right? So, like a lot of these
things that are working in LLMs work in retrieval, too, which is kind of interesting. And you don't
hear a lot of people talking about. Uh so, yeah, in that RL blog post we just we just basically try like a lot of people RL on a search tool, uh but
we haven't seen as much many studies like testing different search tools that you are on. >> Right. >> And so, we simply RL'd on SERP, so like uh
wrapping versus uh versus Exa and found that, you know, uh RL-ing on Exa does way better. Um like, it
both like uses uh fewer calls, so it's more efficient, and then it's like higher performance. Um and this makes sense because, again, like Exa was
designed for agents to use, and so like it just it just allows agents to make more complex queries, like, you really like capture what they actually
want as opposed to having like to compress what they want into like shorter phrases that are more for
traditional search engines. So, that was a cool blog post to explore and think it was really helpful for that. In general, our research is like the
bitter lesson, so just like scaling laws in lots of different directions, some of the ones I mentioned, post-training, pre-training, RL. We've been
pretty under the radar, like, I don't think uh people don't realize how much research we're doing. We
don't publish all of it, obviously, we don't publish much of it. Um but, there is a lot to go and search, and I I don't think people realize that. And
I think we realize it because we've just been obsessed with it, one, and it crazily enough, like, that's just it's what we're required. [laughter]
But, then also, I think the biggest thing is actually we're in uh because of our business model and
who we serve, we've just been pushed in all these crazy directions. Like, we're not serving 2 billion consumers who are kind of all the same, like, 2
billion humans. We're serving, you know, now, you know, over 5,000 businesses that are pushing us in all these crazy directions. Like, just like every
day it's like, why can't this be higher quality over companies or people? Like, why can't this be
faster? Like, why can't why can't the uh the information extraction be even higher quality? And so, like, just we're being pushed in all these crazy
ways. Um and that's why we're exploring all these research directions. Again, like, research always follows need. So, we have insane amounts of needs
at Exa uh to do better, and so that's why we do all this crazy research. >> Yeah. No, that makes
sense. Um so, I guess kind of tied to this is uh I wanted to ask you about how you think about benchmarks and hill climbing. Um and I'll say this sort
of tongue in cheek, but we've noticed that especially maybe among folks in your space, but also in other spaces, right? Um there seems to be like
benchmark maxing or whatever you want to call it. And and you know, of course, no to no surprise, um
everyone is always at the top of their own benchmarks. >> that's how you know it's not So something's wrong. That's how you know. >> [laughter] >>
Exactly. So um and obviously you have this relentless pursuit of ground truth. Um and also, you know, self-improvement, right? I think like you're the
first to admit, "Hey, there are areas we could be better on. We're trying to improve on a like
continuous basis." Um but how do you internally think about what benchmarks matter, what is ground truth for you guys in terms of like, "Hey, we're
actually better on this front, but worse on this front." >> Yeah, yeah, no, 100% the the evals have been benchmarked in retrieval. There aren't too
many evals in retrieval. So that's one problem. There's aren't that many standard third-party retrieval
evals and they've been like benchmarked and they're and they're not really actually good representations of agentic search. Uh like what agents
actually need. And so it is a problem in the industry where like customers can't really know what is true, which is sad. It also uh demonstrates the
need for a really good search engine to distinguish what is true. Uh >> [laughter] >> it's just another
example. Um but yeah, I mean, the ground truth for customers is their own AB tests. And so when we do a like right like literally they are testing us
for the versus other providers uh on their use case. And if they have enough data to do AB tests, that's the best. If they have their there's some
often they make their own evals. So sophisticated customers make their own evals. Super sophisticated
customers run AB tests. And then like customers who just want something might just like look at evals that are published online. Um but certainly for
the sophisticated customers when they test us, uh it becomes a lot clearer who's who's like on the top. Um but yeah, I I I we want to improve this
ecosystem. We want like we want to be like the research lab that like helps improve this ecosystem and
like publishes things. And even if we're not at the top, we want to show. And so you'll you'll see more coming out there. >> So you predicted that
agentic search will be a bigger business than Google search and by the 2030s. Say more about that. What are what trends are you seeing that lead you
to believe this? >> Yeah, I mean just you could get this from basically like estimating the number of
searches. So the number of LM calls and the number percentage of those LM calls that require search, then the cost of the search and then you just
like play it out. And the trend has actually been pretty clear and we have, you know, pitch decks from like years ago where we kind of predict where
things are going. You know, maybe we're off by a quarter here or there, but it's like pretty you know,
it it follows a trend. And so yeah, if you if you follow that trend even conservatively, you get to a massive TAM for agentic search in the 20 I mean
even before like late 2020s and then early 2030s. Basically like the number it's hard to express like how many searches will come from agents, right?
Like humans on average make, you know, a couple searches a day. But agents when everyone has a
personal assistant and every single software tool you use is going to be checking its work with retrieval. Like the number of searches is going to be
we say thousands cuz that is like understandable and grokkable to people, but really it's going to be millions. At some point. It's just going to be
like the world's be filled with search in a way that the world is filled with electricity. It's like
it's a fundamental infrastructure that powers everything. Like information I think less of search is like perfect information. So world will be filled
with like the highest quality information. And so yeah, I mean there is a lot of when the world is filled with something it's usually a a large TAM.
And and yeah, if you just play out the numbers we think it will be bigger than Google Ads in 20 30.
Not not that ads won't be a huge part of the world too. Like it's ads are important for commerce and that might be a percentage of the agentic search
economy, too. But yeah, it's just the numbers here are insane. And it really it comes down to a belief like do you think LLMs will eat the world? Will
eat all software? And like we have always believed that. It seems like very true. It seems even
more true every month. so um yeah. >> What do you think um and there's some debate on this on the LLM side on the on the training side, too, but what
do you think is the bottleneck today versus, let's say, 3 years from now. I feel like 5 years is too far to predict. Um is it Yeah, I think you've
said in the past it's no longer intelligence in terms of bottlenecking search. Like is it data,
accessible data? How do you think about how that evolves? >> Yeah, I mean, well, initially the bottleneck is going to be actually the infrastructure,
which is kind of interesting. No one realizes but like if you if you actually get, for example, 10x, 100x, 1,000x more searches than Google, you the
infrastructure to handle that is insanely large. It just hasn't been built yet. Uh so, we're really
excited to explore all sorts of cool new vector databases that have like super high throughput, for example. Uh things like that. So, that's an
interesting bottle like in the same way there's compute bottlenecks, there'll be like infrastructure bottlenecks. I mean, it'll be solved at some
point. Um Uh yes, and then like other bottlenecks are like data bottlenecks. Like so, agents are increasingly
going to want to ask questions about the world, and that data might not be on the web. It might not even be recorded anywhere. Uh and so, like uh you
know, there will be this trend towards how can we like accumulate all the world's data like literally unearth the world's data. Uh I'm I'm really
excited by this because the world is filled with information, and it's not all recorded. You know, the
history of humanity History of humanity is the world's been, you know, when we even were were hunters and gatherers, there's all sorts of information.
Then they started writing things down, and like the amount of information the world has been like skyrocketing. There's still so much information
that's not recorded. Um you know, you go from all the way from a clay The first clay tablet, or really
the first like uh paintings on on caves would be arguably the first time things were written down. And then like uh and then, you know, clay tablets,
and you know, now I I was in newspapers. And then obviously now we have like the digital age, but there's so much information in your head, uh like
satellite images like that are not just like in the world's soup of information that we could search
over. And like to fully understand the world, fully understand how crops crop yields are going to uh you know, affect you know, some company you know,
like or uh like what are people thinking about the world or how do we unite the world? Like like there requires like understanding the world at a
deeper level. So I think the bottleneck will be data in a lot of ways. And then once you have What the
problem is once you have all this data now the bottleneck will be retrieval. Like imagine like imagine it's just a crazy idea like imagine you know,
we're as we expand as a species like or we're we're thinking like very far in the future like think about like or in the solar system like there's so
many things going on in the world. There's so much data being accumulated. The retrieval over that is
very expensive. And so like they're actually really fundamental they're interesting fundamental questions here like right now the web is like let's
call it a trillion pages that don't matter. What happens if the web were a thousand times bigger like a quadrillion pages mean like everyone started
uploading data. Well then like any search algorithm that works over a trillion pages to no longer like
it might work over a quadrillion pages but it might be a thousand times more expensive and that's not practical because if anything we want search to
get cheaper not more expensive. So what kind of search algorithms will work over a quadrillion pages? These are fascinating questions that like I
don't know if I've ever heard anyone else talk about. >> [laughter] [gasps] >> Um well actually I was
going to say the at least the thinking about when we're living in you know, on Mars or whatnot. There's probably one other founder that has thought
about that extensively. Elon and you know, I listened to your when you went on the Latent Space podcast last year. One of my favorite podcasts and you
talked about actually working at SpaceX and I'm I'm kind of curious how was that experience and like
are there elements of Elon's leadership style that you've taken as you know, CEO of your own company? >> Yeah, well first of all the internship was
magical. So like for example I saw the first landing on the barge. Yeah and like I just outside mission control or like I'm getting tingles just
thinking about it like just seeing that like people coming together to do something magical for like that
was very inspiring and it made a big impact on me. And yeah, I mean in terms of my leadership style yeah, I I to think that I've incorporated some of
what I think are the best aspects of Elon. So, um and you know, for example, like he's very detail-oriented. Like he gets into details of everything.
And like for better or worse, like I do that too at X. Like everything from like the algorithms,
like the vector DB algorithm to like the office space and making sure like every part of the office is like really just like inspires and like
excites. And like shows our passion. And that goes across everything from our marketing to the engineering to to go to market. And and so that that's
exciting. Obviously it doesn't scale. So, one thing I've been learning as we scale the company is like what
details can I choose to go into? And so it's I like the eagle metaphor of like you know, I'm an eagle flying above the company. And then when I see
some detail that I think it should be important to fix, like I go dive down and go into it and then come back up. So, I think Elon has some of those
properties. Obviously Elon is like all in every day for what is it? Like decades. I've only been doing
it for 5 years, but I intend to do it for decades. One last thing that he does really well, which is like he's very good at memetic like names and
like inspiring through like memetic things. Like you know, like you realize that SpaceX's mission is not like improve rockets to get to or like make
rocket travel really fast like good. It's like it's like make humanity interplanetary. Like that's a
really good memetic thing. So, I think a lot about the names of like projects and like when I when I got I you know, I do a a team stand-up every
Monday in front of the whole company. Now we have like this like double floor office where people now are surrounded on the top floor. It's really
cool. It's like a stadium. And I give like a speech and I and I like to like simplify what we're doing into
like what is the core? What is the memetic core? And like come up with a cool name. And that inspires. By the way, a name is really important because
it's when you have a company of a certain size, people can are constantly communicating in ways you're not you're not part of those conversations. And
like the name like really like grounds the mission of that project. >> Yeah. >> This is really
important. Like I I I could think of a project a day of just about a name. Anyway. >> [laughter] >> Well, so I just totally divergent but on the topic
of names one I love goat. Talk a little bit about the symbolism of that and then I mean besides you know greatest of all time obviously but and then
also Exa. You shared before what Exa means but talk more about why you named the company Exa. >>
Yeah, yeah, sure. Okay, so the goat thing is uh >> [laughter] >> basically there was a coffee shop that opened up next door and so many people so like
the mayor of SF kept uh posting about it and so a lot of people go It's really cool coffee shop. Had coffee. Um and so many people were walking by
were like we got to like have them stop and look at what Exa is. So we have some posters about Exa on
the entrance to our office but we were like how do we get them to stop? And so I was like just put a goat I don't know why I thought this but just put
a goat there. >> [laughter] >> And I actually some part of me wanted a real goat. I don't know how we'd maintain the goat. I think the ROI would still
be valuable. But then instead we bought like a a nice like fake goat. >> Uh-huh, yeah, I love that
goat. >> instead and we put like the swag on him so it's like it's a beautiful yeah and I I I people stop by. So like we're actually like tons of
people stop by. Like it's so beautiful even on Saturday and Sunday which is like when the coffee shop is very popular like people come by and I I go
and I say hi to them I talk to them I tell them about Exa. It's actually very beautiful. We're actually
like a important stop for kids on their walk to school. Like you know how when you're a kid you had like those stops that you would like move at ice
cream store. Like the goat is a stop and they're always taking pictures. Very cute. >> Starting the recruiting funnel early. That's good. [laughter]
Yeah. But it's interesting how much those things matter. We've already hired we've hired people
because of the goat. Uh [laughter] anyway um Exa I think it's a great name. I love Exa. Uh one value of Exa is like it's a it's a great prefix. So
like Exa anything Exa data Exa this Exa that. Um but Exa means 10 to 18th which is in contrast to Google which is 10 to the 100th. And the idea there
was like you know we're kind of overwhelmed with information. Even though you could technically find a
lots of things on the web on Google that doesn't mean it's like organized and you get highest quality knowledge. So like >> Totally. >> And here it's
like 10 to the like, you know, 10 to the 18th is like you're you're extracting the most important information. It's still a very large amount of
information, 10 to the 18th, but it's not like overwhelming. Uh that was one of the ideas, yeah. >> Great
name. Uh we love the name as well and the goats. Uh my kids love the goats. Um Uh so, I I wanted to this topic feels a little bit over talked about,
but it's been in the news more recently, so I'm going to bring it up. Um which is I don't know if you saw uh Will Manidis wrote a uh an article
recently called grind slop. And it's almost this backlash of And in fact, I there's a I pulled a quote from
it. He said something about we're witnessing a phenomenon that masquerades as discipline, but represents perhaps the most extravagant squandering of
economic surplus in our civilization's history. Um so, extreme words, right? Um and he talked about um sort of all of these articles that are
glorifying the grind over what they're doing, you know, all these things that um you know, maybe are more
relevant to work-life balance, etc. Um So, I I guess I'll start with saying one of the things that I was actually most impressed and excited by when I
visited Exa at God, I think it was like 8:00 p.m. was that there were a ton of people in the office. Um and it wasn't just that they were there for
face time cuz obviously that's not what you and Jeff care about. Um but they were excited about what
they were working on. And I think really importantly, they were excited by who they were doing it with. Um And so, just say more about like how you
think about that culture of going all in and going after what you guys are, you know, sort of the mission that you just talked about and like what do
you think the important things to balance are in culture? >> Yeah, yeah. I mean, this is critical to
me and it should be critical to any company that wants to do great things. Like, you people need to be super excited about what they're doing and have
a ton of fun, right? So, like, you might have also heard at 8:00 p.m. like people are just laughing. We laugh We laugh a lot at Exa and it's like
almost too loud sometimes and it's like distracting. [laughter] So we have to have like headphones for
everybody. But um But um yeah, like people are it's very important people having fun and that's necessary for doing the best work of your life. And I
also I'd say like having fun just like just good vibes. But also I it's very important that people work on the projects that are most exciting to them
at any point. And so like people especially now with like you know all these AI tools like anyone
especially on engineering side or even on the go to market side or op side everyone can work on whatever they want. So for example like we had someone
who built the vector database who was like hey I want to go train models. I was like okay just go do it. And like he didn't really have experience in
training models but I was like you're really smart you'll learn it in like two weeks. And then
you'll just do amazing work and he's done amazing work. So right so like I make sure everyone's working on exactly what they want to work on and
luckily in search it's like at least in our space like in any direction we improve things like it will be good for the business. So it's like okay.
And it actually happens to turn out somehow that like the what people want to work on and what we need like
perfectly aligns. I don't it's like a magical thing I don't know how that's possible. But um yeah so everyone's working on exactly what they want to
work on. They're also working on multiple projects cuz like AI tools will enable you to be like productive in parallel. And so it's just and then it's
just what we're doing is just super cool like you know. There aren't like it's hard to find maybe
these days like really exciting like humongous projects and missions that other people aren't doing. And this one happens to be like organizing
world's information making perfect search. That's very exciting important for the world but also like really hard problem and engineers are really
excited about that. And on the go to market side it's like we're then selling this search to the whole world
and we're selling to some of the coolest companies and it's very exciting to them. They get to talk to like all these like hot companies and give them
and give them search that really helps their products. And so it's just an exciting space and but I it's very important to me so I hope we maintain
that forever. Um yeah. >> No absolutely. Well maybe just to end on what one last question and I'll
say first of all Exa's hiring and [clears throat] you told me once that you actually still interview every single person at X or who comes to you know
interview for X and so I'm curious what and you've you've attracted some of the most incredible talent. Um Junior, senior, everything in between, very
high slope, very experienced and I'm curious what it is that you look for when you do that last
final interview and um you know how you've been able to get these incredible people to come to X? >> Yeah. Uh I mean people might not like this word
but I look for passion. It's like someone just like a fire in their eye. Like do they really care about what we're doing or some aspect of doing like
I really want to scale this thing or I really want to sell this to everyone because why is that
important? It's It was important five years ago but I think it's especially important now because like the world goes to who is most passionate, who's
most agentic because like you can now do anything. Like you could be whatever you want. Like you know like with agentic tools you could literally like
do anything and what matters most is like how much do you care about the end result? >> Yeah. >>
Um and then like your judgment everything. And so like I I look for the fire in the eye because if you have that fire you could literally do anything.
It's It's It's actually crazy how meritocratic the world is becoming because of these agentic tools filling the gaps. >> Yeah, absolutely. >> Yeah. >>
Well, thank you so much, Will. This was great. Such a fun conversation. Appreciate you and very
excited to be partners. >> Awesome.