Hello and welcome. I'm Brandon and we're going to be talking about building the context engine AI agents need. You're working with a lot of agents and
those agents are doing things that you either don't want them to do or you're tired when they finish trying to do a code output and you go, "No,
Claude. You have made the wrong decisions and you have to keep correcting it." What we'll cover today is
I I like to count down. So, four myths to building a context engine, three hard-earned lessons from my team and doing this, two open-source tools that
we have that are components of a context engine, so you could just take them and run with it if you want to build some of these things internally, and
then I wrote one live demo, but you might get a recorded one depending on the internet. But first,
like some framing. Not long ago, you were the context engine. What I mean by that is every time you had to write code, of course, you thought about
it. So, how did you build context? Day one, you showed up at work and you probably graduated, you've been working on stuff, but you didn't know how
this company worked. So, you did stuff, a lot of stuff, got mentored, you went to planning, you wrote
code, it got rejected, you wrote it again, it got accepted, and all this stuff just built your internal instincts on like how work runs here. And now,
you're really good at your job because you ask good questions and you gathered accurate context. The reality is every new agent you spawn in a
terminal or spin up in Claude's is you on day one. Bright-eyed, you're very technically capable, but you
don't know anything about this place or how it works. So, what we have to do is give these agents context at a speed that's acceptable for us cuz we
got to get stuff done faster than ever before. And the way that we do that is through a context engine. This chart is something that we show almost
every company we work with and it's basically how organizations are adopting AI as we see it over eight
stages adapted from the engineer Bas El Das' work. But in short, on the far left, as you can see underpinning, you are the context engine in these
first AI eras where we had GPT-3.5 doing next token prediction or these types of things. Remember, what happened is you saw that tab complete, you
thought about it and went, "Nah, it's not good code." You hit escape. Sometimes you hit tab and you're
happy. As we moved into more agents, obviously we had the reckoning of Claude Opus in December. We were able to do more techniques and what happens is
teams basically unlock the next level by, you know, doing each of these moments at the bottom. In this phase, there's curated context which I'll talk
about and then finally in our dreamy case of having agents work overnight while we sleep and waking
up to useful PR's, that requires a context engine. What that is that ability for your agent to ask questions of a system that knows all of the context
at your business and can answer all the questions the agent needs as it's running and trying to accomplish the task it's been given. And today, right
now, it's it's you. You're probably the bottleneck where you have to manually get each paper or
manually answer the questions. So, with an engine, this moves faster. So, first on that chart, there's three places that we see major stall outs. The
first is this idea of curated context. A lot of people are doing this in personal productivity. You might have Obsidian or you have a Git repo with a
file tree that's markdown files of stuff that you're working on and you run that locally. That's
great for you, but as soon as you scale to a team, if you make a shared repo available with curated context, again, great, but we all have written
docs before. The moment you put that up there, it starts getting stale and it starts rotting. So, as long as your team's reliant on this curated
static context source, somebody has to maintain it, and that somebody better have the right judgment for the
entire company because if that's the context layer, it's got to be correct. So, this is a real bad trap cuz what happens is you start growing
maintenance on like a static layer instead of just doing more work. Another is the MCP plateau. A lot of people think if I just connect a lot of, you
know, pipes essentially to different data stores, I'm going to get context. But, what happens is the agent
suffers from satisfaction of search. It's a known bio Sorry, it's a known problem from radiology because the bias is when you look for a problem in
radiology, often what would happen is they'd find what looks like a tumor on a scan, they'd stop and say I found the problem. The issue is if you
don't do an exhaustive search, you may have actually missed the root cause. And agents suffer from this.
If they call the Notion MCP to find a paper on Zendesk, and they find the first page that says Zendesk integration, they think they found the answer
and they move forward in implementation. The problem is that may be a 2-year-old document, it may not be correct, the wrong team may have written it.
All these issues basically arise. So, a context engine is able to search over all of the knowledge.
It should be in a way that it can entity relate every single piece so it can relate a Notion doc to a pull request, to a Jira ID, and actually reason
across that in order to give an answer. And then finally, the background agent wall. You have to get context right and dynamically be able to give it
to the agent at its runtime. So, this means connecting typically through API, CLI, or MCP, which are
ways that Unlocked is available, but if you build these internally, you'll want to make those ingress and egress points available as well to your
entire team. Basically, what I'm trying to instill in you is the cost of bad context compounds with every step towards autonomy. So, at the beginning
in tab complete, you saw it was wrong, no problem, you move on. There's not a lot of cost. But, as you
move forward, you're going to hit these doom loops where the agent does something wrong and you correct it. That's a huge waste of your wall clock
time. And finally, as you move forward, basically, if you have overnight running, you just burned a ton of tokens if it didn't know what it was
supposed to be doing or how the shape of your code base works or operates. As our boy Kaparthy said, the gap
is no longer intelligence, it's context. I'm sure that's why we're all here at this conference. So, let's start showing you some fun things. The
problem is access to information does not mean understanding. When you use one of these systems to provide context to your agents or you and your
team, you need to deliver understanding in a tightly scoped token optimized way to not bloat your context
window with useless information cuz then agents may make incorrect choices. So, what's important is again, you have an engine that can reason, it can
decide over here, and then it sends the answer back to the agent that's performing the work that's scoped and precise and dense with high signal. In
this case, I'll show you an outcome, but I ran a task. This was the actual task. And I ran it without
a context engine. And I ran it with our context engine. And to give you a little tease, what happened is without the engine, it passed all our checks,
the code passed, but when our engineer looked at it, it failed miserably. It would have broken silent callers, it would have taken the system down.
So, you'll see the outcome of that later on in the talk. But now, four myths about building context
for agents. People think a naive rag over your context or your docs is an engine, it is not. You need a lot more than that to make it dynamic, to make
it real time, and to make it intelligent and useful. If you just connect enough MCPs, I'm done. Unfortunately not. The satisfaction of search and also
the agent makes the choice of what to call is not a good place to be in because it introduces more
probabilism into an already probabilistic LLM. You don't want that. You want to put it on rails and make sure it finds what it needs. A bigger context
window. We're grateful for a million tokens, but the bigger it gets does not mean if you whack all the data in there, it's going to do the right
thing. Usually that inflates it with a bunch of noise and it does the wrong thing. So, giant context
windows will not save us here. And then finally, if I just put a bunch of rules, that's going to fix it, right? No. How many of you have a Claude MD
that says you must do this and Claude does not do it? Yeah, exactly. Basically, what I'm saying is what your agent can't see is drastically important
to it. So, it knows code that compiles. It's syntactically genius, but underneath there's intent to
these things. Your team wants to move forward. You have a road map. You're going somewhere with your system in order to serve your customers. If
you're just looking at source, that's the what's running today, but it's not where you want to be in the future. And all that information is super
useful for agents when writing code cuz they'll write the right code. They'll achieve the intent of the org.
They'll achieve the business logic. They know the state machine. There's just so much more there. So, why you need a context engine is A, it needs to
understand who you are and where you work in the system because that's a key pivot point in graph traversal to understand that when I say I want to do
a Zendesk implementation, it knows who I am, what team I work on, and what would be relevant for me
as compared to perhaps Susan in my customer service team who's also asking the same question. She has different needs and is probably trying to
implement something differently. It must resolve conflicts. All the time in our system, someone says, "How do I implement Zendesk?" And we find an
architecture diagram that says do A and we find a Slack thread that says do B. Well, who's right? So, the
engine must be able to settle a conflict. One technique is that personalization, so who's who? If the Slack thread is between the CTO and the chief
architect saying do B, but the source code is doing A, the source code's probably wrong. Like, sorry. We'll check a freshness, of course, indicator.
We'll do a bunch of other things, but what will happen is you'll see the Unblocked AI code review bot
would reply to a PR that was open that said we're going to do B and say, "Hey, there's a Slack thread over here saying do A. So, the intent of the PR
is wrong, not just the syntactical accuracy or like is there a bug or a feature or a whatever." Pass respect permissions and governance, so we have
all the security theater you'd expect. And then, of course, deliver the right context to the right
model at the right time. Very important to be able to run it for long-running agents and to do again these overnight tasks or to have stuff fully run
in the cloud and not think about it. This is how our context engine works in a basic diagram. I'll speak to it briefly and then I'll zoom in on those
bottom six boxes, so you'll get a little more data. But in short, on the far left, we use good
old-fashioned APIs and programmatic connections in order to build up our actual ingestion of knowledge. So, you connect all the places your team works
today. You do not change how you work. I want to be very clear. Our belief is that you should not have to. You're already writing stuff in Slack.
You're already writing stuff in Notion, wherever else you do. Just put it there. We ingest all that. We
build our embeddings vectors. We construct many graphs that we use for that graph traversal to give you the right answer. And that all goes into the
box. And then, when you connect to us via your agents or yourself, we have many surface areas, including you can just chat with us in Slack or desktop
app. You can also, of course, use our code review and then our CLI API and MCP to have your coding
agent get all this sweet data. And then, we output those answers when you ask a query. At the bottom, these are the six things that are key, like I
mentioned before, but you need to have unified system context in your engine, because that allows it again to look at data across the entire knowledge
base. You want to be able to relate that Slack thread to that PR. This is the critical component, and
you can only do that if the engine is able to see both sets of data and basically open the links that are all connected in order to bring it together
in order to do a proper assessment of is this valuable data for the task I've been given? And that way the agent can make that call and move forward.
Conflict resolution, again as mentioned, one of our customers has 115,000 repositories and 30,000
developers. The amount of conflicts in their data set is unbelievable. So, you must be able to resolve that to get the best answer, so the agent again
can move forward. Targeted retrieval. With our tool in our MCP, we have targeted links, so you can give a direct link. We'd be able to pull that,
unfurl it, give the full context, and off you go. So, it's not just about these large research packets
and data sets, it's also about targeted and surgical retrieval in an engine as well, in case you have that data when you're running the task.
Governments, it's security, it's important. I hope you didn't use Delve. Token optimization. What's important there is when that response comes back,
you want to make sure that you scope it, as I mentioned again, you might have a ton of data, but you don't
want to put into the context window of the agent that's actually doing the work, useless information. So, the engine needs to take all that data at
the end and then compress it down as best it can to basically using the least amount of tokens to give the most amount of signal to the agent that's
running the job. And then finally that personalized relevance, which I won't bore you with again. So,
what you can have today from us, which I'll demo now, are two open-source tools. One I will show is our social graph builder. This is the one that we
use to construct the personalization graph that I mentioned and pivot against for using just your GitHub data. So, you'll be able to basically pull it
down, point it at a Git repo, and it will generate a graph for you. The other is a tool called our
repo rules agent, which we've open sourced. What it does is you point it at a repo, and it finds every single quad MD, agents MD, or any other method
of that, looks at all the rules encoded in those, assesses them, dedupes them, and also warns you about which ones you're saying are severe, you must
do or not. Because when you run these things again with a context engine, those are great insights
to how the team wants the system to work and wants the agents to behave. So, it's really important to ingest that type of data for your engine. And
then I'll show you we have our open core for our skills that come with Unblocked show you how to use an engine, and we have a cookbook that has really
varied recipes that you can take a peek at and see how people get leverage out of our system to run
again in CI, cloud, background, overnight, whatever you need. This is the social comment network. So, this graph, this is that open source tool. I ran
it on our actual repository. We have a dev called Rasheen the machine. As you can see, the size of his circle is he ships a lot. So, if we look at
Rasheen, you can see who he most often interacts with on the right-hand side, as well as obviously the
graph. And what this teaches us is if Rasheen, for example, says something on a PR that's related to integrations and ingestion, he is an expert. So,
the relevancy of his words are more important in this area of the codebase. This is useful for a good context engine. Just to flip through, you'll
also find experts graph. This is another key piece of data, as I mentioned, who the experts are in your
company is a great way to pivot that table to confirm and to do conflict resolution. So, I'll scroll down. For example, Matthew owns our front end. We
actually have a bus factor risk here, which is why it says one. But if he talks about front-end changes, we treat that with much more scrutiny, and we
take it more seriously. You'll find other fun things as part of this tool, like where your teams
are, what your peer tables are, and who reviews whose PRs. But this tool is fun for you if you want to take a look and use it in your own systems. Our
other tool I have running in an actual Claude code session. But this is an example. I ran our repo rules agent, and I got it to just show me a subset
of some of our rules, so you can see its structure. These are musts. These are the types of calls
that we're asking for. This is what it's for, the language we use it against, and where it sources. But basically, having this table again, you can
assess the rules that your team has put in. You can deconflict them. You can get rid of dupes. And then you can also use this again in a context
engine to make sure it behaves how you and the teams want it to behave. So, these are two really useful
components of building your own context engine that we wanted to give away in case some of you'd like to experiment. If you don't want to experiment
and skip to winning, call us at Unblocked. Another example of actually using this in production, it looks like the internet's not here, so I'm using
pre-recorded sessions. I apologize. But what I've done here is using a skill called Ask Unblocked. I
asked a question about the Source Mark engine, which is a service in our actual code base. What happens is then Claude code is able to just call our
MCP, get the data it needs, and then give me an answer. It tried to draw a mermaid diagram, but Claude decided not to render it. You can see it gives
a bunch of optimizations, but what's more important, because you don't know my source code, is you
can see the sources of where all that information came from in order for the agent to construct this answer, which includes, if I hyperlink here, a
Slack conversation with the chief architect about the system, as well as, of course, prior PRs and other prior I've also used this for things like
doing our planning. I mentioned Zendesk 20 times. You can see we use the Unblock research command skill
to figure out how to do an implementation for Zendesk. For us, again, we do a lot of programmatic work to do all the suggestion to get a useful uh
tool for you. So, the amount of work that it takes to do a new integration is quite high. In this one run, this is the quality of a plan you would get
when you have something like a context engine. It actually finds everything without you having to,
again, babysit that agent to get that first plan, correct it, tell it where it should have gone, give it more data. It just figures it out. So, what
happens from here is if you have rich, hydrated context, the quality of that agent starts here, and then think like a waterfall, every choice it makes
when it has the right data, it's going to make better next choices. Because if it gets the right
data up front, when it sends background agents to correct the code base and read files, it sends them to the right place. So, now it's getting the
right data about your patterns, and it continues to make better choices. So, ultimately, the output you get is not only correct in terms of quality,
but it happens faster, often 80% faster in our controlled tests in terms of human wall clock time, and
typically 50% less token cost to get the job done.
Sorry, we'll also show you that cookbook cuz I promised. So, the skills repo is Unblock {slash} skills. You can read about the very specific skills
that we've designed that come with our engine. Again, just take a peek, they're open core, and maybe adapt your own. And then in the cookbook, this
explicitly goes through very rich recipes, if you want, for human in the loop, but most people use our
headless ones because they put Unblock as a context engine inside of CI runners. So, when a customer reports a bug, we can actually validate that bug
is real in real time because it that CI runner that looks at that bug report from a customer is spun up in a environment that also has access to our
MCP and it uses that to gather data to confirm that the bug the customer has told us is real before
moving it to an issue backlog. I'm sure we've all been there where a customer said something isn't working and unfortunately it was just a skill
issue. This is a great way to prevent your team from thrashing on things they don't need. We also connect to DataDog and Sentry. So of course we have
incident investigations, evidence gathering and other tools that we use for incident response. So those
are all Gibbs, the two open source tools and of course two open core skills and cookbooks for you to learn from and to see how it engine can serve
you. Remember earlier I talked about a task that I gave Coda or sorry Claude code to do. This is the output and full honesty what happened is I took
the output of both. So a run where Claude code did the whole work by itself with MCPs and then one where
it had Unblock and then I gave both outputs to Claude and said write me a report. This is what it came out with. What matters here is with Unblock it
was 10 million tokens to get the job done. Without Unblock it cost 21 million tokens and we got something that failed our quality bars across the
board. It literally broke stuff. If that had gone to main it would have taken down our entire services
is the short story. You don't want that. So giving a context engine makes it happen faster. The time it took for this with Unblock was 25 minutes.
Without was 2 hours and 32 minutes because it kept doing things an engineer sat down and went that's dumb and corrected it and corrected it and
corrected it until it came to an outcome that we were looking for. These are our hard lessons. We first
optimized for access not understanding. In order to serve our customers we quickly started just connecting to more data sources and giving more pipes
to our agent to try to do stuff. And that's when we found that known bias of satisfaction of search. We also used to hide conflicts. At the beginning,
trying to settle which is right and which is not is very hard. So, until we had good signal data,
we just figured, "Oh, well, I mean, if there's conflicting docs, maybe it's a 50/50 coin flip, serve the document." No, that uh that causes bad
outcomes. So, don't do that. Uh it's important to have source de-conflation and techniques to do that in your engine. And then finally, we thought
it'd be really cool for latency for our customers to cache answers that were correct. We have a QA product
you can use in Slack or Microsoft Teams that most of our engineering orgs that buy our product give to their sales, customer success, and other
support teams, so they can just ask unblocked questions about what's happening. So, they would then say, "This is a great answer." We'd cache it, and
then we'd return it. The reality is that's a bad idea because if you cache an answer, it's also now
immediately becoming stale. It's suffering from rot and decay, and though it was right today, your team just shipped 12 PRs, and now it's wrong. So,
it's important to compute those answers every time. This is where we see the most AI-forward teams leveraging context engines. Think of Stripe with
their minions, they have a context engine. Think of Ramp and Spect, they also have a context engine.
Many teams use this type of capability, and we are obviously trying to make it available to everyone. Agent planning, as shown, the higher quality
plan with the higher quality context hydration at the beginning leads to better outcomes. Ticket enrichment, if you have a large ticket backlog, it's
great to run an autonomous job over that to gather all the data from your context engine, and then
enrich that ticket with useful information cuz the team who picks up the ticket's going to be way happier with you. Triage operations, again, as
mentioned, there's tons of opportunity for you while an incident is going on to have background agents running, doing evidence gathering, checking
blast radiuses. And if they have access to a context engine, it's incredible what they can find. We use it
in every single one. There are three more. Incident management, again, very clear. Code review, I both use Unblocked in my local code reviews before I
push to remote. And then we also have Unblocked as a native integration via GitHub or GitLab to run code reviews for you. So, I review it locally with
Unblocked, and then I also get reviewed again at remote by Unblocked. It's great, feels good. And
then and finally, as mentioned, customer success and sales, as well as product teams, they all constantly need to know what's been happening in the
product. And I'm sure as all of your velocities have increased in terms of delivering new capabilities, it's hard for those teams to keep up. It's
hard for them to know what's running in the product, what's real. So, giving them a context engine they
can just chat with leaves A, the burden off of your developers that question, so you can keep shipping, but B, typically gives them the best
understanding because an engine can see everything. And often as a human, I know what I know, but I don't know what I don't know. So, there's lots
going on in a business I may be unaware of. So, an engine gives a more complete response typically compared to
anyone else on our team. Maybe the CEO knows better. Sorry, Dennis. How do teams make it their own? They build skills on top of it. Again, we have
reference material for you. They devise workflows to just automate running this headlessly or again in cloud or background agent capabilities. Our
cookbook will help show you some of those. And then they use those headless recipes to move towards having
parallel agents running locally with them because you have less babysitting to do to carry and feed for each session. And of course, to move you into
full cloud. It's better if you could just run it over there, let it do its thing, and it can get to the source of truth, whatever it needs. Basically,
the whole point of this is AI-generated code should feel like it was written by someone who has
been on your team for years. By using experts graphs, by distilling who knows what, by being able to see everything at the same time and reason about
it, you get better outcomes, and every PR will look like your best engineer on that part of your system wrote it.