um more recently he's been working in two main areas one in resource efficient learning for large Foundation models and the other one from marrying uh
deep models with uh traditional symbolic decision making which he'll be talking about today so please take it away okay cool cool yeah thanks for
introducing me uh my name is I'm a resarch scientist from meta AI far uh so I've been working on many
different directions uh this morning I actually give a talk in CCH uh in Jas class about understanding Transformers understanding new networks and uh
we have a understanding to basically to actually improve the existing systems for training efficiency that part was not covered uh because of time
limitations and the second uh component that I'm working on is the decision making so how can we
leverage de models uh in order to make decision making much more efficient and how can we uh make AI guided design much more uh reliable and useful uh
in many different scenario right so I think this today I'm going to talk a little bit about the second part uh so let's start from there so we all
know that L model has been taking over the world uh so have been used in many different scenarios
since 20 22 23 right so people start to use it for com generation a agent a compil summarization all different scenarios and uh and I think just today
m i released Lama 3 uh I and I strongly encourage everyone to use it uh it's very strong and uh and also very small as well and has AE vocabulary size
uh and can be um used in the different scenarios right so uh everything sounds very good right so
and then people may ask about the question what model cannot do well yet right so uh people actually ask about this question because we want to make
the model stronger in the future people also want to ask about problem because we want to understand what's going on inside larer models right so
there's a lot of different questions we really want to think about so here I give you one example about
uh the task called travel planning uh so this example actually uh was brought about by one of our previous uh paper called travel planner this a joint
collaboration uh with Ohio State University and in that paper we actually uh find there's one Benchmark uh we construct so that the uh even the most
art model cannot do very well so the task is very simple right so it's basically we do travel
planning uh we ask the model to do some kind of planning in the following right I'm going to maybe Hawaii different places for two days and I have a
budget of $1,000 I want to stay there for three days and I have some preferences in restaurant and also hotels and please give me an optimal itinerary
so that's the question that usually can ask so then what happened here is that for this paper uh we
have um uh follow this kind of paradig we start with some like two use right so we call different tools to get information necessary and after that
use the information to do planning and also to do search and finally there some plan the plan is to say I have to be at some places day and then take
a fly to somewh else today too Etc right so that's the plan surprisingly if you want to use SL model
to do that uh it actually give you pretty bad performance right so I we tested like two different kind of scenarios one scenario is that okay I just
like we just like L model to do everything for us right so we what we do is like we ask model to use the uh API we provided um to to call all these
informations and then use the information to plan the travel in this situations you're going to see
that the gb4 two turble is going to give you 0.6 percentage of accuracy as it's basically uh very small it's less than 1% right so in many cases it
doesn't really well it doesn't do very well um and uh there's another uh way of doing evalution it's called so planning in which uh we already provide
all the ground Truth uh to use and uh which means that the ground truth to use give you all the
information that you need you order to plan optimally and then uh we ask them all to BR the travel right so this actually give you uh slightly higher
numbers which is 4.4% but still are very low compared to humans right so this actually give you this kind of like idea that still we cannot do uh good
solutions that satisfy many constraint yet right so I think usually one simpon that uh you actually
see is that model will try to find a solution uh that is good for one of the constraint but not the others right so we will forget about oh I should
satisfy budgets and satisfy the date and satisfy for example the hotel INF preferences Etc so it's actually hard problem to deal with it's also a very
hard problem for uh for reasoning as well right so another example I will give you is that uh this
kind of a very simple example and people often use in Twitter x uh to discuss about limitations uh this is about like how can language model solve the
Tic Tac Toe problem Tik Tac Toe we all know it's a very simple game and uh people play stones in this 3x3 grid and uh whoever get the first three uh
ston line up in either horizontal diagonal and vertical wings but unfortunately if you play this
with car three oppus which is BAS start out approach it doesn't really do uh very well right so basically for example this one it actually realize it
seems like this is a draw actually it's not jaw uh because given the uh bar where you can see I'm going to X on the top the bottom left corner I going
to win the game I but for unfortunately the representation uh in the uh in the El model does not
really see that and you see it's a jaw right so it's actually a big problems right so uh and also for the example to the right uh you also will see
similar uh situations right so uh it's U realize that the model think okay I place an Old Stone to the left in order to block attack but the attack is
not happening on the leftmost uh vertical uh lines but on the middle lines right so you can still win
the game by PR stone at the bottom middle line so uh it's very clear that still the St model cannot do things very well okay so uh then like how we
can deal with that so is there a way we can make the model stronger and more efficient so there are several different ways of doing that so in this uh
lecture I'm going to talk about three different parts uh so the first part is about uh how can we
leverage existing solers and using this server to provide data and use data to train the model so that the model become stronger right and the second
is that uh maybe we can actually train a system that is end to end differentiable combining solers and D models so that the system may become very
capable in uh having uh good solutions that by all the constraints there and finally there's another
case uh that we can actually find some analytical structure of the Sol solution and we can use deep models to uh simulate and also to uh to basically
fit this analytical solution so that it will give you a better solution uh and in VI of the solvers so I'm going to talk three different parts uh in
this presentation and uh and let's go one by one okay let's start with the first part uh which is the
better Transformers with data from solers and traditional solware actually did very well in different tasks right so for example the tasks uh with
maze navigation and also with soang uh what people usually do is that um forget about reinforcement for everything like that we just um uh just check
like what's the uh check Bic put everything into uh kind of symbolic form and you call the source and
the S will automatically give you a pass right and that pass will give you Optimal Solutions using for example a star or different kind of planning
algorithms right so very simple so you don't need any knowledge on reinforcement on like lar models so on so task the goal is here is to uh actually
put uh the two boxes into the two destinations right so uh but you can only push the box but you cannot
pull it once the Box get into trapped into a corner then you're dead you have to bories and you find what is the best way to put all the boxes into
that destinition so for this task you can also uh find a way uh using solvers to uh to solve these problems of course like the more the larger the
problem is the complicated it would be right so it also depends on like how you're going to write
theistic functions in order to make the uh solver much more efficient right so this for this task usually for Transformers it does not work that well
uh but in our paper we actually come up with an approach uh that uh can actually increase the uh import performance of Transformers uh substantially
and also reduce the data usage right by using the trajectory or the search dynamics that is being
created by the sers okay so let's start with the very basic stuff right so let's say what how The solware Works uh in this kind of scenarios right so
first of all uh you might start with this kind of very simple cases uh you are you are uh starting point and also goals right so and then you may have
two Wars right the war is one is in 21 and the other is in zero2 and your goal is to find where is
the optimal pass from the starting point all the way to the goal so uh if you call the sers uh you actually will have um uh a few passs that you can
uh yeah so you can you can basically do right so you start from uh you start from this starting point and then you create uh States the state is 02
which is starting here and then uh you close this uh the text the states and then it will actually goes
to uh one uh one zero right so uh from starting from here you can expand your tree uh all over the place you can expand to the left to the right or
you can expand to the do for the botton if you expand to the right because there's a wall okay you know that your expansion fails you won't expand
there but you can expand downwards so it will creat another um another uh notes right so you can also
close that notes and keep expanding so here you can expand two different directions right so expand to the right or expand to the bottom and this will
create two different uh uh States uh and then you can continue doing that until you find uh you find that one expansion actually leads to the goal
right so what you find the one expansion Le you to go then when you can do you can do back tracing uh
you actually will find Optimal pass from that right so basically uh this is optimal pass and this entire procedure is called the search Dynamics uh so
you basically follow this bread first or whatever characteristics in order to create a sequence of nodes until you actually find definations so it's
very clear and uh what we actually do here is that while the server can and give you this sequence
of trace and then optimal plan uh we can actually train a Transformer model and transform model will basically take uh The Prompt which is basically
the description of the language of description of the environment right so T is the description of enironment this basically says oh it start from the
starting point is here 0 two and the goal is one uh one zero or Z one and the wall there has two
walls here and this is B description of envirment and the trace uh for this entire search and the final Optimum uh trajectories right so that's
basically the the idea and then uh you can basically train a model called search augmented models right so here basically there a comparison so if you
just do prompt and plan this is basically called a solution only models the model does not really go
through the trace in order to uh predict the optimum plan but the model basically take the prompt which is the description of the environment all the
way to the plan without any intermediate uh tokens right so this is called the solution only models and the other is called search argumented models
in which uh when you are training an a regressive Transformers you start starting from this prompt
you first predict the trace and then given the prompt and Trace predict the plan okay so that's basically the idea and uh how you're going to uh do
this training we are actually using an encode decoder uh structures so we first use a prompt to encode and you get a representation of the prompt so
the prompt here basically represents the environment and uh once you have the representation of it and
then you use decoder which is also Auto regressive models to plan recursively until uh you get uh either the plan in the search only model or the
trace and the plan in the search augmented models so that's basically the way we are training the models and uh so this basically give you like how
many tokens you will con to have so plan only uh it only will occupy 100 to 400 tokens but if you do trace
and plan usually this will be much longer um because the trace is often taking a long time to generate right so um but interestingly uh you actually
see there actually a interesting curve this is kind of scaling curve right so our search Orting model is much more data efficient and also parameter
efficient here um if you just use the search only model uh with 175 million parameters uh the curve
look like that right the performance is really bad at the beginning because you don't have enough training sequence you really get to almost close to
100% of a 30 x 30 m task when uh the number of training sequence reach 1 million okay that's the Baseline however if you try search of augmented
models even with just 15 million parameters you are able to uh get very high performance even with very
few amount of data right so it basically starting from here and then of course the model will become better uh when uh the uh when the data and they
become like a larger right so that's a that's an interesting uh Behavior here right so uh it's and also it doesn't really matter like what are how
many parameters we have so you have like a large parameters and still performance are reasonable good uh
but um if you compare this with the search only model of course it's more data efficient and at the same time it's also par efficient as well right so
in some sense you can be regarded as kind of advanced version of chain of ss right so uh question sorry I'm a little bit lost here so when you're
doing a you got to provide that each also consistent Adis are you providing theistic over here or is
like Transformer learning theistic from planning or creating or are you giving theistic to yeah so here we basically train the model with AAR
trajectory so AAR trajectory is created by a specific herti okay and so that sameti is being fixed okay the is being fixed yeah so this is going to be
the first Next Step yes so very definitely can be important and we actually did some small experiment but
not systematics doesn't seem to be changing a
lot yeah I think that's some magic here I we don't really have a clear idea right so uh there's some performance job uh maybe this because there need
to some tuning in term of the parameters you know to get things better but we are using the same parameters here yeah okay uh if there's no question
then we can continue uh so and on soan we actually get the very similar uh curves right so uh if you
start from the small models um the small models uh with search augmented uh you actually get much better performance right so with search Only
Solution only model the performance is worse right so of course that's the uh that's the current Trend I question yeah so when you do this Trace roll
out the Transformers window can actually attend over the entire Trace yes exactly yeah so I was wondering
how you yeah we don't M times how many I think it's for like it should be three times yeah you can definitely do more I mean the reason why it's uh
it's not touching so this is a missing data point here is that if you have large parameters and also the contact window is huge then you might spend a
lot of time that yeah this actually the the cons of the approach yeah okay so now we know like okay
seems like these kind of models can do very well in term interior learning uh when we want to use few data in order to get optimal plans right so I
mean then the next question is how to go beyond right so we don't want to just do materi learning we want to do something that is more exciting than
that right so uh then we actually go to the second stage which is the fine tuning stage so in ter of f
State what happen is that we really want to make sure that the model still can output optimal plan but you want to get Optimum plan with shorter Chace
right so basically you do fine tuning in this kind of loss with this kind of reward functions the re function basically tell you okay the shorter the
trace is the better it will be when you train models so this is actually called a search forer uh
that's an improvement of the augmented models which is basically by imitation so uh it's interesting to see that the search form can actually do uh
even better than the search argumented models right so we can basically do this search former in multiple itations right so for every situation we are
doing called X expert situation so we use the current sear model to generate Auto traces and also
their plans and pick those traces and plan in which the plans are optimal but the traces are shorter than uh than basically the baselines right so the
reason why we can do that is because the search augumented model is train on nondeterministic version of a star and so that every time the trory is
different but it will lead to Optimum plans and second the Circ op model can be generated in a
reversal Manner and by varing the temperature you will get different traces uh at different Generations right and for this different traces you pick
one that shorter but still give you option plan and use this amount of data to retrain or to fine tune the search forer uh for several steps and
you're going to see that uh after several step of training the average sequence length uh per test Tas
actually get shrinked right so more and more and more so that's that's actually good so this mean that it actually find some algorithms that is
shorter in term of is in term of the trajectories uh in order to get the optimum plan right so that's that's a good thing it means that the actually
find something uh that is uh new than a search any questions can you train like aw or something like which
can exact you the symbolic representation of this kind of STI is kind of better than this a star fixed stick yes yes yes so that's the and this is
some like some numbers right so um if you start with the search augmented models right this is basically the number of solutions uh number of
trajectories you can see this are solved so by Sol we mean that trory may not be optimal but it's feedable and
it's actually can solve the problems right optimal means that the actual output plan is optimal right so know that optimality score is less than 100%
because for all these Transformer based models there's no guarantee about optimality so of course there will be some probability that some of the uh
final plan is not optimal but that's fine uh as long as it's differentiable then we can use different
fine tune method to make them even better so you can see that after three steps of the uh the the fine tuning uh the solved the amount of solved cases
actually getting higher and the same time the optimal also getting higher a little bit so um and this is can you can see this is actually comparable
to a model that with solution only so uh this is in some sense pretty nice okay so uh and this is
Inon task and we can also test like whether uh the actual uh boost strapping uh will actually reduce the lens of the of the intermediate Trace right
so this is basically in order to measure that we Define a metric called improved lens ratio IR R so uh it's basically very simple B you have your
current ratio the current like search lens the search Dynamics lens in ter in term of tokens number
tokens divided by the Dynamics lens that is computed from AA so this number should be greater than one I mean if it's larger than one it means that we
getting uh we're getting like a more efficient search results right so I mean every equation you see like the the ratio become larger and larger which
means that we somehow find a way to reduce the search Trace at the same time we still get either
solve op solved plan Optimum so that's that's a good thing okay any questions yeah so in the previous slide you showed like that when you shorter the
paths it gets slightly worse is there some sort of pattern to the paths that it gets wrong like how close are they to the yeah so we actually have
some study on like the failure cases yeah right so in some cases the reason why it goes wrong is that
the transform seem to misremember some of the per specifications right say oh here is actually a war but actually find a p across the wall this kind
as inedible but there's some like ining issues uh in the representation yeah that is one that is one way you can you can see that yeah and in some
other cases it does not really give you optim solution because you think like a one block is actually
War but actually it's not war it's actually possible pass so by doing that you actually you won't be able to get the timate
uh so you want to predict the trade so that it becomes shorter but at the same time you want the final optimal plan is still optimal final plan is still op
uh so for this one we don't have any uh specific form you want to follow you want the tra to follow you just want to be shorter that's that's that's a
goal yes yeah Yeah so basically we don't really know like what is the shortest tra should be but we just say the tra should be shorter and shorter
yeah so I think that's the question that's because there's no guarantee in term of a star right so uh so I won't say it's admissible it just like give
you a better way of planning but it's possible that the planning may not optimal may not be may not
have guarantees in the future there may be a way you can make sure are guaranteed yeah right now that's the case Okay so that's the first part and uh
the second part I'm going to tell you uh how we can train the models with solers uh that part we actually have a serious of work uh collaborating with
the University of South California about how we combine sers with deep models together um so for
this one uh we basically want to like think about a little bit on like industrial uh kind of provs right so um so think about like we want to optimize
a nonlinear objectives but with comori constraints right so uh here's like a figure demonstrating what's happening right so you have some non function
that's May complicated and then uh if you would just want to find Optimal path optimal trajectory
optimal like points maybe that's fine you can just follow green sense uh but you have constraint basically say oh you also have feasible region that
is specified by uh comori uh equations so this might uh include for example linear constraint nonlinear constraint or maybe even like integer
constraints this called control constraint but for this kind of problem it's it's very tricky to get a
solution uh because uh you cannot use you can use for example General nonlinear optimization s like for example like evolutional algorithms right but
evolution algorithm is very slow it take forever to get any solutions and and plus you also want some feasibility uh which is also pretty complicated
and on the other hand this kind of problem actually has very many real word domain applications
right so uh if you think about like a guide design in many different domains that is going Beyond larg Lang models or CS Department then you actually
see many problems like that right so you want to do planning in this kind of complicated situations and everything is non linear but you also have
constraint right so in order to get things to work so uh you have also a problem like that so in other
case in this case what kind of thing we can do right so here's one example is actually happening when we actually want to run a training uh for large
models right so uh here is the example is here it's very simple you have K tables each table uh put will be put into uh n gpus so this device is
actually gpus and you want to put K table into n gpus so that uh each GPU will have a memory capacity and
you don't want to uh go beyond this capacity and at the same time you cannot split the table into two parts you can either place table in GP one or
gp2 but not either of them by splitting this table into two so that's basically the the constraint and the final like uh objective function uh the
latency is actually a neuron Network it's a it's a nonlinear optim nonlinear linear cost right so then
it's become in some sense very complicated right so here's the formulation right so x j means like you put uh table x uh to device J right so this is
the DI XI J of course XI has to be binary you cannot split a table into two parts it's binary so and sum over j x i j has to be one because a table
has to be play somewhere so the summation of across all device has to be one and uh if you place the
uh the objects into each of the gpus and the capacity cannot be uh cannot be exceeded right so uh if you sum of I I is all tables sum of I on the on
the memory requirement then the summation has to be smaller than or equal to some capacity MJ so basically that's the that's the that's the constraint
and the object function is nonlinear because it may have a lot of communication cost or different
nonlinearities uh due to uh caching loading Etc so this is estim can be estimated by uh in Industry designs so you can see the prop itself has com
constraint it has nonlinear constraint uh it has a linear constraint it has a inequality constraint and it has a uh basically binary uh constraints
right so for the for this problems right if this loss function is linear then you can actually call
existing sers right so we have a lot of servers on already on the market right so Kobi is one I mean skip is also one they have been engineer to death
for decades so that the formance is super good uh you can actually call them in short amount of time and you it will give you a Solutions a solution
will be useful for your problems right so what but what happen to nonlinear case right so if this a
is really like some compated functions if the eror function is has some analytical form right so a has some kind of form has to be log exp whatever
then it's possible that you can still call sers there's a nonlinear sofware that try to leverage the property of each functions there to set up low up
and lower Bound in order to do branching uh but you can definitely do that but if the area is
actually estimated by new network then what can we do so usually this is the kind of problem that people cannot deal with so um in our paper um the we
actually find a new way to do that um it's called Circle and that paper actually uh is uh is last year neurs and actually one outstanding paper in one
of the workshops um about discr oranization so what the idea is about is that the original problem
is nonlinear right so you have a f function that's nonlinear X is the domain that you want to optimize and Y is the problem specification so why may
mean that the memory capacity of each dpus is how many dpu you have Etc and uh this basically uh is optimization but X subject to some Comal
constraints is how to solve but what we can do here is that we can actually predict a surrogate cost C that c
is a function of the problem specification why such that with this C cost the problem become linear and then you can solve that with existing solvers
right so I want to say that again you want to find a surrogate cost C so that the solution you obtain from the surrogate optimization also minimize
original provin yeah you're basically predicting which surface of the polytop you yeah so to some
extent yes but we basically leave the machine to Fig out sure right it's possible that the exact poop is hard to predict but you can predict the rough
Direction so that the solware take the rest of it right yeah okay then the that's the uh that's the then like the challeng is how you find the right
objectives right so and uh what we can do with that how we can actually deal with that we can
actually do green descent uh in order to uh get the solutions so here is basically the surrogate learning how the learning is done right so you
actually have this kind of computational pattern so started from c c will goes to the server and the server will basically go the solution the
solution you put a solution into objectives and you want to minimize that objectives right so you have thetion
graph but for the what we can do is I mean if you ask anyone from Pyon team from any kind of different era say okay by pration right so that's what it
is uh so what we do is we back prate all the place right so you assume that function is differentiable so you canate here and your solver can also be
a differenti um by having uh by basically doing tricks sole the simplest trick would be you can do
some fin difference you can get gradient FR it but people recently in Rec the paper they actually show you don't need to find difference you can
actually do just one path of server you get a gent as well you can do vation then you actually get uh some kind of gr respect to C the con
coefficients then you can basically do this uh intuitively on the Fly and then you automatically get a coefficient C
that will lead to a solution that minimize objectives that may be nonon okay so that's basically the idea and this is called a circle zero why it's
called Circle zero this because there's no uh neur networks right so everything is uh I mean f is fixed neur networks we treat this as a the kind of
smooth functions and there's no new networks right so you basically learn everything on the fly when a
new optimization problem comes but now you can do something better as CLE prior what that mean is that you instead of learning this coefficients C you
can assume that c is actually coming from new network and you actually B prate through that new network in order to get its own parameters right so
the reason why we want to add another new Hook is that you want to generalize across multiple problem
descriptions so given the current problem description y you figure out there's a c that can be useful for C coefficients but when in the future
there's a new Pro description y Prime what happens right so uh if you follow Circle zero then you have to do by pration again that's not efficient but
if we already have a new network and we can actually train this neur in for example 100 problem that we
already know so that the neur actually have a coefficient have a parameter CA that can correctly predict right C and then you can use the neur to for
new problems that have never seen before so this is what's happening here right so you have a for one provin uh with pering Yi why the pro
descriptions and then you can train this notti that for all this 100 problems you don't even need to know the
optimal uh optimal solution you just keep training because it will automatically figure out what is the optimal Solutions in the training procedure
itself once you train this model then you have a good uh CA so this CA has been learned so you have a CA here so once the ca is here then once you
have a new uh test uh examples y test so y test is the kind of the description of the test you plug this
into the neur netork and combined with the learn parameter CA it will actually give you a test uh coefficient C that's likely to work reasonably well
and then uh you can plug this C test directly into the solware it will give you a solution a solution is uh very likely to be a good solution uh
according to the pr distribution right and finally you can combine these two together right you can say
okay you first start with this c0 which is basically be predicted by a w test and once you have the c0 and you can keep doing this circle zero kind of
B propagation to to basically continue improving uh the the surrogate coefficients to make the uh objective function even better so that's called a
circle hybrid any questions
yeah oh yeah yeah so there's two components first of all it's whether it's feasible or not right for the feasibility you can actually prove it's
feasible because any solution that come out of the solver has to be a feable solution so that part is guaranteed right and the second part is this F
function because it's convex nonconvex it's complicated we don't know we only what we only know is that
this F function as long as it's small we get we have in yeah okay cool I hear question same answer what is the benefit of learning oppos to the
solution ability satisfy con yeah so first of all it's a very to S constraint because if you go to the sare it's guaranteed that this x star always s
constraints right so if you predict this x directly from Network then it will take a long time for you to
get this you have to put NE examples you have to basically set clear boundaries and at the end you might get like maybe 90% of solution correct but
the 10% is not corre that's the first thing the second thing I think that this is also an answer to eon's question is that if you want to for example
this is the figure that shows intuitively what's happening right so for example op solution is in the
star but if you go through the solver then we don't need to precisely predict optimal solution we just we predict some kind of Direction rough
Direction okay this direction that roughly can find that direction may be much easier to to predict than the actual op Solution on the predict right
and then you basically rely on the solver to correct that to to actually make the solution optimal and also
feasible okay so uh that's basically the idea so basically to do cap uh we have C zero uh that does not need any offline training data and just solve
a single problem in on the flly by propagation right but this might not be efficient and you also have supp prior then try to use offline data to
quickly solve a problem at test time when just one Sol a call we train a model we not work we got a
thetical we get a a parameter CA and then we use the parameter to directly predict the C and you call the solver once and you get the solution or you
can get uh do like a circle hybrid basic which mean that you first do offline training and then on the Fly fine tuning the serate so you can do both
okay so let's go back to this inviting table sheding uh problem so that problem actually is the first
problem that we are met with this is actually collaboration with production group in which they actually have the real problem to deal with right so
we actually use the public data deep learning recommendation models and uh we actually compare this with multiple uh existing Solutions right the Big
G is one which is kind of a uh crappy uh baselines but we also compare St art which is actually our
approach as well using reinforcement called dreamshot um so uh we actually can see there's actually a difference uh in term of uh so just do this one
so yeah so first of all there's three differents right so there's a circle zero Circle PRI Circle hyd these three are actually our approach so it
corresponds to all the uh all the like do either like slash line or horizontal lines right so there's
two different metric one is uh how good the solution is right so you really need the solution to be as small the loss to be as small as possible and
the second is that you really want the deployment time to be as small as possible right so um first the solution let's check that the solution loss
for solution loss uh you're going to see that uh the circle hybrid and also uh the the circle uh Circle
zero they actually have pretty small uh loss compared to other approaches right especially when uh we deal with like larger settings so here like a 16
15 means like there 16 60 tables you want to deal with so go from left to right the size of the table become larger and larger right so and the
problem become also a larg scale so you have this so This these two actually has good uh s solution loss
after optimization and circle uh prior has a higher loss because it's a one shot approach right so you predict the C and call solvers and that's it
will have higher uh results but still this is actually pretty good compared to all baselines so if you go to the next step table on the deployment
time on deployment time side Circle pry is the fastest right so uh a dream chart is actually also pretty
fast as well because it's also a one short prediction right Circle PR Circle pry is fast because it just predict one C and then call solvers right so
uh but uh on the other hand like Circle PRI still has a pretty good uh actual Solutions right so there's a balance between the two you can always
basically adjust the balance by calling for example a circle hybrid but with fewer number of inations
right so you will have a basically balance between the two and uh and you'll see like both the circle prior and also uh yeah and this actually give
you like a very small uh very small amount of time used to deploy this can be very useful because you really need to make the very important decisions
in a very short amount time in order to move all the tables to different gpus and the trend models
right so if you spend like for example like a 10 one minutes to do that then it's better to just use gy approach that may be a much faster to make
decisions and it may have some extra overhead uh to train models so the laty is in seconds or yeah the latency is in uh it should be in seconds yeah
okay so if the run time takes I see the run time takes 50 seconds it doesn't yeah I remember I think
this should be no so this should be Sor this should be millisecond Mill yeah yeah millisec yeah okay cool uh okay so uh then we go to the next uh Pro
uh that design problems this is another industry design problem that we can leverage uh our C approach right so for this one uh what we do is uh
there's actually a very small device uh it's probably 100 Nom by 100 Nom right so and the goal here is to
design the pattern within this small image so that the when the light goes from the top it will actually change the path according to their wavelength
that's basically the the proportionalities uh that could be depending on different design there could be different uh kind of tools like that so for
example this one by designing this material like that you can actually bend the light uh if the
light come from the left it will bend all the way to the top right so for this one it's called a beam splitter right so one light come from the left
it will be split into two beam of light on the right so uh and also for this kind of for finally for this one this called the wavelength division
multipliers multiplier uh this mean that you have a light with some wav length goes from the from the
left it will depend on the lower length it will either go on the top or on the bottom so people may Wonder like why not use some uh prism to do that
but the question answer here that this the size is small it's kind of 100 n ners nanometers it's very small 10 to the minus 9 meters so you can you
have to design something fancier in order to uh get this to work rather than using a prism a prison
will be very much larger
yeah you can do that yeah so and there There's a constraint you cannot put arbitary patters there because uh there's some constraint on the
manufacturing so each pattern if you put them there it has to be at least some size Yeah so basically you can you can see each pattern can be brushed
basically you need data brush and that brush can reconstruct all these patterns but really want this brush to
have a smaller kind of diameters you cannot be smaller than that yeah okay and the objective function here is actually uh quite complicated right so
you first need to simulate this entire device because everything follows Electric magnetic waves uh and once you follow that then you will get um uh
you get spectrums of this device and then there's some complicated functions in order to minimize uh
so that the device will give you uh the right answers so for example for this one you really want the wavelength wav length one to go on the left W
two go on the right so in order to measure that you want to measure like okay how much energy you receive here and how many energy you see there for
two different frequencies and then you come up with some complicated loss functions in order to make
sure that by maximizing it or minimizing it uh Your Design goal is met and that's the basically the loss functions and uh feable solution is actually
uh you to satisy uh satisfy some constraint that it has to be the union of the brush pattern so there's a brush pattern in the device in the actual
machine and then the uni of brush pattern is is basically something that you can you can actually you
can actually manufacture uh by your machines so this is actually a linear constraint if you write them down correctly and everything now it become a
linear constraint but with non convex optimization problems right so uh then what happens here is that you want to still measure the two quantities
right so one is the solution loss and the other is deployment uh run time so here because we are
looking for anything uh that is feasible uh so there's uh there's there a there a constraint and also there's there criterias so as long as it's
feedable then we actually think this is good uh so um we actually see that uh the circle zero and circle hybrid they actually do very good job and
they get zero loss here and they get pretty small loss uh in uh two of the complicated design problems so uh
if you compare this with baselines yeah and uh for deployment time deployment run time I of course the circle prior is the fastest because it will
give you uh the prediction instantly uh but on the other hand the cirle hybrid is a bit slow but still faster than existing baselines and also the CLE
price final results is also good ter of the loss there's a tradeoff if you run this for sufficient
long time you will be getting is there like a simpler domain specific Baseline other than pass through yeah so uh for this one we basically have to
ask the domain experts right the the passwor is the one that they provide with us yeah so there could be other baselines um yeah so if you basically
print out the curve you see like the curve actually much smoother on ccle hybrid right so it's actually
goes down very quickly rather compared to Cle Zer even with ccle zero it's actually converge faster than the original prop just understand the last
yeah solver only gives Solutions what does it mean for the solution to uh you mean what do we mean by the solution to be feedable
yeah yeah yeah so basically this is the criteria for solution so basically the solution need to be constructed by a brush which mean that you cannot
have a so if there a do here it has to be at Le something but the solver guarantees that you get aible solution right yeah so how could so the next
slide how did you have were
Sol Uh Oh you mean you mean this one I see I see I see I think this is basically like there's a solution loss um there's some constraint that is
feedable you can guarantee but some constraint actually put into the loss functions yeah yeah so basically there will be some solutions that is U uh
first of all the solution the this is the feedable solution that you can put but there are some other
constraint that need to be put into the loss functions but in that case like it's not U it's not feable yeah okay cool that's good question uh okay so
uh the conclusion for this part is that uh we can actually handle industry level applications with different transitions right so um we really need to
um uh we need to basically like a find like a linear subate in order to get high quantity
solutions to comori noncom organizations and then we have a circle that can be useful uh for uh this kind of settings and we propose multiple settings
for Circle including Circle Prior Circle zero and also Circle hybrid okay of course there are limitations right so this is actually uh this is the
next this is the Fout paper that we did last year and published in Europe 2023 uh this is done by uh
one of my post off and uh basically the limitation here is obviously uh you really need to require uh the B gradiant of the object functions right so
in order to do by propagation and the second is that you also want the server to be propagable as well so you want this two and you also want to call
the sers doing the training because maybe sometime it's pretty slow so in order to deal with that we
actually have an idea called using another circuit a circuit top of circuit right so you have a Sate model M and M actually will cover this U uh comp
this function composition right so you have F compos G this is some function you really want to optimize so this is the one that you want to optimize
and then uh you have another tget model M to cover that and M can be differentiable so then you can
back progate everything through rather than going through f and g in a separate steps so that's very simple idea and uh how you estimate the CATE
Closs M it's very simple right so you basically do some kind of sampling in your space and then you try to match uh the function shape within the
space and this space is basically uh constructed by you first do you you first like estimate like what is
the good CA which is the parameter for the predicting uh the surrogate cost right so and within this range of the ca star and you want to find a Sur
cost uh M which is parameterized by W regarding to F composed with G so it's very simple so then what we do here is like we uh we basically like use
exis data to refine this m and then use given the M uh you have uh you you use the m to basically
learn uh the setas right so uce this iteratively so we have done multiple experiments including exist experiments of circle uh but we also do an real
experiments which is portolio optimization and uh we all know that you want to find kind of like a configuration of your stock so that you maximize
return so that's basically the idea and uh in this paper uh we actually think about like an objective
function that is non quadratic it's beyond Quadra I so that's uh in some sense you want to basically for maximize the screw or maximize some kind of
third order or first order terms in order to make sure that portfolio behaves well but in that case the loss function will become complicated and at
the same time you still have the constraint so you cannot buy too much for one stock right so and all
the stock if you add together has to be S to your total assets uh perform right so that's something like that and we actually can see that our
approach um which is called a lenser uh can also give you reasonable good performance uh with fewer number of function cost right so um I think this
is actually understandable because we have a surate cost and that surate cost in many cases can replace the
function uh call the solver call during the training right so but sometimes solver's call may be very expensive right so you call a solver it may
takes like several minutes uh in order to get some numbers right so in that case it will be very slow if you incorporate a circle pipeline but with
lenser uh because we have a surate cost M and that M actually uh can give you a good estimate of f compos
G and you can bu progress through M you don't need to call the Sol anymore so this actually give you good performance um so you can see that for
lenser uh we actually give you have fewer number of function calls uh while the performance which is the loss function actually goes down yeah
similarly yeah okay that's basically one extension of a circle there's another extension of circle uh which is
to generate like diverse Solutions uh so um so for design problems sometimes like okay we only want one solution but sometimes you actually want many
solutions right so it's possible that we want not only one but T Solutions some of them each of them cover a different portion of the space and you
want diverse Solutions but I we also want divers solution to optimize the nonlinear objectives and Ste
uh these objectives come up with not come toal constraints right so you have three components you really want to satisfy at the same time so it
actually makes this more complicated so what can we do with that uh so for this problem we actually deal with the following loss functions so the loss
function actually composed of this additional group loss the group loss basically tells that uh I want to
also optimize some um optimize some uh finalize some design so that collectively uh they are minimizing some group loss right so obviously and you
also have individual loss uh which may uh correspond to your previous loss functions uh for previous nonlinear objective functions that we mentioned
before and of course because we want direct solution then we really want Genera models right so you have
now you can put diffusion model you can whatever gen model in uh that models we all give you some generation in latent space and then the latent space
which is basically design space in our previous uh notes going through a solware and it will give you a solution so uh if you do that then uh you
actually were able to do some good designs on top of that so we actually test s t tasks so we still
test yeah okay yeah so have a generative model that generates an initial solution yeah so lat space yeah so the lat space is basically the surate
space c in the previous uh in size yeah okay yeah so it will go through a solware and give you a solution so the general generates c yeah yeah so
previously this C was either optimized or predicted by new network yes but now because you want have a
diverse solution so you really want models to generate that and then the training of this generator model is you get like in the next slide you get
like maps from Warcraft yeah and then I I and then I backrop through the solware to get a c yeah and then that I fit the general model to that c yes
yes yes that's right that's right that's right yeah so for this one for example like if you if you
think about this game level design right you want to you want the game design to to Maxim minimize or maximize some objectives right so for example
the you want the playing PA to be maximized right or you want to uh but still feedable right so you don't want the level to be inedible which mean
that you won't be able to finish your goals but also want your game level to be diverse you want people
to play with like interesting game level which have never seen before yeah so that's basically the task that we can deal with right so and also in the
Worldcraft Man map generation uh we want to gener diverse maps and at the same time you also want to minimize the pass of each of the diverse uh
diverse Maps right so um and for inverse for of course you want to find different designs and all can
satisfy the same constraint and some designs may have nice properties that you might might be looking for yeah so that's the so we actually test three
different domains in the paper the paper is not on the archive yet uh it's actually in the I yeah it's it's in the submission site hopefully we get
accept I don't know see yeah so uh there are some results um yeah so for if you just do again plus mp
uh then it does not give any it basically does seem to be very uh diverse right so uh you you basically like get like a symbol like basic one or two
blocks there and uh by M mean like it actually give you eable Solutions and there's no many strong diversities um but if we actually use our approach
um we actually get pretty good diversities and coverage that good question so so in these games
especially if they're complicated games um there may be the maps the constraints that are active or inactive May themselves be diverse yeah for some
maps I want to make sure that it has a narrow whatever right yeah that's right that's right so it's possible that by changing that constraint maybe it
can be done but here all the constraints are fixed they're just like the base Atomic constraints of
this of this environment yes yes but you still wanted to be diverse right so for any kind of reasons right so for beautiful reason for reason that you
want to find good Solutions something like that and the good thing about the generation is that the generation can actually give you uh solutions that
is uh guaranteed to be feedable because everything that is generated is going through the solware
the solare give you feable Solutions so uh and uh so this is actually the solutions you can basically there several metric right so first is
uniqueness right so unique mean that how diverse it is if you check you find two solution that are identical then you have this kind of uniqueness
right so and these two are actually covering like the train distributions and uh the final one is like a
solution loss you really want the solution loss to be lower and to finish constraints yeah okay so final going to pass three I'm not sure people are already tired questions yeah
yeah yeah exactly yeah of course L program is much faster right much faster to solve and cority may be very slow 10 times 100 times slower right and
in many cases I don't think uh you can actually Leverage The Power of new networks right if newwork already can predict some corners that is good
enough then you don't need a CTIC program yeah so there a trade-off usually like I think the point here
is that this firworks works for any arbitary sers uh and arbit networks so if you want you can play with different combination of them and find which
one is the best yeah got about 15 minutes oh 15 minutes left okay cool cool okay so I have the S second the final part uh that's kind of replace all
with the models uh that is actually a very interesting collaboration with antenna group uh in matter
and we are trying to solve a linear power differential equations uh it's it's an interesting experience and I'm actually one of the first auor the for
derive all equations and I learned some a lot of D ipd stuff um so uh so for this one okay so I think people especially people in Academia they like a
lot about a for science right so for a for science there's a lot of linear PD problem you want to
solve right this include he equations reading equations Max equations Etc so um these are very tricky to simulate accurately and efficiently right so
can we do better so the background of this problem is the inter design problem so in this problem what we do is that we have a design um kind of like
a two dimensional uh image for as a design of the Anna so two Dimension basically say that which
part you should put the metals which part you should put with isolation isolated materials Etc and with this design the goal is to make sure that uh
the final uh frequency response has to uh be in some specific forms so that's basically the design problems right so you see there's two dips here the
two dips basically correspond to strong absorptions at the specific frequencies right so you want
design we want the antenna to be super strong in absorption for 2.4 gabz and also like 5 gz right so you want the antenna to basically absorb this
signals uh in uh in these two frequency range uh so in that case uh you really have a basically invert problem you finally figure out what is the best
Edge so that uh the final frequen response look like that right so it's something that not
optimization problems but one way you can do that is that very simple right so you start from simulating the design of intenna you can try to try
different approach of the design and use some simulators the simulator will tell you given the design what is the best uh what's the frequ response to
design right so there's a mapping and once you have the mapping then uh you can basically find okay
which mapping fit to your actual design go very well and then you come back and check okay this is actually is the best one you can go right so very
simple but then the question is like the simulation is slow so you have to go through like numerical simulation you have to go through from CST solers
and may take some time maybe one minutes two minutes or for complicated design problem it might take
like given hours in order to get the actual frequeny response there so this is actually very slow then what we do is that we can actually um derive
some kind of theoretical structures of the uh of these kind of mappings right so for linear PDS it turn out that there's actually a theoretical uh
parametrical form uh for uh for for the for the any quantities uh that is appearing in the for transform
of the of the equ right so um for any linear coefficient B1 B2 so this five hat is basically like any uh field uh that is transformed by uh for
transform and uh as long as the linear combination uh the ratio of them uh if you consider ratio of them then this ratio actually uh will look like
the following right so it's actually is a uh is a is a polinomial and it's actually rational polinomial uh
structure so it will be the division of two polinomial because the division of two polinomial you can actually write them down according to the basic
fundamental theorem of algebra right so it will be decomposed into zeros and the PO in the product forms so you have to the kind of structures for any
linear IDs uh so because you have that then what we do is like because the antenna coefficient S11
can be written down as also a uh quadratic so some kind of like a uh potion form right so with these two terms and each of the Z here is the impedance
and which is also the quotient of the voltage and current and current and the voltage and current these two actually integral of the field strength in
the Le Dynamics right so everything seems to be aligned with the condition of the theorems right
so everything is linear and it's quotients of the two linear so because of that you can assume that this S11 uh if you take the log is actually can be
decomposed into uh this kind of structures so you have zeros and you have also pools and then you also have the coefficients so for the all these
three numbers we can actually predict these three numbers uh with any new networks that you want to you
want to use right from Divine choices by doing that you actually automatically uh get rid of uh the simulation and can come with a good uh Sate cost
uh for uh for the for this design problems right so then what we do here is that we start from some image B representation of the designs of the
antennas and then uh we go going through some uh Transformers and it will basically directly predict uh c
z and p uh from this Transformers right so that's very simple and once you have that you can use existant data set to fit that right so uh you start
with um some ex existing like material commercial simulators and give them that you have a pair of H and the final S11 so you have a mapping so the
mapping may be expensive to get but after you run a sufficient round of simulations you might get like
for example a data set of 48k or some data set that's may be many expensive but still fine uh to use then you can use the data set to fit to the
previous uh Transformers so that you get surate models so um I think here's a I will basically so this is the visualization about circuit models uh so
this CDP model that we develop actually has a much smoother frequency response uh compared to the model
that does not respect any of the smooth properties of the uh of the F response right so if you just predict this one uh using any arbitrary networks
then what you see is you're going to have this a these humps right so that's not smooth enough uh but our approach because we follow the uh linear PD
structure it actually give you much smoother frequen response after prodution so in term of
performance um we actually do two different evaluations one is called a static evaluation which means that you just chest test what's the loss in the
test set right so um and uh basically like you want this loss to be small as possible but you also tested the dynamic evaluation which mean that we
already have this C model we can use model to search a good design and see where this design is really
good in the practice Yeah so in static evaluation we really see that uh first of all we find like the transformer actually helps a lot in term of the
loss and second uh we find that um we should be able to uh use the CDP architecture compared to Raw architectures because uh if you compare the raw
with CVP you see there's a gap uh between the two ones yeah so and uh and for CDP there's coefficients
which is how many R how many col how many uh Roots how many like zeros and poles right so this number is high par into T but overall seems like the
loss is reduced which is good so and finally we can also do Dynamic evaluations so we can use C model as Circ model and couple with search here search
uh we just basically do a very simple reinforcement name uh search with these value functions as the
CDP models and we are able to find that the percentage of success in any of the top three uh design is actually much higher than the RO models okay so
that's pretty much it uh if you have any questions let me know so hopefully we still have some time to answer questions all
right any questions from the
class maybe they want to know what it fixs the internet thata oh okay uh I think it's okay A a tricky question I think it's yeah I do have a question
um so uh this kind of design still this kind of work still requires a lot of I guess um modeling work with a domain expert yeah one of the I guess
maybe one of the advantages of using llms and and large Vision language Vision models is the
opportunity to ex streamline significantly yeah that aspect have you thought about that at all yeah so I think this something that I forgot to mention
in talk but that's a perfect question to connect that so first of all people may say this is a deep model so where is the AR right so I think ARS can
definitely be inserted in many different places right so for Circle what happened here uh is that
if you check the formulations right so the formulation yeah so yeah circle prior so you can see like the formulation this but if you can imagine that
this one and this why can be any of Tex so once you replace with any kind of models then what happen that you want to describe a problem language and
it will basically get cross bym and that can give you
that's can that so the last chapter we talked was replacing the solers with models yeah the you gave kind of very specific to the an problem what
youles replacing first was very arbitrary yeah exactly specific set's what's getting placing the solware here arbitrarily with oh I think first of all
I still think like the solware can be pretty important because the model right now cannot still give
you precise Solutions especially when so I think that first of all you put a solver there and then you can solve your problem that's good and secondly
long run if you want to computer place the solver then there could be a lot of timeas right so one possibility will be obviously you can replace the
solver you can generate from Sol and hopefully the data can be very useful for training models
definitely so people talk about data all the time and this is the one way to J that's definitely one way but in the long run what should we do is that
if you can understand what's going on Transformer training then we might be able to understand what it can do not do and find efficien so in that case
there could be maybe like a more deeper integration which solve any last question yep