That is really That is a cute airplane. Actually, oh my god. No, it's not. The feet coming out of it are really really concerning. Today, we're going
to be looking at a one-bit image model created by Prism ML. Now, you may be familiar with this name because they came out with something a little
while back where if you Google search them, you see this video right here from this gigachad. No, I'm in
a mood. It was called Bonzai 8B and it was basically like a one-bit 8 billion parameter model and it actually performed quite impressively considering
the massive reduction in size from the model that it was based on. So, for today's video, they have released an image model that kind of uses some of
the same magic based off of the Flux Klein 4B model. I think it's called Klein. I will verify that
if I was incorrect. So, this image model is very exciting because this is going to not only allow for resource constrained devices to actually
generate image models using an offline model such as this, but it is going to be significantly faster than perhaps some of the things we're used to in
terms of how long it takes to generate an image. So, this is quite exciting and for today's video, we're
going to be taking a look at and testing the newly released Bonzai image from Prism ML. So, to begin, please do feel free to subscribe as I do want
that 100k plaque, but also we can see in this model collection, there is a bunch of different models here shown. Now, first and foremost, the big
differentiator and something that is nice to see is the ones that are titled MLX are for Apple silicon
machines and that's nice because those have become significantly more popular in terms of using local AI. So, it's good to see support out of the gate
for that. Additionally, we have different versions of these. So, we have one-bit and two-bit and really I think the simplest way to show the
difference between the two of those is if we click on the two-bit version right here, we see that the
transformer for this specific two-bit version is 1.21 GB down from 7.75 GB for the FP16 Flux 2 Klein 4B transformer, which is what this model is
basically down sized from unscientifically. And if we click on the one-bit version right here, we can see that the transformer is 0.93 GB, so 930 MB
basically, down from that same 7.75 GB size. Additionally to that, they also show following the size This
is ternary, so it is minus one, zero, and plus one transformer weights with FP16 group-wise scaling. Um we can probably ignore that for now, but the
difference between this two-bit where it mentions this right here and the one-bit right here, we can see this is just binary transformer weights with
FP16 group-wise scaling. So, you can see kind of some of the difference there, at least just in terms
of how this is actually shown right here in terms of the number arrangement, where there's three there and there's two here, unscientifically. But, it
showcases um the one-B as well, and I or one-bit as I do want to test this because I think it's very interesting to see the maximum size reduction and
how it would perform. Now, for this video, I would imagine there's going to be a decent amount of
folks out there who may have a Windows-based gaming PC that would perhaps be interested in trying this. I do want to actually go through the setup
steps from scratch here on a Windows machine, which I am currently on. So, for that, I'm just going to scroll down to one of these random repos right
here, whether it be the one-bit or the two-bit variation, and they do have a link in the resources
section for the demo repository. And if we click that, we will get to one command setup for Windows or Mac or Linux. Now, as a prerequisite, we do
want to direct our attention to right here in the quick start section, where there is this PowerShell command block, but underneath that there is a
link right here to this windows.markdown file, and this has all of the setup prerequisites that your
system is going to need in order to allow this to actually work smoothly. If you are unsure about some of these things, which I wouldn't blame you
because this is not necessarily like totally beginner-friendly right here, I would say my best advice would be to just copy the raw file for this,
paste it into your favorite LLM, and then ask it to walk you through ensuring these things are set up on
your Windows system, and it will very likely get you to a point where you can then go follow the instructions that we're about to run through without
encountering really any issues, fingers crossed. Now, to begin, we're going to be using PowerShell on Windows. So, in order to get to that, you can
just do the Windows search thing right here, and then type PowerShell, and you will go to that, and it
will open PowerShell on your system. So, once PowerShell is open, we're going to click the green code button right here for a drop-down, and we're
going to copy the URL right there, and then we're going to hop back into PowerShell, and we're going to type git clone, and then we're going to paste
in the URL that we just copied, and this will download or clone this repository onto our local system.
Following that, our next step is going to be changing our working directory into this new Bonsai Image Demo repository. So, we're just going to type
Bonsai, and if you start typing it out tab, it will very likely auto-fill the location that you're trying to go in, and then you can just press enter
from there just to streamline things or make them a bit quicker. So, once we've cloned this, our next
step is just going to be to copy-paste some of these commands into PowerShell, namely the first one right here, and if we right-click in there, it
should paste in. The next thing you're going to skip is this right here. This will set a fake or improper Hugging Face token, so even though these
models are public and not gated, if you accidentally paste this in and then try to run the download
script, it won't even get to the point where it downloads because it will just see you trying to use an invalid token. So, you want to ensure that you
do not run that command, so basically skip this right here. Our next step is going to be to copy-paste this setup script right here. And assuming
everything is all set with the prerequisites on your system, you should just see some like decent or
happy green and blue text as well as some progress for things that are going on. This is just basically setting up your environment with a lot of the
dependencies that are going to be needed for this. Now, to that point, I do want to just quickly make note that something they have done here that is
very good is they have essentially also done something to protect against some of the recent
security issues that have happened with like supply chain attacks and things. So, this actually has a flag that sets the minimum age of things that it
will install to ensure that no zero-day thing gets like pulled as part of this environment, which is nice to see. Additionally to that, because this
is using like different kernels and things, if we scroll down right here, they talk about like some
of that stuff. Always be cautious about installing like different kernels and pretty much anything on your system, but just make note of that. Not to
say there's anything suspicious about this, but it's always good to at least use burner machines or just be conscientious if you're using your main
PC. Though, the next thing, and if you do get that red scary-looking text there that appeared, you can
just kind of ignore it. The next thing is we would run this specific command right here, which would do a demo generation for the model. Now,
unfortunately, we're seeing that there is a bit of an issue when we just paste that in verbosely. So, if we scroll up, we'll get back to it, and then
let's just do we'll change this. So, instead of just this one flag here, which is messing things up, we're
going to do that and then type prompt. And that should Okay, I messed this up. My apologies. Okay, so basically, you're just going to want to have two
little dashes in front of prompt right there, and we're going to see that this should then hypothetically begin the generation sample for this model.
Unfortunately, it doesn't, and it's saying you don't have any models downloaded, and you need to
actually download them. So, our next step is going to be to actually run the script to download the model weights and everything like that, so that
this sample generation then does work. So, in order to download the model that we're going to need, we can type the {slash} scripts and then {slash}
download {underscore} model .ps1 and then we're going to type the name of the model if we want to
specifically choose one. So, basically as we see right here, but don't just for both to copy paste these specific commands because they won't work for
PowerShell. But, um I do want to try both of these. However, I will try the ternary one first because it will have higher quality and that's what it
default kind of ships with. So, you should actually not even need to do anything following that.
Does anyone see the issue there? So, {dot} {slash} scripts and then download model.ps1 and then we'll see this will go ahead and start downloading
things right here um from hugging face, which will take however long depending on internet connection, etc. And once the model has been downloaded,
we'll get the message that it's been saved to that specific location and then we can once again try to
run the sample prompt from before keeping in mind that we need kind of like this {dash} {dash} prompt in instead of what's listed there, or else it
will get upset. Now, hypothetically, fingers crossed, assuming everything should have worked, okay? So, we still have a bit of a hang up here. Okay,
so the issue that we're seeing right here is essentially it's still trying to validate the path for the
binary model as well as the ternary one, even though we're just trying to use the ternary model. So, for this there is basically a flag that we can
set. Um I will just move my head out of the way real quick and leave this on screen for a bit of time so anyone can just see this right here. And
basically we're just setting this as an environment variable pointing the binary path to the same ternary
one. So, keep that in mind if you do try to try the binary model instead, you will need to undo this or invert it if you've downloaded that one first
and want to use it. But, then once we've done that, we should now hypothetically be able to run our test generation prompt one more time and this time
it should work without any issues. Okay, we're seeing a bit more. Now, I wonder if I can catch it
in time. I just tried to use a Linux command. So, let me do We'll see if we have any VRAM utilization right now. Okay, not currently. So, it's still
loading everything. There we go. If I run this once more now, we should be able to see. And in terms of system specifications, this is running on a
5090 mobile, but >> [laughter] >> um so, we can see right here the card was tapped when I happened to
run this NVIDIA-SMI check. It was using all 175 W and around 5.5 gigs of RAM. It should now be done. Okay. And in our outputs folder here, as it shows
right there this location, we now see the output for the image that we received right here, which is a little ice-capped bonsai tree or something of
the sort. So, our final step before just jumping into some actual fun testing of this model is to
run the web UI just to ensure everything works smoothly with that, and it will be much better to test it through that. Now, they do give you run
scripts.serve.sh if you want to use the studio. If you run that specific command on Windows, it will not work, and it will be upset. So, we're going
to do scripts Okay. And it's saying npm is not found. It has not been bundled. Okay, so this error that
we're seeing right here actually stems from if we scroll all the way up, when we got the first like scary red text error right here, this actually
stopped the setup script from continuing what it was going to be supposed to do, which would culminate in making sure that everything was set up here,
like npm was all found and stuff like that. So, it stopped it from proceeding, and that's why this
error actually said did setup run successfully. Now, in order to fix this, there's going to be another big command that we need to paste. So, I'm
going to paste that, but then also showcase it right here. Okay, so hypothetically, if we click that URL now, we should see a pretty front end, and
then we can go over good what specifically needed to be done here in order to fix the previous setup
issue. Um and again, this is why I wanted to do the Windows installation because it's going to be more of a pain, but it's likely going to be the
thing that folks who just want to experiment around with AI are going to be using. So, in order to ensure that does not happen, so the commands that
were run are shown right here. And these are things you'll have to type into your terminal or PowerShell
instance in order to get this working. So, the first one, we're setting an environment variable to basically skip the portion of the GPU setup that
caused the error in the first place up here, which prevented this from actually continuing to set all of the dependencies up for the web URL. So, we
set that, and then we're also skipping the download portion cuz we've already handled that manually.
And that is also why we had to have that issue. So, it's kind of a roundabout setup way, but just keep in mind this will get you working. And then
finally, once we have set both of those things to essentially be skipped from within the setup script, we will then run the setup script once more,
which will then continue and set up everything that we would have needed dependency-wise for the actual
front end to work. So, this is kind of a roundabout like installation tutorial, but I wanted to actually provide some educational value in this video
beyond just testing this model cuz it's pretty cool, and folks may be interested in playing with it. All right. So, now that the web UI is up and
running, we can also toggle between dark and light. I'll probably keep it on dark mode just to kind of
save the viewers' eyes. I personally kind of like the light theme with this, but dark is probably preferable for filming a video. And then obviously,
going to start out with the tried-and-true Agent Poopman movie poster. Now that I've tried to provide some educational value for this video, I feel
totally fine with just jumping into like chaotic generations cuz I do find them funny, but also Okay.
He said Poopman. Now, again, we're not going to be expecting like state-of-the-art image generation things. I don't know that this is going to be any
good at all at doing text, but the whole draw of this is the fact that the model has been shrunk down so much and is also able to be run so quickly
and things like that, which is pretty cool. Okay, so again, not necessarily the best and perhaps the
prompt that I'd given it was also somewhat lackluster. Let's try just some of these sample generations, which are likely Okay, that's actually quite
impressive. Ink wash style gentle shadows. I like that. I think we're going to we're going to use that for a few things. Ink wash style gentle
shadows. Let's see how well it transpo Okay, all right, so oftentimes you'll see perhaps some of the
specifically chosen demos do the best job of showcasing the model in its prime. Um this is kind of concerning because this looks like World War II
footage on this screen and I don't know why it chose to do that. So let's let's just try like a cute dog. Perfect. Okay, very good. So it's very good
at doing animals in the style. That's actually not bad at all. Let's see if it has any understanding of
different dog breeds. So let's do a cute German Shepherd. Fantastic. Well, more or less German Shepherd-like. All right, and what are some other dogs?
Corgi. All right. And just look how quick. These are basically like instant. Like yes, this is a pretty like Also, that's a fantastic corgi. This is
like a pretty beefy GPU at least for a laptop. It's a you know, 5090 mobile. But >> [laughter] >>
what if we changed cute like a a hideous you know, hideous. I can't spell. Here we go. Not that we need to with AI now, but Oh, okay. Yeah, cuz dogs
can't be hideous. They're all cute. This is good. These are like I'm not going to say that cuz people will get mad if I say like I'm going to put
these on Etsy as hand drawings. A cute mobster. I am at my peak when it comes to the creative generation.
[laughter] >> Okay, that's like a It's like a old school mobster like I'm thinking like let's do a cute gangster. Oh, pretty much the same exact
thing. So, it must tokenize mobster and gangster in like the same understanding. Don't I'm It's not, but a cute What's something else that like just
would not be cute? Powerlifter. >> [laughter] >> All right, ink wash style I am a big fan of
apparently, which is something I've just learned. Also, there's adjustable resolution, so we could do posters, we could do quick. So, the quick is
just smaller resolution, so less like compute needs to occur. You know what? I'm going to Let's do a poster. Let's do a big poster. And we see it
takes a little longer because of like a It's doing more. All right, so now we'll be able to keep an eye
just for like one sample generation. Yes, that works. Thank you. I hate auto tabbing stuff. Okay, so that was using around 8600 MB of RAM, so 8.6 gigs
of RAM. We'll generate again and then we'll take a peek. So, this will be updating every second right here in PowerShell. So, okay. And it gave us the
same thing, but just to showcase some of the RAM. Let's change Okay. Let's do standing. And then
we'll take a a look back at the GPU utilization and things. That is really That is a cute airplane. I don't know what I had in mind, but Okay, I don't
actually Oh my god. No, it's not. So, >> [laughter] >> the feet coming out of it are really really concerning, and that is because I had changed this
to Oh, I forgot to Let's just try that once more cuz that was actually like something terrifying
the more I looked at it. There we go. Significantly better and cuter. This thing's really good at this ink wash style. All right, a glass terranium
city at night miniature streets glowing storefronts. Let's see what's up with this. Terrarium, not terranium. That's pretty good. A glass terrarium
powerlifter.
Oh, glowing storefronts. Might mess this up. This could be concerning. No, fantastic. Look at this. This is like a text to like Christmas tree
ornament thing. Basically, you could make it. Somewhat concerning. The glowing storefronts um We'll just stick with the glass terrarium and we'll see
what we get. >> [laughter] >> Oh, okay. Concerning. Oh, and we have to keep in mind the resolution here is
set larger. That is fantastic. That is a cute puppy with a glass terrarium uh surrounding it. Let's change this back. Let's just do one to one 1024 by
- And again, the speed here is nuts. This is a quick GPU, but still the speed here is nuts. Now, okay. What I like to do because we are using the
turnery model right here, which means that this one is a bit larger. So, this is a 1.58 bit
technically. I want to try the one-bit one as well. It will be less quality like capable, but it will use less RAM and it will be very interesting to
see just that working as well. I've now swapped this to work with the binary model and something I want to quickly touch on because we did see some
VRAM utilization and I find it can sometimes be confusing because you see highlighted here like, "Oh,
1.21 gigs." So, it'll probably just use like 1 to 2 gigs of video RAM. That is not specifically the case and does show why we were seeing more VRAM
usage than just the model size itself. Basically, it's using a bunch of additional components here like a text encoder and VAEs and things like this.
So, there are additional pieces to this to allow it to function in the way we're using it beyond just
that massively shrunk down model file right here with the 1.21 gig diffusion transformer. So, that is why this may seem a little confusing at first
glance is like, "Oh, okay. Well, I thought it would only take like 1 or 2 gigs of VRAM, but now even with the binary one loaded in, we're still
sitting at around 4.6 gigs of VRAM being utilized. So, just to kind of try to make sense of that at least in
a simpler way, I do want to try this one more time but in watching NVTop right here. So, we can see that with the binary version of the model we're
using around 5.15 gigs of VRAM. Now, it is possible that this issue that showcased right here is something that may prevent this from showing up. I
don't think it will, but I do just want to showcase the like generation VRAM use for this binary model
now. Okay? And we will inevitably expect to get a decrease in the quality of the results because the binary model is shrunk down even more. And now
we're sitting at around 8.5 gigs of VRAM uses as well. But after that first load, even though we're at the higher VRAM utilization point again, we see
things are happening significantly, significantly faster now that like the model's actually loaded in
with all of the pieces that it needs. Oh, good. And our session history is still there. So, just clicking generate here now though, the speed that
this is happening at is really quite something. And this is at the one-to-one. So, this is at 1024 by 1024 and it's just so, so quick. And we can kind
of compare some of these with the results that we saw with the ternary version of the model, which is
inevitably going to be a bit higher quality. Let's just see if we can reuse this specific prompt. Good. And this is with the binary one. I don't know
how much of a difference we'll see with something small like ink wash like this, but okay. And there's a decrease in quality comparatively. This was
definitely a bit better compared to if I can try to side by side it somewhat that. So, so now we're
just going to do some random image generation tests with prompts that are a little more intricate. Keep in mind this is the one bit version, so this
is the smallest of the released versions right here. Okay, and this is a retro basement LAN party. And again, we're going to notice that the quality
here is not necessarily the exciting thing about this demo. It is just the massive reduction in size
based off of the Flux Klein 4B model that this is based on that is really kind of cool. And this is similar to how the demo of the one bit 8 billion
parameter model was as well. Where some of the results were not necessarily great if compared to other larger things, but when factoring in the actual
like technical coolness of this, it's still impressive to see like it still actually has some level
of capability. So, okay. That is our basement LAN party. Let's just try another one. The different seed so we don't get the exact same image. Oh, wow.
That one's kind of scary. >> [laughter] >> I'll try one more. Okay. That's probably one of the better ones cuz there's less people visible, so it's
not as easy to be like uh All right, so here is an Agent Poopman movie poster and I am going to just
make it the maximum height or size for a poster in terms of the resolution and we'll see what we get. This is a like a vintage spy comedy one. That is
really not bad. If you ignore the text, this is actually I would go out on a limb and say kind of impressive considering adherence to the style that
was denoted in this prompt of a vintage comic poster. This is actually Agent Poopman. >> [laughter]
Close enough. Anyone who watches the channel will know what this is supposed to say. And thank you. I actually I'd like to try one more of these.
That one's even closer there. Okay, good. Helicopters, cars, buildings, explosions, lasers. Pretty much what you'd expect. So, now we're still trying
the same movie poster thing, but this is going to be a more modern take on it. Also, I have realized
that I inverted the specific resolution that would have made this a poster in the traditional sense, but again, we see the prompt adherence changes
massively. Let me go and actually fix that. I apologize for my incompetence. Now, we'll get something that looks like a more traditional movie poster.
Okay, very good. Do one more. I like it. It's actually like if we were to remove all the text and
things like this, these would be almost like passable. Let's now try something that's just totally out of scope, a clean modern infographic explaining
how GPUs work with a Yeah, okay. So, it got the overall like shape of what one would expect for this, but again, this is a prompt that is massively um
overcomplicated. Let's just double the amount of steps and see if we see any noticeable level of
difference. That should be a with the same seed, so. Okay, a bit better. So, that's good to see. Goopless work, >> [laughter] >> but definitely a bit
more legible overall. Then finally, I'd like to try a full comic book page, which again is going to be a bit outside the scope. >> [laughter] >>
That's terrifying. It's It's funny cuz like in one It's scary, but at the same time it's actually kind of
impressive all things considered, so. That one was just kind of I'm going to I'm going to just move on. So, for the final thing, I've swapped back to
the ternary model, which is higher quality, and I'm going to rerun some of the things that we tested earlier just to see if we notice any difference.
This will be a little slow for the first time that we try it since it's freshly loaded in. Okay,
definitely more action going on there, I can see. Now, I do also want to try something I did before with the infographic where we just double the
number of steps to see the side-by-side noticeable difference in quality between four and eight steps. Okay. I think that's Oh, wow. >> [laughter] >>
That's the thing things of nightmares. Now, let's try the retro LAN party again. Okay, it seems a bit
more detailed I think were I to take a guess. And then just out of curiosity, we'll double the number of steps. We'll change roll the dice to change
the seed and then we'll run it once more. Wow. Significantly better. That's actually impressive. Now, our agent Poopman poster again. This one is
still going to be the landscape wide screen. Wow, the text there is significantly improved. This weird
helicopter staff thing he's holding is quite something. But overall, definitely an increase in quality. Let's just jump up to eight steps here and try
side-by-side just to see what we get. I'd go out on a limb here and say we almost might get fully legible movie title text with this result. >>
[laughter] >> Close. Agent Pumar. But like, you know, not bad. Look, there's even like a visible ring on
the finger. There's a defined watch face. The buildings are more realistic. Now, I want to try the infographic again on eight steps with the ternary
model. Okay. Definitely more defined in terms of the GPU cores and things of the sort. And then I'll just do a couple random ones for good measure. A
disgusting meal on eight steps. We'll see what we get. Yeah, all right. I can I can see that. This is
actually a fairly high-quality portrait photo of a person. So, I'm impressed. I regrettably can't show the prompt, but it did a good job here. This is
actually impressive. All right. And now for the last thing, just trying a portrait photo of a person high-quality 4K Ultra HD. And this is again on
eight steps. Okay, really not bad at all. The wrinkles in the shirt fabric are more or less
realistic. There is a visible button there with some like movement of the shirt past where the button would be in real life. That's actually pretty
cool. Okay, I've said this like four times. The last thing I want to do is just generate that in one step and we'll see. Okay, and that adds up. So,
very cool. And we'll just do it in four steps, which is the default, just to see our middle ground of
what we would have received there. Very similar. Cool. So, overall, that is going to conclude our first look and test of the Bonzai image binary model
or a one-bit image model, as I will likely include in the title. I did want to spend a little time in this video just showcasing how to actually get
this set up and running on a Windows machine, being that it can be somewhat hair-pullingly
frustrating. So, hopefully that is of assistance to folks who are interested in using this on a gaming PC they might have or something of the sort.
Overall, I'd say it's cool. It definitely is extremely fast and I like the ability to try the different models as well, so that ternary one, which is
a little larger, but a little more performant in terms of its um quality versus the binary one, which
is just really cool to see. So, this is interesting and Prism ML has had some cool stuff, namely the actual like eight-bit parameter model,
eight-billion parameter one-bit model was very, very cool because it actually worked surprisingly well considering the compression in size. So, it's
cool to see some additional things because this is like early work into hopefully being able to run really
impressive things on hardware that does not have very impressive price tags. So, it's always nice to see new things like this. I always find there's a
lot of interest in these sorts of things. So, I wanted to cover this and overall, that's going to conclude our first look and test of Bonzai image
ternary and binary. So, if you have any questions, please feel free to leave them in the comments and
thanks for watching.