Watch this. I'm just going to tell my agent what I want. Get me Quen 3 4 billion parameter model, the MLX version in 3bit. There it is. Q134 billion.
I'm not even giving it the exact model name. The transcription even messed it up. Let's go. Got it. Already on your shelf. No download needed. It just
found the right one, grabbed it, and loaded it up. Gave me the location if I want to use it in
another tool or gave me the exact command I need to run that. And check out where it found the file. It found it on this external Aacasis drive under
MLX community quen 34B3bit. Notice that's not the hugging face cache with gibberish folder names and blobs and weird file names and references and
snapshots. I can't stand that. Instead, it's exactly in the right place where it should be named the
way I want it. And this is how I get my LLM and my image and video models now. And I think by the end of this video, you're going to like this method
so much that you're going to end up using this, too. Now, here's the thing. This right here is the brand new M5 Max. I'm not bragging or flexing or
anything. I just need it. Okay. And I did get this one with 8 TB of storage. And I only have 1.8 TB
left on this thing. The reason I'm running out of space is, can you guess? Yeah. Models, local, image generation weights, video generation weights,
multi-quant variants of the same model because I want to be able to compare. I need it for this channel. Hundreds of gigabytes just evaporate. And the
worst part is half of these are the same files I already have on another machine somewhere else or
even on the same machine in multiple different locations. I just couldn't find them. So I redownloaded them. So being that I'm a software developer, I
built a tool for this. Not by hand. Come on. It's open source. It's called Model Shelf. And it ships with an agent that teaches Claude Code or really
any agent that can run a shell command to use it automatically. I'm going to walk you through that.
But first, the hardware that makes this whole workflow actually viable. Now, you might have seen a bunch of my videos on these little mini Aasacasis
things. And so did Aasis. So, they were kind enough to sponsor this video and ship me the new Aasis TB504. This is their new Thunderbolt 5 80 Gbit per
second 4bay M.2 NVME drive external enclosure. It is a hefty boy. There's a lot more detail on the
Kickstarter page of this. Uh, by the way, the Kickstarter is just how some brands launch their new products now. Even established brands do it.
Pricing, launch updates, all of that will be in the description down below. But here's how I describe this thing. It's not just another SSD enclosure.
Let's plug this in. And I want you to listen to the sound right now. This thing sits just between the
portable powered SSD that we have here and a full NAS rig that most of us don't actually want to deal with. Now I've turned it on and I can't hear a
single thing. That's because this thing is actually completely fanless. I'm going to bring this up to the mic so you can hear this. Yeah, there's zero
sound coming here. It's kind of like using the MacBook Air in that you can be absolutely quiet cuz
there's no fans. And this is very different than that dash I showed in a video last year where I had to move it away from my desk in order to even be
able to use it because it was so loud. The other unique thing about this one is that it gives each bay its own 80 gigabit lane, not split four ways,
each one. But how is this relevant to what we're talking about today? Here's the thing about local
LLMs. Quen 3 14 billion parameter model at infor quantization is 8 GB. A 70 billion parameter class quantization is 40 plus gigabytes. Image and video
weights can get up to hundreds. And if you're doing rag data sets, vector databases, there going to be gigs upon gigs. So yeah, this is an external
enclosure that can do all kinds of things like 8K video, big photo libraries, but that's not why
we're here today. And as soon as I plug this thing in, there are my four drives that I've loaded up in there available to use right away. Here's just
one of the drives. Yeah, we're getting up there. 6,500 right 6,000 read around there. And that's Blackmagic disc speed, which is generally a little
bit slower than Amorphice. So, here is Amorphice 6900 read and 6600, right? And they're running at the
same time on two different discs. So, this thing is fast and quiet, and it's got two 80 GB down ports, so you can use them to plug in monitors, for
example, which is not something you usually see in enclosures like this. Now, a fast drive doesn't alone solve the actual problem. Where do we put our
models? How does the agent find them? How do you download models? And how do you keep track of them?
Well, before model shelf, I used to just create a models folder and hope to remember. But I knew there had to be a better way. And that's what this
is. This is the GitHub repo, and you can install it for yourself with just one command. It's right here in the documentation actually. Model shelf at
Alexiskin 1/Modshelf. This will do two things. one is it'll get that tool model shelf, but it'll also
add the skill that I mentioned before. So the second you install it, cloud code already knows how to use it. You don't have to teach the agent
anything. But if you don't have cloud code, if you just rather use model shelf, you can still use it with your favorite Python environment. Once it's
installed, you can run model shelf in it and boom, this will tell me what has model shelf already installed
on it. for example, it's already installed locally on this machine, but we have a few new drives right here where it's not yet. So, let's pick one of
those. And if I go to that drive, there it is. Model shelf. It's got models. It's got the directory structure, your Llama CPP models, your MLX models,
and your other models, safe tensors. If you're not using a Mac, for example, you want to use VLM or
Llama CPP on another machine, but that uses safe tensors file type. There's no path to copy and paste. You're done. Now, here's the part that makes
the agent in the scale really useful. You might have asked an agent previously for a model, right? And it says something like, "Oh, that model doesn't
exist because it's training data is maybe 6 months old." Yeah. Well, with model shelf, the agent
doesn't argue. It actually checks the live hub. The hub is the source of truth, not the model's memory. Get me Quen 3.5, the 9 billion parameter model
in 4bit quadization for llama CPP. By the way, I'm using Super Whisper. Not sponsored or anything, but I kind of like it holding down the button and
speaking to it. It's just so easy. However, it doesn't always get things right. Like Qen instead of
QWEN and Llama CPP is spelled wrong. But anyway, let's see if it gets it. So, there it goes. It loaded the skill. The input is loose. So, I'll search
the Hive first. Found it and found it exactly what it needs, what format it needs. It's going to grab Unsllo version of it because it's the cleanest
GGUF repo with the most downloads. And now it's grabbing it because I don't have it anywhere local.
But if you try this with cloud code without model shelf, it'll tell you confidently that model does not exist. Unless you of course ask it to check
the hub first, but model shelf takes care of that for you. Done. Look at that. It gave me the full path. So now all I got to do is just plug that into
my Llama CPP. It even gave me the CLI command if I want to run Llama CBP and Llama server commands
too, pointing to the exact place where that model lives. now which is going to be on this cases drive one under GGUF unsloth there it is squen 3.59b
there's the gguf file now you can use model shelf with any drives internal external or a combination but here's where the 4 bay architecture of this
aasis actually pays off each bay shows up as its own volume on the Mac in this case not one big stripe
you could we'll get to that in a moment but now it's four independent drives each one with its own 80 GB lane to the controller. And you don't have to
fill all four on day one. Start with one or two drives and add more as your model library grows or use some drives for some things for like MLX. Uh
use another drive for GGUF. Run it with Llama CLI and give me a test prompt. And there it goes. In
one paragraph, explain what 4-bit quantization does to a large language model and what the main trade-off is. So yeah, it started thinking, but let's
increase the token limit so we actually get an answer. And while that's happening, I want to keep an eye on this. Yeah. So there it goes. It went from
68 GB usage on the system to about 76, which makes perfect sense. And if we check the GPU history,
there's that little blue spike that represents the GPU being used. So because this is Thunderbolt 5, your model loads instantly into memory from the
Aacasis drive, the external drive used by the LLM runtime directly from memory. So now it's loaded in and that is a really fast transfer. Here's our
final answer. 4-bit quantization compresses a large language model blah blah. Now the internal drive
is still faster on this machine because this is an insane M5 max with 8 tab. So I'm getting like 15,000 megabytes per second which is insane. So it's
going to be a little bit faster. For example, a 70 billion parameter model is going to be a few seconds later time to first token than if the 70
billion weights lived on my internal SSD. But that's the trade-off. Okay, so each bay is fast, but here's
where it gets interesting. I put model shelf on three of these drives, and I put a couple of different models on each one. List the models that are
available. Model shelf is going to look through everything that's available that has model shelf initialized and give me the listing of all the local
models that are available. There it is. Quen 3.59 billion for Llama CPP. Gemma 34B for Llama CPP on
another drive. Quen 34B3bit MLX on a third drive. And on the local shelf, I had Barttowski's small LM2 installed. So even though I have my models in
multiple different places, as long as model shelf knows about it, it can find them all and let your agent know about it. So the agent doesn't care
about which bay a model lives on. It just asks model shelf, gets a path, and uses it. and any one of
them could move to another Mac and still works. Now, here's the stuff I actually care about. First, the paths are clean. It's GGF, then on Sloth or
Bowski or whoever makes the models, the publisher, then Quen, the name of the model, I should say, the repo name, and finally, we get the file. That's
literally the model's address on HuggingFace. Do I need to remind you what HuggingFace's own cache
looks like for models? Look at this. Models dash Quen Dash. What? What is this? And then you have blobs. What are these files? Those are your model
weights, but how do you even make sense of them? If I need to point Llama CBP or MLX at one of these, where do I point it at? This always bothered me.
There's also refs main. There's snapshots. That's where the configs are. It's just a mess. Oh, and
the best part is there's no server running in the background. No MCP, no demon, nothing sitting in my menu bar. is just a shell command that spits out
JSON. Oh, and one more thing. If I unplug this, if it's not plugged in, the agent doesn't just quietly dump 8 gigs onto my internal SSD anywhere and
act like everything is fine. It'll stop and tell me the drive is not mounted or give me the option
to use the internal. It just won't do it without my permission. Now, here is one more fun trick we can do with this enclosure. And if you remember my
video from last year where I had three of these external drives all plugged into my Mac at the exact same time with three cables, I was able to RAID
zero them. So have them all treated as one big drive in RAID zero configuration made them all
essentially faster and have more space together. For AI storage, RAID zero is the right call. And there's a few reasons for that. One, models are
regeneratable. Every weight on your shelf can be downloaded from the hub. There's no original you can lose. So redundancy, which other RAID options
give you, like RAID 1, RAID 5, RAID 6, RAID zero doesn't give you redundancy. It gives you speed. So if
you lose one drive, it's all gone. But redundancy isn't the problem in this case with models. Two, inference is bandwidth bound. So loading a 70
billion parameter model into VRAM means moving 40 gigs through storage controller as fast as possible. We saw that the internal drive gets up to
15,000 megabytes per second. Rate zero with four drives should give you theoretically more than that. I'm
about to try that. And three, I already mentioned if a stripe fails, you just redownload the model. For video projects or photos, don't do that. You
want to use RAID 5 or six for that. Those files are irreplaceable. So models, that's fine. Here I am in dis utility on a Mac. I go to RAID assistant.
I pick striped RAID zero and then select all my drives TB504. Let's go with APFS and chunk size of
256 and create the RAID array. Now, this will erase all my models that are on there already, but I'm curious about the speed here. RAID created
successfully. And there it is right there. TB504. Let's check out our disc speed. Well, it's about the same. But I guess it was worth the test anyway.
Now, you know, I can just initialize model shelf on it. Get me Gemma 3 4 billion parameter model in MLX
format uh at 4bit quantization. There it is. It's grabbing it building the structure getting all the files from the hub and I mean hugging face hub.
You don't leave that comment. So now if I want to use the model it's there and available and all I got to do is just tell my agent go ahead. You were
supposed to record all that. So there we go. There we go. Resolve still works and the agent doesn't
care. So, remember where we started? 8 terabytes out of space, drowning in models. Well, not anymore. They're going to live on the TB504 now. My
internal drive will have space. The agent will find all the models wherever they sit. Model shelf is open source. You can check it out. The Aasis
TB504 is on Kickstarter. Both links are down below. There's an early bird special while it lasts. Let me know
in the comments how you store models right now and if you think you're going to use model shelf. If you like this video, you're going to like this one
over here. Thanks for watching and I'll see you next time.