Hello everyone. This conversation is about the dangers of information leakage from your LLM through shared GPU hardware. If you run a model in the cloud on a GPU shared with other tenants, someone nearby can learn certain things about it. They can determine what model you are running, how quickly it responds, and even what type of work it is currently doing, without touching the data, just by observing time. That's what this conversation is about.
What a roommate on the same equipment can learn about your LLM just from the time characteristics and what you can do about it . To start, I'm Branislav Brzak, a cybersecurity researcher at Stelthium. At Stelthium, we specialize in ensuring the security and surveillance of environments where AI systems operate. My area of interest is low-level security research related to GPUs. That's what this conversation covers .
When you share GPUs between tenants, you expect complete isolation, but the isolation provided may not be sufficient. In the case of this attack, the attacker doesn't even need special permissions, such as root access. It also doesn't need any traditional exploits. The only thing it does is measure the operating time of parts of the equipment that remain common to tenants. This is a demonstration video of what the victim and attacker control panels look like.
On the left we have the Quant 3 model performing inference. In the case, the attacker uses a side channel into which Quant 3 "leaks" while running. From this, it determines what kind of model it is and what text it is currently generating. These are just the main points, but not the whole story. We can determine the model, architecture, and depth with 97% accuracy.
We can determine the decoding speed with an accuracy of 99. 3%. We can determine when and how much, load periods , number of tokens, and response length. We can't determine exactly what type of work is being done: code, creativity, prose, JSON, or search. What we cannot know is the prompts themselves , the answers, or the content.
Only execution time. And all this on real H100s serving real LLMs. We use Nvidia hardware. GPU isolation is achieved using Multi-Instance GPU technology, or MIG for short. Each isolated MIG segment has its own physical compute units and dedicated VRAM.
This provides hardware isolation for these parts. In our example, the LLM we are servicing runs on a segment with three compute units and 48 GB of VRAM. The attacker is running on only one compute unit and 10 GB of VRAM. The computing units, memory capacity, bandwidth quotas, and fault tolerance are isolated. The L2 cache is also split for each instance.
There is no displacement between instances here. So, in that sense, there is nothing in common . The actual leaks are memory barriers, or membars, that operate at the GPU-wide level . The single instance barrier still affects every L2 partition on the map. Time is the only thing that is not isolated.
So when one tenant is busy, the others work just a little slower. The key point is that the GPU-wide memory barrier operates in every L2 partition on the card. The next slide bases the entire attack on this single fact. The vector is that the attacker only reads his own memory. It measures the time of only its own readings.
He never touches the memory of the victim. There are three steps. Loading from its own small buffer, measuring loading time and repeating every 5 microseconds. This primitive itself leaked through membar in MIG—not our discovery. Credit goes to the researchers behind the article " Behind Bars" on Usenix Security.
Many thanks for their work. Now try to look at the time difference. To the naked eye, comparing your tenant's downtime and LLM's maintenance, you won't even be able to tell the difference. The median latency is only 3 microseconds higher. However, for an attacker, those 3 microseconds are all it takes.
This is enough to gather all the information we have already talked about. Quantri with 8 billion parameters has been working on the victim segment for this entire time. All the data obtained were taken from a real model. Above— GPU activity over time, one burst per token. Below is the frequency spectrum of the attacker's own latency trace .
There is a fundamental frequency of 881 hertz, which is simply tokens per second, plus harmonics, since each token is a mixture of individual frequencies, not one pure one. This is a real trace, one token in the time domain. The token shows a total of 39 peaks. The peaks with green tops are layer challenges, while the orange ones are additional processing between layers. In the following slides, we will demonstrate how we determine the actual number of layers, even with these additional peaks.
Here we look at the decoding speed. Only by measuring time did we determine the token velocity blindly. So, speed itself is something that an attacker can accurately recover. But speed alone is not a clear reflection. Quantri 8 billion and Llama 3 8 billion differ by only 1.
6% in token velocity. So just by speed you would think it was the same model. This narrows the search, but does not provide an exact answer. We need something more precise, the number of layers. That's what the next few slides are about.
We use the same convolution and averaging method as before. This time we run it once for the model, not once for the token. The result is a number of bits that almost exactly matches an integer. The average deviation from the integer over 22 measurements is only 0. 028.
This is a hint. If it were just a fixed hardware artifact divided by different token rates, we would expect fractions, not whole numbers. Each count corresponds to the number of layers in the model plus one to three additional peaks. Each model retains its own signal imprints, combinations of tempo and harmonic structure. To be clear, this is not a trained machine learning classifier.
There is no training set, no labels , no configuration. It's less than 30 lines of code. The method checks each candidate depth from 16 to 48. It measures how clearly the expected signature of that depth appears in the actual trace . Then he chooses the one that fits best.
For a quantized model with 8 billion parameters, 36 layers confidently win by a margin of 4. 5 times over the next candidate . Because it is a best- fit test and not a lookup table, it also works with models that have never been seen before. This applies to both proprietary and unreleased models. Because he didn't need to "see" anything in advance .
And it works surprisingly well. The layer classifier also outputs a confidence level. According to our testing results, only two answers with the lowest confidence level turned out to be incorrect. This same method also recognizes retrained models and installations that use the same base model. This is because the architecture and number of layers remain the same.
让我们以三个相关的模型为例。Meta 的 Llama 3（80亿参数）拥有32层结构。同样来自 Meta 的 Llama 3.1（80亿参数）使用了相同的32层基础架构，仅进行了少量更新。而 DeepSeek 的 R1（仍为 Llama 80亿参数）则是对此版本的改进。
各层的计时差异仅在中位延迟上体现为1.6纳秒，其余信号看起来基本相同。尽管如此，我们的分类器仍然能够区分它们。现在我们将在同一通道上提出一个更深层次的问题：它不是关于模型是什么，而是关于它当前正在做什么。我们考虑四种类型的工作负载：散文、代码、JSON 和搜索。基于过去5秒的计时数据，我们每半秒做出一次判断。在40个半秒的判断中，有31次是正确的，并且通过多数投票正确识别了四个完整响应。这与上一视频中使用的分类器相同，包括所有的错误情况。
值得注意的是一个条件：这是在启用推测解码（speculative decoding）的情况下测量的。这是 VLLM 中的一个选项，而非默认设置。其机制在于：文本的可预测性会影响每一步接受的 token 草稿数量。这改变了步骤执行时间，而我们记录的正是这一信号。我们测试了11种配置：密集模型、专家混合（expert mix）、BF16 和 FP8。每次出现的图表都相同，只是强度不同。GPT（20亿参数）的离散度为70.2%，而 GLM（49亿参数）仅为0.9%。
在11次运行中，散文是最快的，尽管随机猜测也只有四分之一的几率。假设攻击者已经从前面的幻灯片知道了模型类型，并且推测解码已启用，它还可以确定正在生成何种类型的文本。现在我们将共享芯片从方程中完全排除。对于 MIG（多实例 GPU），攻击者必须位于相同的物理晶体上，但此攻击不需要这样做。
这里的设置是一台通过 NVLink 交换机连接的八块 H100 GPU 服务器。受害者（Victim）在 GPU 0和1上运行张量并行（tensor parallelism）。攻击者只需要同一服务器上的任何其他 GPU，在本例中是 GPU 2。通过 NVSwitch 进行的点对点访问是一个完全预期的功能，这是多 GPU 任务的标准原则，而非漏洞。MIG和张量并行也不能在同一设备上共存，所以这实际上是不同类别的部署，而不仅仅是同一攻击的重演。
问题出现在 GPU 2上。它通过 NVLink监控实际上位于 GPU 0上的内存，并测量路径上每次转换的时间。空闲状态的转换大约需要890纳秒。当受害者开始工作时，这一时间增加到300微秒。这是三次“镜像”的半秒记录，你甚至可以用肉眼数出 token 的数量。与 MIG相比，我们失去的是层位信息（layer bits）。在这个距离下，它低于噪声水平。因此，由于 NVLink 的存在，留下的足迹主要是时间性的，而非基于层数。
我们在相同的服务设置下测试了五个模型。只有模型本身揭示了它的身份，这种方法给出了79.2%的准确率。它在预期会出现的时间分析失败的地方失败了：Llama 3.5（80亿参数）和 R1 distill（80亿参数）是解码频率相差3Hz的模型，因此在 MIG 可以通过层数轻松区分它们的地方，这两个模型却融合了。结论是 NVLink中的信噪比比 MIG 差，因为层位信息丢失了。但较差的信噪比并不意味着安全。79%对比20%的基线——这仍然是真实的足迹，而非噪声。
首先是一些好消息：我们工作中的产品在安装后立即检测到了此攻击。Stealthium 监控每个 GPU上的每次内核启动，分析可疑工作负载，并通过完整的通知管道运行——并非每家供应商都能做到这一点。正是这使得此案脱颖而出。一个核心被重启了约3,000次，每次访问相同的40千字节内存，而不是转向新数据。
在整个过程中，只有1千字节的数据被发送回 CPU，因为有效载荷本身就是执行时间，而不是结果。在实时情况下，从开始到结束不到600毫秒，响应 token 少于50个。甚至在答案只输入了一半时我们就发现了这一点。因此得出三个主要结论：第一，放置位置是操作员实际可以控制的因素，所以不要将不受信任租户的 MIG 片段放在敏感模型旁边。
第二，要注意妥协风险。MIG消除了节点之间的连接，这意味着 MIG 上没有张量并行。另一方面，通过 NVLink进行分片将泄漏限制为仅时间参数，而非层级数据。第三，密切关注 GPU 本身。确保主机基础设施积极检查硬件上运行的恶意任务。这正是 Stealthium所做的，直接从内核行为检测此类请求。
共享 GPU 意味着只有运行时才能提供模型、速度和负载信息，即使在没有漏洞利用的 NVLink情况下也是如此。Stealthium 实时看到这种情况的发生。大多数组织仍然缺乏这种级别的可见性。再次感谢 Behind Bars的研究人员，这份报告完全基于他们的工作。谢谢大家观看。