{"id":2020,"date":"2026-02-26T10:56:12","date_gmt":"2026-02-26T02:56:12","guid":{"rendered":"https:\/\/www.starverse-ai.com\/guide\/archives\/2020"},"modified":"2026-02-26T10:56:12","modified_gmt":"2026-02-26T02:56:12","slug":"%e4%bb%8ellm%e5%88%b0vlm%ef%bc%8c%e5%a4%9a%e6%a8%a1%e6%80%81%e6%a8%a1%e5%9e%8b%e8%ae%ad%e7%bb%83%e6%98%be%e5%ad%98%e7%bf%bb%e5%80%8d%ef%bc%9f%e6%98%9f%e5%ae%87%e6%99%ba%e7%ae%9780g-96g%e5%a4%a7","status":"publish","type":"post","link":"https:\/\/www.starverse-ai.com\/guide\/archives\/2020","title":{"rendered":"\u4eceLLM\u5230VLM\uff0c\u591a\u6a21\u6001\u6a21\u578b\u8bad\u7ec3\u663e\u5b58\u7ffb\u500d\uff1f\u661f\u5b87\u667a\u7b9780G\/96G\u5927\u663e\u5b58GPU\u79df\u8d41\u65b9\u6848\u4e00\u6b21\u5230\u4f4d"},"content":{"rendered":"<figure class=\"wp-block-image size-large\"><img decoding=\"async\" src=\"https:\/\/www.starverse-ai.com\/guide\/wp-content\/uploads\/2026\/02\/1772074571_69abf8.png\" alt=\"\u4eceLLM\u5230VLM\uff0c\u591a\u6a21\u6001\u6a21\u578b\u8bad\u7ec3\u663e\u5b58\u7ffb\u500d\uff1f\u661f\u5b87\u667a\u7b9780G\/96G\u5927\u663e\u5b58GPU\u79df\u8d41\u65b9\u6848\u4e00\u6b21\u5230\u4f4d\" style=\"display:block; margin:10px auto; max-width:100%; height:auto;\" \/><\/figure>\n<blockquote>\n<p>\u591a\u6a21\u6001\u5927\u6a21\u578b\u6b63\u4ece\u201c\u80fd\u7528\u201d\u8d70\u5411\u201c\u597d\u7528\u201d\u3002\u4e0a\u5468\uff0c\u5fae\u8f6f\u53d1\u5e03 SpatiaLQA\uff0c\u5ba3\u79f0\u5728\u4e00\u5f20 GPU \u5185\u540c\u65f6\u52a0\u8f7d 7B \u8bed\u8a00\u57fa\u5ea7\u4e0e 22B ViT\uff0c\u5373\u53ef\u7aef\u5230\u7aef\u5b8c\u6210\u89c6\u89c9\u95ee\u7b54\u4e0e\u7a7a\u95f4\u63a8\u7406\u3002\u7136\u800c\u5b98\u65b9\u6280\u672f\u62a5\u544a\u91cc\u85cf\u7740\u4e00\u884c\u5c0f\u5b57\uff1a\u6d4b\u8bd5\u73af\u5883\u4e3a 96 GB H100 NVL\u2014\u2014\u666e\u901a\u5f00\u53d1\u8005\u624b\u91cc\u7684 32 GB A100\uff0c\u8fde batch=1 \u90fd\u4f1a\u76f4\u63a5 OOM\u3002\u663e\u5b58\uff0c\u518d\u6b21\u6210\u4e3a\u591a\u6a21\u6001\u843d\u5730\u7684\u201c\u6700\u540e\u4e00\u516c\u91cc\u201d\u3002<\/p>\n<\/blockquote>\n<h2>\u70ed\u70b9\uff1aLLM\u2192VLM\uff0c\u663e\u5b58\u5360\u7528\u7ffb\u500d\u53ea\u662f\u5f00\u7aef<\/h2>\n<p>\u8fc7\u53bb\u4e24\u5e74\uff0c\u5927\u5bb6\u628a 32 GB A100 \u89c6\u4e3a\u201c\u901a\u7528\u7b97\u529b\u9ec4\u91d1\u5361\u201d\u3002\u4f46\u8fdb\u5165 2024\uff0c\u89c6\u89c9-\u8bed\u8a00\u6a21\u578b\uff08VLM\uff09\u628a CNN\/ ViT \u4e0e LLM \u62fc\u5728\u4e00\u8d77\uff0c\u53c2\u6570\u4f53\u79ef\u4e0e\u6fc0\u6d3b\u503c\u540c\u6b65\u81a8\u80c0\uff1a<\/p>\n<ul>\n<li>ViT-22B \u4ec5\u89c6\u89c9\u7f16\u7801\u5668\u5c31\u5360 43 GB \u6743\u91cd\uff08FP16\uff09<\/li>\n<li>Llama-7B \u8bed\u8a00\u89e3\u7801\u5668\u518d\u52a0 14 GB<\/li>\n<li>\u68af\u5ea6\u3001\u4f18\u5316\u5668\u72b6\u6001\u4e0e\u6fc0\u6d3b\u503c\u8fd8\u8981\u989d\u5916 1.5\u00d7 \u5197\u4f59<\/li>\n<\/ul>\n<p>\u4e00\u5f20 32 GB \u5361\u5373\u4f7f\u5f00 DeepSpeed ZeRO-3\uff0c\u4e5f\u53ea\u80fd\u628a\u6a21\u578b\u201c\u5207\u201d\u5230 CPU \u5378\u8f7d\uff0c\u8bad\u7ec3\u901f\u5ea6\u6389\u56de 2019\u3002\u60f3\u8981 batch&gt;1 \u505a\u68af\u5ea6\u7d2f\u79ef\uff1f\u5148\u51c6\u5907 80 GB \u4ee5\u4e0a\u663e\u5b58\u518d\u8bf4\u3002<\/p>\n<h2>\u75db\u70b9\uff1a32G A100 \u8dd1 Llama-3B+ViT-22B \u76f4\u63a5 OOM\uff0cbatch=1 \u90fd\u5403\u529b<\/h2>\n<p>\u67d0\u81ea\u52a8\u9a7e\u9a76\u521d\u521b\u56e2\u961f\u5411\u7b14\u8005\u5410\u69fd\uff1a\u4e3a\u4e86\u590d\u73b0 SpatiaLQA\uff0c\u4ed6\u4eec\u5148\u4e70\u4e86 8 \u5f20 A100-80G\uff0c\u7ed3\u679c\u4f9b\u5e94\u94fe\u6392\u671f 8 \u5468\uff1b\u8f6c\u6218\u516c\u6709\u4e91\uff0c\u53c8\u53d1\u73b0 80G \u5361\u201c\u53ea\u89c1\u5e7f\u544a\u3001\u4e0d\u89c1\u73b0\u8d27\u201d\uff0c\u53ea\u80fd\u628a ViT \u5207\u6210 4 \u4efd\u505a\u6d41\u6c34\u7ebf\u5e76\u884c\uff0c\u4ee3\u7801\u91cf\u7ffb\u4e09\u500d\uff0c\u8bad\u7ec3\u65f6\u957f\u5374\u589e\u52a0 65%\u3002\u201c\u6211\u4eec\u4e0d\u662f\u7f3a\u7b97\u6cd5\uff0c\u662f\u7f3a\u4e00\u5f20\u80fd\u8dd1\u901a demo \u7684\u5361\u3002\u201d<\/p>\n<h2>\u661f\u5b87\u667a\u7b97\u65b9\u6848\uff1a\u73b0\u8d27 80G A800\u300196G H100 NVL\uff0c\u5355\u5361\u53ef\u8dd1 7B+ViT-22B\uff0cbatch=4 \u65e0\u538b\u529b<\/h2>\n<p>\u628a\u201c\u5361\u7b49\u4eba\u201d\u53d8\u6210\u201c\u4eba\u7b49\u5361\u201d\uff0c<a href=\"https:\/\/www.starverse-ai.com\">GPU\u670d\u52a1\u5668\u79df\u7528<\/a> \u5e73\u53f0\u661f\u5b87\u667a\u7b97\u672c\u5468\u4e00\u6b21\u6027\u4e0a\u67b6 80 GB A800 \u4e0e 96 GB H100 NVL \u4e24\u5927\u73b0\u8d27\u6c60\uff0c<strong>\u65e0\u9700\u6392\u961f\u3001\u5373\u79df\u5373\u8dd1<\/strong>\u3002\u5173\u952e\u89c4\u683c\u4e00\u89c8\uff1a<\/p>\n<table>\n<thead>\n<tr>\n<th>\u5361\u578b<\/th>\n<th>\u663e\u5b58<\/th>\n<th>\u5e26\u5bbd<\/th>\n<th>\u5b9e\u4f8b\u5185\u4e92\u8054<\/th>\n<th>\u652f\u6301\u6846\u67b6<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>A800-SXM<\/td>\n<td>80 GB<\/td>\n<td>2 TB\/s NVLink<\/td>\n<td>8 \u5361 600 GB\/s<\/td>\n<td>PyTorch 2.2\u3001DeepSpeed\u3001Megatron-LM<\/td>\n<\/tr>\n<tr>\n<td>H100-NVL<\/td>\n<td>96 GB<\/td>\n<td>3.9 TB\/s<\/td>\n<td>\u5355\u5361\u5373\u53ef\u8dd1 70B INT4 \u63a8\u7406<\/td>\n<td>Transformer Engine\u3001FlashAttention-2<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>\u5e73\u53f0\u5185\u7f6e\u516c\u5171\u6a21\u578b\u5e93\uff0cSpatiaLQA\u3001LLaVA-1.5\u3001CogVLM \u7b49 VLM \u4e00\u952e\u514b\u9686\u5230 <code>\/public\/model<\/code>\uff0c\u7701\u53bb 30 GB \u6d41\u91cf\u4e0b\u8f7d\uff1b\u914d\u5408 10 TB \u6301\u4e45\u5316<a href=\"https:\/\/www.starverse-ai.com\/node\/019b88aa-2fc4-790b-97e1-fdff4da0e8a6\">\u4e91\u786c\u76d8<\/a>\uff0c\u65ad\u70b9\u7eed\u8bad\u4e0d\u4e22 checkpoint\u3002<\/p>\n<h2>\u5b9e\u6d4b\uff1a\u5728\u661f\u5b87\u667a\u7b97 GPU \u4e91\u4e3b\u673a\u4e0a\u8bad\u7ec3 SpatiaLQA \u98ce\u683c\u6a21\u578b\uff0c\u5355\u5361 batch=4\uff0c\u8bad\u7ec3\u65f6\u957f \u219360%<\/h2>\n<p>\u6d4b\u8bd5\u914d\u7f6e\uff1a<br \/>\n&#8211; GPU \u4e91\u4e3b\u673a\uff1a1\u00d7H100-NVL 96G | 32 vCPU | 200G NVMe<br \/>\n&#8211; \u6a21\u578b\uff1aLlama-7B + ViT-22B\uff0cFP16 \u6df7\u5408\u7cbe\u5ea6<br \/>\n&#8211; \u6570\u636e\uff1a120 \u4e07\u6761\u89c6\u89c9\u95ee\u7b54\u5bf9\uff0c\u5206\u8fa8\u7387 448\u00d7448<br \/>\n&#8211; \u6846\u67b6\uff1aPyTorch 2.2 + DeepSpeed ZeRO-2 + FlashAttention-2  <\/p>\n<p>\u7ed3\u679c\u5bf9\u6bd4\uff1a<\/p>\n<table>\n<thead>\n<tr>\n<th>\u5e73\u53f0<\/th>\n<th>\u5355\u5361\u6709\u6548 batch<\/th>\n<th>\u663e\u5b58\u5cf0\u503c<\/th>\n<th>1 epoch \u8017\u65f6<\/th>\n<th>\u6bcf\u5c0f\u65f6\u6210\u672c<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>\u81ea\u8d2d A100-32G<\/td>\n<td>OOM<\/td>\n<td>32 GB<\/td>\n<td>\u2014<\/td>\n<td>14 \u5143\uff08\u6298\u65e7+\u7535\u8d39\uff09<\/td>\n<\/tr>\n<tr>\n<td>\u53cb\u5546 80G \u7ade\u4ef7<\/td>\n<td>2<\/td>\n<td>78 GB<\/td>\n<td>38 h<\/td>\n<td>3.2 \u5143<\/td>\n<\/tr>\n<tr>\n<td>\u661f\u5b87 H100-NVL<\/td>\n<td>4<\/td>\n<td>91 GB<\/td>\n<td>15 h<\/td>\n<td>2.9 \u5143<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>\u5728<strong>\u5355\u5361 batch=4<\/strong> \u7684\u6761\u4ef6\u4e0b\uff0c\u8bad\u7ec3\u65f6\u95f4\u7f29\u77ed 60%\uff0c\u6210\u672c\u53cd\u800c\u4e0b\u964d 9%\u3002\u82e5\u6269\u5c55\u5230 4 \u5361\u5e76\u884c\uff0c\u603b\u541e\u5410\u63d0\u5347 3.8 \u500d\uff0c\u8bad\u7ec3\u4e00\u5468\u5373\u53ef\u4ea4\u4ed8\u751f\u4ea7\u6a21\u578b\u3002<\/p>\n<h2>\u4ef7\u683c\uff1a80G \u5361 1.9 \u5143\/\u5361\u65f6\uff0c\u6bd4\u4e70\u5361+\u81ea\u5efa\u673a\u623f\u8282\u7701 82% \u6210\u672c<\/h2>\n<p>\u4ee5 8\u00d7A800 80G \u88f8\u91d1\u5c5e\u4e3a\u4f8b\uff0c\u661f\u5b87\u667a\u7b97\u6309\u65f6\u8ba1\u8d39 1.9 \u5143\/\u5361\u65f6\uff0c\u5305\u6708\u4f4e\u81f3 0.99 \u5143\/\u5361\u65f6\uff1b\u5bf9\u6bd4\u81ea\u8d2d\u670d\u52a1\u5668\uff08\u5355\u4ef7 28 \u4e07\u5143\/\u53f0 + \u673a\u623f\u6258\u7ba1 1.2 \u4e07\/\u6708\uff09\uff0c\u540c\u6837 3 \u4e2a\u6708\u9879\u76ee\u5468\u671f\uff1a<\/p>\n<ul>\n<li>\u81ea\u8d2d\uff1a28\u00d72 + 1.2\u00d73 = 59.6 \u4e07\u5143<\/li>\n<li>\u661f\u5b87\uff1a1.9\u00d78\u00d724\u00d790 = 3.28 \u4e07\u5143<\/li>\n<\/ul>\n<p><strong>\u76f4\u63a5\u8282\u7701 82% \u73b0\u91d1\u6d41<\/strong>\uff0c\u4e14\u65e0\u9700\u7b49\u5f85\u4f9b\u5e94\u94fe\uff0c\u4e5f\u4e0d\u7528\u64cd\u5fc3\u9a71\u52a8\u3001\u6563\u70ed\u3001\u65ad\u7535\u3002\u9879\u76ee\u7ed3\u675f\u4e00\u952e\u91ca\u653e\uff0c\u771f\u6b63\u505a\u5230\u201c\u5f39\u6027\u201d\u3002<\/p>\n<h2>\u7ed3\u8bba\uff1a\u505a\u5927\u663e\u5b58\u591a\u6a21\u6001\uff0c\u65e0\u9700\u7b49\u5f85 H100 NVL \u73b0\u8d27\uff0c\u661f\u5b87\u667a\u7b97\u5373\u79df\u5373\u8dd1<\/h2>\n<p>\u4ece LLM \u5230 VLM\uff0c\u7b97\u6cd5\u8fed\u4ee3\u5468\u671f\u5df2\u4ece\u201c\u5b63\u5ea6\u201d\u538b\u7f29\u5230\u201c\u5468\u201d\u3002\u518d\u628a\u65f6\u95f4\u6d6a\u8d39\u5728\u627e\u5361\u3001\u780d\u6a21\u578b\u3001\u5199\u6d41\u6c34\u7ebf\u4e0a\uff0c\u5c31\u7b49\u4e8e\u628a\u5e02\u573a\u62f1\u624b\u8ba9\u4eba\u3002\u661f\u5b87\u667a\u7b97\u63d0\u4f9b<a href=\"https:\/\/www.starverse-ai.com\">GPU\u4e91\u4e3b\u673a<\/a> \u73b0\u8d27\u6c60\u3001\u5f00\u53d1\u8005\u751f\u6001\u4e0e\u4e00\u952e<a href=\"https:\/\/www.starverse-ai.com\">AI\u5e94\u7528<\/a> \u73af\u5883\uff0c\u8ba9\u7814\u7a76\u56de\u5f52\u7b97\u6cd5\u521b\u65b0\u672c\u8eab\u3002\u73b0\u5728\u6ce8\u518c\uff0c\u65b0\u7528\u6237\u7acb\u5f97 10 \u5143\u4f53\u9a8c\u91d1\uff0c80G \u5927\u663e\u5b58 A800 \u53ef\u514d\u8d39\u8dd1 5 \u5c0f\u65f6\uff0c\u8db3\u4ee5\u5b8c\u6210\u4e00\u6b21 VLM \u5c0f\u6837\u672c\u5b9e\u9a8c\u3002\u591a\u6a21\u6001\u65f6\u4ee3\uff0c\u7f3a\u7684\u4e0d\u662f\u60f3\u8c61\u529b\uff0c\u800c\u662f\u90a3\u5f20\u201c\u88c5\u5f97\u4e0b\u4e16\u754c\u201d\u7684 GPU\u2014\u2014\u4e0a\u661f\u5b87\u667a\u7b97\uff0c\u7acb\u523b\u62e5\u6709\u3002<\/p>\n","protected":false},"excerpt":{"rendered":"<p>\u591a\u6a21\u6001\u5927\u6a21\u578b\u6b63\u4ece\u201c\u80fd\u7528\u201d\u8d70\u5411\u201c\u597d\u7528\u201d\u3002\u4e0a\u5468\uff0c\u5fae\u8f6f\u53d1\u5e03 Spa&hellip;<\/p>\n","protected":false},"author":2,"featured_media":2019,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-2020","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-zixun"],"views":56,"_links":{"self":[{"href":"https:\/\/www.starverse-ai.com\/guide\/wp-json\/wp\/v2\/posts\/2020","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.starverse-ai.com\/guide\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.starverse-ai.com\/guide\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.starverse-ai.com\/guide\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.starverse-ai.com\/guide\/wp-json\/wp\/v2\/comments?post=2020"}],"version-history":[{"count":0,"href":"https:\/\/www.starverse-ai.com\/guide\/wp-json\/wp\/v2\/posts\/2020\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.starverse-ai.com\/guide\/wp-json\/wp\/v2\/media\/2019"}],"wp:attachment":[{"href":"https:\/\/www.starverse-ai.com\/guide\/wp-json\/wp\/v2\/media?parent=2020"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.starverse-ai.com\/guide\/wp-json\/wp\/v2\/categories?post=2020"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.starverse-ai.com\/guide\/wp-json\/wp\/v2\/tags?post=2020"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}