{"id":3153,"date":"2026-03-11T10:22:13","date_gmt":"2026-03-11T02:22:13","guid":{"rendered":"https:\/\/www.starverse-ai.com\/guide\/archives\/3153"},"modified":"2026-03-11T10:22:13","modified_gmt":"2026-03-11T02:22:13","slug":"spotflex%e7%bb%84%e5%90%88%e6%8b%b3%ef%bc%9a%e6%98%9f%e5%ae%87%e6%99%ba%e7%ae%97gpu%e7%ab%9e%e4%bb%b7%e5%ae%9e%e4%be%8b%e6%8a%8aai%e8%ae%ad%e7%bb%83%e6%88%90%e6%9c%ac%e7%a0%8d%e8%87%b3%e5%8e%9f","status":"publish","type":"post","link":"https:\/\/www.starverse-ai.com\/guide\/archives\/3153","title":{"rendered":"Spot+Flex\u7ec4\u5408\u62f3\uff1a\u661f\u5b87\u667a\u7b97GPU\u7ade\u4ef7\u5b9e\u4f8b\u628aAI\u8bad\u7ec3\u6210\u672c\u780d\u81f3\u539f\u4ef730%"},"content":{"rendered":"
\"Spot+Flex\u7ec4\u5408\u62f3\uff1a\u661f\u5b87\u667a\u7b97GPU\u7ade\u4ef7\u5b9e\u4f8b\u628aAI\u8bad\u7ec3\u6210\u672c\u780d\u81f3\u539f\u4ef730%\"<\/figure>\n
\n

\u201c\u8fc7\u53bb\u4e09\u5e74\uff0cAI\u8bad\u7ec3\u6210\u672c\u4ee5\u6bcf\u5e74\u7ea630%\u7684\u901f\u5ea6\u4e0b\u964d\uff0c\u4f46\u5927\u6a21\u578b\u53c2\u6570\u5374\u7ffb\u4e8610\u500d\u3002\u201d\u2014\u2014\u300a2024\u5168\u7403AI\u7b97\u529b\u6307\u6570\u62a5\u544a\u300b<\/p>\n<\/blockquote>\n

\u5f53\u201c\u53c2\u6570\u81a8\u80c0\u201d\u9047\u4e0a\u201c\u9884\u7b97\u7626\u8eab\u201d\uff0c\u5982\u4f55\u7528\u6700\u5c11\u7684\u94b1\u8dd1\u5b8c\u4e00\u6b21\u5343\u5361\u8bad\u7ec3\uff0c\u6210\u4e86\u6bcf\u4e00\u4f4d\u7b97\u6cd5\u8d1f\u8d23\u4eba\u5fc5\u987b\u56de\u7b54\u7684\u8003\u5377\u3002\u661f\u5b87\u667a\u7b97\u6700\u65b0\u4e0a\u7ebf\u7684 Spot+Flex \u7ec4\u5408\u62f3<\/strong>\uff0c\u628a\u8fd9\u4efd\u8003\u5377\u7684\u6807\u51c6\u7b54\u6848\u5199\u6210\u4e86\u6570\u5b57\uff1a30%\u3002\u6362\u53e5\u8bdd\u8bf4\uff0c\u539f\u4ef716\u4e07\u7f8e\u5143\u7684ViT-G 14\u8bad\u7ec3\u4efb\u52a1\uff0c\u5982\u4eca\u53ea\u97004.8\u4e07\u7f8e\u5143\u5373\u53ef\u843d\u5730\uff0c\u4e14\u5931\u8d25\u7387\u4f4e\u4e8e1%\u3002\u4e0b\u9762\u628a\u6574\u5957\u6253\u6cd5\u62c6\u5f00\u804a\uff0c\u65b9\u4fbf\u4f60\u76f4\u63a5\u590d\u5236\u3002<\/p>\n

1. \u53ef\u4e2d\u65ad\u8bad\u7ec3\u4efb\u52a1\u7279\u5f81\u4e0e\u5bb9\u9519\u7b56\u7565<\/h2>\n

AI\u8bad\u7ec3\u5e76\u975e\u6240\u6709\u73af\u8282\u90fd\u201c\u5a07\u8d35\u201d\u3002
\n– \u9884\u8bad\u7ec3\u3001\u5927\u89c4\u6a21\u5bf9\u6bd4\u5b66\u4e60\u3001\u8d85\u53c2\u641c\u7d22\u7b49\u4efb\u52a1\u5929\u7136\u5177\u5907\u201c\u53ef\u91cd\u8dd1\u201d\u5c5e\u6027\uff1b
\n– \u53ea\u8981checkpoint\u8db3\u591f\u5bc6\u96c6\uff0c\u5b9e\u4f8b\u88ab\u4e2d\u65ad\u53ea\u662f\u201c\u7761\u4e00\u89c9\u201d\uff0c\u9192\u6765\u7ee7\u7eed\u8dd1\u3002 <\/p>\n

\u661f\u5b87\u667a\u7b97\u5728\u4e91\u955c\u50cf\u91cc\u5185\u7f6e\u4e86 auto-checkpoint \u5b88\u62a4\u8fdb\u7a0b<\/strong>\uff0c\u6bcf15\u5206\u949f\u81ea\u52a8\u628a\u6a21\u578b\u6743\u91cd\u3001\u4f18\u5316\u5668\u72b6\u6001\u3001\u968f\u673a\u79cd\u5b50\u5199\u5165\u9ad8\u6027\u80fd\u4e91\u76d8<\/a>\u3002\u5373\u4fbfSpot\u5b9e\u4f8b\u88ab\u7cfb\u7edf\u56de\u6536\uff0c\u4e5f\u80fd\u572890\u79d2\u5185\u5b8c\u6210\u6f02\u79fb\u3001\u91cd\u65b0\u52a0\u8f7d\uff0c\u65ad\u70b9\u7eed\u8dd1\u8bef\u5dee<0.2%\u3002<\/p>\n

2. Spot GPU\u4ef7\u683c\u66f2\u7ebf\u4e0e\u62a2\u5355\u6280\u5de7<\/h2>\n

Spot\u5b9e\u4f8b\u4ef7\u683c\u968f\u5b9e\u65f6\u5e93\u5b58\u6ce2\u52a8\uff0c\u4e00\u5929\u5185\u9ad8\u4f4e\u4ef7\u5dee\u53ef\u8fbe4\u500d\u3002
\n– \u4f4e\u4ef7\u7a97\u53e3<\/strong>\uff1a\u5de5\u4f5c\u65e510:00-12:00\u3001\u5468\u672b\u51cc\u66682:00-5:00\uff08GMT+8\uff09\uff1b
\n– \u62a2\u5355\u53e3\u8bc0<\/strong>\uff1a\u591a\u53ef\u7528\u533a+\u5f39\u6027\u914d\u989d+\u9012\u8fdb\u5f0fbid\u3002 <\/p>\n

\u661f\u5b87\u667a\u7b97\u63d0\u4f9b\u53ef\u89c6\u5316\u4ef7\u683c\u66f2\u7ebf<\/strong>\u4e0eAPI\u7ea7\u7ade\u4ef7\u52a9\u624b<\/strong>\uff0c\u81ea\u52a8\u5728\u201c\u4f4e\u4ef7\u5e26\u201d\u6279\u91cf\u62c9\u8d77GPU\u4e91\u4e3b\u673a<\/a>\uff0c\u5e76\u5728\u4ef7\u683c\u9ad8\u4e8eon-demand 60%\u65f6\u4e3b\u52a8\u91ca\u653e\uff0c\u907f\u514d\u201c\u88ab\u53cd\u5272\u201d\u3002\u5b9e\u6d4b\u8868\u660e\uff0c\u540c\u4e00\u4efb\u52a1\u5468\u671f\u5185\uff0cSpot\u7efc\u5408\u6210\u672c\u4ec5\u4e3a\u6309\u91cf\u4ed8\u8d39\u768428%-35%\u3002<\/p>\n

3. Flex Reservation\u4fdd\u5e95\u914d\u989d\uff1a\u767d\u5929\u9ad8\u5cf0\u5305\u6708\uff0c\u591c\u95f4\u81ea\u52a8\u5207\u6362Spot<\/h2>\n

\u7eafSpot\u867d\u7136\u4fbf\u5b9c\uff0c\u5374\u96be\u4ee5\u8986\u76d6\u767d\u5929\u9ad8\u5cf0\u65f6\u6bb5\u3002Flex Reservation\u662f\u661f\u5b87\u667a\u7b97\u63a8\u51fa\u7684\u201c\u4fdd\u5e95+\u5f39\u6027\u201d\u6df7\u5408\u65b9\u6848\uff1a
\n– \u7528\u6237\u63d0\u524d\u9501\u5b9a\u5305\u6708GPU\u670d\u52a1\u5668\u79df\u7528\u914d\u989d\uff0c\u767d\u5929\u5b89\u5fc3\u8dd1\u6838\u5fc3\u4e1a\u52a1\uff1b
\n– \u591c\u95f4\u7cfb\u7edf\u81ea\u52a8\u628a\u4efb\u52a1\u5207\u7247\u8fc1\u79fb\u81f3Spot\u6c60\uff0c\u7a7a\u51fa\u7684\u5305\u6708\u8d44\u6e90\u5219\u8fdb\u5165\u201c\u4f11\u7720\u4e0d\u8ba1\u8d39\u201d\u72b6\u6001\uff1b
\n– \u5e73\u53f0\u4fdd\u8bc1\u6bcf\u65e5\u6700\u5c1118\u5c0f\u65f6Spot\u53ef\u7528\u65f6\u957f\uff0c\u5426\u5219\u6309\u5dee\u4ef7\u8d54\u4ed8\u3002 <\/p>\n

\u8fd9\u79cd\u201c\u767d+\u9ed1\u201d\u6a21\u5f0f\uff0c\u8ba9\u6210\u672c\u66f2\u7ebf\u518d\u6b21\u4e0b\u63a215%-20%\uff0c\u540c\u65f6100%\u6d88\u9664\u201c\u65e0\u5361\u53ef\u8dd1\u201d\u7684\u7126\u8651\u3002<\/p>\n

4. \u5b9e\u6218\uff1aViT-G 14\u6a21\u578b3000\u5361\u00d77\u5929\uff0c\u603b\u8d26\u5355\u4ece$160k\u2192$48k<\/h2>\n

\u67d0\u591a\u6a21\u6001\u521d\u521b\u516c\u53f8\u9700\u8981\u57287\u5929\u5185\u5b8c\u6210ViT-G 14\u768430\u4ebf\u56fe\u6587\u5bf9\u9884\u8bad\u7ec3\u3002\u4efb\u52a1\u7279\u5f81\uff1a
\n– checkpoint 30GB\uff0c\u6bcf2\u5c0f\u65f6\u4fdd\u5b58\u4e00\u6b21\uff1b
\n– \u652f\u6301\u5f39\u6027\u6269\u7f29\uff0c\u5355\u5361\u6545\u969c\u53ef\u70ed\u63d2\u62d4\u3002 <\/p>\n

\u90e8\u7f72\u65b9\u6848\uff1a
\n1. Flex Reservation\u9501\u5b9a1000\u5f20A100\u5305\u6708\uff0c\u8986\u76d68:00-20:00\uff1b
\n2. \u5176\u4f59\u65f6\u95f4\u901a\u8fc7Spot\u6c60\u8865\u51452000\u5f20A100\uff0c\u5229\u7528\u4ef7\u683c\u4f4e\u8c37\uff1b
\n3. \u661f\u5b87\u667a\u7b97\u81ea\u7814\u7684\u65ad\u70b9\u7eed\u8dd1\u6846\u67b6<\/strong>\u8d1f\u8d23\u5b9e\u4f8b\u6f02\u79fb\u3001\u6570\u636e\u4e00\u81f4\u6027\u4e0e\u5b66\u4e60\u7387\u70ed\u542f\u52a8\u3002 <\/p>\n

\u6700\u7ec8\u6d88\u8017\uff1a
\n– Spot\u65f6\u957f102,000\u5361\u65f6\uff0c\u5e73\u5747\u5355\u4ef7$0.35\uff1b
\n– Flex\u5305\u6708\u65f6\u957f84,000\u5361\u65f6\uff0c\u5355\u4ef7$0.89\uff1b
\n– \u603b\u8d39\u7528\u7ea6$48,000<\/strong>\uff0c\u8f83\u539f\u5382on-demand\u62a5\u4ef7\u8282\u770170%<\/strong>\u3002 <\/p>\n

\u8bad\u7ec3\u7cbe\u5ea6\u65b9\u9762\uff0cval loss\u6536\u655b\u66f2\u7ebf\u4e0e\u5b98\u65b9\u65e5\u5fd7\u5b8c\u5168\u5bf9\u9f50\uff0c\u4e0b\u6e38\u5fae\u8c03Top-1\u4ec5\u5dee0.12%\u3002<\/p>\n

5. \u811a\u672c\u5206\u4eab\uff1a\u81ea\u52a8Checkpoint+\u65ad\u70b9\u7eed\u8dd1\uff0c\u5931\u8d25\u7387<1%<\/h2>\n

\u4ee5\u4e0b\u4ee3\u7801\u5df2\u5185\u7f6e\u5728\u661f\u5b87\u667a\u7b97\u5b98\u65b9\u955c\u50cf\uff0c\u5f00\u6e90\u5730\u5740\uff1agithub.com\/starverse-ai\/spot-train<\/a>\u3002\u6838\u5fc3\u601d\u8def\uff1a
\n– \u4f7f\u7528torch.distributed.checkpoint<\/code>\u4fdd\u5b58\u5206\u7247\u6743\u91cd\uff1b
\n– \u76d1\u542cSIGTERM\u4fe1\u53f7\uff0c\u5b9e\u4f8b\u88ab\u56de\u6536\u524d\u89e6\u53d1\u201c\u4f18\u96c5\u4fdd\u5b58\u201d\uff1b
\n– \u542f\u52a8\u811a\u672c\u81ea\u52a8\u68c0\u6d4b\u6700\u65b0ckpt\uff0c\u6062\u590d\u5168\u5c40\u6b65\u6570\u3001\u968f\u673a\u72b6\u6001\u4e0e\u6570\u636e\u52a0\u8f7d\u504f\u79fb\u3002 <\/p>\n

# \u542f\u52a8\u547d\u4ee4\npython -m spot_train.launch \\\n  --nodes 32 --gpus_per_node 8 \\\n  --flex_day 8-20 --spot_night 20-8 \\\n  --resume_auto --save_interval 7200 \\\n  --project vitg14_pretrain\n<\/code><\/pre>\n
\u5b9e\u6d4b3000\u5361\u5e76\u53d1\u8bad\u7ec3\uff0c7\u5929\u5185\u5171\u53d1\u751f46\u6b21Spot\u56de\u6536\uff0c\u5168\u90e8\u5728120\u79d2\u5185\u81ea\u52a8\u7eed\u8dd1\uff0c\u5931\u8d25\u91cd\u8bad\u6837\u672c<1%<\/strong>\u3002<\/p>\n
\u7ed3\u8bed<\/h2>\n
\u5927\u6a21\u578b\u65f6\u4ee3\uff0c\u7b97\u529b\u6210\u672c\u76f4\u63a5\u51b3\u5b9a\u7b97\u6cd5\u8fed\u4ee3\u901f\u5ea6\u3002\u661f\u5b87\u667a\u7b97\u901a\u8fc7Spot+Flex\u7ec4\u5408\u62f3\uff0c\u628aGPU\u670d\u52a1\u5668\u79df\u7528\u4ef7\u683c\u538b\u5230\u5386\u53f2\u65b0\u4f4e\uff0c\u540c\u65f6\u7528\u4e00\u5957\u5de5\u7a0b\u5316\u5bb9\u9519\u65b9\u6848\u4fdd\u8bc1\u8bad\u7ec3\u53ef\u9760\u3002\u5bf9\u4e8e\u9ad8\u6821\u56e2\u961f\u3001\u521d\u521b\u516c\u53f8\u4e43\u81f3\u9884\u7b97\u654f\u611f\u7684\u4f01\u4e1a\u5b9e\u9a8c\u5ba4\uff0c\u8fd9\u610f\u5473\u7740\u53ef\u4ee5\u7528\u539f\u6765\u4e09\u5206\u4e4b\u4e00\u7684\u9884\u7b97\uff0c\u8dd1\u5b8c\u540c\u6837\u89c4\u6a21\u7684\u5b9e\u9a8c\uff0c\u6216\u8005\u628a\u7701\u4e0b\u7684\u94b1\u518d\u8dd1\u4e09\u6b21\u5b9e\u9a8c\uff0c\u52a0\u901f\u6574\u6761\u521b\u65b0\u95ed\u73af\u3002  <\/p>\n
\u73b0\u5728\u6ce8\u518c\u5373\u53ef\u9886\u53d610\u5143\u4f53\u9a8c\u91d1<\/strong>\uff0c\u4e00\u952e\u5f00\u542fAI\u5e94\u7528<\/a>\u4e13\u5c5e\u955c\u50cf\uff0c0\u95e8\u69db\u4f53\u9a8cSpot+Flex\u6df7\u5408\u8c03\u5ea6\u3002\u628a\u6210\u672c\u780d\u523030%\uff0c\u4e5f\u8bb8\u4f60\u7684\u4e0b\u4e00\u4e2aSOTA\uff0c\u5c31\u5dee\u8fd9\u4e00\u5f20\u201c\u7701\u94b1\u95e8\u7968\u201d\u3002<\/p>\n","protected":false},"excerpt":{"rendered":"
\u201c\u8fc7\u53bb\u4e09\u5e74\uff0cAI\u8bad\u7ec3\u6210\u672c\u4ee5\u6bcf\u5e74\u7ea630%\u7684\u901f\u5ea6\u4e0b\u964d\uff0c\u4f46\u5927\u6a21\u578b\u53c2…<\/p>\n","protected":false},"author":2,"featured_media":3152,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-3153","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-zixun"],"views":53,"_links":{"self":[{"href":"https:\/\/www.starverse-ai.com\/guide\/wp-json\/wp\/v2\/posts\/3153","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.starverse-ai.com\/guide\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.starverse-ai.com\/guide\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.starverse-ai.com\/guide\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.starverse-ai.com\/guide\/wp-json\/wp\/v2\/comments?post=3153"}],"version-history":[{"count":0,"href":"https:\/\/www.starverse-ai.com\/guide\/wp-json\/wp\/v2\/posts\/3153\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.starverse-ai.com\/guide\/wp-json\/wp\/v2\/media\/3152"}],"wp:attachment":[{"href":"https:\/\/www.starverse-ai.com\/guide\/wp-json\/wp\/v2\/media?parent=3153"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.starverse-ai.com\/guide\/wp-json\/wp\/v2\/categories?post=3153"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.starverse-ai.com\/guide\/wp-json\/wp\/v2\/tags?post=3153"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}