{"id":957,"date":"2024-01-27T04:45:35","date_gmt":"2024-01-27T04:45:35","guid":{"rendered":"https:\/\/sparkonelabs.com\/home\/?p=957"},"modified":"2024-05-04T15:37:38","modified_gmt":"2024-05-04T15:37:38","slug":"gpu-server-for-llms-sd-pipelines-etc","status":"publish","type":"post","link":"https:\/\/sparkonelabs.com\/home\/2024\/01\/27\/gpu-server-for-llms-sd-pipelines-etc\/","title":{"rendered":"GPU Server for LLMs, SD Pipelines, etc."},"content":{"rendered":"\n

So, in experimenting with these neural network trained models, it has become apparent that either one must buy a lot of expensive individual GPU cards to achieve the core density and VRAM necessary for interesting experiments like training or local model services; or, one can set up a server with several slightly older model GPUs and GPU-Accelerators. <\/p>\n\n\n\n

https:\/\/www.linkedin.com\/pulse\/7-best-gpus-deep-learning-ai-2023-ashwani-patel<\/a> – Overview of some GPU cards, recommends Nvidia K80 and P100. Apparently P100 supports nvlink.<\/p>\n\n\n\n

https:\/\/h20195.www2.hp.com\/v2\/GetDocument.aspx?docname=4AA7-3070ENW<\/a> – General takeaway is that a z840, properly configured, can nvlink two Nvidia Quadro p6000s. This would result in 48gb vram, enough to run a 70b model.<\/p>\n\n\n\n

Or possibly 2 x NVidia Quadro p8000 cards, as these have Tensor Cores and possibly ray tracing, facilitating higher order matrix operation capability. <\/p>\n\n\n\n

NOTES (5\/4\/24):<\/p>\n\n\n\n

The following hardware has been obtained for the purposes of a general server:<\/p>\n\n\n\n

    \n
  1. HP z840 Workstation<\/li>\n\n\n\n
  2. 2x e5-4669 v4 22 core CPUs<\/li>\n\n\n\n
  3. 4TB SSD<\/li>\n\n\n\n
  4. 1x HMABAGR7A4R4N-VN<\/a> Sk-Hynix 128gb DDR4 LRDIMM 2666Mhz<\/li>\n\n\n\n
  5. 1x 128GB Hynix ECC 2666MHz PC4-21300 LRDIMM HMABAGL7C4R4N-VN ECC Memory<\/a><\/li>\n<\/ol>\n\n\n\n

    Thus at this point, the matter of filling the remaining 14 ram module slots is in order. Due to human error, the part numbers of the RAM DIMMs do not match completely, and HMABAGR7A4R4N-VN<\/a> will need to fill the odd numbered slots, while HMABAGL7C4R4N-VN<\/a> will need to fill the even numbered slots. Either this or vice versa should satisfy compatibility requirements. <\/p>\n","protected":false},"excerpt":{"rendered":"

    So, in experimenting with these neural network trained models, it has become apparent that either one must buy a lot of expensive individual GPU cards to achieve the core density and VRAM necessary for interesting experiments like training or local model services; or, one can set up a server with several slightly older model GPUs<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"nf_dc_page":"","om_disable_all_campaigns":false,"pagelayer_contact_templates":[],"_pagelayer_content":"","_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_hide_featured_image":false,"_jetpack_featured_image_duplicate_dismissed":0,"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[1],"tags":[],"class_list":["post-957","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"yoast_head":"\nGPU Server for LLMs, SD Pipelines, etc. - SparkOne() Labs Official Site<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/sparkonelabs.com\/home\/2024\/01\/27\/gpu-server-for-llms-sd-pipelines-etc\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"GPU Server for LLMs, SD Pipelines, etc. - SparkOne() Labs Official Site\" \/>\n<meta property=\"og:description\" content=\"So, in experimenting with these neural network trained models, it has become apparent that either one must buy a lot of expensive individual GPU cards to achieve the core density and VRAM necessary for interesting experiments like training or local model services; or, one can set up a server with several slightly older model GPUs\" \/>\n<meta property=\"og:url\" content=\"https:\/\/sparkonelabs.com\/home\/2024\/01\/27\/gpu-server-for-llms-sd-pipelines-etc\/\" \/>\n<meta property=\"og:site_name\" content=\"SparkOne() Labs Official Site\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/profile.php?id=61551444286908\" \/>\n<meta property=\"article:published_time\" content=\"2024-01-27T04:45:35+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-05-04T15:37:38+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/i0.wp.com\/sparkonelabs.com\/home\/wp-content\/uploads\/2024\/01\/logo_sparkone_12.png?fit=1152%2C896&ssl=1\" \/>\n\t<meta property=\"og:image:width\" content=\"1152\" \/>\n\t<meta property=\"og:image:height\" content=\"896\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"admin\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@sparkonelabs\" \/>\n<meta name=\"twitter:site\" content=\"@sparkonelabs\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"admin\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"1 minute\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/sparkonelabs.com\\\/home\\\/2024\\\/01\\\/27\\\/gpu-server-for-llms-sd-pipelines-etc\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/sparkonelabs.com\\\/home\\\/2024\\\/01\\\/27\\\/gpu-server-for-llms-sd-pipelines-etc\\\/\"},\"author\":{\"name\":\"admin\",\"@id\":\"https:\\\/\\\/sparkonelabs.com\\\/home\\\/#\\\/schema\\\/person\\\/ee3bd21f007818ac04abf178e749642c\"},\"headline\":\"GPU Server for LLMs, SD Pipelines, etc.\",\"datePublished\":\"2024-01-27T04:45:35+00:00\",\"dateModified\":\"2024-05-04T15:37:38+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/sparkonelabs.com\\\/home\\\/2024\\\/01\\\/27\\\/gpu-server-for-llms-sd-pipelines-etc\\\/\"},\"wordCount\":263,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\\\/\\\/sparkonelabs.com\\\/home\\\/#organization\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/sparkonelabs.com\\\/home\\\/2024\\\/01\\\/27\\\/gpu-server-for-llms-sd-pipelines-etc\\\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/sparkonelabs.com\\\/home\\\/2024\\\/01\\\/27\\\/gpu-server-for-llms-sd-pipelines-etc\\\/\",\"url\":\"https:\\\/\\\/sparkonelabs.com\\\/home\\\/2024\\\/01\\\/27\\\/gpu-server-for-llms-sd-pipelines-etc\\\/\",\"name\":\"GPU Server for LLMs, SD Pipelines, etc. - SparkOne() Labs Official Site\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/sparkonelabs.com\\\/home\\\/#website\"},\"datePublished\":\"2024-01-27T04:45:35+00:00\",\"dateModified\":\"2024-05-04T15:37:38+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/sparkonelabs.com\\\/home\\\/2024\\\/01\\\/27\\\/gpu-server-for-llms-sd-pipelines-etc\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/sparkonelabs.com\\\/home\\\/2024\\\/01\\\/27\\\/gpu-server-for-llms-sd-pipelines-etc\\\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/sparkonelabs.com\\\/home\\\/2024\\\/01\\\/27\\\/gpu-server-for-llms-sd-pipelines-etc\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/sparkonelabs.com\\\/home\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"GPU Server for LLMs, SD Pipelines, etc.\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/sparkonelabs.com\\\/home\\\/#website\",\"url\":\"https:\\\/\\\/sparkonelabs.com\\\/home\\\/\",\"name\":\"SparkOne() Labs Official Site\",\"description\":\"SparkOne Labs is a Science, Art and Technology blog. Email: sparkonelabs@sparkonelabs.com\",\"publisher\":{\"@id\":\"https:\\\/\\\/sparkonelabs.com\\\/home\\\/#organization\"},\"alternateName\":\"SparkOne Labs\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/sparkonelabs.com\\\/home\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/sparkonelabs.com\\\/home\\\/#organization\",\"name\":\"SparkOne() Labs\",\"alternateName\":\"SparkOne Labs\",\"url\":\"https:\\\/\\\/sparkonelabs.com\\\/home\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/sparkonelabs.com\\\/home\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/sparkonelabs.com\\\/home\\\/wp-content\\\/uploads\\\/2024\\\/01\\\/logo_sparkone_12.png\",\"contentUrl\":\"https:\\\/\\\/sparkonelabs.com\\\/home\\\/wp-content\\\/uploads\\\/2024\\\/01\\\/logo_sparkone_12.png\",\"width\":1152,\"height\":896,\"caption\":\"SparkOne() Labs\"},\"image\":{\"@id\":\"https:\\\/\\\/sparkonelabs.com\\\/home\\\/#\\\/schema\\\/logo\\\/image\\\/\"},\"sameAs\":[\"https:\\\/\\\/www.facebook.com\\\/profile.php?id=61551444286908\",\"https:\\\/\\\/x.com\\\/sparkonelabs\"]},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/sparkonelabs.com\\\/home\\\/#\\\/schema\\\/person\\\/ee3bd21f007818ac04abf178e749642c\",\"name\":\"admin\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/bc8a8c84ca952e624e7407ec63589bc91e7996574dee7a01c13a96c536c341e9?s=96&d=mm&r=g\",\"url\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/bc8a8c84ca952e624e7407ec63589bc91e7996574dee7a01c13a96c536c341e9?s=96&d=mm&r=g\",\"contentUrl\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/bc8a8c84ca952e624e7407ec63589bc91e7996574dee7a01c13a96c536c341e9?s=96&d=mm&r=g\",\"caption\":\"admin\"},\"sameAs\":[\"https:\\\/\\\/sparkonelabs.com\\\/home\"],\"url\":\"https:\\\/\\\/sparkonelabs.com\\\/home\\\/author\\\/admin\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"GPU Server for LLMs, SD Pipelines, etc. - SparkOne() Labs Official Site","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/sparkonelabs.com\/home\/2024\/01\/27\/gpu-server-for-llms-sd-pipelines-etc\/","og_locale":"en_US","og_type":"article","og_title":"GPU Server for LLMs, SD Pipelines, etc. - SparkOne() Labs Official Site","og_description":"So, in experimenting with these neural network trained models, it has become apparent that either one must buy a lot of expensive individual GPU cards to achieve the core density and VRAM necessary for interesting experiments like training or local model services; or, one can set up a server with several slightly older model GPUs","og_url":"https:\/\/sparkonelabs.com\/home\/2024\/01\/27\/gpu-server-for-llms-sd-pipelines-etc\/","og_site_name":"SparkOne() Labs Official Site","article_publisher":"https:\/\/www.facebook.com\/profile.php?id=61551444286908","article_published_time":"2024-01-27T04:45:35+00:00","article_modified_time":"2024-05-04T15:37:38+00:00","og_image":[{"width":1152,"height":896,"url":"https:\/\/i0.wp.com\/sparkonelabs.com\/home\/wp-content\/uploads\/2024\/01\/logo_sparkone_12.png?fit=1152%2C896&ssl=1","type":"image\/png"}],"author":"admin","twitter_card":"summary_large_image","twitter_creator":"@sparkonelabs","twitter_site":"@sparkonelabs","twitter_misc":{"Written by":"admin","Est. reading time":"1 minute"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/sparkonelabs.com\/home\/2024\/01\/27\/gpu-server-for-llms-sd-pipelines-etc\/#article","isPartOf":{"@id":"https:\/\/sparkonelabs.com\/home\/2024\/01\/27\/gpu-server-for-llms-sd-pipelines-etc\/"},"author":{"name":"admin","@id":"https:\/\/sparkonelabs.com\/home\/#\/schema\/person\/ee3bd21f007818ac04abf178e749642c"},"headline":"GPU Server for LLMs, SD Pipelines, etc.","datePublished":"2024-01-27T04:45:35+00:00","dateModified":"2024-05-04T15:37:38+00:00","mainEntityOfPage":{"@id":"https:\/\/sparkonelabs.com\/home\/2024\/01\/27\/gpu-server-for-llms-sd-pipelines-etc\/"},"wordCount":263,"commentCount":0,"publisher":{"@id":"https:\/\/sparkonelabs.com\/home\/#organization"},"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/sparkonelabs.com\/home\/2024\/01\/27\/gpu-server-for-llms-sd-pipelines-etc\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/sparkonelabs.com\/home\/2024\/01\/27\/gpu-server-for-llms-sd-pipelines-etc\/","url":"https:\/\/sparkonelabs.com\/home\/2024\/01\/27\/gpu-server-for-llms-sd-pipelines-etc\/","name":"GPU Server for LLMs, SD Pipelines, etc. - SparkOne() Labs Official Site","isPartOf":{"@id":"https:\/\/sparkonelabs.com\/home\/#website"},"datePublished":"2024-01-27T04:45:35+00:00","dateModified":"2024-05-04T15:37:38+00:00","breadcrumb":{"@id":"https:\/\/sparkonelabs.com\/home\/2024\/01\/27\/gpu-server-for-llms-sd-pipelines-etc\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/sparkonelabs.com\/home\/2024\/01\/27\/gpu-server-for-llms-sd-pipelines-etc\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/sparkonelabs.com\/home\/2024\/01\/27\/gpu-server-for-llms-sd-pipelines-etc\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/sparkonelabs.com\/home\/"},{"@type":"ListItem","position":2,"name":"GPU Server for LLMs, SD Pipelines, etc."}]},{"@type":"WebSite","@id":"https:\/\/sparkonelabs.com\/home\/#website","url":"https:\/\/sparkonelabs.com\/home\/","name":"SparkOne() Labs Official Site","description":"SparkOne Labs is a Science, Art and Technology blog. Email: sparkonelabs@sparkonelabs.com","publisher":{"@id":"https:\/\/sparkonelabs.com\/home\/#organization"},"alternateName":"SparkOne Labs","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/sparkonelabs.com\/home\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/sparkonelabs.com\/home\/#organization","name":"SparkOne() Labs","alternateName":"SparkOne Labs","url":"https:\/\/sparkonelabs.com\/home\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/sparkonelabs.com\/home\/#\/schema\/logo\/image\/","url":"https:\/\/sparkonelabs.com\/home\/wp-content\/uploads\/2024\/01\/logo_sparkone_12.png","contentUrl":"https:\/\/sparkonelabs.com\/home\/wp-content\/uploads\/2024\/01\/logo_sparkone_12.png","width":1152,"height":896,"caption":"SparkOne() Labs"},"image":{"@id":"https:\/\/sparkonelabs.com\/home\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/profile.php?id=61551444286908","https:\/\/x.com\/sparkonelabs"]},{"@type":"Person","@id":"https:\/\/sparkonelabs.com\/home\/#\/schema\/person\/ee3bd21f007818ac04abf178e749642c","name":"admin","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/bc8a8c84ca952e624e7407ec63589bc91e7996574dee7a01c13a96c536c341e9?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/bc8a8c84ca952e624e7407ec63589bc91e7996574dee7a01c13a96c536c341e9?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/bc8a8c84ca952e624e7407ec63589bc91e7996574dee7a01c13a96c536c341e9?s=96&d=mm&r=g","caption":"admin"},"sameAs":["https:\/\/sparkonelabs.com\/home"],"url":"https:\/\/sparkonelabs.com\/home\/author\/admin\/"}]}},"jetpack_sharing_enabled":true,"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/sparkonelabs.com\/home\/wp-json\/wp\/v2\/posts\/957","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/sparkonelabs.com\/home\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/sparkonelabs.com\/home\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/sparkonelabs.com\/home\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/sparkonelabs.com\/home\/wp-json\/wp\/v2\/comments?post=957"}],"version-history":[{"count":4,"href":"https:\/\/sparkonelabs.com\/home\/wp-json\/wp\/v2\/posts\/957\/revisions"}],"predecessor-version":[{"id":1234,"href":"https:\/\/sparkonelabs.com\/home\/wp-json\/wp\/v2\/posts\/957\/revisions\/1234"}],"wp:attachment":[{"href":"https:\/\/sparkonelabs.com\/home\/wp-json\/wp\/v2\/media?parent=957"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/sparkonelabs.com\/home\/wp-json\/wp\/v2\/categories?post=957"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/sparkonelabs.com\/home\/wp-json\/wp\/v2\/tags?post=957"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}