{"id":2770,"date":"2024-07-03T14:31:17","date_gmt":"2024-07-03T14:31:17","guid":{"rendered":"https:\/\/fadyanwar.com\/?p=2770"},"modified":"2024-07-03T14:31:20","modified_gmt":"2024-07-03T14:31:20","slug":"running-a-llama-model-locally","status":"publish","type":"post","link":"https:\/\/fadyanwar.com\/index.php\/2024\/07\/03\/running-a-llama-model-locally\/","title":{"rendered":"Running a LLaMa Model Locally"},"content":{"rendered":"\n

Something special about running an open source LLM model <\/a>locally. Few years ago, one would be too excited for merely calling GPT’s API using free credits after months of being on a waiting list just to get access. Now, I get to run such models locally as shown below, with no subsciption and with no waiting. Isn’t this truely exciting?<\/p>\n\n\n\n

\"\"<\/figure>\n","protected":false},"excerpt":{"rendered":"

Something special about running an open source LLM model locally. Few years ago, one would be too excited for merely calling GPT’s API using free credits after months of being on a waiting list just to get access. Now, I get to run such models locally as shown below, with no subsciption and with no […]<\/p>\n","protected":false},"author":1,"featured_media":2771,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_editorskit_title_hidden":false,"_editorskit_reading_time":0,"_editorskit_is_block_options_detached":false,"_editorskit_block_options_position":"{}","_vp_format_video_url":"","_vp_image_focal_point":[],"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[1],"tags":[],"class_list":["post-2770","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-technology"],"yoast_head":"\nRunning a LLaMa Model Locally - Fady Anwar<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/fadyanwar.com\/index.php\/2024\/07\/03\/running-a-llama-model-locally\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Running a LLaMa Model Locally - Fady Anwar\" \/>\n<meta property=\"og:description\" content=\"Something special about running an open source LLM model locally. Few years ago, one would be too excited for merely calling GPT’s API using free credits after months of being on a waiting list just to get access. Now, I get to run such models locally as shown below, with no subsciption and with no […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/fadyanwar.com\/index.php\/2024\/07\/03\/running-a-llama-model-locally\/\" \/>\n<meta property=\"og:site_name\" content=\"Fady Anwar\" \/>\n<meta property=\"article:published_time\" content=\"2024-07-03T14:31:17+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-07-03T14:31:20+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/fadyanwar.com\/wp-content\/uploads\/2024\/07\/photo1719947525.jpeg\" \/>\n\t<meta property=\"og:image:width\" content=\"948\" \/>\n\t<meta property=\"og:image:height\" content=\"738\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Fady Anwar\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@fadyanwar\" \/>\n<meta name=\"twitter:site\" content=\"@fadyanwar\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Fady Anwar\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"1 minute\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/fadyanwar.com\\\/index.php\\\/2024\\\/07\\\/03\\\/running-a-llama-model-locally\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/fadyanwar.com\\\/index.php\\\/2024\\\/07\\\/03\\\/running-a-llama-model-locally\\\/\"},\"author\":{\"name\":\"Fady Anwar\",\"@id\":\"https:\\\/\\\/fadyanwar.com\\\/#\\\/schema\\\/person\\\/b66e3277ceba346f7053a83464e90b03\"},\"headline\":\"Running a LLaMa Model Locally\",\"datePublished\":\"2024-07-03T14:31:17+00:00\",\"dateModified\":\"2024-07-03T14:31:20+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/fadyanwar.com\\\/index.php\\\/2024\\\/07\\\/03\\\/running-a-llama-model-locally\\\/\"},\"wordCount\":65,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\\\/\\\/fadyanwar.com\\\/#\\\/schema\\\/person\\\/b66e3277ceba346f7053a83464e90b03\"},\"image\":{\"@id\":\"https:\\\/\\\/fadyanwar.com\\\/index.php\\\/2024\\\/07\\\/03\\\/running-a-llama-model-locally\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/i0.wp.com\\\/fadyanwar.com\\\/wp-content\\\/uploads\\\/2024\\\/07\\\/photo1719947525.jpeg?fit=948%2C738&ssl=1\",\"articleSection\":[\"Technology\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/fadyanwar.com\\\/index.php\\\/2024\\\/07\\\/03\\\/running-a-llama-model-locally\\\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/fadyanwar.com\\\/index.php\\\/2024\\\/07\\\/03\\\/running-a-llama-model-locally\\\/\",\"url\":\"https:\\\/\\\/fadyanwar.com\\\/index.php\\\/2024\\\/07\\\/03\\\/running-a-llama-model-locally\\\/\",\"name\":\"Running a LLaMa Model Locally - Fady Anwar\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/fadyanwar.com\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/fadyanwar.com\\\/index.php\\\/2024\\\/07\\\/03\\\/running-a-llama-model-locally\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/fadyanwar.com\\\/index.php\\\/2024\\\/07\\\/03\\\/running-a-llama-model-locally\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/i0.wp.com\\\/fadyanwar.com\\\/wp-content\\\/uploads\\\/2024\\\/07\\\/photo1719947525.jpeg?fit=948%2C738&ssl=1\",\"datePublished\":\"2024-07-03T14:31:17+00:00\",\"dateModified\":\"2024-07-03T14:31:20+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/fadyanwar.com\\\/index.php\\\/2024\\\/07\\\/03\\\/running-a-llama-model-locally\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/fadyanwar.com\\\/index.php\\\/2024\\\/07\\\/03\\\/running-a-llama-model-locally\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/fadyanwar.com\\\/index.php\\\/2024\\\/07\\\/03\\\/running-a-llama-model-locally\\\/#primaryimage\",\"url\":\"https:\\\/\\\/i0.wp.com\\\/fadyanwar.com\\\/wp-content\\\/uploads\\\/2024\\\/07\\\/photo1719947525.jpeg?fit=948%2C738&ssl=1\",\"contentUrl\":\"https:\\\/\\\/i0.wp.com\\\/fadyanwar.com\\\/wp-content\\\/uploads\\\/2024\\\/07\\\/photo1719947525.jpeg?fit=948%2C738&ssl=1\",\"width\":948,\"height\":738},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/fadyanwar.com\\\/index.php\\\/2024\\\/07\\\/03\\\/running-a-llama-model-locally\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/fadyanwar.com\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Running a LLaMa Model Locally\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/fadyanwar.com\\\/#website\",\"url\":\"https:\\\/\\\/fadyanwar.com\\\/\",\"name\":\"Fady Anwar\",\"description\":\"\",\"publisher\":{\"@id\":\"https:\\\/\\\/fadyanwar.com\\\/#\\\/schema\\\/person\\\/b66e3277ceba346f7053a83464e90b03\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/fadyanwar.com\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":[\"Person\",\"Organization\"],\"@id\":\"https:\\\/\\\/fadyanwar.com\\\/#\\\/schema\\\/person\\\/b66e3277ceba346f7053a83464e90b03\",\"name\":\"Fady Anwar\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/a9172040bbc3bbe24fb49d59dac20da030af1f5ff628126c979a1d4b71eaed41?s=96&d=mm&r=g\",\"url\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/a9172040bbc3bbe24fb49d59dac20da030af1f5ff628126c979a1d4b71eaed41?s=96&d=mm&r=g\",\"contentUrl\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/a9172040bbc3bbe24fb49d59dac20da030af1f5ff628126c979a1d4b71eaed41?s=96&d=mm&r=g\",\"caption\":\"Fady Anwar\"},\"logo\":{\"@id\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/a9172040bbc3bbe24fb49d59dac20da030af1f5ff628126c979a1d4b71eaed41?s=96&d=mm&r=g\"}}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Running a LLaMa Model Locally - Fady Anwar","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/fadyanwar.com\/index.php\/2024\/07\/03\/running-a-llama-model-locally\/","og_locale":"en_US","og_type":"article","og_title":"Running a LLaMa Model Locally - Fady Anwar","og_description":"Something special about running an open source LLM model locally. Few years ago, one would be too excited for merely calling GPT’s API using free credits after months of being on a waiting list just to get access. Now, I get to run such models locally as shown below, with no subsciption and with no […]","og_url":"https:\/\/fadyanwar.com\/index.php\/2024\/07\/03\/running-a-llama-model-locally\/","og_site_name":"Fady Anwar","article_published_time":"2024-07-03T14:31:17+00:00","article_modified_time":"2024-07-03T14:31:20+00:00","og_image":[{"width":948,"height":738,"url":"https:\/\/fadyanwar.com\/wp-content\/uploads\/2024\/07\/photo1719947525.jpeg","type":"image\/jpeg"}],"author":"Fady Anwar","twitter_card":"summary_large_image","twitter_creator":"@fadyanwar","twitter_site":"@fadyanwar","twitter_misc":{"Written by":"Fady Anwar","Est. reading time":"1 minute"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/fadyanwar.com\/index.php\/2024\/07\/03\/running-a-llama-model-locally\/#article","isPartOf":{"@id":"https:\/\/fadyanwar.com\/index.php\/2024\/07\/03\/running-a-llama-model-locally\/"},"author":{"name":"Fady Anwar","@id":"https:\/\/fadyanwar.com\/#\/schema\/person\/b66e3277ceba346f7053a83464e90b03"},"headline":"Running a LLaMa Model Locally","datePublished":"2024-07-03T14:31:17+00:00","dateModified":"2024-07-03T14:31:20+00:00","mainEntityOfPage":{"@id":"https:\/\/fadyanwar.com\/index.php\/2024\/07\/03\/running-a-llama-model-locally\/"},"wordCount":65,"commentCount":0,"publisher":{"@id":"https:\/\/fadyanwar.com\/#\/schema\/person\/b66e3277ceba346f7053a83464e90b03"},"image":{"@id":"https:\/\/fadyanwar.com\/index.php\/2024\/07\/03\/running-a-llama-model-locally\/#primaryimage"},"thumbnailUrl":"https:\/\/i0.wp.com\/fadyanwar.com\/wp-content\/uploads\/2024\/07\/photo1719947525.jpeg?fit=948%2C738&ssl=1","articleSection":["Technology"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/fadyanwar.com\/index.php\/2024\/07\/03\/running-a-llama-model-locally\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/fadyanwar.com\/index.php\/2024\/07\/03\/running-a-llama-model-locally\/","url":"https:\/\/fadyanwar.com\/index.php\/2024\/07\/03\/running-a-llama-model-locally\/","name":"Running a LLaMa Model Locally - Fady Anwar","isPartOf":{"@id":"https:\/\/fadyanwar.com\/#website"},"primaryImageOfPage":{"@id":"https:\/\/fadyanwar.com\/index.php\/2024\/07\/03\/running-a-llama-model-locally\/#primaryimage"},"image":{"@id":"https:\/\/fadyanwar.com\/index.php\/2024\/07\/03\/running-a-llama-model-locally\/#primaryimage"},"thumbnailUrl":"https:\/\/i0.wp.com\/fadyanwar.com\/wp-content\/uploads\/2024\/07\/photo1719947525.jpeg?fit=948%2C738&ssl=1","datePublished":"2024-07-03T14:31:17+00:00","dateModified":"2024-07-03T14:31:20+00:00","breadcrumb":{"@id":"https:\/\/fadyanwar.com\/index.php\/2024\/07\/03\/running-a-llama-model-locally\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/fadyanwar.com\/index.php\/2024\/07\/03\/running-a-llama-model-locally\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/fadyanwar.com\/index.php\/2024\/07\/03\/running-a-llama-model-locally\/#primaryimage","url":"https:\/\/i0.wp.com\/fadyanwar.com\/wp-content\/uploads\/2024\/07\/photo1719947525.jpeg?fit=948%2C738&ssl=1","contentUrl":"https:\/\/i0.wp.com\/fadyanwar.com\/wp-content\/uploads\/2024\/07\/photo1719947525.jpeg?fit=948%2C738&ssl=1","width":948,"height":738},{"@type":"BreadcrumbList","@id":"https:\/\/fadyanwar.com\/index.php\/2024\/07\/03\/running-a-llama-model-locally\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/fadyanwar.com\/"},{"@type":"ListItem","position":2,"name":"Running a LLaMa Model Locally"}]},{"@type":"WebSite","@id":"https:\/\/fadyanwar.com\/#website","url":"https:\/\/fadyanwar.com\/","name":"Fady Anwar","description":"","publisher":{"@id":"https:\/\/fadyanwar.com\/#\/schema\/person\/b66e3277ceba346f7053a83464e90b03"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/fadyanwar.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":["Person","Organization"],"@id":"https:\/\/fadyanwar.com\/#\/schema\/person\/b66e3277ceba346f7053a83464e90b03","name":"Fady Anwar","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/a9172040bbc3bbe24fb49d59dac20da030af1f5ff628126c979a1d4b71eaed41?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/a9172040bbc3bbe24fb49d59dac20da030af1f5ff628126c979a1d4b71eaed41?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/a9172040bbc3bbe24fb49d59dac20da030af1f5ff628126c979a1d4b71eaed41?s=96&d=mm&r=g","caption":"Fady Anwar"},"logo":{"@id":"https:\/\/secure.gravatar.com\/avatar\/a9172040bbc3bbe24fb49d59dac20da030af1f5ff628126c979a1d4b71eaed41?s=96&d=mm&r=g"}}]}},"jetpack_featured_media_url":"https:\/\/i0.wp.com\/fadyanwar.com\/wp-content\/uploads\/2024\/07\/photo1719947525.jpeg?fit=948%2C738&ssl=1","jetpack_sharing_enabled":true,"post_mailing_queue_ids":[],"_links":{"self":[{"href":"https:\/\/fadyanwar.com\/index.php\/wp-json\/wp\/v2\/posts\/2770","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/fadyanwar.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/fadyanwar.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/fadyanwar.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/fadyanwar.com\/index.php\/wp-json\/wp\/v2\/comments?post=2770"}],"version-history":[{"count":1,"href":"https:\/\/fadyanwar.com\/index.php\/wp-json\/wp\/v2\/posts\/2770\/revisions"}],"predecessor-version":[{"id":2772,"href":"https:\/\/fadyanwar.com\/index.php\/wp-json\/wp\/v2\/posts\/2770\/revisions\/2772"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/fadyanwar.com\/index.php\/wp-json\/wp\/v2\/media\/2771"}],"wp:attachment":[{"href":"https:\/\/fadyanwar.com\/index.php\/wp-json\/wp\/v2\/media?parent=2770"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/fadyanwar.com\/index.php\/wp-json\/wp\/v2\/categories?post=2770"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/fadyanwar.com\/index.php\/wp-json\/wp\/v2\/tags?post=2770"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}