• kamenLady.
    link
    fedilink
    English
    21 year ago

    i see… I’ll have to ramp up my hardware exponentially …

    • @PeterPoopshit@lemmy.world
      link
      fedilink
      English
      5
      edit-2
      1 year ago

      Use llama cpp. It uses cpu so you don’t have to spend $10k just to get a graphics card that meets the minimum requirements. I run it on a shitty 3.0ghz Amd 8300 FX and it runs ok. Most people probably have better computers than that.

      Note that gpt4all runs on top of llama cpp and despite gpt4all having a gui, it isn’t any easier to use than llamacpp so you might as well use the one with less bloat. Just remember if something isn’t working on llamacpp, it’s also going to not work in exactly the same way on gpt4all.

        • @NotMyOldRedditName@lemmy.world
          link
          fedilink
          English
          3
          edit-2
          1 year ago

          Check this out

          https://github.com/oobabooga/text-generation-webui

          It has a one click installer and can use llama.cpp

          From there you can download models and try things out.

          If you don’t have a really good graphics card, maybe start with 7b models. Then you can try 13b and compare performance and results.

          Llama.cpp will spread the load over the cpu and as much gpu as you have available (indicated by layers that you can set on a slider)