I just learned about LocalAI.

I was in the right state of mind when I skimmed through a how-to article to run a GenAI locally with just CPUs. I felt a little savvy getting it running. Then I've stuggled 'cause I wanted more RAM in my machine, but that didn't end well. While scouring for image gen with only CPUs I found that there's a site dedicated to this!

With one simple Docker container, a whole slew of LLM and image gen and other stuff I haven't even poked at yet.

docker run -d -v /local/path:/build/models -p 8080:8080 --name localai localai/localai:latest

Of course, /local/path should be a real place on the host system. The persistent storage is necessary, because the first thing it does is slurp down a bunch of models. While this could end up inside the container just fine, it would be lost and have to happen with every redeployment of the container. I was a little worried for my new HDD, and that it wouldn't necessarily respect my RAM, since I was unable to use the larger LLM in the other container. It only downloaded 11GB of data, though, which is less than the 18GB for the couple LLMs I've added to my Ollama container. It only installed a few models to get started, and has over 400 others available from the UI, and documentation suggests that others can be found and used from places like Hugging Face.

After letting it sit for a while, I was able to make it tell me a short story, generate a bash script, suggest a recipe. I even got it to make me a couple goofy pictures.

I asked it to make a "child eating ice cream in the style of Mona Lisa" and got this.

I also asked it to make a "man in prison garb drinking coffee in the style of Don Martin" and go this.

The one with ice cream certainly looks a lot more like Mona Lisa than the one with coffee looks like Fester Bestertester, but maybe that's not all Don Martin drew. I'm guessing his face is a little distorted by the steam from the coffee.

The promise of the LocalAI project is that all of the stuff is running on my system. I did see the CPU spike on the server, and not a lot of traffic changed on the router, so I'm guessing it's pretty true.

Maybe I'm late to the party, but I'm here, and am starting to fit all the pieces together better.