RTX Spark: Local AI Coding Fails, I’ve Had (More Than) a Few ⭐

I've spent many days--too many days--trying to find a local AI coding experience on the RTX Spark that works consistently and well. And while I've had some minor victories here and there, I've done nothing but fail to date in the sense that I'm looking for a repeatable process that always works.

When I got started testing the RTX Spark computers, I assumed that I would quickly move across several different local AI use cases like general usage, software development, creative (image, video, etc.), productivity, agents, and whatever else. General usage, where you use local AI like any AI chatbot, is straightforward--like the Tolkien books and Mexico City itineraries I mentioned in RTX Spark: Real-World Local AI ⭐--and while that was a minor but nice success, I've really gotten stuck on the software development bit.

There are many reasons for that, including that some AI front-ends and/or local AI models will just use (and, if necessary, install and configure) whatever apps, frameworks, and other technologies are necessary to get the job done, while others will just give you the source code and tell you to copy and paste it into Visual Studio or whatever else manually. The former approach isn't just superior, it's a necessity as being the middleman between the AI and the multitude of errors that your developer environment absolutely will throw up, no matter how simple the project, makes that process a complete non-starter.

But it's not just that. Local AI models often come in multiple versions, each with its own quality/version and model size choices. If you're looking for a local model that's good for software development, can install, configure, and use all the relevant developers tools itself, and one that works well on your particular hardware, well, that's on you. It's all on you, really. From start to finish.

So I've been experimenting a lot. And, to be clear, failing a lot too. This morning, for what seems like the umpteenth time, I decided to try yet again and to mix up the tools and models I use yet again. This is pretty much what I've been doing for the past few weeks, almost every single day.

Here's today's experiment: LLama Windows and some range of model choices, and an opportunity for you to vicariously feel the pain.

As you may recall, Hugging Face timed the release of Llama Windows to the launch of the RTX Spark computers earlier this month. It's a graphical front-end to the llama.cpp open source AI inference engine, a de facto standard that's used by popular local AI apps like LM Studio Bionic and Ollama. And that's important because llama.cpp is a command line interface (CLI) that most will find complex to use.

You can find llama.cpp on GitHub and it's the right option if you're in too good a mood. On a Windows PC, you install llama.cpp from the command line, of course, but oddly you don't use the Windows Package Manager (winget); instead, you use the Invoke-RestMethod command in PowerShell or, more...