Little Tech, Little AI ⭐

Big Tech and Big AI are over-hyping AI with a “bigger is better” message that only the biggest tech companies in the world with the biggest and most expensive and inefficient datacenters can deliver on the promises of this technology.

But sometimes less is more. There are Little Tech cloud AI alternatives that use open source models or anonymized access to Big AI models. And there are Little AI alternatives that let you use small language models (SLMs) locally, on-device. Both are evolving to the point where they are, or soon will be, “good enough” alternatives to Big Tech and Big AI. And while both should be broadly interesting to just about anyone, I feel that these Little Tech and Little AI solutions should be particularly compelling to those who fear, hate, or distrust AI today.

? Little Tech

I identified the Little Tech companies and products I trust last summer, and it’s no surprise that several of them are pursuing standalone AI solutions in the form of chatbots or, if they’re browser makers, AI that integrates directly into their products.

Proton and DuckDuckGo are among those with standalone chatbots with a privacy focus.

Proton Lumo is open source, secure, and private, it doesn’t log your chats, can’t share data with others, and doesn’t serve ads. It uses open source AI models like Mistral Nemo, Mistral Small 3, and Nvidia OpenHands 32B, plus its own custom models, and not Big AI models. The free version can be used without an account with limits, though signing in will save your chat history and provide more chats. And the paid version is $12.99 per month, though it’s on sale as I write this for $9.99 per month. There’s also a version for businesses.

DuckDuckGo’s Duck.ai provides smaller Big AI models like Anthropic Claude Haiku 3.5, Meta Llama 4 Scout, and OpenAI GPT-5 mini to all users, and some bigger models like Anthropic Claude Sonnet 4.5, Meta Llama 4 Maverick, and OpenAI GPT-5.1 to paying customers. But access to these models is anonymized, of course, and DuckDuckGo also offers a web search engine with integrated AI Answers and an eponymous web browser, both of which are likewise private and secure. Anyone can use all of these products and services for free, with the expected limits on Duck.ai. But DuckDuckGo also offers a Privacy Pro subscription for $9.99 per month or $99.99 per year that provides anonymized access to those bigger AI models in addition to the VPN, personal information removal, and identity theft restoration services.

For those who prefer to access AI through a web browser, almost every Little Tech web browser maker is doing something in this space. The notable exception, of course, is Vivaldi, which said in August that it will not add AI features to its browser because users who need that functionality can find it on the web or through extensions.

Brave is the only web browser I can recommend without hesitation or caveat and, like DuckDuckGo, it offers an in-house private search engine, Brave Search, which is also adding AI functionality like Answer with AI and Chat mode. Brave’s in-browser AI is called Leo and it is predictably private, anonymous, and secure. It integrates with Brave Search, of course, and it provides free access to smaller Big AIs like Alibaba Qwen 14B, Anthropic Claude 3.5 Haiku, Google Gemma 12B, and Meta Llama 3.1 8B with limits. Those who pay for a Leo AI Premium subscription at $149.99 per year or $14.99 per month gain access to higher rate limits and bigger models like Anthropic Claude Sonnet 4 and DeepSeek R1. And Brave also offers an interesting Bring Your Own Model (BYOM) option that lets users integrate Leo with Big AI in the cloud (which you pay for separately) and/or local SLMs (which are free).

Opera makes one of my favorite web browsers, Opera Air, but its Opera AI service is free with limits to anyone in any of its web browsers—so, Opera One and Opera GX, too—and it’s now using Google Gemini behind the scenes. You don’t have to sign in to use it, but if you need higher limits or more advanced capabilities, Opera now offers its Neon agentic web browser for $20 per month. Agentic web browsers are still a bit iffy for all the obvious security and privacy reasons, but if you’re interested in this functionality, I trust Opera more than, say, Perplexity, which makes an excellent web browser of its own called Comet.

More is on the way. Mozilla is working on an opt-in AI infrastructure for its Firefox web browser called AI Window that its new CEO says will come with an “AI kill switch” to instantly disable all AI features. And then there are startups like The Browser Company, which was recently acquired in what I view as a problematic development, though its Dia web browser is only available on the Mac for now.

? Little AI

If you don’t mind a bit of complexity and some functional shortcomings, running small language models (SLMs) locally delivers some big benefits, too. It’s more private, works offline, and might save you money if you would otherwise pay a monthly fee to Big Tech or Big AI companies. And while SLMs are understandably not as powerful as the large language models (LLMs) that power ChatGPT, Gemini, and other big, cloud-based AIs, they’re getting better all the time. Given the speed at which AI advances, these SLMs will be good enough soon enough.

Because of this value proposition, I think of this technology as Little AI, in keeping with my use of the term Little Tech to describe personal technology solutions that tend to come from smaller, independent companies that still value their users’ needs, haven’t enshittified their offerings, and don’t lock us into one-way ecosystems.

I know this term isn’t perfect, and I know that most would just refer to this type of AI as local AI. There’s also some irony (or even hypocrisy) in the fact that many SLMs come from Big Tech companies like Google (Gemma) and Microsoft (Phi). But bear with me here. I think little is important.

Consider my use of Typora and iA Writer, both of which are lightweight apps that create Markdown documents, a plain text format with lightweight markup for formatting (similar to XML and HTML but simpler). This isn’t an affectation. For the previous 30 years, I had used Microsoft Word, an overwrought, bloated monstrosity that iA describes as “not a writing app but an elaborate layout program for paper or PDF documents.” For decades, I had observed that even I, a professional writer, probably used less than 5 or 10 percent of Word’s features. And I had wanted to find something that was both smaller and simpler but still met my needs. Something … little.

I think of local AI, which I will call Little AI, the same way. Big Tech, in collusion with Big AI, has sold the world on so-called foundation and frontier models, huge AI solutions that can only be made by Big AI and can only run on the massive and expensive infrastructure that only Big Tech can provide. But as DeepSeek and many open source models show us, AI is just like any other technology. It can be done more inexpensively. It can be more lightweight and more efficient.

It may also be better.

Those gigantic LLMs don’t just require massive infrastructure and energy use, they require massive amounts of data to train the models to know everything there is to know. Big Tech and Big AI have created what are essentially hive minds polluted by all the contradictory information they found online and, in many cases brazenly stole. It is this massive data set that causes hallucinations, the pre-AI slop buzzword that AI critics still cling to as the reason to ignore this technology.

Big Tech and Big AI need investors, shareholders, and the world at large to believe that the magic of AI can only be achieved using the expensive, sprawling infrastructure that only they can provide. But even they have admitted that the best way to stop hallucinations is to ground the AI in small, specific sets of data. An AI grounded in the known-correct documentation for a programming language will be more accurate–and, go figure, faster and more efficient—than some gigantic frontier model trained on all the world’s data.

In fact, there is a new line of thinking that the future of AI is tied up in the next logical step in that reality. Small AI models, SLMs, grounded in specific and accurate data sets are better at solving specific tasks. So perhaps what’s really needed is a lot of very small AI models, each tailored to a specific task. And then some orchestrator that will accept a user or other input and assign specific sub-tasks to specific SLMs to get to whatever answer or other outcome.

If this sounds familiar, it’s because there are no new ideas. Also, this is what Microsoft did with Copilot+ PC. The first of these PCs shipped with over 40 SLMs on disk about a year and a half ago, and I would bet money that the SLM count is much higher today. That Copilot+ PCs have or haven’t resonated with consumers is sort of beside the point, but maybe this is why the perception is negative: There’s no single big AI feature on them that benefits most customers, there are instead dozens of small AI features powered by small AI models, each plugging away without fanfare and delivering small benefits. It’s boring. Things that just work often are.

I’m not an expert in this area, but I wouldn’t be surprised to see Microsoft and other Big Tech and Big AI companies take this approach in the cloud. It would be more efficient, saving them money. And it would likely work better than the current system. But that’s not what I care about per se. What I care about is applying this “life always finds a way” reality to AI that can run locally on our PCs and other devices. It is inevitable that SLMs will get more capable. That they will become good enough. And when this happens, Little AI becomes not just a reality but, for many, the preferred way to do things.

We’re not there yet. This is still early days, and this is still the type of thing that will be of more interest to enthusiasts than mainstream users. But again, this is AI, and AI moves fast. When will it cross that line? When will then be now? Soon.

But there’s no reason to wait. We are enthusiasts after all. And if you have concerns about the AI that’s being foisted on us now through Copilot, ChatGPT, or whatever else, it may not be a bad idea to check in from time to time on Little AI and see where it’s at. You may be surprised.

There are many ways to run Little AI models locally on your own PC. Most of them are technical and complex, and I’m not at the point where I can recommend one over the others or even have suggestions about which models to try. But the one I’ve used the most so far is called Jan.ai. This is a locally installable app for Windows, Mac, and Linux that presents a familiar chatbot user interface and can connect to a long list of locally installable SLMs and then use them like any other chatbot.

If anything, the list of SLMs is too long, and it needs some curation and/or ranking system, not to mention filtering for those SLMs that work against an NPU or GPU instead of the main processor. But it’s a start. And a safe and private way to experiment, whether you’re a doubter or a true believer.

There are other solutions that accomplish the same thing. I’ve used the integrated AI functionality in Visual Studio Code as a chatbot, for example, and there are other standalone local AI apps like LM Studio (which has a native Windows 11 on Arm client) and AnythingLLM that I am experimenting with as well.

? Final thoughts

The problem with AI isn’t technical, it’s human nature. Big Tech and Big AI are over-hyping AI and under-delivering. And in doing so, they’re doing what might be irreparable harm to AI as a term: Thanks to the incessant marketing and forced usage, and the tornado of noise from deniers and fear mongerers, AI has a bad rap. But I see AI as nothing more than a series of features and agentic AI as background processes, and in that light AI should be seen for what it is. Just more technology. This isn’t something to fear, it’s something to harness as appropriate. And perhaps some combination of Little Tech and Little AI will put it over the top for you.