You can download Large Language Models (LLMs) on your computer and use them without the internet, if you want. I’ve been playing with open source models. Some I downloaded locally (which basically means on my computer) from ollama, some through Application Programming Interface (API). It took me a while to familiarise myself with the setup and different models.
They’re not quite the ChatGPT experience yet, mostly because of the LLM’s size and your system’s memory. But there’s massive scope for personalisation: integrations with other software, long-term memory, and most importantly, they’re free and private. Fast-forward to what is becoming valuable in the AI world: your data. Yes. They are using it and you are not protected or getting paid for it.
The catch? You need a lot of compute power for a good enough model. Ollama’s local models almost destroyed my GPU. While DeepSeek and Qwen’s smaller models were amazingly open source and performed all functions that ChatGPT or Claude could, the smaller local models couldn’t handle the coding development work I needed.
2. Automation:
This is the most hyped AI product: automating processes using AI. A lot of AI agencies are selling these apart from chatbots: personal assistants, lead generation, email management, calendar scheduling.
Source: n8n
I set up a few automations for myself but I’m yet to close in on what I actually need agents for. And it seems like every day there are new AI agents with more advanced functionalities. Very interesting and entertaining space.
I would compare this to graphic design tools in the 90s or social media management in the 2000s. While these tools were all the rage, the real value remained at the macro-strategy level.
Anyway, I bet most influencers, are making more from their Skool communities than actually selling these agents. But there’s still scope for specialisation.
3. RAG (Retrieval-Augmented Generation):
This is a topic of its own. Basically, you can build up your AI’s memory and then exchange and even add information. You can use it for basic functions like fetching relevant information from documents, or feeding instructions to perform functions based on context you’ve provided.
Different parameters, components, and models in a simple RAG application. Source: Gradient Flow
I made a basic one for myself to test out. My experiments all ended with the free tiers of GPT’s OpenRouter, Groq, Gemini 2.5 Pro and Flash models. Qwen’s API needed my card even though they claimed a free tier. Broadening the context is what I’m struggling with. Like if you build a bot for appointment setting, it can only do that in intelligent ways. You give it more functionalities and it will work within those boundaries. You can program it to do those exact operational functions.
My goal was to make it give me strategic outputs.
But I have to preempt what I tell the agent to do. It can’t be random and I can’t expect the AI to know what I want. For RAG to work strategically within a given academic context, it will either come back empty handed or hallucinate on its own. Then it gets deeper and beyond regular RAG, you give it tools and so on, for example search tools.
The LLMs I find are built with these typical limitations: Western-European frameworks that dominate higher-level institutions and analytical thinking. I’ve been trying to get AI to stop defaulting to certain perspectives, to align with specific lines of academic thinking, but there’s bias baked in. There are uncensored models though, which are fun to experiment with but they will fry my GPU.
Although I found this to be an evolving space, with the evolution of GraphRAG and other hybrids this remains an interesting space to watch. And this will have more utilities to come, example Google’s NotebookLM. I made one for myself and tested it but I could not produce a divinely-oriented non-secular output because of LLM bias.
My attempt to create a realtime GraphRAG that can give answers inspired by the divine. Source: self
4. Open-source Versus Closed Source:
This is one of those spaces I keep going down the rabbit hole of. Basically: zero ads, forget paying for software, decentralised social media. I think this is the direction we’re moving in or at least I hope we do or we will be forced to. Obviously not anytime soon because monopolies like private OpenAI will find ways to distract and keep the sheep in the present status quo. But it’s inevitable.
Source: awesome-selfhosted repo
Especially for people like us who don’t want to pay $200 to OpenAI for using their tech. Every software you can imagine has an open source GitHub repository you can clone. You need some technical expertise, but if you have the testicular tenacity, you can vibe code your way to write code, or feed it the code and ask what it means, edit, iterate, copy and paste. You have been warned.
I found myself doing this way too much. It’s time-energy-consuming. It’s more important to actually spend that time on what you’re trying to achieve instead of trying out different stuff and going down the rabbit hole of figuring out the code. Too many times I kept finding myself trying to sort out small issues instead of building an overall MVP.
The Real Problem: Too Many Unfinished Projects
So many projects started and never completed. Maybe I underestimated the project. Maybe I overestimated my skills and available resources. Or didn’t understand the stack enough before starting. There’s a fear of actually finishing something, you know?
What Actually Sells
The intelligent chatbot is actually the easiest and most sold product. It takes some back and forth with the client to understand what the customer is looking for, but programming a simple meeting scheduler or AI assistant is not the hard part. The challenge is building the appropriate knowledge base and user experience. If someone wants to buy for their business, it can be built.
What we need to do first is create ‘ecosystems’. That is an offering.
Strategic Thinking
We must start by asking: what is valuable to my business that I need? Then create it. The cut-paste solutions of just automation crafts are just tools. Showing a demo of an ecosystem, now that’s a working use case.
Example of an automation: a social media agent that emails you content it’s going to post for you. You reply “approved” and it’s done, on all your platforms. Or reply with “make these changes” and it sends back revised content.
Example of an ecosystem: builds a business solution that uses multiple digital avenues to transform outputs for business. Here the focus is on evolving technology. Build versus buy. Evolving tech like real-time data use. Basically, for businesses, return on investment, efficiency and ease of use and user interface that will matter eventually.
Where We Are Now
Creating AI Agents or “digital workers.” These are software applications designed to do tasks and share the workload. For example, that social media agent I mentioned. Once set up, all you need to do is approve the content. Like an agency model, understanding the basics of how you can use AI and keep selling it. The next step would be to create a SaaS.
Models have almost plateaued with slightly better efficiency than the last wave of AI-models.
The gap is closing fast. While with some AI, you can’t even tell anymore. The intelligent use of AI as a tool, that’s what separates those who orchestrate from those who get orchestrated.
The big LLM companies continue dominating the game. Now it’s open season for low to no budget builders. But are the builders the farmers or are they being farmed?