Can a 1.4GB LLM Be My Local Search Engine? (spoiler: yes)

One of the main LLM use cases is often pitched as a search engine replacement and it got me thinking.. Can a small locally hosted LLM replace google when I'm on the go?

TL;DR - Yes! 

Though LLM stands for Large Language Models, some are surprisingly small.

To get started, I installed Ollama, which makes tinkering with a huge number of open-source models incredibly easy. I wanted to find a model that's small enough so it can run smoothly on my laptop while being actually useful for coding.

After some tinkering, I settled on qwen3:1.7b. The 1.7b version is only 1.4GB so it fits well in my laptop's RAM and it's good enough for real coding tasks!

Now, I can just spin up a local LLM in my termial (!!) and ask it random stuff while coding:

 

For simple search tasks (and even a bit of debugging), it's as good as the full blown web based LLMs. It won't tell me today's news, but for quick code snippets, syntax and library definitions, it's perferct.

(Sidenote - I recommend disabling the thinking mode because this model is an overthinker.)

Popular posts from this blog

TL;DR Remote work key-points from GitLab handbook

Image
I'm trying to understand how to make my work more async and started reading about it from companies that seem to be working this way for a while now. These are the keypoints from Gitlab's thorough handbook  - it's my interpretation of the key concepts behind the handbook. Communicate with the intent for one reply, meaning that the sent message should contain all of the necessary information for the recipient to act upon it without back and forth questions. Unblocking yourself - Always have multiple tasks in the pipeline so when you get stuck waiting for one task to be reviewed, you can work on another. Refrain from using Slack because of it's instant-messaging like figure. Choose the correct medium for the job. sometimes a recorded short video is better than a wall of text. One source of truth (for gitlab it's gitlab) - I don't agree with that because different systems can organize data in different ways. Strong documentations Communicate your work preferences -...
Read more

Digital detox with a custom Android launcher

Image
Lately I find myself doom-scrolling way too much. Seems like it's a global pandemic so I searched for ways that will make my phone less appealing and stumbled upon this app: The design is super simple, nothing flashy, it's purpose built to making my phone look boring. Right after the initial installation, I found myself struggling with the limitation of maximum 8 apps in the home screen. I was aiming for ultra simplicity but I didn't want to sacrifice productivity in the process. Since the OLauncher project is open source ( https://github.com/tanujnotes/Olauncher ), I could tinker with it and up the number to a whopping 12 apps! 4 hour of random tasks: Install Android Studio Figure out the Kotlin code structure - last time I checked (~2012), android used Java Fix build errors Edit the code to allow 4 more apps - it's all const resources so instead of modifying it to use for loops and auto generate as many apps as the user wants, I followed the footsteps of the original ...
Read more