Tag: LLM

On running GPT-OSS-20B on my Macbook Pro

I’m been running gpt-oss-20b on my Macbook Pro for some time now. I typically run different LLMs on my Macbook using ollama as the basis for this. While I’ve tried various models, this model from OpenAI is the one I found that gave me the best answers to the questions I’ve been giving it. I even prefer the answers I get with gpt-oss-20b on my Mac over those I get from larger SaaS models provided by OpenAI, Anthropic and Google.

I’ve found it very good when it comes to asking it questions concerning lifestyle changes. For instance, I asked it for:

  • ideas on how to celebrate a big birthday
  • a 12 month plan to do Swedish death cleaning on my house
  • a 12 month plan to get my estate in order
  • a 12 month plan to get back in shape after a long layoff

I’ve found it not very good at all with questions around food and music:

  • coming up with music playlists based on certain criteria (over a third of the songs were hallucinations!)
  • coming up with spice mixes for cooking meats (the proportions were way off compared to tested recipes)

Like all LLMs, I found it works well as a means to generate ideas. It works ok or poorly when you start expecting exactitude in the answers. YMMV.

Other things I like about gpt-oss-20b running on ollama on my Macbook:

  • I can see it go through it’s reasoning when it comes up with the answer, which is a nice feature.
  • For some reason the default output format is one to my liking, and I can easily move it into Microsoft Word or Apple Pages for further formatting
  • I especially like it because I can ask personal questions and not have those being captured by a service in the cloud. It provides an extra layer of privacy.

I am running gpt-oss-20b and not gpt-oss-120b due to the capacity of my Macbook Pro. Indeed, even gpt-oss-20b takes up a fair amount of resources on my Macbook, despite it having a GPU to do the heavy lifting. I am not in a rush for these answers though, so it is fine. I’d run 120b if I could, but I have been happy with the response with 20b.

If you have a machine with a GPU and enough RAM and disk space, consider downloading ollama and this model and give it a go. Let me know what you discover!

 

How generative AI works (it’s not designed, at least the way you think)

How generative AI works is likely not how you think it works.

What led me to state this was two discussions I’ve had this week about the “design” of gen AI. I think the “design” conclusion that people come up with is based on emergent behaviours of the system. You can shape this behaviour in a number of ways, based on the data you feed the system or some ways you configure the software being trained. However at best you are influencing the behaviour of the system, vs designing the behaviour of the system.

In some ways it’s like taking a bucket of seeds and tossing them around a large area of a field. If you took only seeds of one or two flowers and distributed the seeds so that only these one or two flowers and grew there, you could say you designed the garden to grow these flowers. Likewise, if you divided up the land into rows and planted one type of seed in each row, you can say you designed the garden. However if you have a wide range of seeds included in your bucket and you don’t target the planting of the seeds but just toss them into the ground, it will no longer be considered designed.

That’s why I think gen AI is not really designed. It’s a alot like a big bucket of random seeds not planted in any order. What design you see there is likely how you look at it.

P.S. If you want to explore more on how gen AI works, see this. For a great example of how a gen AI system is built from the ground up, see this.

 

A guide to generative AI and LLM (large language models), February 2025


I decided to go through all my posts on AI and pull out information that would be useful to anyone wanting to learn more about generative AI (often referred to as gen AI or genAI) and the LLMs they run. If you have used chatGPT, you have used genAI. But there’s much more to the technology than what you find on that site. To see what I mean, click on any of the blue underlined text and you will be taken to a site talking about something to do with gen AI.

Enjoy!

Tutorials/Introductions: for people just getting started with gen AI, I found these links useful: how generative AI works, what is generative AI, how LLMs works,  sentence word embeddings which kinda shows  how LLM works, best practices for prompt engineering with openai api,  a beginners guide to tokens, a chatGPT cheat sheet,  demystifying tokens: a beginners guide to understanding AI building blocks, what are tokens and how to count them, how to build an llm rag pipeline with llama 2 pgvector and llamaindex and finally this: azure search openai demo. (Some of these are introductory for technical people – don’t worry if you don’t understand all of them.)

For people who are comfortable with github, this is a really good repo / course on generative AI for beginners. (and check out these other repositories here, too). This here on the importance of responsible AI. and here’s a step by step guide to using generative AI in your business, here.

Prompts and Prompt Engineering: if you want some guidance on how best to write prompts as you work with gen AI, I recommend this, this, this, this, this, this, this, and this.

Finally:  Here’s the associated press AI guidelines for journalists. This here’s a piece on how the  Globe and Mail is using AI in the newsroom. Here’s a how-to on using AI for photo editing. Also, here’s some advice on writing better ChatGPT prompts. How Kevin Kelly is using  AI as an intern, as told to Austin Kleon. A good guide on  how to use AI to do practical stuff.

Note: AI (artificial intelligence) is a big field incorporating everything from vision recognition to game playing to machine learning and more. Generative AI is a part of that field. However nowadays when we talk of AI people usually mean gen AI. A few years ago it was machine learning and before that it was expert systems. Just something to keep in mind as you learn more about AI and gen AI in particular.

 

Forget ChatGPT. Now you can build your own large language model (LLM) from scratch

Yep, it’s true. If you have some technical skill, you can download this repo from github: rasbt/LLMs-from-scratch: Implementing a ChatGPT-like LLM in PyTorch from scratch, step by step and build your own LLM.

What I like about this is that it demystifies LLMs. LLMs aren’t magic, they aren’t Skynet and they’re not some sentient being. They’re software. That’s all.

So ignore all the hype and handwaving about LLMs and go make your own.

Prefer to read it in dead tree form? You can get the book here.

Will AI tools based on large language models (LLMs) become as smart or smarter than us?

With the success and growth of tools like ChatGPT, some are speculating that the current AI could lead us to a point where AI is as smart if not smarter than us. Sounds ominous.

When considering such ominous thoughts, it’s important to step back and remember that Large Language Model (LLM) are tools based in whole or in part on machine learning technology. Despite their sophistication, they still suffer from the same limitations that other machine learning technologies suffer, namely:

    • bias
    • explainability
    • overfitting
    • learning the wrong lessons
    • brittleness

There are more problems than those for specific tools like ChatGPT, as Gary Marcus outlines here:

  • the need for retraining to get up to date
  • lack of truthfulness
  • lack of reliability
  • it may be getting worse due to data contamination (Garbage in, garbage out)

It’s hard to know if current AI technology will overcome these limitations. It’s especially hard to know when orgs like OpenAI do this.

My belief is these tools will hit a peak soon and level off or start to decline. They won’t get as smart or smarter than us. Not in their current form. But that’s based on a general set of experiences I’ve acquired from being in IT for so long. I can’t say for certain.

Remain calm. That’s my best bit of advice I have so far. Don’t let the chattering class get you fearful. In the meanwhile, check out the links provided here. Education is the antidote to fear.