Run gpt-oss-20b on a 16GB Mac

[slyapustin]

I would like to have an option to run gpt-oss-20b on a 16GB Mac

As per @ggerganov tweet it's now possible and it works fine for me when I tested.

To run gpt-oss-20b on a 16GB Mac use these commands:

brew install llama.cpp
llama-server -hf ggml-org/gpt-oss-20b-GGUF --n-cpu-moe 12 -fa -c 32768 --jinja --no-mmap

[igardev]

@slyapustin Here is how to use gpt-oss with your command for Llama Vscode Agent:

• Create new tools model
  From Llama-vscode menu select Tools models... -> Add tools model. For local start command enter "llama-server -hf ggml-org/gpt-oss-20b-GGUF --n-cpu-moe 12 -fa -c 32768 --jinja --no-mmap" and for endpoint "http://localhost". Alternatively, if you start the server outside llama-vscode, you could leave local start command empty and enter the appropriate endpoint.

• Select the new tools model
  From Llama-vscode menu select Tools models... -> Select/start tools model.and choose your new model

• Start using Llama Vscode Agent
  From Llama-vscode menu select Show Llama Agent and start using it. All the tools will work, except code_search tool. To make this tool work as well - select a chat model and embeddings model (this will consume additional RAM memory). If you prefer, you could disable this tool - click button with a picture of an instrument (near "Chat History" button), uncheck the tool and click OK button.

I hope this will help.