jpmcb/nvim-llama

github github
ai
stars 37
issues 2
subscribers 4
forks 1
CREATED

2023-08-26

UPDATED

3 months ago


🦙 nvim-llama

Llama 2 and llama.cpp interfaces for Neovim

🏗️ 👷 Warning! Under active development!! 👷 🚧

Installation

Use your favorite package manager to install the plugin:

Packer

  use 'jpmcb/nvim-llama'

lazy.nvim

{
    'jpmcb/nvim-llama'
}

vim-plug

Plug 'jpmcb/nvim-llama'

Setup & configuration

In your init.vim, setup the plugin:

require('nvim-llama').setup {}

You can provide the following optional configuration table to the setup function:

local defaults = {
    -- See plugin debugging logs
    debug = false,

    -- Build llama.cpp for GPU acceleration on Apple M chip devices.
    -- If you are using an Apple M1/M2 laptop, it is highly recommended to
    -- use this since, depending on the model, may drastically increase performance.
    build_metal = false,
}

Models

Llama.cpp supports an incredible number of models.

To start using one, you'll need to download an appropriately sized model that is supported by llama.cpp.

The 13B GGUF CodeLlama model is a really good place to start: https://huggingface.co/TheBloke/CodeLlama-13B-GGUF

In order to use a model, it must be in the llama.cpp/models/ directory which is expected to be found at ~/.local/share/llama.cpp/models.

The following script can be useful for downloading a model to that directory:

LLAMA_CPP="~/.local/share/nvim/llama.cpp"
MODEL="codellama-13b.Q4_K_M.gguf"

pushd "${LLAMA_CPP}"
    if [ ! -f models/${MODEL} ]; then
        curl -L "https://huggingface.co/TheBloke/CodeLlama-13B-GGUF/resolve/main/${MODEL}" -o models/${MODEL}
    fi
popd

In the future, this project may provide the capability to download models automatically.

License

This project is dual licensed under MIT (first party plugin code) and the Llama 2 license. By using this plugin, you agree to both terms and assert you have already have your own non-transferable license for Llama 2 from Meta AI.