Getting Started with Instructlab: Installing on Fedora 40

/ Christopher Paquin

Prerequisite – Install python 3.11. The default on Fedora 40 is python 3.12

$ sudo dnf -y install python3.11.x86_64 python3.11-devel.x86_64  -y

Install automake and gcc (GNU Compiler Collection)

$ sudo dnf install make automake gcc gcc-c++

Add the following env vars to your .bashrc, and source

export CUDA_HOME=/usr/local/cuda
export PATH=${CUDA_HOME}/bin:${PATH}
export LD_LIBRARY_PATH=${CUDA_HOME}/lib64:$LD_LIBRARY_PATH

Create your virtual env and activate

$ python3.11 -m venv --upgrade-deps venv
$ source venv/bin/activate

Note that every time you want to use instructlab cli (ilab) post-install you must source the file above. venv/bin/activate

We then run the 3 commands shown below. Apparently there is a known bug and we will not pick up these required packages as needed, so we need to install them manually.

$ pip install --upgrade pip wheel setuptools
$ pip install torch
$ pip install flash-attn

The install guide found here has you run this command to clear out the pip package cache (for this one package. Not required for first install, but may be on subsequent runs.

$ pip cache remove llama_cpp_python

Now install instructlab

$ pip install 'instructlab[cuda]' -C cmake.args="-DLLAMA_CUDA=on" -C cmake.args="-DLLAMA_NATIVE=off"

Install vllm.

NOTE: The vllm install is currently failing with this error – ImportError: libcudnn.so.8: cannot open shared object file: No such file or directory. I am looking to find a resolution or work-around

$ pip install vllm@git+https://github.com/opendatahub-io/vllm@2024.08.01

Resources

  1. https://docs.instructlab.ai/getting-started/linux_nvidia/
  2. https://github.com/vllm-project/vllm
  3. https://www.redhat.com/en/topics/ai/what-is-vllm
  4. https://github.com/opendatahub-io

Leave a Reply