Figure 1. Table from LLaMa’s paper showing zero-shot performance benchmarks for all 4 sizes of the LLaMa models (7B, 13B, 33B and 60B parameters).
wsl --import ffai-textgen <install location> <path and name of downloaded file>
wsl --import ffai-textgen C:\WSL D:\WSL\FollowFoxAI-textgen-WSL.tar.gz
wsl -d ffai-textgen --user felix
wsl --list --all
wsl --export <distribution name> <path and name of backup file>
wsl --export Ubuntu-20.04 D:\WSL\backed-up-ubuntu.tar.gz
wsl --unregister <distribution name>
wsl --import <distribution name> <install location> <path and name of backup>
wsl -d <distribution name> --user <user name>
wsl --import MyUbuntu C:\WSL D:\WSL\backed-up-ubuntu.tar.gz
wsl -d MyUbuntu --user felix
wsl --install -d Ubuntu-20.04
wget https://developer.download.nvidia.com/compute/cuda/repos/wsl-ubuntu/x86_64/cuda-wsl-ubuntu.pin
sudo mv cuda-wsl-ubuntu.pin /etc/apt/preferences.d/cuda-repository-pin-600
wget https://developer.download.nvidia.com/compute/cuda/11.7.0/local_installers/cuda-repo-wsl-ubuntu-11-7-local_11.7.0-1_amd64.deb
sudo dpkg -i cuda-repo-wsl-ubuntu-11-7-local_11.7.0-1_amd64.deb
sudo cp /var/cuda-repo-wsl-ubuntu-11-7-local/cuda-*-keyring.gpg /usr/share/keyrings/
sudo apt-get update
sudo apt-get -y install cuda
wget https://repo.anaconda.com/archive/Anaconda3-2023.03-Linux-x86_64.sh
bash ./Anaconda3-2023.03-Linux-x86_64.sh
conda create -n textgen python=3.10.9
conda activate textgen
pip3 install torch torchvision torchaudio
sudo apt update
sudo apt upgrade
conda update --all -y
git clone https://github.com/oobabooga/text-generation-webui.git
cd text-generation-webui
mkdir repositories
cd repositories
git clone https://github.com/facebookresearch/xformers.git
git clone https://github.com/oobabooga/GPTQ-for-LLaMa.git -b cuda
cd xformers
pip3 install -r requirements.txt
cd ../..
pip3 install -r requirements.txt
cd repositories
cd GPTQ-for-LLaMa
pip3 install -r requirements.txt
python setup_cuda.py install
pip install -i https://test.pypi.org/simple/ bitsandbytes-cuda117
cd ../xformers
git submodule update --init --recursive
pip3 install -e .
python3 -m xformers.info
Figure 2. Here we see Vicuna-7B-1.1 strongly defending its host GPU. We disagree on that one.
conda activate textgen
cd ~/text-generation-webui
python3 server.py --wbits 4 --groupsize 128 --model_type LLaMA --xformers --chat