JetStream inference at TPU

Reference

step-by-step

1. TPU 띄우기

2. jetstream 받고 설치

git clone https://github.com/google/jetstream-pytorch.git
cd jetstream-pytorch
git checkout jetstream-v0.2.3
source install_everything.sh

• 추가로 설치하면 더 빠른 다운로드

pip install hf_transfer
echo 'export PATH=~/.local/bin:$PATH' >> ~/.bashrc
echo 'export HF_HUB_ENABLE_HF_TRANSFER=1' >> ~/.bashrc
source .bashrc

3. HF 로그인 & 모델 다운로드

pip install -U "huggingface_hub[cli]"
huggingface-cli login
huggingface-cli download google/gemma-7b-pytorch --local-dir $input_ckpt_dir