编程开源技术交流,分享技术与知识

网站首页 > 开源技术 正文

安装Lora模型训练服务kohya_ss的两种方式

wxchong 2024-08-12 02:22:00 开源技术 31 ℃ 0 评论

方法一(以root账号运行):

1、下载源码

git clone https://github.com/bmaltais/kohya_ss.git

2、下载基础环境镜像

docker pull nvcr.io/nvidia/pytorch:23.04-py3

3、运行基础环境镜像(保证第1步下载的源码位于宿主机的/data目录下)

docker run -itd --restart=always --name pytorch-test --gpus all -p 6667:6667 --entrypoint='bash' -v /data:/opt nvcr.io/nvidia/pytorch:23.04-py3

4、进入基础环境镜像内部

docker exec -it pytorch-test bash

5、依次执行如下命令

先修改apt源

cp /etc/apt/sources.list /etc/apt/sources.list.bak
vim /etc/apt/sources.list
deb http://mirrors.aliyun.com/ubuntu/ focal main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ focal main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ focal-security main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ focal-security main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ focal-updates main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ focal-updates main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ focal-proposed main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ focal-proposed main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ focal-backports main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ focal-backports main restricted universe multiverse

然后

切换到/opt/kohya_ss目录下,开始安装依赖:

apt update && apt-get install -y software-properties-common

add-apt-repository ppa:deadsnakes/ppa && \
    apt update && \
    apt-get install -y git curl libgl1 libglib2.0-0 libgoogle-perftools-dev \
    python3.10-dev python3.10-tk python3-html5lib python3-apt python3-pip python3.10-distutils && \
    rm -rf /var/lib/apt/lists/*

update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.10 3 && \
    update-alternatives --set python3 /usr/bin/python3.10 && \
    update-alternatives --set cuda /usr/local/cuda-11.8

curl -sS https://bootstrap.pypa.io/get-pip.py | python3

python3 -m pip install wheel

cd /opt/wsb/kohya_ss
python3 -m pip install -r requirements.txt
python3 -m pip install -r requirements_linux_docker.txt

python3 -m pip uninstall -y pillow && \
    CC="cc -mavx2" python3 -m pip install -U --force-reinstall pillow-simd

6、退出容器,并保存镜像

exit
docker commit xxxx my-registry/pytorch:23.04-py3

7、将kohya_ss源码目录下的Dockerfile文件内容改成如下

FROM my-registry/pytorch:23.04-py3 as base
ENV DEBIAN_FRONTEND=noninteractive
ENV TZ=Asia/Shanghai
WORKDIR /app
RUN python3 -m pip install wheel
# Todo: Install torch 2.1.0 for cu121 support (only available as nightly as of writing)
## RUN python3 -m pip install --pre torch ninja setuptools --extra-index-url https://download.pytorch.org/whl/nightly/cu121
# Todo: Install xformers nightly for Torch 2.1.0 support
## RUN python3 -m pip install -v -U git+https://github.com/facebookresearch/xformers.git@main#egg=xformers
ADD . ./
# Install requirements
COPY ./requirements.txt ./requirements_linux_docker.txt ./
COPY ./setup/docker_setup.py ./setup.py
RUN python3 -m pip install -r ./requirements_linux_docker.txt
RUN python3 -m pip install -r ./requirements.txt
# Replace pillow with pillow-simd
RUN python3 -m pip uninstall -y pillow && \
 CC="cc -mavx2" python3 -m pip install -U --force-reinstall pillow-simd
# Fix missing libnvinfer7
RUN ln -s /usr/lib/x86_64-linux-gnu/libnvinfer.so /usr/lib/x86_64-linux-gnu/libnvinfer.so.7 && \
 ln -s /usr/lib/x86_64-linux-gnu/libnvinfer_plugin.so /usr/lib/x86_64-linux-gnu/libnvinfer_plugin.so.7
STOPSIGNAL SIGINT
ENV LD_PRELOAD=libtcmalloc.so
ENV PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python
RUN python3 -m pip install --upgrade requests

8、根据Dockerfile编译成镜像

docker build --rm -t kohya_ss-web:1.0.0 -f Dockerfile .

9、运行容器

docker run -itd --restart=always --name kohya_ss-web --gpus all -p 6667:6667 --entrypoint='bash' -v /data:/opt kohya_ss-web:1.0.0

10、进入容器,启动kohya_ss WebUI

docker exec -it kohya_ss-web bash
nohup python3 kohya_gui.py --listen 0.0.0.0 --server_port 6667 --headless > app.log 2>&1 &

方法二(以appuser账号运行):

1、下载源码

git clone https://github.com/bmaltais/kohya_ss.git

2、进入kohya_ss目录

cd kohya_ss

3、编译镜像,会生成名叫kohya-ss-gui:latest的镜像

docker compose build

4、运行镜像

docker run -itd --restart=always --name kohya_ss-web --gpus all -p 6667:6667 --entrypoint='bash' -v /data:/opt kohya_ss-web:1.0.0

5、进入容器,启动kohya_ss WebUI

docker exec -it kohya_ss-web bash
nohup python3 kohya_gui.py --listen 0.0.0.0 --server_port 6667 > app.log 2>&1 &

Tags:

本文暂时没有评论,来添加一个吧(●'◡'●)

欢迎 发表评论:

最近发表
标签列表