在Windows中使用Docker搭建大模型RAG

本文将使用Windows环境来构建基于Qwen2-7b大模型的小型本地知识库。涉及到的内容包括：

Windows环境的Docker部署；
Qwen2-7b大模型的获取；
基于Ollama的大模型部署；
Open-webui的前台展示

07 摄于广州黄埔·鱼珠码头 909功勋试验舰

系统准备

部署WSL

需要先启用“适用于 Linux 的 Windows 子系统”可选功能，然后才能在 Windows 上安装 Linux 分发。

以管理员身份打开 PowerShell（“开始”菜单 >“PowerShell” >单击右键 >“以管理员身份运行”），然后输入以下命令：

# 启用WSL
dism.exe /online /enable-feature /featurename:Microsoft-Windows-Subsystem-Linux /all /norestart
# 安装虚拟化
dism.exe /online /enable-feature /featurename:VirtualMachinePlatform /all /norestart
# 下载内核包并安装
https://wslstorestorage.blob.core.windows.net/wslblob/wsl_update_x64. msi

# 重启主机
Restart-Computer

# 升级wsl，否则会有"WslRegisterDistribution failed with error: 0x800701bc"报错
wsl --update
# 将wsl版本默认为2
wsl --set-default-version 2

# 安装默认Ubuntu
wsl --install

配置Ubuntu

# 禁用开机提示
touch . hushlogin

# 系统更新
sudo apt update -y
sudo apt install -y vim htop iotop lnav
sudo apt install -y wget curl git

显卡信息

sujx@alienware:~$ nvidia-smi
Wed Aug 21 15:56:03 2024
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550. 73. 01              Driver Version: 552. 12         CUDA Version: 12. 4     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp. A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 4050 ...    On  |   00000000:01:00. 0  On |                  N/A |
| N/A   49C    P8              5W /   45W |     832MiB /   6141MiB |      1%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

安装Docker

下载安装文件
安装完成
配置代理

配置CUDA

大模型部署

下载模型

# 通过构建魔搭下载器来获取大模型
docker pull python:3. 10-slim
docker run --rm -it -v `pwd`:/models python:3. 10-slim bash
sed -i 's/snapshot.debian.org/mirrors.tuna.tsinghua.edu.cn/g' /etc/apt/sources.list.d/debian.sources
pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple
cd /models
pip install modelscope

# 1. 5B下载
modelscope download --model=qwen/Qwen2-1. 5B-Instruct-GGUF --local_dir . qwen2-1_5b-instruct-q5_k_m.gguf
# 7B下载
modelscope download --model=qwen/Qwen2-7B-Instruct-GGUF --local_dir . qwen2-7b-instruct-q5_k_m.gguf

Ollama部署

docker pull ollama/ollama:0. 3. 6

# 默认 CPU 模式运行
# docker run -d -v ollama:/root/. ollama -p 11434:11434 --name ollama ollama/ollama

# Nvidia GPU 模式运行
docker run -d --gpus=all -v .\ollama:/root/. ollama -p 11434:11434 --name ollama ollama/ollama:0. 3. 6

# 进入Docker
docker exec -it ollama bash
mkdir ollama
cd ollama/
mv ../. ollama/qwen2-7b-instruct-q5_k_m.gguf ./
echo "FROM ./qwen2-7b-instruct-q5_k_m.gguf" > Modelfile
cd
ollama create custom_qwen2-7b -f ~/. ollama/Modelfile

导入

# 模型转换
root@055b6d9444a9:~/. ollama# ollama create custom_qwen2-7b -f ~/. ollama/Modelfile
transferring model data 100%
using existing layer sha256:7aeb11bb9b36d47625bd61514a3ac085826537144b375bb1d66b42738127e425
using autodetected template chatml
using existing layer sha256:f02dd72bb2423204352eabc5637b44d79d17f109fdb510a7c51455892aa2d216
using existing layer sha256:78a522a17dc3f34b94876c309ad592f7ff4ff1e029777eeac93774d69dd11bc9
writing manifest
success

# 查看模型信息
root@055b6d9444a9:~# ollama show custom_qwen2-7b
  Model
        arch                    qwen2
        parameters              7. 6B
        quantization            Q5_K_M
        context length          32768
        embedding length        3584

# 转换完成的模型       
root@055b6d9444a9:~# du -hs ~/. ollama/models/
5. 1G    /root/. ollama/models/

演示

docker pull soulteary/ollama:0. 3. 0-fix
docker run --gpus all -d -v .\ollama:/root/. ollama -p 11434:11434 --name qwen2 soulteary/ollama:0. 3. 0-fix 

sujx@alienware:~$ curl http://192. 168. 1. 158:11434/api/generate -d '{
  "model": "custom_qwen2-7b",
  "prompt":"天空为什么那么蓝?"
}'
{"model":"custom_qwen2-7b", "created_at":"2024-08-21T18:09:41. 363290951Z", "response":"天空", "done":false}
{"model":"custom_qwen2-7b", "created_at":"2024-08-21T18:09:41. 422390576Z", "response":"之所以", "done":false}
{"model":"custom_qwen2-7b", "created_at":"2024-08-21T18:09:41. 482391648Z", "response":"呈现", "done":false}
{"model":"custom_qwen2-7b", "created_at":"2024-08-21T18:09:41. 542272685Z", "response":"蓝色", "done":false}
{"model":"custom_qwen2-7b"， "created_at":"2024-08-21T18:09:41. 603602263Z"， "response":"，"， "done":false}
{"model":"custom_qwen2-7b", "created_at":"2024-08-21T18:09:41. 664370661Z", "response":"主要是", "done":false}
{"model":"custom_qwen2-7b", "created_at":"2024-08-21T18:09:41. 721054611Z", "response":"由于", "done":false}
{"model":"custom_qwen2-7b", "created_at":"2024-08-21T18:09:41. 779412138Z", "response":"大气", "done":false}
{"model":"custom_qwen2-7b", "created_at":"2024-08-21T18:09:41. 8385991Z", "response":"中的", "done":false}
{"model":"custom_qwen2-7b", "created_at":"2024-08-21T18:09:41. 896342916Z", "response":"气体", "done":false}
{"model":"custom_qwen2-7b", "created_at":"2024-08-21T18:09:41. 95431912Z", "response":"和", "done":false}
{"model":"custom_qwen2-7b", "created_at":"2024-08-21T18:09:42. 010744209Z", "response":"微", "done":false}
{"model":"custom_qwen2-7b", "created_at":"2024-08-21T18:09:42. 068093671Z", "response":"小", "done":false}
{"model":"custom_qwen2-7b", "created_at":"2024-08-21T18:09:42. 12515422Z", "response":"颗粒", "done":false}
{"model":"custom_qwen2-7b", "created_at":"2024-08-21T18:09:42. 182568345Z", "response":"对", "done":false}
{"model":"custom_qwen2-7b", "created_at":"2024-08-21T18:09:42. 239366307Z", "response":"太阳", "done":false}
{"model":"custom_qwen2-7b", "created_at":"2024-08-21T18:09:42. 297282361Z", "response":"光", "done":false}
{"model":"custom_qwen2-7b", "created_at":"2024-08-21T18:09:42. 353023896Z", "response":"的", "done":false}
{"model":"custom_qwen2-7b", "created_at":"2024-08-21T18:09:42. 410579873Z", "response":"散", "done":false}
{"model":"custom_qwen2-7b", "created_at":"2024-08-21T18:09:42. 46952615Z", "response":"射", "done":false}
{"model":"custom_qwen2-7b", "created_at":"2024-08-21T18:09:42. 525942839Z", "response":"作用", "done":false}
{"model":"custom_qwen2-7b", "created_at":"2024-08-21T18:09:42。 583992362Z", "response":"。", "done":false}
{"model":"custom_qwen2-7b", "created_at":"2024-08-21T18:09:42. 64235722Z", "response":"太阳", "done":false}
{"model":"custom_qwen2-7b", "created_at":"2024-08-21T18:09:42. 701759517Z", "response":"光", "done":false}
{"model":"custom_qwen2-7b", "created_at":"2024-08-21T18:09:42. 759330313Z", "response":"是由", "done":false}
{"model":"custom_qwen2-7b", "created_at":"2024-08-21T18:09:42. 817788108Z", "response":"各种", "done":false}
{"model":"custom_qwen2-7b", "created_at":"2024-08-21T18:09:42. 874783492Z", "response":"不同", "done":false}
{"model":"custom_qwen2-7b", "created_at":"2024-08-21T18:09:42. 935173302Z", "response":"波", "done":false}
{"model":"custom_qwen2-7b", "created_at":"2024-08-21T18:09:42. 992110059Z", "response":"长", "done":false}
{"model":"custom_qwen2-7b", "created_at":"2024-08-21T18:09:43. 052014607Z", "response":"的", "done":false}
{"model":"custom_qwen2-7b", "created_at":"2024-08-21T18:09:43. 112184454Z", "response":"光线", "done":false}
{"model":"custom_qwen2-7b", "created_at":"2024-08-21T18:09:43. 170437415Z", "response":"组成的", "done":false}
{"model":"custom_qwen2-7b", "created_at":"2024-08-21T18:09:43. 230450769Z", "response":"复合", "done":false}
{"model":"custom_qwen2-7b", "created_at":"2024-08-21T18:09:43. 288458072Z", "response":"光", "done":false}
{"model":"custom_qwen2-7b"， "created_at":"2024-08-21T18:09:43. 347953774Z"， "response":"，"， "done":false}
{"model":"custom_qwen2-7b", "created_at":"2024-08-21T18:09:43. 407144815Z", "response":"包含了", "done":false}
{"model":"custom_qwen2-7b", "created_at":"2024-08-21T18:09:43. 466536397Z", "response":"从", "done":false}
{"model":"custom_qwen2-7b", "created_at":"2024-08-21T18:09:43. 524233057Z", "response":"紫", "done":false}
{"model":"custom_qwen2-7b", "created_at":"2024-08-21T18:09:43. 582222113Z", "response":"到", "done":false}
{"model":"custom_qwen2-7b", "created_at":"2024-08-21T18:09:43. 63959678Z", "response":"红", "done":false}
{"model":"custom_qwen2-7b", "created_at":"2024-08-21T18:09:43. 697854731Z", "response":"的各种", "done":false}
{"model":"custom_qwen2-7b", "created_at":"2024-08-21T18:09:43. 755939647Z", "response":"颜色", "done":false}
{"model":"custom_qwen2-7b", "created_at":"2024-08-21T18:09:43。 813553529Z", "response":"。", "done":false}

前端部署

MaxKB

MaxKB是国内开源的一款基于 LLM 大语言模型的开源知识库问答系统，旨在成为企业的最强大脑。

启动

# 通过Docker拉取镜像
docker pull cr2. fit2cloud.com/1panel/maxkb
# 拉起服务
docker run -d --name=Maxkb -p 8080:8080 -v ~/. maxkb:/var/lib/postgresql/data cr2. fit2cloud.com/1panel/maxkb

# 默认登录信息
http://127. 0. 0. 1:8080
用户名：admin
默认密码：MaxKB@123..

应用

添加应用

配置模型

这里需要注意两点：
1. 添加大模型的地址不能使用127. 0. 0. 1或者localhost，可以用实际主机获取地址
2. 由于我们使用的是自定义过的大模型，需要配置自定义模型名称

测试模型
另外问题测试
后台管理
添加知识库
导入Ansible面试题
关联应用和知识库
自我展示

Open-webUI

open-webUI 是另一个著名的LLM前端项目，它是一个类似chatGPT风格的、自托管的大模型前端项目，可以对接Ollama这样的对话大模型，也可以对接SD这样的文生图大模型、TTS-OpenedAI这样的文生语音大模型。

启动

# 拉取镜像
docker pull ghcr.io/open-webui/open-webui:main
# 拉起服务
docker run -d -p 3000:8080 -e OLLAMA_BASE_URL=http://172. 16. 10. 110:11434 -v open-webui:/app/backend/data --name open-webui ghcr.io/open-webui/open-webui:main

应用

默认没有用户，需要注册一个
界面和功能较为简单