Windows环境下搭建Qwen开发环境

1 参考文献

【官方指引】https://qwen.readthedocs.io/en/latest/

【ModelScope训练】https://modelscope.cn/docs/%E4%BD%BF%E7%94%A8Tuners

【CUDA下载安装教程】https://blog.csdn.net/changyana/article/details/135876568

【安装cuDNN】https://developer.nvidia.com/rdp/cudnn-archive

【安装PyTorch】https://pytorch.org/

【安装Ollama】https://ollama.com/download

2 基础环境

2.1 安装CUDA

【查看显卡驱动】nvidia-smi

【验证CUDA安装】nvcc -V

首先查看NVIDIA显卡对应的CUDA版本号，然后根据此版本号下载对应版本的Toolkit

2.2 安装cuDNN

把下载的cudnn压缩包进行解压，在cudnn的文件夹下，把bin，include，lib文件夹下的内容对应拷贝到cuda相应的bin，include，lib下即可，最后安装完成。

2.3 安装PyTorch

# 验证PyTorch是否与CUDA兼容

import torch

print(torch.\_\_version\_\_)
print(torch.cuda.is\_available())

3 通过Ollama调用模型

Ollama是一种比较简单方便的本地运行方案。通过Ollama官网直接安装，可搜索支持的大模型：https://ollama.com/search?q=qwen

4 Qwen官方指引

Qwen官方指南详细说明了支持的各种方案，比查看各种网络博客说明更清晰。

5 自定义训练

5.1 参考ModelScope指引

\# 自定义ModelScope模型缓存路径
export MODELSCOPE\_CACHE\=/Users/kuliuheng/workspace/aiWorkspace/Qwen

 1 # A100 18G memory
 2 from swift import Seq2SeqTrainer, Seq2SeqTrainingArguments 3 from modelscope import MsDataset, AutoTokenizer 4 from modelscope import AutoModelForCausalLM 5 from swift import Swift, LoraConfig 6 from swift.llm import get\_template, TemplateType 7 import torch 8 
 9 pretrained\_model = 'qwen/Qwen2.5-0.5B-Instruct'
10 
11 
12 def encode(example):
13     inst, inp, output = example\['instruction'\], example.get('input', None), example\['output'\]
14     if output is None:
15         return {}
16     if inp is None or len(inp) == 0:
17         q = inst
18     else:
19         q = f'{inst}\\n{inp}'
20     example, kwargs = template.encode({'query': q, 'response': output})
21     return example
22 
23 
24 if \_\_name\_\_ == '\_\_main\_\_':
25     # 拉起模型
26     model = AutoModelForCausalLM.from\_pretrained(pretrained\_model, torch\_dtype=torch.bfloat16, device\_map='auto', trust\_remote\_code=True)
27     lora\_config = LoraConfig(
28                     r=8,
29                     bias='none',
30                     task\_type="CAUSAL\_LM",
31                     target\_modules=\["q\_proj", "k\_proj", "v\_proj", "o\_proj"\],
32                     lora\_alpha=32,
33                     lora\_dropout=0.05)
34     model = Swift.prepare\_model(model, lora\_config)
35     tokenizer = AutoTokenizer.from\_pretrained(pretrained\_model, trust\_remote\_code=True)
36     dataset = MsDataset.load('AI-ModelScope/alpaca-gpt4-data-en', split='train')
37     template = get\_template(TemplateType.chatglm3, tokenizer, max\_length=1024)
38 
39     dataset = dataset.map(encode).filter(lambda e: e.get('input\_ids'))
40     dataset = dataset.train\_test\_split(test\_size=0.001)
41 
42     train\_dataset, val\_dataset = dataset\['train'\], dataset\['test'\]
43 
44     train\_args = Seq2SeqTrainingArguments(
45         output\_dir='output',
46         learning\_rate=1e-4,
47         num\_train\_epochs=2,
48         eval\_steps=500,
49         save\_steps=500,
50         evaluation\_strategy='steps',
51         save\_strategy='steps',
52         dataloader\_num\_workers=4,
53         per\_device\_train\_batch\_size=1,
54         gradient\_accumulation\_steps=16,
55         logging\_steps=10,
56 )
57 
58     trainer = Seq2SeqTrainer(
59         model=model,
60         args=train\_args,
61         data\_collator=template.data\_collator,
62         train\_dataset=train\_dataset,
63         eval\_dataset=val\_dataset,
64         tokenizer=tokenizer)
65 
66     trainer.train()

View Code

（1）官方示例代码中没有写 __main__ 主函数入口，实际运行时发现会报错提示说：子线程在主线程尚未完成初始化之前就运行了。所以这里就补齐了一个主函数入口

（2）官方代码中没有针对 ‘qwen/Qwen2.5-0.5B-Instruct’ 模型代码，运行时target_modules会提示错误，需要指定模型中实际存在的模块名才行。这里有个技巧，通过打印模型信息可以看到实际的层级结构：

from modelscope import AutoModelForCausalLM

model\_name \= 'qwen/Qwen2.5-0.5B-Instruct'
model \= AutoModelForCausalLM.from\_pretrained(model\_name)
print(model)

得到如下结果：

Qwen2ForCausalLM(
  (model): Qwen2Model(
    (embed\_tokens): Embedding(151936, 896)
    (layers): ModuleList(
      (0\-23): 24 x Qwen2DecoderLayer(
        (self\_attn): Qwen2SdpaAttention(
          (q\_proj): Linear(in\_features\=896, out\_features=896, bias=True)
          (k\_proj): Linear(in\_features\=896, out\_features=128, bias=True)
          (v\_proj): Linear(in\_features\=896, out\_features=128, bias=True)
          (o\_proj): Linear(in\_features\=896, out\_features=896, bias=False)
          (rotary\_emb): Qwen2RotaryEmbedding()
        )
        (mlp): Qwen2MLP(
          (gate\_proj): Linear(in\_features\=896, out\_features=4864, bias=False)
          (up\_proj): Linear(in\_features\=896, out\_features=4864, bias=False)
          (down\_proj): Linear(in\_features\=4864, out\_features=896, bias=False)
          (act\_fn): SiLU()
        )
        (input\_layernorm): Qwen2RMSNorm((896,), eps=1e-06)
        (post\_attention\_layernorm): Qwen2RMSNorm((896,), eps=1e-06)
      )
    )
    (norm): Qwen2RMSNorm((896,), eps=1e-06)
    (rotary\_emb): Qwen2RotaryEmbedding()
  )
  (lm\_head): Linear(in\_features\=896, out\_features=151936, bias=False)
)