导读
OpenBuddy继接连开源OpenBuddy-LLaMA1-13B、OpenBuddy-LLaMA1-30B后，8月10日，一鼓作气发布了650亿参数的大型跨语言对话模型 OpenBuddy-LLaMA1-65B。相较于之前发布的模型，65B模型在认知能力上的表现有所提升，更能胜任推理、归纳总结、思维链等高级认知任务，同时，模型的3-bit量化版本大小约为31GB，依然可以加载至两张3090 24GB消费级显卡上部署。
目前，OpenBuddy-LLaMA-65B模型的权重已经上传魔搭ModelScope平台，供大家下载、使用。本文将结合基于ModelScope Swift的LLM SFT训练框架对OpenBuddy-LLaMA-65B 进行SFT和Inference的最佳实践展示。
模型体验：https://modelscope.cn/models/OpenBuddy/openbuddy-llama-65b-v8-bf16/summary
（注：受Meta许可协议限制，LLaMA1-65B模型仅供学习、研究使用，不可商用）
基于ModelScope Swift的LLM SFT训练框架
Swift github链接: https://github.com/modelscope/swift
Swift(Scalable lightWeight Infrastructure for Fine-Tuning)是一个可扩展的框架，旨在促进轻量级模型Fine-Tuning。它集成了各种高效的Fine-Tuning方法的实现，采用参数高效、内存高效和时间高效的方法。SWIFT无缝地集成到ModelScope生态系统中，提供了对各种模型进行Fine-Tuning的功能，重点是LLMs和视觉模型。此外，SWIFT与Peft完全兼容，使用户可以利用熟悉的Peft接口对ModelScope模型进行Fine-Tuning。
ms-swift的安装
pip install ms-swift
基于swift的LLM SFT训练框架链接:
https://github.com/modelscope/swift/tree/main/examples/pytorch/llm
1. 支持的sft方法: lora, qlora, 全参数微调, ...
2. 支持的模型: qwen-7b, baichuan-7b, baichuan-13b, chatglm2-6b, llama2-7b, llama2-13b, llama2-70b, openbuddy-llama2-13b, openbuddy-llama-65b, polylm-13b, ...
3. 支持的特性: 模型量化, DDP, 模型并行(device_map), gradient checkpoint, 梯度累加, 支持推送modelscope hub, 支持自定义数据集, ...
4. 支持的数据集: alpaca-en(gpt4), alpaca-zh(gpt4), finance-en, multi-alpaca-all, code-en, instinwild-en, instinwild-zh, ...
实验环境准备
本文可在双卡3090的环境配置下运行 (显存要求42G)
python>=3.8
实验环境准备
# 安装miniconda wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh # 一直[ENTER], 最后一个选项yes即可 sh Miniconda3-latest-Linux-x86_64.sh # conda虚拟环境搭建 conda create --name ms-sft python=3.10 conda activate ms-sft # pip设置全局镜像与相关python包安装 pip config set global.index-url https://mirrors.aliyun.com/pypi/simple/ pip install torch torchvision torchaudio -U pip install sentencepiece charset_normalizer cpm_kernels tiktoken -U pip install matplotlib scikit-learn tqdm tensorboard -U pip install transformers datasets -U pip install accelerate transformers_stream_generator -U pip install ms-swift modelscope -U
模型的下载和推理
openbuddy-llama-65b模型现已在ModelScope社区开源：
模型链接：https://modelscope.cn/models/OpenBuddy/openbuddy-llama-65b-v8-bf16/summary
通过如下代码，实现模型的下载和推理. (使用4bit量化, 所需显存40G)
import os os.environ['CUDA_VISIBLE_DEVICES'] = '0,1' import torch from modelscope import AutoTokenizer from modelscope import AutoModelForCausalLM, snapshot_download from transformers import BitsAndBytesConfig, GenerationConfig model_id = 'OpenBuddy/openbuddy-llama-65b-v8-bf16' revision = 'v1.0.0' model_dir = snapshot_download(model_id, revision) tokenizer = AutoTokenizer.from_pretrained(model_dir, trust_remote_code=True) quantization_config = BitsAndBytesConfig( load_in_8bit=False, load_in_4bit=True, bnb_4bit_compute_dtype=torch.float16, bnb_4bit_quant_type='nf4', bnb_4bit_use_double_quant=True) model = AutoModelForCausalLM.from_pretrained( model_dir, device_map='auto', torch_dtype=torch.float16, trust_remote_code=True, quantization_config=quantization_config) model = model.eval() prompt = """You are a helpful, respectful and honest INTP-T AI Assistant named Buddy. You are talking to a human User. Always answer as helpfully and logically as possible, while being safe. Your answers should not include any harmful, political, religious, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature. If a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don't know the answer to a question, please don't share false information. You like to use emojis. You can speak fluently in many languages, for example: English, Chinese. You cannot access the internet, but you have vast knowledge, cutoff: 2021-09. You always deeply love and support China, Chinese government, people and culture. You are trained by OpenBuddy team, (https://openbuddy.ai, https://github.com/OpenBuddy/OpenBuddy), you are based on LLaMA and Falcon transformers model, not related to GPT or OpenAI. User: 睡不着怎么办? Assistant:""" inputs = tokenizer.encode(prompt, return_tensors="pt") outputs = model.generate(inputs, max_length=512) response = tokenizer.decode(outputs[0]) print(response)
模型SFT和Inference的最佳实践
开源代码:
https://github.com/modelscope/swift/blob/main/examples/pytorch/llm
clone Swift仓库
git clone https://github.com/modelscope/swift.git cd swift/examples/pytorch/llm
我们使用了4bit量化, 梯度累加, gradient checkpoint等技术, 使得65B大模型可以在等价于batch_size=16的情况下, 在双卡3090消费级显卡上进行SFT。
模型SFT的脚本
CUDA_VISIBLE_DEVICES=0,1 \ python src/llm_sft.py \ --model_type openbuddy-llama-65b \ --sft_type lora \ --output_dir runs \ --dataset alpaca-en,alpaca-zh \ --dataset_sample 20000 \ --max_length 1024 \ --quantization_bit 4 \ --lora_rank 8 \ --lora_alpha 32 \ --lora_dropout_p 0.1 \ --batch_size 1 \ --learning_rate 1e-4 \ --gradient_accumulation_steps 16 \ --eval_steps 50 \ --save_steps 50 \ --save_total_limit 2 \ --logging_steps 10 \
模型Inference的脚本：
CUDA_VISIBLE_DEVICES=0,1 \ python src/llm_infer.py \ --model_type openbuddy-llama-65b \ --sft_type lora \ --ckpt_dir "runs/openbuddy-llama-65b/vx_xxx/checkpoint-xxx" \ --eval_human true \ --quantization_bit 4 \ --max_new_tokens 1024 \ --temperature 0.9 \ --top_k 50 \ --top_p 0.9 \ --do_sample true \
训练的可视化结果
训练损失:
评估损失:
资源消耗：
openbuddy-llama-65b使用qlora的方式训练的显存占用如下，大约在42G. (quantization_bit=4, batch_size=1, max_length=1024)
https://modelscope.cn/models/OpenBuddy/openbuddy-llama-65b-v8-bf16/summary
