LLaMA-Factory实战推理

LLaMA-Factory官网：https://github.com/hiyouga/LLaMA-Factory

安装环境

git clone https://github.com/hiyouga/LLaMA-Factory.git cd LLaMA-Factory/ conda create -n py310 python=3.10 conda activate py310 

按照llama-factory要求的标准格式组织数据集，保存成一个文件，比如下面这种.json文件：

[ { "instruction": "user instruction (required)", "input": "user input (optional)", "output": "model response (required)", "system": "system prompt (optional)", "history": [ ["user instruction in the first round (optional)", "model response in the first round (optional)"], ["user instruction in the second round (optional)", "model response in the second round (optional)"] ] } ] 

大模型选择：

LLaMA、Mistral、Mixtral-MoE、Qwen、Yi、Gemmha、Baichuan、ChatGLM、Phi

推理模型

CUDA_VISIBLE_DEVICES=0 python cli_demo.py \--model_name_or_path path_to_llama_model \--adapter_name_or_path path_to_checkpoint \--template default \--finetuning_type lora 

总结：

有效的微调已成为大型语言模型适应特定任务的必要条件之一。随着 Llama-Factory 的引入，这一全面的框架让训练更加高效，用户无需编写代码即可轻松为超过 100 个 LLMs 定制微调。