本地搭建【文档助手】大模型版(LangChain+llama+Streamlit)

本地搭建【文档助手】大模型版(LangChain+llama+Streamlit)

    正在检查是否收录...

概述

本文的文档助手就是:我们上传一个文档,然后在对话框中输入问题,大模型会把问题的答案返回。

安装步骤

先下载代码到本地

LangChain调用llama模型的示例代码:https://github.com/afaqueumer/DocQA(代码不是本人写的,尊重原创)

git clone https://github.com/afaqueumer/DocQA.git 
环境安装
双击 setup_env.bat 
如果没反应可能是缺少环境,打开控制台手动执行一下,缺python或者pip的自己根据报错下载一下

如果llama-cpp-python安装报错
(1)需要下载Visual Studio
(2)打开Visual Studio,工具,获取工具和功能

(3)等待下载完,重新运行setup_env.bat

如果还有报错【error C2061: 语法错误:】,那么可能是Visual Studio的版本太低了,我一开始用的是2019版本,后来换成了2022

更新为2022之后重复上面操作

下载一个靠谱的模型
https://huggingface.co/TheBloke/Llama-2-7B-GGUF
本文用的是:TheBloke/Llama-2-7B-GGUF中的最精简版
进入DocQA,修改app.py

原始

llm = LlamaCpp(model_path="./models/llama-7b.ggmlv3.q4_0.bin") embeddings = LlamaCppEmbeddings(model_path="models/llama-7b.ggmlv3.q4_0.bin") 

这里改成自己下载的模型地址,比如:llama-2-7b.Q2_K.gguf

llm = LlamaCpp(model_path="../llama.cpp/models/7B/llama-2-7b.Q2_K.gguf") embeddings = LlamaCppEmbeddings(model_path="../llama.cpp/models/7B/llama-2-7b.Q2_K.gguf") 
运行
双击run_app.bat 
测试

准备好一个txt文档

As of October this year, there were nearly 2,500 geographical indication products in China The reporter learned from the State Intellectual Property Office that in recent years, the quantity and quality of China's geographical indication products have risen rapidly. As of October this year, China has approved a total of 2,495 geographical indication products, and approved 7,013 geographical indications to be registered as collective trademarks and certification trademarks. In 2021, the direct output value of GI products exceeded 700 billion yuan. In recent years, the State Intellectual Property Office has conscientiously implemented the decisions and arrangements of the CPC Central Committee and the State Council, actively and steadily promoted the unified acceptance channels, unified special signs, unified announcements, unified protection and supervision, unified foreign cooperation and other work, and further improved the system of protection, management and application of geographical indications. In terms of institutional construction, the State Intellectual Property Office issued the "14th Five-Year Plan for the Protection and Use of Geographical Indications", formulated and issued a unified special indication for geographical indications, revised and issued the "Measures for the Protection of Foreign Geographical Indication Products", and launched the legislative work on geographical indications; In the year, a total of 1,416 cases of infringement of geographical indications were investigated and dealt with across the country, involving an amount of 9.28 million yuan and a fine of 13.023 million yuan 

上传到页面中

没有GPU的痛苦,运行太慢了

注意:别用中文问,这个模型好像不支持中文,换一个支持中文的模型就行了

llamacoderapappmacgitdocintelofficefaqpythongithubllmcto文档助手gpu大模型huggingfacelangchainstemlcmtpuapicloneurl
  • 本文作者:李琛
  • 本文链接: https://wapzz.net/post-5656.html
  • 版权声明:本博客所有文章除特别声明外,均默认采用 CC BY-NC-SA 4.0 许可协议。
本站部分内容来源于网络转载,仅供学习交流使用。如涉及版权问题,请及时联系我们,我们将第一时间处理。
文章很赞!支持一下吧 还没有人为TA充电
为TA充电
还没有人为TA充电
0
  • 支付宝打赏
    支付宝扫一扫
  • 微信打赏
    微信扫一扫
感谢支持
文章很赞!支持一下吧
关于作者
2.3W+
5
0
1
WAP站长官方

微软Copilot Pro来了:个人用户也能在Word里用GPT-4,20美元/月

上一篇

Portkey AI Gateway:一个连接多种人工智能模型的开源工具

下一篇
  • 复制图片
按住ctrl可打开默认菜单