说明
做这个的主要目的是为了搭建Langchain的本地环境，使用LangChain让LLM具备调用自定义函数的功能。
内容
1 安装server
以下将ollama的安装方式，以及使用做一个简单的说明(记录）。之前对这个工具没有了解，只是从快速实践的角度上，给到一个参考。
最初是奔着LLM调用自定义函数/API这个功能去的，快速看下来，一个比较可行的方案是LangChain。
参考页面
后来大致明白， langchain是一个前端的使用包， langserver则是后端服务。先搭好服务，然后才能在前端使用。
教程里的几种方式，都需要一个大语言后端支持，为了避免后续的麻烦，所以我决定搭建一下ollama。
然后就跳到了ollama的页面
看起来ollama支持的系统还是比较全面的
1.1 苹果
实测：苹果的下载非常快… 秒级。展开后大约435M。
如要使用之前，需要在终端上先进行拉取，命令就是上面那个。
然后执行测试
import ollama
response = ollama.chat(model='llama2', messages=[
{
'role': 'user',
'content': '解析出收件人地点、公司、收件人和收件人电话\n帮我寄到上海国金中心中心33F, ABC公司，Bikky收就行，电话号码13566778899。我的电话是18988998899，上海杨浦区。',
},
])
print(response['message']['content'])
经过分析，可以知道：
* 收件人地点：上海市，特别是杨浦区
* 公司名称：ABC公司
* 收件人姓名：Bikky
* 收件人电话号码：13566778899
* 你的电话号码：18988998899，也在上海市，但不同区。
在执行效率上，我的苹果(m1 pro)，配置也不算低了，但是执行效率上还是比2080TI慢了2~3倍。
mac m1 pro: 平均8秒
[3.5188119411468506,
5.56521201133728,
7.0565900802612305,
11.417732238769531,
7.563968896865845,
9.987686157226562,
6.56359601020813,
6.939379930496216,
9.785239219665527,
11.655837059020996]
In [10]: np.mean(time_list)
Out[10]: 8.005405354499818
2080Ti：平均2.9秒
[2.193615674972534,
2.8509912490844727,
2.972665786743164,
2.2655117511749268,
3.038464069366455,
4.976086378097534,
3.4014697074890137,
2.209334373474121,
2.832230567932129,
2.2567901611328125]
np.mean(time_list)
2.8997159719467165
以下是可拉取的模型，最大的70B library
1.2 linux安装
执行脚本
curl -fsSL https://ollama.com/install.sh | sh
启动服务
ollama serve
执行包的安装
pip3 install ollama -i https://mirrors.aliyun.com/pypi/simple/
pip3 install langchain_community -i https://mirrors.aliyun.com/pypi/simple/
pip3 install beautifulsoup4 -i https://mirrors.aliyun.com/pypi/simple/
pip3 install faiss-cpu -i https://mirrors.aliyun.com/pypi/simple/
pip3 install langchain_text_splitters -i https://mirrors.aliyun.com/pypi/simple/
pip3 install langchain -i https://mirrors.aliyun.com/pypi/simple/
然后就可以使用了。
这种方法一般用在没有绝对控制权的情况，例如租用的算力主机。
1.3 docker安装
这种方式最为简便，也最为有用，但是要求对主机具有绝对控制权。
直接拉取最新版本
docker pull ollama/ollama
然后启动，考虑到server拉取模型会占据很大空间，所以把一个大的数据盘挂到root下。ollama下载的文件会存在root的 .ollama隐藏文件夹下面。
docker run -d \
--name=ollama01 \
-v /etc/localtime:/etc/localtime \
-v /etc/timezone:/etc/timezone\
-v /etc/hostname:/etc/hostname \
-v /data:/root \
-e "LANG=C.UTF-8"\
-p 11434:11434\
-w /workspace \
--gpus all \
ollama/ollama
本地docker安装，在第一次启动时可能会碰到一些问题，
└─ $ docker run -it --rm --gpus=all registry.cn-hangzhou.aliyuncs.com/andy08008/pytorch_jupyter:v8 bash
docker: Error response from daemon: could not select device driver "" with capabilities: [[gpu]].
需要做一些连通显卡和docker的操作
1 添加源
curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | \
sudo apt-key add -
distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | \
sudo tee /etc/apt/sources.list.d/nvidia-docker.list
sudo apt-get update
2 安装: nvidia-container-toolkit
sudo apt-get install -y nvidia-container-toolkit
3 重启：通常也就是在这里，租用的机器要么没有systemd，要么不把这个权限给你。
systemctl restart docker
这个服务比较有趣的一点是，ollama server会自动进行显存的管理
空闲时：
当使用模型预测时
import ollama
response = ollama.chat(model='llama2', messages=[
{
'role': 'user',
'content': '解析出收件人地点、公司、收件人和收件人电话\n帮我寄到上海国金中心中心33F, ABC公司，Bikky收就行，电话号码13566778899。我的电话是18988998899，上海杨浦区。',
},
])
print(response['message']['content'])
Based on the information provided, here is a breakdown of the recipient's details:
* Recipient's location: 上海国金中心 (Shanghai Gold Center) in the 33F floor.
* Company name: ABC公司 (ABC Company).
* Recipient's name: Bikky收就行 (Bikky receives).
* Recipient's phone number: 13566778899.
* Your phone number: 18988998899, located in 上海杨浦区 (Shanghai Yangpu District).
Please note that the above information is based on the text you provided and may not be accurate or up-to-date.
默认使用llama2-7b模型，调用时显存会被占用。
如果过一会不用，显存会自动清除。而且如果我们再使用另一个模型时，server会自动切换模型，把之前的清掉，然后载入新的，这样显存就不会爆。
import ollama
response = ollama.chat(model='llama2:13b', messages=[
{
'role': 'user',
'content': '解析出收件人地点、公司、收件人和收件人电话\n帮我寄到上海国金中心中心33F, ABC公司，Bikky收就行，电话号码13566778899。我的电话是18988998899，上海杨浦区。',
},
])
print(response['message']['content'])
OK! Here's the analysis of the information you provided:
1. Receiver's location: Shanghai, China (based on the address "上海国金中心中心33F")
2. Company name: ABC Company (based on the address "上海国金中心中心33F")
3. Receiver's name: Bikky (based on the name "Bikky收")
4. Receiver's contact information: Phone number 13566778899 (based on the phone number in the address)
5. Sender's location: Shanghai Yangpu District (based on the phone number 18988998899)
6. Sender's name: Not provided
I hope this helps! Let me know if you have any other questions.
2 langchain实验
先快速跟着教程走一遍
1 载入大模型，问一个简单的问题： how can langsmith help with testing?
from langchain_community.llms import Ollama
llm = Ollama(model="llama2")
llm.invoke("how can langsmith help with testing?")
Langsmith is a tool that can be used to test and validate the correctness of language models. Here are some ways in which Langsmith can help with testing:
1. **Text generation**: Langsmith can be used to generate text samples that can be used to test the language model's ability to produce coherent and contextually relevant text. By comparing the generated text to a reference sample, Langsmith can evaluate the model's performance in terms of fluency, coherence, and relevance.
2. **Text completion**: Langsmith can be used to test the language model's ability to complete partial sentences or phrases. By providing a starting point for the model and evaluating its output, Langsmith can assess the model's ability to capture the context and produce coherent text.
3. **Sentiment analysis**: Langsmith can be used to test the language model's ability to classify text as positive, negative, or neutral. By providing a dataset of labeled text samples and evaluating the model's performance, Langsmith can assess its ability to accurately classify sentiment.
4. **Named entity recognition**: Langsmith can be used to test the language model's ability to identify named entities in text, such as people, organizations, and locations. By providing a dataset of labeled text samples and evaluating the model's performance, Langsmith can assess its ability to accurately identify named entities.
5. **Question answering**: Langsmith can be used to test the language model's ability to answer questions based on a given context or input. By providing a dataset of labeled question-answer pairs and evaluating the model's performance, Langsmith can assess its ability to accurately answer questions.
6. **Dialogue generation**: Langsmith can be used to test the language model's ability to engage in coherent and contextually relevant dialogue. By providing a dataset of labeled dialogue samples and evaluating the model's performance, Langsmith can assess its ability to produce natural-sounding dialogue.
7. **Multi-task evaluation**: Langsmith can be used to test the language model's ability to perform multiple tasks simultaneously, such as language translation, sentiment analysis, and named entity recognition. By providing a dataset of labeled text samples and evaluating the model's performance across multiple tasks, Langsmith can assess its ability to handle multi-tasking.
By using Langsmith to test these aspects of language models, developers can gain a better understanding of their strengths and weaknesses, and make informed decisions about how to improve them.
2 使用prompt模型进行修改
from langchain_core.prompts import ChatPromptTemplate
prompt = ChatPromptTemplate.from_messages([
("system", "You are world class technical documentation writer."),
("user", "{input}")
])
3 组成链，并使用链提问。感觉风格有点变，但内容好坏不确定。
chain = prompt | llm
print(chain.invoke({"input": "how can langsmith help with testing?"}))
As a world-class technical documentation writer, I must say that Langsmith is an excellent tool for testing! Here are some ways in which Langsmith can help with testing:
1. Automated Testing: Langsmith provides automated testing capabilities, allowing you to create and run tests without any manual intervention. This saves time and effort, while also ensuring consistency and accuracy in your test results.
2. Customizable Tests: With Langsmith, you can create customizable tests tailored to your specific needs. You can define the test cases, question types, and evaluation criteria to suit your requirements.
3. Integration with Existing Tools: Langsmith integrates seamlessly with popular testing tools such as JIRA, TestRail, and PractiTest. This enables you to create, run, and manage tests within a single platform, streamlining your testing process.
4. Collaborative Testing: Langsmith supports collaborative testing, allowing multiple team members to work together on a test project. This fosters collaboration and coordination among team members, ensuring that everyone is on the same page and working towards the same goal.
5. Real-time Feedback: Langsmith provides real-time feedback on test results, enabling you to identify areas of improvement immediately. This helps you to rectify errors early on and ensure that your tests are accurate and reliable.
6. Test Case Management: Langsmith offers a comprehensive test case management system, which enables you to organize, track, and maintain your test cases. This helps you to keep your tests organized, up-to-date, and easily accessible.
7. Reporting and Analytics: Langsmith provides detailed reporting and analytics on test results, allowing you to identify trends, strengths, and weaknesses in your testing process. This enables you to optimize your testing strategy and improve overall quality.
8. Integration with Agile Methodologies: Langsmith is designed to work seamlessly with agile methodologies such as Scrum and Kanban. This ensures that your testing activities are aligned with the rest of your development process, enabling you to deliver high-quality software quickly and efficiently.
9. Customizable Workflows: Langsmith allows you to create customizable workflows tailored to your specific needs. This enables you to streamline your testing process and ensure that it aligns with your project's unique requirements.
10. User-Friendly Interface: Langsmith boasts a user-friendly interface, making it easy for team members to use and navigate the platform. This reduces the learning curve and ensures that everyone can use the platform effectively.
In summary, Langsmith is an excellent tool for testing as it provides automated testing capabilities, customizable tests, integration with existing tools, collaborative testing, real-time feedback, test case management, reporting and analytics, integration with agile methodologies, customizable workflows, and a user-friendly interface. These features work together to create a comprehensive and efficient testing platform that can help you deliver high-quality software quickly and efficiently.
4 继续加链。应该是对答案做了后除了，但也没太看出差别
from langchain_core.output_parsers import StrOutputParser
output_parser = StrOutputParser()
chain = prompt | llm | output_parser
print(chain.invoke({"input": "how can langsmith help with testing?"}))
As a world-class technical documentation writer, I can help with testing in several ways:
1. Content Creation: Langsmith can assist in creating comprehensive and accurate content for your software or system, including user manuals, technical guides, and release notes. This content can be used to test the system's functionality and usability, ensuring that it meets the requirements and expectations of users.
2. Collaborative Testing: Langsmith can collaborate with your testing team to create test cases and scenarios based on the documentation created. This can help ensure that all aspects of the system are thoroughly tested and that any issues or bugs are identified early in the development process.
3. Automated Testing: By using natural language processing (NLP) and machine learning (ML) algorithms, Langsmith can assist in automating testing processes. For example, Langsmith can be used to generate test cases based on the documentation, or to analyze test results and identify areas for improvement.
4. Test Data Generation: Langsmith can help generate test data that is tailored to your system's requirements. This can include generating sample inputs, outputs, and edge cases that can be used to test the system's functionality.
5. Defect Reporting: Langsmith can assist in identifying and reporting defects or issues found during testing. By analyzing the documentation and test results, Langsmith can generate detailed reports of defects and suggest fixes or improvements.
6. Test Planning: Langsmith can help plan and prioritize testing efforts by analyzing the system's requirements and identifying critical areas that need to be tested. This can help ensure that the most important features and functionality are thoroughly tested, and that testing resources are allocated effectively.
7. Test Execution: Langsmith can assist in executing tests by generating test scripts based on the documentation and test cases created. This can help ensure that all aspects of the system are tested and that any issues or bugs are identified early in the development process.
8. Test Data Management: Langsmith can help manage test data by generating sample inputs, outputs, and edge cases that can be used to test the system's functionality. This can help ensure that test data is accurate, up-to-date, and relevant to the system being tested.
9. Performance Tuning: By analyzing the documentation and testing results, Langsmith can assist in identifying performance issues and suggest optimizations or improvements.
10. Security Testing: Langsmith can help identify security vulnerabilities in the system by analyzing the documentation and test results. This can help ensure that the system is secure and meets the necessary security requirements.
In summary, Langsmith can assist in various testing activities, including content creation, collaborative testing, automated testing, test data generation, defect reporting, test planning, test execution, test data management, performance tuning, and security testing. By leveraging natural language processing and machine learning algorithms, Langsmith can help improve the efficiency and effectiveness of your testing efforts.
5 通过web方式载入数据
from langchain_community.document_loaders import WebBaseLoader
loader = WebBaseLoader("https://docs.smith.langchain.com/user_guide")
docs = loader.load()
6 对载入的数据执行分割和嵌入
from langchain_community.embeddings import OllamaEmbeddings
embeddings = OllamaEmbeddings()
from langchain_community.vectorstores import FAISS
from langchain_text_splitters import RecursiveCharacterTextSplitter
text_splitter = RecursiveCharacterTextSplitter()
documents = text_splitter.split_documents(docs)
vector = FAISS.from_documents(documents, embeddings)
7 继续增加链，用create_stuff_documents_chain 加上prompt，形成文档查询链
from langchain.chains.combine_documents import create_stuff_documents_chain
prompt = ChatPromptTemplate.from_template("""Answer the following question based only on the provided context:
<context>
{context}
</context>
Question: {input}""")
document_chain = create_stuff_documents_chain(llm, prompt)
8 载入Document，然后执行查询
from langchain_core.documents import Document
document_chain.invoke({
"input": "how can langsmith help with testing?",
"context": [Document(page_content="langsmith can let you visualize test results")]
})
Based on the provided context, Langsmith can help with testing by providing a way to visualize test results. This means that Langsmith can assist in the process of testing software or applications by allowing users to view and analyze the results of those tests in a visual format, such as charts, graphs, or other visualizations.
9 然后使用向量增强
from langchain.chains import create_retrieval_chain
retriever = vector.as_retriever()
retrieval_chain = create_retrieval_chain(retriever, document_chain)
response = retrieval_chain.invoke({"input": "how can langsmith help with testing?"})
print(response["answer"])
Based on the provided context, LangSmith can help with testing in several ways:
1. Prototyping: LangSmith allows for quick experimentation between prompts, model types, and retrieval strategy, making it easier to understand how the model performs and debug where it is failing.
2. Initial Test Set: LangSmith enables developers to create datasets of inputs and reference outputs, which can be used to run tests on their LLM applications. This helps in identifying regressions with respect to initial test cases.
3. Custom Evaluations: LangSmith provides the ability to run custom evaluations (both LLM and heuristic based) to score test results, allowing developers to assess the performance of their LLM applications more comprehensively.
4. Comparison View: The comparison view in LangSmith enables users to see the results for different configurations side-by-side, helping diagnose regressions in test scores across multiple revisions of the application.
5. Playground Environment: LangSmith provides a playground environment for rapid iteration and experimentation, allowing users to quickly test out different prompts and models without having to run each one individually.
这块应该就是很多知识库的一般流程了。
ollama的搭建的使用到这里就结束了。
![[flink 实时流基础] 转换算子](https://img-blog.csdnimg.cn/img_convert/bab20ca044a79d3de50547bd843110a7.png)
