项目名称：基于RAG的对于特定汽车的助手
报告日期：2024年8月18日
项目负责人：黄浩宇
### 项目概述
本项目的目标是开发一个基于RAG技术的文字智能对话机器人，旨在帮助用户解决使用车辆时遇到的问题，尤其是涉及导航系统的使用、功能设置等方面。通过结合RAG模型的强大信息检索能力和交互功能，用户能够更加便捷地获取所需信息，提高行车安全性和便利性。
项目的应用场景包括：
1. **导航与路线规划**：帮助用户通过语音或文本输入快速获取驾驶路线。
2. **车载系统功能查询**：用户可以通过对话机器人了解车辆功能的详细信息和操作指南。
3. **实时帮助与故障排查**：在行车过程中，用户可以实时获取解决车辆故障或使用问题的指导。
### 技术方案与实施步骤
#### 模型选择：本项目采用了Meta公司的`meta/llama-3.1-405b-instruct`模型，结合RAG（Retrieval-Augmented Generation）技术。选择该模型的理由包括：
- **高效的信息检索能力**：RAG模型能够通过检索外部文档或数据库，生成更加精确和上下文相关的回答。
- **强大的生成能力**：`meta/llama-3.1-405b-instruct`模型具备强大的自然语言生成能力，能够理解复杂的查询并提供准确的回答。
- **灵活的集成**：RAG模型能够与不同的数据源灵活集成，如车辆用户手册和实时导航数据，为用户提供全面的支持。
#### 数据的构建
数据构建过程中，我们主要使用了两份关于Honda HR-V车辆的用户手册，对这些文档进行了向量化处理。数据向量化的方法主要包括：
- **使用词向量模型**：对手册中的文本进行词向量化，以便RAG模型能够高效地检索相关信息。
- **构建知识库**：将向量化后的数据存储在一个知识库中，供RAG模型进行检索。通过这种方式，模型可以实时提供与用户查询相关的精确信息。
#### 实施步骤：
1. **环境搭建**：
· 安装必要的软件，包括Python、PyTorch等机器学习库。
· 配置开发环境，确保模型训练和推理能够顺利进行。
· 部署语音识别和自然语言处理工具，如Whisper模型，用于语音输入的处理。
2. **代码实现**：
Step 1 - 使用NVIDIA_API_KEY：
import getpass
import os
if os.environ.get("NVIDIA_API_KEY", "").startswith("nvapi-"):
print("Valid NVIDIA_API_KEY already in environment. Delete to reset")
else:
nvapi_key = getpass.getpass("NVAPI Key (starts with nvapi-): ")
assert nvapi_key.startswith("nvapi-"), f"{nvapi_key[:5]}... is not a valid key"
os.environ["NVIDIA_API_KEY"] = nvapi_key
from langchain_nvidia_ai_endpoints import ChatNVIDIA
ChatNVIDIA.get_available_models()
Step 2 - 初始化SLM
这里我们使用 meta/llama-3.1-405b-instruct
llm = ChatNVIDIA(model="meta/llama-3.1-405b-instruct", nvidia_api_key=nvapi_key, max_tokens=512)
result = llm.invoke("explain w to change the spair tayar of Honda HRV ho2021?")
print(result.content)
Step 3 - 初始化ai-embed-qa-4向量模型
from langchain_nvidia_ai_endpoints import NVIDIAEmbeddings
embedder = NVIDIAEmbeddings(model="ai-embed-qa-4")
Step 4 - 获取文本数据集
import os
from tqdm import tqdm
from pathlib import Path
# Here we read in the text data and prepare them into vectorstore
ps = os.listdir("./zh_data/")
data = []
sources = []
for p in ps:
if p.endswith('.pdf'):
path2file="./zh_data/"+p
with open(path2file,encoding="utf-8") as f:
lines=f.readlines()
for line in lines:
if len(line)>=1:
data.append(line)
sources.append(path2file)
Step 5 - 进行一些基本的清理并删除空行
documents=[d for d in data if d != '\n']
len(data), len(documents), data[0]
Step 6a - 将文档处理到 faiss vectorstore 并将其保存到磁盘
# Here we create a vector store from the documents and save it to disk.
from operator import itemgetter
from langchain.vectorstores import FAISS
from langchain_core.output_parsers import StrOutputParser
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.runnables import RunnablePassthrough
from langchain.text_splitter import CharacterTextSplitter
from langchain_nvidia_ai_endpoints import ChatNVIDIA
import faiss
# 只需要执行一次，后面可以重读已经保存的向量存储
# text_splitter = CharacterTextSplitter(chunk_size=400, separator=" ")
# docs = []
# metadatas = []
# for i, d in enumerate(documents):
# splits = text_splitter.split_text(d)
# #print(len(splits))
# docs.extend(splits)
# metadatas.extend([{"source": sources[i]}] * len(splits))
# store = FAISS.from_texts(docs, embedder , metadatas=metadatas)
# store.save_local('./zh_data/nv_embedding')
Step 6b - 重读之前处理并保存的 Faiss Vectore 存储
# Load the vectorestore back.
Store = FAISS.load_local("./zh_data/nv_embedding", embedder,allow_dangerous_deserialization=True)
Step 7- 提出问题并基于meta/llama-3.1-405b-instruct模型进行RAG检索
retriever = store.as_retriever()
prompt = ChatPromptTemplate.from_messages(
[
(
"system",
"Answer solely based on the following context:\n<Documents>\n{context}\n</Documents>",
),
("user", "{question}"),
]
)
chain = (
{"context": retriever, "question": RunnablePassthrough()}
| prompt
| llm
| StrOutputParser()
)
chain.invoke("explain w to change the spair tayar of Honda HRV ho2021?")
### 项目成果与展示
#### 应用场景展示
本项目的智能对话机器人主要应用于车辆导航和系统查询场景，如帮助用户设置导航目的地、了解车辆功能、以及处理常见问题。这些功能极大地提高了用户的驾驶体验和安全性。
#### 功能演示
retriever = store.as_retriever()
prompt = ChatPromptTemplate.from_messages(
[
(
"system",
"Answer solely based on the following context:\n<Documents>\n{context}\n</Documents>",
),
("user", "{question}"),
]
)
chain = (
{"context": retriever, "question": RunnablePassthrough()}
| prompt
| llm
| StrOutputParser()
)
chain.invoke("explain how to change the spair tayar of Honda HRV 2021?")
回答：
Changing a spare tire on a Honda HR-V 2021 is a relatively straightforward process. Here's a step-by-step guide to help you through it:
**Important:**
* Make sure you're in a safe location, away from traffic, and turn off the engine.
* Engage the parking brake and put the transmission in Park (automatic) or first gear (manual).
* Turn on the hazard lights.
* Familiarize yourself with the location of the spare tire, tools, and jacking points on your Honda HR-V.
**Location of the spare tire and tools:**
* The spare tire is located under the floor panel in the cargo area.
* The tools ( lug wrench, jack, and wheel chocks) are stored in a bag attached to the cargo area's side panel.
**Step-by-Step Instructions:**
1. **Loosen the lug nuts**:
* Before you jack up the vehicle, loosen the lug nuts on the flat tire with the lug wrench.
* Turn them counterclockwise until they're loose, but do not remove them yet.
2. **Jack up the vehicle**:
* Position the jack under the jacking point nearest to the flat tire.
* Raise the jack until the flat tire is off the ground, but not so high that the vehicle is unstable.
3. **Remove the lug nuts and flat tire**:
* Completely remove the lug nuts from the wheel hub and set them aside in a safe place.
* Carefully pull the flat tire straight off the wheel hub and set it aside.
4. **Install the spare tire**:
* Place the spare tire onto the wheel hub and hand tighten the lug nuts.
* Make sure the tire is properly seated and centered on the hub.
5. **Tighten the lug nuts**:
* Lower the vehicle to the ground using the jack.
* Use the lug wrench to tighten the lug nuts in a star pattern (tightening one lug nut a little, then moving to the next one, and so on).
* Make sure they're snug, but do not overtighten.
6. **Check the tire pressure**:
* Make sure the spare tire is inflated to the recommended pressure, which can be found on the tire's sidewall or in the owner's manual.
7. **Drive carefully**:
* Drive carefully to a tire repair shop to have the flat tire repaired or replaced.
* Do not drive at high speeds or for an extended period with the spare tire.
**Additional Tips:**
* If you're not comfortable changing the spare tire, call for roadside assistance or seek help from a professional mechanic.
* Make sure to regularly check the spare tire's air pressure and condition to ensure it's ready for use in case of an emergency.
* The spare tire is only intended for temporary use and should be replaced with a full-size tire as soon as possible.
Remember to always refer to your Honda HR-V's owner's manual for specific instructions on how to change a spare tire on your vehicle. If you have any questions or concerns, feel free to ask!
### 项目总结与展望
#### 项目评估
整体上，项目成功实现了预期的功能目标，特别是在车辆导航和功能查询方面表现出色。然而，仍存在复杂使用环境以及复杂的问题下的表现有待提升，
#### 未来方向
未来可以考虑添加语音识别技术，并探索更多的多模态交互场景。同时，可以考虑集成更多的车辆数据源，提供更全面的智能服务。
### 附件与参考资料
车辆手册1）
https://techinfo.honda.com/rjanisis/pubs/OM/AH/AT7A2121NM/enu/AT7A2121NM.PDF
车辆手册2）
https://techinfo.honda.com/rjanisis/pubs/OM/AH/AT7A2121OM/enu/AT7A2121OM.PDF