In the previous article AI Learning Record – Using and Fine-tuning ChatGLM2-6B, we separately introduced the ChatGLM2 project, which is a class of chatGPT projects. And in this article, we will introduce the LangChain-ChatChat personal knowledge management library based on ChatGLM2 (or other chat systems) on GitHub.
Its characteristic is that it can
- Switching quickly between different chat systems, such as between chatGPT and chatGLM.
- Can upload files and build an intelligent retrieval system. Such as for law or medicine.
Being able to build one’s own knowledge base is the biggest advantage, and it does seem to have that effect, but I haven’t delved into using it yet. The search is a bit too fast, unlike typical AI training, so we’ll have to wait and see how effective it is.
Required Hardware
It is recommended to have a graphics card with at least 16GB of memory, but it seems that 6GB should also be sufficient if you lower the specifications. However, I have not tried it myself. If there is no GPU, the system requires at least 32GB of RAM, but the speed will be very slow. An i7-12700K processor can output about two characters per second.
Basic environment installation
Some basic environment settings (such as Anaconda and shared scripts) have already been written in the article on “Common Operations“. Please take a look first to ensure that all commands can run correctly.
Create a Conda environment
Due to the different dependencies of each project, an environment will be set up for each case here.
|
1 2 |
conda create -n langchat python=3.9 conda activate langchat |
Source code download and installation environment
Install the following packages.
|
1 2 3 4 |
git clone https://github.com/chatchat-space/Langchain-Chatchat.git cd Langchain-Chatchat pip install -r requirements.txt echo "conda activate langchat" > env.sh |
Download model and embedding
Due to downloading large files with Git, it is necessary to support LFS. You can successfully install it by following the steps in this article: Supporting LFS with Git.
|
1 2 |
curl -s https://packagecloud.io/install/repositories/github/git-lfs/script.deb.sh | sudo bash sudo apt-get install git-lfs |
Download the model and embeddings.
|
1 2 |
git clone https://huggingface.co/THUDM/chatglm2-6b git clone https://huggingface.co/moka-ai/m3e-base |
Edit configuration file
|
1 2 |
cp configs/server_config.py.example configs/server_config.py cp configs/model_config.py.example configs/model_config.py |
First, copy the example configuration file. Then modify model_config.py and server_config.py with the following difference in their diffs.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 |
14c14,29 < "m3e-base": "m3e-base", --- > "ernie-tiny": "nghuyong/ernie-3.0-nano-zh", > "ernie-base": "nghuyong/ernie-3.0-base-zh", > "text2vec-base": "shibing624/text2vec-base-chinese", > "text2vec": "GanymedeNil/text2vec-large-chinese", > "text2vec-paraphrase": "shibing624/text2vec-base-chinese-paraphrase", > "text2vec-sentence": "shibing624/text2vec-base-chinese-sentence", > "text2vec-multilingual": "shibing624/text2vec-base-multilingual", > "text2vec-bge-large-chinese": "shibing624/text2vec-bge-large-chinese", > "m3e-small": "moka-ai/m3e-small", > "m3e-base": "moka-ai/m3e-base", > "m3e-large": "moka-ai/m3e-large", > "bge-small-zh": "BAAI/bge-small-zh", > "bge-base-zh": "BAAI/bge-base-zh", > "bge-large-zh": "BAAI/bge-large-zh", > "bge-large-zh-noinstruct": "BAAI/bge-large-zh-noinstruct", > "text-embedding-ada-002": os.environ.get("OPENAI_API_KEY") 23a39,44 > "chatglm-6b": { > "local_model_path": "THUDM/chatglm-6b", > "api_base_url": "http://localhost:8888/v1", # "name"修改為fastchat服務中的"api_base_url" > "api_key": "EMPTY" > }, > 25,28c46,86 < "local_model_path": "chatglm2-6b", < "api_base_url": "http://localhost:8888/v1", # "name"修改? FastChat 服?中的"api_base_url" < "api_key": "EMPTY" < }, --- > "local_model_path": "THUDM/chatglm2-6b", > "api_base_url": "http://localhost:8888/v1", # URL需要與運行fastchat服務端的server_config.FSCHAT_OPENAI_API一致 > "api_key": "EMPTY" > }, > > "chatglm2-6b-32k": { > "local_model_path": "THUDM/chatglm2-6b-32k", # "THUDM/chatglm2-6b-32k", > "api_base_url": "http://localhost:8888/v1", # "URL需要與運行fastchat服務端的server_config.FSCHAT_OPENAI_API一致 > "api_key": "EMPTY" > }, > > # 調用chatgpt時如果報出： urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='api.openai.com', port=443): > # Max retries exceeded with url: /v1/chat/completions > # 則需要將urllib3版本修改為1.25.11 > # 如果依然報urllib3.exceptions.MaxRetryError: HTTPSConnectionPool，則將https改為http > # 參考https://zhuanlan.zhihu.com/p/350015032 > > # 如果報出：raise NewConnectionError( > # urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x000001FE4BDB85E0>: > # Failed to establish a new connection: [WinError 10060] > # 則是因為內地和香港的IP都被OPENAI封了，需要切換為日本、新加坡等地 > > # 如果出現WARNING: Retrying langchain.chat_models.openai.acompletion_with_retry.<locals>._completion_with_retry in > # 4.0 seconds as it raised APIConnectionError: Error communicating with OpenAI. > # 需要添加代理訪問(正常開的代理軟件可能會攔截不上)需要設置配置openai_proxy 或者 使用環境遍歷OPENAI_PROXY 進行設置 > # 比如: "openai_proxy": 'http://127.0.0.1:4780' > "gpt-3.5-turbo": { > "api_base_url": "https://api.openai.com/v1", > "api_key": os.environ.get("OPENAI_API_KEY"), > "openai_proxy": os.environ.get("OPENAI_PROXY") > }, > # 線上模型。當前支持智譜AI。 > # 如果沒有設置有效的local_model_path，則認為是在?模型API。 > # 請在server_config中為每個在?API設置不同的連接埠 > # 具體註冊及api key獲取請前往 http://open.bigmodel.cn > "chatglm-api": { > "api_base_url": "http://127.0.0.1:8888/v1", > "api_key": os.environ.get("ZHIPUAI_API_KEY"), > "provider": "ChatGLMWorker", > "version": "chatglm_pro", # 可選包括 "chatglm_lite", "chatglm_std", "chatglm_pro" > }, |
|
1 2 3 4 |
16c16 < "host": "0.0.0.0", --- > "host": DEFAULT_BIND_HOST, |
The diff file looks a bit messy. In model_config.py, only m3e-base and llm_model_dict only retain chatglm2-6b in embedding_model_dict. While in server_config.py, it allows connections from external machines.
Initialization and Startup
“Initiate knowledge base initialization.”
|
1 |
python init_database.py --recreate-vs |
Press one key to start all services.
|
1 |
python startup.py -a |
Start
Seeing the above picture means that it has been successfully started, and then use the browser to connect to the web page via http://IP:8501.
Home Page
On the left, you can choose to use a conversational mode, including LLM conversation (like chatGPT), knowledge base QA, or search engine. The number of historical dialogue rounds refers to the number of previous questions and answers that it will reference. On the right is the question and answer area.
Knowledge Base Management
Entering the knowledge base management, you can perform operations such as creating and uploading files. Let’s try downloading the Copyright Act from the Ministry of Justice link. Downloading a PDF should work.
Alright, switching back to the dialogue mode and selecting Knowledge Base Q&A.
I asked “Will I be punished for distributing pirated software?” and “What is the range of possible sentences?”, and he spoke knowledgeably about it. After carefully reading one of them, I found out that it was indeed correct!
“Violation of Article 81-1 may be punished by imprisonment for a term not exceeding one year, detention, or a fine of not less than NT$250,000 and not more than NT$2,500,000, in addition to any other punishment provided by law.”
|
1 2 3 4 5 6 7 8 9 10 |
第 80-1 條 著作權人所為之權利管理電子資訊，不得移除或變更。但有下列情形之一者，不在此限： 一、因行為時之技術限制，非移除或變更著作權利管理電子資訊即不能合法利用該著作。 二、錄製或傳輸系統轉換時，其轉換技術上必要之移除或變更。 明知著作權利管理電子資訊，業經非法移除或變更者，不得散布或意圖散布而輸入或持有該著作原件或其重製物，亦不得公開播送、公開演出或公開傳輸。 第 96-1 條 有下列情形之一者，處一年以下有期徒刑、拘役，或科或併科新臺幣二萬元以上二十五萬元以下罰金： 一、違反第八十條之一規定者。 二、違反第八十條之二第二項規定者。 |
Conclusion
These two articles really amazing, it has only been a short time since the last research and there is already a powerful retriever that can be used privately! AI’s progress is really fast~
