Wendada: A Large-scale Language Model Invocation Platform

This project aims to achieve efficient content generation for specific contexts while considering the limited computing resources of individuals and small to medium-sized enterprises, as well as concerns about knowledge security and privacy. To achieve this goal, the platform integrates the following capabilities:

1. Knowledge Base: Supports integration with various sources such as [Local Offline Vector Libraries]rtst mode, [Local Search Engines]Fess Mode and online search engines.
2. Multiple Large Language Models: Currently supports offline deployment models like chatGLM-6B\chatGLM2-6B、chatRWKV、llama系列(not recommended for Chinese users)、moss(not recommended)、baichuan(requires lora for optimal results)、Aquila-7B、InternLM, and online API access to openai api and chatGLM-130b api。
3. Auto Scripts: By developing JavaScript scripts as plugins, the platform is extended to include various automation features, such as customizing dialogue flow, accessing external APIs, and switching LoRA models online.
4. Other Practical Capabilities: Dialogue history management, intranet deployment, multi-user simultaneous usage, etc.

Communication QQ Groups: LLM Usage and General Discussion Group162451840;Knowledge Base Usage Discussion Group 241773574(full, please go to QQ Channel for discussion)full, please go to QQ Channel for discussion744842245；QQ Channel

• Wendada: A Large-scale Language Model Invocation Platform
• Installation and Deployment
  • Description of Various Models
  • Lazy Package
  • Baidu Cloud
  • Quark
  • Introduction
  • Manual Installation
  • Step 1: Install Dependencies
  • Step 2: Download Models
  • Step 3: Parameter Configuration
• Auto
  • List of Auto Development Functions
  • Code Segments Related to Auto Development
  • Partial Built-in Auto Usage Instructions
• Knowledge Base
  • rtst mode
  • Using Fine-tuned Models to Improve Knowledge Base Answer Accuracy
  • Models
  • Fess Mode
  • Knowledge Base Debugging
  • Cleaning Knowledge Base Files
• Model Configuration
  • chatGLM-6B/chatGLM2-6B
  • chatRWKV
  • torch
  • cpp
  • Aquila-7B
• Secondary Development Based on this Project
• wenda-webui
• Word Document Software Integration

Installation and Deployment

Description of Various Models

Quark

Link:https://pan.quark.cn/s/c4cb08de666e Extract Code:4b4R

Introduction

Default parameters work well on devices with 6GB VRAM. The latest version of the lazy package has integrated a one-click update feature, which is recommended to be updated before use.

Usage steps (using the glm6b model as an example): 1. Download the lazy package and model. The model can be downloaded using the built-in script from HuggingFace (HF) or from a cloud drive. 2. If CUDA 11.8 is not installed, download and install it from the cloud drive. 3. Double-click on 运行GLM6B.bat (Run GLM6B.bat). 4. If you need to generate an offline knowledge base, refer to知识库。

Manual Installation

PS:Be sure to check example.config.yml,which provides more detailed explanations of each feature!

Step 1: Install Dependencies

Common dependencies:pip install -r requirements/requirements.txt Configure according to the知识库being used.

Step 2: Download Models

Download the corresponding model based on your needs.

Recommended to use the RWKV-4-Raven-7B-v11 model for chatRWKV or chatGLM-6B.

Step 3: Parameter Configuration

Renameexample.config.yml to config.yml and fill in the model download location and other necessary information according to the parameter instructions inside the file.

Auto

Auto functionality is achieved through JavaScript scripts, which are injected into the program as plugins using Greasemonkey scripts or placed directly in the autos directory. This extends Wendada with various automation features.

List of Auto Development Functions

func.push({
    name: "Name",
    question: async () => {
        let answer=await send(app.question)
        alert(answer)
    },
})

Add content in the lower tab:

app.plugins.push({ icon: 'note-edit-outline', url: "/static/wdnote/index.html" })

Search in a specific RTST knowledge base:

find_in_memory = async (s, step, memory_name) => {
   response = await fetch("/api/find_rtst_in_memory", {
      method: 'post',
      body: JSON.stringify({
         prompt: s,
         step: step,
         memory_name: memory_name
      }),
      headers: {
         'Content-Type': 'application/json'
      }
   })
   let json = await response.json()
   console.table(json)
   app.zhishiku = json
   return json
}

Upload to a specific RTST knowledge base:

upload_rtst_zhishiku = async (title, txt,memory_name) => {
   response = await fetch("/api/upload_rtst_zhishiku", {
      method: 'post',
      body: JSON.stringify({
         title: title,
         txt: txt,
         memory_name: memory_name
      }),
      headers: { 'Content-Type': 'application/json' }
   })
   alert(await response.text())
}

Save a specific RTST knowledge base:

save_rtst = async (memory_name) => {
   response = await fetch("/api/save_rtst_zhishiku", {
      method: 'post',
      body: JSON.stringify({
         memory_name: memory_name
      }),
      headers: { 'Content-Type': 'application/json' }
   })
   alert(await response.text())
}

Access SD_agent:

response = await fetch("/api/sd_agent", {
   method: 'post',
   body: JSON.stringify({
         prompt: `((masterpiece, best quality)), photorealistic,` + Q,
         steps: 20,
         // sampler_name: "DPM++ SDE Karras",
         negative_prompt: `paintings, sketches, (worst quality:2), (low quality:2), (normal quality:2), lowres, normal quality, ((monochrome)), ((grayscale)), skin spots, acnes, skin blemishes, age spot, glans`
   }),
   headers: {
         'Content-Type': 'application/json'
   }
})
try {
   let json = await response.json()
   add_conversation("AI", '![](data:image/png;base64,' + json.images[0] + ")")
} catch (error) {
   alert("Failed to connect to the SD API. Please make sure the agents library is enabled and set the SD API address to 127.0.0.1:786.")
}

Partial Built-in Auto Usage Instructions

Call the Stable Diffusion interface for drawing using the agents module (see example.config.yml <Library>)

Auto examples

Knowledge Base

The knowledge base principle is to generate some prompt information and insert it into the dialogue after searching, so that the model becomes aware of the knowledge base data. rtst modeThe knowledge base principle is to generate some prompt information and insert it into the dialogue after searching, so that the model becomes aware of the knowledge base data. Fess Mode (similar to a local search engine) and bing mode use search engines to obtain answers.

To prevent excessive GPU memory consumption and limitations on model comprehension, the inserted data cannot be too long. Therefore, there are limits on the number of characters and the number of entries. This issue can be resolved by enhancing the knowledge base with Auto.

In normal usage, enabling the knowledge bas