modify: README 补充混合部署与进程插件隔离架构(动态路由注册/容器部署/命令入口)
This commit is contained in:
@@ -16,6 +16,9 @@
|
|||||||
| **动态路由生成** | 扫描 `config/workers/*/*.toml` 自动注册 HTTP 路由 |
|
| **动态路由生成** | 扫描 `config/workers/*/*.toml` 自动注册 HTTP 路由 |
|
||||||
| **Worker 框架** | `@rpc_method` 装饰器 + `Plugin` 基类统一生命周期 + Pydantic 模型绑定 |
|
| **Worker 框架** | `@rpc_method` 装饰器 + `Plugin` 基类统一生命周期 + Pydantic 模型绑定 |
|
||||||
| **启动契约校验** | TOML 路由 ↔ `@rpc_method` ↔ 生成模型三方一致性校验,fail fast |
|
| **启动契约校验** | TOML 路由 ↔ `@rpc_method` ↔ 生成模型三方一致性校验,fail fast |
|
||||||
|
| **混合部署** | inline(网关进程内)与 external(独立进程)两种插件模式按插件粒度共存 |
|
||||||
|
| **动态路由注册** | external 插件经 Zenoh 注册路由,网关动态挂载/摘除 HTTP 端点 |
|
||||||
|
| **进程插件隔离** | 进程插件与网关零导入关系,仅经 Zenoh 通讯,可独立打包到不同容器 |
|
||||||
| **自动化模型生成** | `gen_models` 根据 TOML 配置生成请求/响应模型 |
|
| **自动化模型生成** | `gen_models` 根据 TOML 配置生成请求/响应模型 |
|
||||||
| **插件脚手架** | `gen_plugin` 一键生成风格统一的 Worker 插件骨架 |
|
| **插件脚手架** | `gen_plugin` 一键生成风格统一的 Worker 插件骨架 |
|
||||||
| **JWT 鉴权** | Rust 实现,路由级 `auth_required` 控制 |
|
| **JWT 鉴权** | Rust 实现,路由级 `auth_required` 控制 |
|
||||||
@@ -172,18 +175,27 @@ npm run dev # http://localhost:5173
|
|||||||
|
|
||||||
```mermaid
|
```mermaid
|
||||||
graph TB
|
graph TB
|
||||||
HTTP[HTTP 客户端] -->|REST API| FastAPI[FastAPI 路由<br/>TOML 自动生成]
|
HTTP[HTTP 客户端] -->|REST API| API[FastAPI 动态路由]
|
||||||
MQTT[MQTT 客户端] -->|chongming/+/+| Bridge[MQTT Bridge<br/>纯客户端模式]
|
MQTT[MQTT 客户端] -->|chongming/+/+| BRIDGE[MQTT Bridge<br/>纯客户端模式]
|
||||||
|
|
||||||
FastAPI --> WM[WorkerManager<br/>随机负载均衡 + 超时重试]
|
REG[RouteRegistry<br/>订阅 chongming/routes/**] -->|动态挂载/摘除 HTTP 路由| API
|
||||||
Bridge --> WM
|
API --> WM[WorkerManager<br/>随机负载均衡 + 超时重试]
|
||||||
|
BRIDGE --> WM
|
||||||
|
|
||||||
WM -->|Zenoh Query| W1[Worker 副本 1]
|
subgraph GW[网关进程 chongming]
|
||||||
WM -->|Zenoh Query| W2[Worker 副本 2]
|
API
|
||||||
WM -->|Zenoh Query| W3[Worker 副本 N]
|
REG
|
||||||
|
BRIDGE
|
||||||
|
WM
|
||||||
|
end
|
||||||
|
|
||||||
W1 & W2 & W3 -->|插件业务逻辑| DB[(SQLite)]
|
WM -->|Zenoh Query| IW[inline Worker 副本<br/>网关进程内加载]
|
||||||
W1 & W2 & W3 -->|AI 调用| AI[豆包 API]
|
WM -->|Zenoh Query| EW[external Worker 进程<br/>独立容器,零网关依赖]
|
||||||
|
EW -.->|发布路由描述符<br/>chongming/routes/**| REG
|
||||||
|
|
||||||
|
IW --> DB[(SQLite)]
|
||||||
|
EW --> DB
|
||||||
|
EW -->|AI 调用| AI[豆包 API]
|
||||||
```
|
```
|
||||||
|
|
||||||
**启动流程**:
|
**启动流程**:
|
||||||
@@ -192,10 +204,14 @@ graph TB
|
|||||||
uv run serve → FastAPI lifespan:
|
uv run serve → FastAPI lifespan:
|
||||||
1. setup_storage() — 本地文件系统存储
|
1. setup_storage() — 本地文件系统存储
|
||||||
2. setup_zenoh() — Zenoh peer 会话
|
2. setup_zenoh() — Zenoh peer 会话
|
||||||
3. setup_worker_manager() — WorkerManager + 定期 liveliness 刷新
|
3. setup_mqtt_broker() — 内嵌 Broker(子进程 + 文件锁单例)
|
||||||
4. load_workers() — 动态 import 插件 → 启动 Worker 协程
|
4. setup_worker_manager() — WorkerManager + 定期 liveliness 刷新
|
||||||
5. register_routes() — 扫描 config/workers/ 生成 HTTP 路由
|
5. load_workers() — 按插件 [workers].load_mode 分派:
|
||||||
6. setup_mqtt() — 内嵌 Broker(子进程)+ MQTT Bridge
|
inline → 进程内 import 插件 → 启动 Worker 协程
|
||||||
|
external → 跳过(独立进程运行),启动 RouteRegistry 订阅 chongming/routes/**
|
||||||
|
6. validate_contracts() — inline 插件三方契约校验,失败终止启动
|
||||||
|
7. register_routes() — inline 插件盘扫生成 HTTP 路由
|
||||||
|
8. setup_mqtt() — MQTT Bridge(inline 主题静态收集 + external 主题动态推送)
|
||||||
```
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
@@ -247,6 +263,41 @@ async def upload(file: UploadFile = File(...)):
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
## 进程插件与混合部署
|
||||||
|
|
||||||
|
两种插件运行模式**按插件粒度共存**,在 `config/workers/<name>/config.toml` 的 `[workers]` 节声明:
|
||||||
|
|
||||||
|
```toml
|
||||||
|
load_mode = "inline" # 默认:插件在网关进程内加载
|
||||||
|
load_mode = "external" # 独立进程运行,经 Zenoh 注册路由
|
||||||
|
```
|
||||||
|
|
||||||
|
**inline 插件**(`src/plugins/`,chongming-plugins 仓库):
|
||||||
|
|
||||||
|
- 网关进程内 import;路由由网关盘扫 TOML 生成;契约校验在网关启动时执行
|
||||||
|
|
||||||
|
**external 进程插件**(本地开发目录 `workers/`,不入库):
|
||||||
|
|
||||||
|
- 与网关**零导入关系**:只依赖 `chongming-worker` SDK + zenoh,仅经 Zenoh 通讯
|
||||||
|
- 路由注册:worker 启动时 put 描述符到 `chongming/routes/<type>/...`,退出时 delete;每 10s 周期重发布自愈
|
||||||
|
- 服务发现:liveliness `worker/<type>/<id>`(与 inline 副本同一机制,网关无需改动)
|
||||||
|
- worker 进程自读 TOML 并在**自己进程内**做契约校验(`chongming_worker.contracts`);
|
||||||
|
网关 RouteRegistry 订阅描述符后动态挂载/摘除 HTTP 路由(type 级 2s 防抖)
|
||||||
|
- 代码结构与 inline 插件完全一致(Plugin 基类 + router/ + models/),示例见本地 `workers/demo/`
|
||||||
|
- 运行:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
uv run worker_serve --type demo # 本地调试
|
||||||
|
chongming-worker --type demo # 容器入口(chongming-worker SDK console script)
|
||||||
|
```
|
||||||
|
|
||||||
|
- 生产部署:进程插件与网关**分别打包到不同容器**
|
||||||
|
- 网关容器:`chongming` + 依赖
|
||||||
|
- 进程插件容器:插件代码 + `chongming-worker` + 依赖
|
||||||
|
- 隔离性由 `test_worker_isolation.py` 保证(阻断 chongming 导入 + AST 检查 import 语句)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## MQTT 协议
|
## MQTT 协议
|
||||||
|
|
||||||
### Topic 约定
|
### Topic 约定
|
||||||
@@ -342,7 +393,8 @@ uv run serve # 启动开发服务
|
|||||||
uv run gen_plugin <name> # 生成统一风格插件骨架
|
uv run gen_plugin <name> # 生成统一风格插件骨架
|
||||||
uv run python -m chongming.scripts.gen_models # 生成 Pydantic 模型
|
uv run python -m chongming.scripts.gen_models # 生成 Pydantic 模型
|
||||||
uv run python -m chongming.scripts.build # 构建打包(PyInstaller/Nuitka)
|
uv run python -m chongming.scripts.build # 构建打包(PyInstaller/Nuitka)
|
||||||
uv run worker_serve --type <name> # 以独立进程运行 Worker(load_mode=external)
|
uv run worker_serve --type <name> # 独立 Worker 进程(本地调试,load_mode=external)
|
||||||
|
chongming-worker --type <name> # 独立 Worker 进程(容器入口)
|
||||||
uv pip install -e utils/<name> # 安装/更新工具包
|
uv pip install -e utils/<name> # 安装/更新工具包
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -369,7 +421,7 @@ uv pip install -e utils/<name> # 安装/更新工具包
|
|||||||
| **chongming-jwt** | `utils/jwt/` | Rust + PyO3 | JWT 编解码,支持 HS256/384/512 + RS256/384/512 |
|
| **chongming-jwt** | `utils/jwt/` | Rust + PyO3 | JWT 编解码,支持 HS256/384/512 + RS256/384/512 |
|
||||||
| **chongming-password** | `utils/password/` | Rust + PyO3 | Argon2id 密码哈希与验证 |
|
| **chongming-password** | `utils/password/` | Rust + PyO3 | Argon2id 密码哈希与验证 |
|
||||||
| **chongming-request** | `utils/request/` | Python | Zenoh RPC 直连客户端 |
|
| **chongming-request** | `utils/request/` | Python | Zenoh RPC 直连客户端 |
|
||||||
| **chongming-worker** | `utils/worker/` | Python | Worker 框架核心:装饰器、运行时、配置加载 |
|
| **chongming-worker** | `utils/worker/` | Python | Worker 框架核心:装饰器、运行时、契约校验、独立进程入口(`chongming-worker`) |
|
||||||
| **chongming-stream-manager** | `utils/stream_manager/` | Python | ffmpeg + MediaMTX RTSP 流代理管理 |
|
| **chongming-stream-manager** | `utils/stream_manager/` | Python | ffmpeg + MediaMTX RTSP 流代理管理 |
|
||||||
| **bytedance** | `utils/bytedance/` | Python | 字节跳动 AICC 安全通信 SDK(Jeddak 等) |
|
| **bytedance** | `utils/bytedance/` | Python | 字节跳动 AICC 安全通信 SDK(Jeddak 等) |
|
||||||
|
|
||||||
@@ -421,6 +473,7 @@ uv run gen_plugin <name> # 生成统一风格插件骨架
|
|||||||
uv run python -m chongming.scripts.gen_models # 生成 Pydantic 模型
|
uv run python -m chongming.scripts.gen_models # 生成 Pydantic 模型
|
||||||
uv run build pyinstaller # PyInstaller 打包
|
uv run build pyinstaller # PyInstaller 打包
|
||||||
uv run build nuitka # Nuitka 打包
|
uv run build nuitka # Nuitka 打包
|
||||||
|
chongming-worker --type <name> # 独立 Worker 进程(进程插件容器入口)
|
||||||
uv pip install -e utils/<name> # 安装/更新工具包子包
|
uv pip install -e utils/<name> # 安装/更新工具包子包
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -451,19 +504,6 @@ uv pip install -e utils/<name> # 安装/更新工具包子包
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 工具包
|
|
||||||
|
|
||||||
| 包名 | 说明 | 技术栈 |
|
|
||||||
| -------------------- | --------------------------------------------- | ----------- |
|
|
||||||
| `chongming-config` | TOML 配置加载、TypedDict 类型定义 | Python |
|
|
||||||
| `chongming-database` | 异步 SQLite(aiosqlite + SQLModel) | Python |
|
|
||||||
| `chongming-jwt` | JWT 编码/验证(HS256/384/512, RS256/384/512) | Rust + PyO3 |
|
|
||||||
| `chongming-password` | Argon2id 密码哈希与验证 | Rust + PyO3 |
|
|
||||||
| `chongming-request` | Zenoh Worker 客户端(负载均衡、超时重试) | Python |
|
|
||||||
| `chongming-worker` | Worker 框架核心(`@rpc_method`、运行时) | Python |
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 更多文档
|
## 更多文档
|
||||||
|
|
||||||
- [配置说明](config/README.md)
|
- [配置说明](config/README.md)
|
||||||
|
|||||||
Reference in New Issue
Block a user