465 lines
14 KiB
Markdown
465 lines
14 KiB
Markdown
# Sophon Chakcy 插件管理系统使用文档
|
||
|
||
## 系统概述
|
||
|
||
Sophon Chakcy 是一个基于 Sophon SAIL 的插件化目标检测框架，核心是动态插件管理系统，而非具体的检测算法。YOLOv8 仅作为预置的插件示例，系统支持任意检测算法的插件化扩展。
|
||
|
||
### 核心特性
|
||
|
||
- **动态插件加载**：运行时根据名称动态加载对应检测器
|
||
- **统一配置管理**：集中管理所有插件配置
|
||
- **资源虚拟化**：支持 SQLite VFS 打包模型和配置
|
||
- **模块化设计**：插件间互相独立，易于扩展和维护
|
||
|
||
## 核心架构
|
||
|
||
```text
|
||
sophon_chakcy/
|
||
├── get_detector.py # 核心插件管理器
|
||
├── __init__.py # 工具函数
|
||
└── build.py # 插件打包工具
|
||
```
|
||
|
||
## 关键组件解析
|
||
|
||
### 1. 插件管理器 (get_detector.py)
|
||
|
||
```python
|
||
def get_detector(name: str, is_vfs: bool = False):
|
||
"""核心插件加载函数
|
||
|
||
Args:
|
||
name: 插件名称（在config.json中定义）
|
||
is_vfs: 是否从VFS加载资源
|
||
|
||
Returns:
|
||
tuple: (检测器类, 配置字典)
|
||
"""
|
||
```
|
||
|
||
**工作原理：**
|
||
|
||
1. 根据插件名称查找 `config.json` 中的配置
|
||
2. 动态导入对应的 Python 模块
|
||
3. 加载插件配置文件
|
||
4. 返回检测器类和配置对象
|
||
|
||
### 2. 配置文件结构
|
||
|
||
```json
|
||
{
|
||
"model_map": {
|
||
"yolov8": {
|
||
"name": "yolov8",
|
||
"module": "YOLOv8.yolov8_detector", # 模块导入路径
|
||
"config": "configs/yolov8_config.json" # 插件配置文件
|
||
}
|
||
}
|
||
}
|
||
```
|
||
|
||
## 插件接口开发指南
|
||
|
||
### 插件接口规范
|
||
|
||
每个插件必须提供以下接口：
|
||
|
||
#### 1. 配置类（必需）
|
||
|
||
```python
|
||
from dataclasses import dataclass
|
||
from typing import Dict, Any
|
||
|
||
@dataclass
|
||
class PluginConfig:
|
||
"""插件配置基类"""
|
||
bmodel_path: str
|
||
classes: list
|
||
|
||
@classmethod
|
||
def from_dict(cls, data: Dict[str, Any]) -> "PluginConfig":
|
||
"""从字典创建配置对象"""
|
||
return cls(**data)
|
||
```
|
||
|
||
#### 2. 检测器类（必需）
|
||
|
||
```python
|
||
class Detector:
|
||
"""插件检测器基类"""
|
||
|
||
def __init__(self, config_dict: Dict[str, Any], is_vfs: bool = False,
|
||
vfs_path: str = "./application.svfs", dev_id: int = 0,
|
||
conf_thresh: float = 0.25, nms_thresh: float = 0.7):
|
||
"""
|
||
初始化检测器
|
||
|
||
Args:
|
||
config_dict: 配置字典
|
||
is_vfs: 是否使用VFS
|
||
vfs_path: VFS文件路径
|
||
dev_id: 设备ID
|
||
conf_thresh: 置信度阈值
|
||
nms_thresh: NMS阈值
|
||
"""
|
||
self.config = PluginConfig.from_dict(config_dict)
|
||
# 其他初始化逻辑...
|
||
|
||
def detect_single_image(self, image_path: str, output_dir: str = "./results") -> list:
|
||
"""单张图像检测接口"""
|
||
pass
|
||
|
||
def detect_images_in_directory(self, input_dir: str, output_dir: str = "./results"):
|
||
"""批量图像检测接口"""
|
||
pass
|
||
```
|
||
|
||
### 创建新插件的完整流程
|
||
|
||
#### 步骤1：创建插件目录结构
|
||
|
||
```text
|
||
my_detector/
|
||
├── __init__.py
|
||
├── my_detector.py # 主检测器文件
|
||
├── preprocess.py # 预处理模块（可选）
|
||
├── postprocess.py # 后处理模块（可选）
|
||
└── utils.py # 工具函数（可选）
|
||
```
|
||
|
||
#### 步骤2：实现检测器类
|
||
|
||
```python
|
||
# my_detector/my_detector.py
|
||
import os
|
||
import numpy as np
|
||
from dataclasses import dataclass
|
||
from typing import List, Dict, Any
|
||
|
||
import sophon_chakcy.sail as sail
|
||
from .preprocess import Preprocessor
|
||
from .postprocess import Postprocessor
|
||
|
||
@dataclass
|
||
class MyDetectorConfig:
|
||
"""自定义检测器配置"""
|
||
bmodel_path: str
|
||
classes: List[str]
|
||
input_size: tuple = (640, 640)
|
||
mean: List[float] = None
|
||
std: List[float] = None
|
||
|
||
@classmethod
|
||
def from_dict(cls, data: Dict[str, Any]) -> "MyDetectorConfig":
|
||
return cls(**data)
|
||
|
||
class Detector:
|
||
"""自定义检测器实现"""
|
||
|
||
def __init__(self, config_dict: Dict[str, Any], is_vfs: bool = False,
|
||
vfs_path: str = "./application.svfs", dev_id: int = 0,
|
||
conf_thresh: float = 0.25, nms_thresh: float = 0.7):
|
||
|
||
# 解析配置
|
||
self.config = MyDetectorConfig.from_dict(config_dict)
|
||
|
||
# 加载模型
|
||
if is_vfs:
|
||
from sqlite_vfs.core import SQLiteVFS
|
||
vfs = SQLiteVFS(vfs_path)
|
||
bmodel_bytes = vfs.read_file(self.config.bmodel_path)
|
||
bmodel_size = vfs.get_file_info(self.config.bmodel_path)["file_size"]
|
||
self.net = sail.Engine(bmodel_bytes, bmodel_size, dev_id, sail.IOMode.SYSO)
|
||
else:
|
||
self.net = sail.Engine(self.config.bmodel_path, dev_id, sail.IOMode.SYSO)
|
||
|
||
# 初始化组件
|
||
self.preprocessor = Preprocessor(self.config)
|
||
self.postprocessor = Postprocessor(
|
||
conf_thresh=conf_thresh,
|
||
nms_thresh=nms_thresh,
|
||
classes=self.config.classes
|
||
)
|
||
|
||
# 其他初始化...
|
||
print(f"插件 '{self.__class__.__name__}' 初始化完成")
|
||
|
||
def detect_single_image(self, image_path: str, output_dir: str = "./results"):
|
||
"""实现单图检测逻辑"""
|
||
# 1. 读取图像
|
||
# 2. 预处理
|
||
# 3. 推理
|
||
# 4. 后处理
|
||
# 5. 返回结果
|
||
pass
|
||
|
||
def detect_images_in_directory(self, input_dir: str, output_dir: str = "./results"):
|
||
"""实现批量检测逻辑"""
|
||
for img_file in os.listdir(input_dir):
|
||
img_path = os.path.join(input_dir, img_file)
|
||
self.detect_single_image(img_path, output_dir)
|
||
```
|
||
|
||
#### 步骤3：创建插件配置文件
|
||
```json
|
||
// configs/my_detector_config.json
|
||
{
|
||
"bmodel_path": "models/my_model.bmodel",
|
||
"classes": ["cat", "dog", "person"],
|
||
"input_size": [640, 640],
|
||
"mean": [0.485, 0.456, 0.406],
|
||
"std": [0.229, 0.224, 0.225]
|
||
}
|
||
```
|
||
|
||
#### 步骤4：注册插件
|
||
|
||
修改 `config.json` 添加新插件：
|
||
|
||
```json
|
||
{
|
||
"model_map": {
|
||
"yolov8": {
|
||
"module": "YOLOv8.yolov8_detector",
|
||
"config": "configs/yolov8_config.json"
|
||
},
|
||
"my_detector": {
|
||
"module": "my_detector.my_detector",
|
||
"config": "configs/my_detector_config.json"
|
||
},
|
||
"faster_rcnn": {
|
||
"module": "faster_rcnn.detector",
|
||
"config": "configs/faster_rcnn_config.json"
|
||
}
|
||
}
|
||
}
|
||
```
|
||
|
||
#### 步骤5：测试插件
|
||
|
||
```bash
|
||
# 使用新插件
|
||
python main.py --name my_detector --input ./test.jpg
|
||
```
|
||
|
||
### 插件打包与部署
|
||
|
||
#### 1. 打包工具详解 (build.py)
|
||
|
||
构建工具执行以下任务：
|
||
|
||
1. **读取配置**：解析 `config.json` 中的所有插件配置
|
||
2. **打包模块**：将插件模块打包到 SQLite 模块库
|
||
3. **打包资源**：将模型和配置文件打包到 VFS
|
||
4. **生成部署包**：创建完整的可部署包
|
||
|
||
#### 2. 打包命令
|
||
|
||
```bash
|
||
# 基本打包
|
||
python -m sophon_chakcy.build
|
||
|
||
# 自定义输出路径
|
||
python -m sophon_chakcy.build --vfs_path my_app.svfs --mbank_path my_plugins.mbank
|
||
|
||
# 详细日志
|
||
python -m sophon_chakcy.build --verbose
|
||
```
|
||
|
||
#### 3. 部署结构
|
||
|
||
```text
|
||
部署目录/
|
||
├── main.py # 主程序（使用VFS和模块库）
|
||
├── application.svfs # 虚拟文件系统
|
||
│ ├── config.json
|
||
│ ├── configs/yolov8_config.json
|
||
│ ├── configs/my_detector_config.json
|
||
│ ├── models/yolov8n.bmodel
|
||
│ └── models/my_model.bmodel
|
||
└── plugins.mbank # Python模块库
|
||
├── YOLOv8
|
||
├── my_detector
|
||
└── faster_rcnn
|
||
```
|
||
|
||
## YOLOv8 示例
|
||
|
||
### 配置文件
|
||
|
||
在 `./config.json` 文件中添加对应的配置
|
||
|
||
```json
|
||
{
|
||
"file_system": {
|
||
"type": "vfs"
|
||
},
|
||
"vfs": {
|
||
"file_path": "./application.svfs"
|
||
},
|
||
"model_map": {
|
||
"yolo_example": {
|
||
"name": "yolo_example", # 插件名称
|
||
"module": "YOLOv8.yolov8_detector", # Detector 类模块位置
|
||
"config": "resources/yolov8_config.json" # 插件配置路径
|
||
}
|
||
}
|
||
}
|
||
```
|
||
|
||
编写插件配置 `resources/yolov8_config.json`
|
||
```json
|
||
{
|
||
"bmodel_path": "resources/yolov8s_int8_1b.bmodel", # 模型路径
|
||
"classes": [ # 检测类型
|
||
...
|
||
],
|
||
"colors": [ # 边框颜色
|
||
[56, 0, 255],
|
||
...
|
||
]
|
||
}
|
||
```
|
||
|
||
将模型放置在配置对应的路径下，YOLOv8 插件代码可参考 `./YOLOv8` （注意需实现配置类和 Detector 类，且 Detector 所在路径需与 config.json 中的配置保持一致）
|
||
|
||
### 测试
|
||
|
||
执行 download.sh 脚本下载测试图片
|
||
|
||
```bash
|
||
./scripts/download.sh
|
||
```
|
||
|
||
执行测试
|
||
|
||
```bash
|
||
python main.py --name yolo_example --input ./resources/datasets/test/3.jpg
|
||
```
|
||
|
||
结果
|
||
|
||
```text
|
||
[BMRT][bmcpu_setup:406] INFO:cpu_lib 'libcpuop.so' is loaded.
|
||
bmcpu init: skip cpu_user_defined
|
||
open usercpu.so, init user_cpu_init
|
||
[BMRT][load_bmodel:1084] INFO:Loading bmodel from [resources/yolov8s_int8_1b.bmodel]. Thanks for your patience...
|
||
[BMRT][load_bmodel:1023] INFO:pre net num: 0, load net num: 1
|
||
加载模型: resources/yolov8s_int8_1b.bmodel
|
||
模型信息:
|
||
输入形状: [1, 3, 640, 640]
|
||
输入类型: Dtype.BM_FLOAT32
|
||
colors数量: 81
|
||
classes类别数: 81
|
||
|
||
处理图像: ./resources/datasets/test/3.jpg
|
||
Open /dev/jpu successfully, device index = 0, jpu fd = 33, vpp fd = 34
|
||
图像大小: 854x480
|
||
原始检测数量: 11
|
||
过滤后数量 (置信度 > 0.25): 11
|
||
|
||
绘制 11 个检测结果:
|
||
--------------------------------------------------------------------------------
|
||
检测 1: person | 置信度: 0.9176 | 位置: [ 74, 163, 209, 422] | 大小: 135x 259 | 颜色: [226, 255, 0]
|
||
检测 2: person | 置信度: 0.9176 | 位置: [ 728, 142, 849, 472] | 大小: 121x 330 | 颜色: [226, 255, 0]
|
||
检测 3: person | 置信度: 0.8510 | 位置: [ 342, 181, 404, 314] | 大小: 62x 133 | 颜色: [226, 255, 0]
|
||
检测 4: person | 置信度: 0.8510 | 位置: [ 539, 152, 640, 428] | 大小: 101x 276 | 颜色: [226, 255, 0]
|
||
检测 5: skis | 置信度: 0.8196 | 位置: [ 99, 384, 221, 464] | 大小: 122x 80 | 颜色: [132, 0, 255]
|
||
检测 6: person | 置信度: 0.8196 | 位置: [ 494, 172, 571, 379] | 大小: 77x 207 | 颜色: [226, 255, 0]
|
||
检测 7: skis | 置信度: 0.6078 | 位置: [ 450, 403, 728, 447] | 大小: 278x 44 | 颜色: [132, 0, 255]
|
||
检测 8: skis | 置信度: 0.4471 | 位置: [ 439, 370, 588, 403] | 大小: 149x 33 | 颜色: [132, 0, 255]
|
||
检测 9: skis | 置信度: 0.3922 | 位置: [ 365, 311, 404, 322] | 大小: 39x 11 | 颜色: [132, 0, 255]
|
||
检测 10: car | 置信度: 0.2941 | 位置: [ 1, 215, 33, 248] | 大小: 32x 33 | 颜色: [0, 37, 255]
|
||
检测 11: skis | 置信度: 0.2510 | 位置: [ 364, 307, 404, 322] | 大小: 40x 15 | 颜色: [132, 0, 255]
|
||
--------------------------------------------------------------------------------
|
||
|
||
检测结果已保存到: ./detection_results/detected_3.jpg
|
||
```
|
||
|
||
|
||
|
||
### 打包
|
||
|
||
```bash
|
||
python -m sophon_chakcy.build
|
||
```
|
||
```
|
||
清理旧的构建目录: build
|
||
开始构建...
|
||
配置文件加载成功
|
||
VFS初始化成功: application.svfs
|
||
模块打包器初始化成功: plugins.mbank
|
||
添加文件失败 config.json: UNIQUE constraint failed: files.path
|
||
添加文件: config.json
|
||
文件大小: 0.00 MB
|
||
数据库文件: application.svfs
|
||
导出文件失败 config.json: [Errno 2] No such file or directory: ''
|
||
已保存到: config.json
|
||
已保存到: application.svfs
|
||
配置文件已添加到VFS
|
||
打包模块: YOLOv8.yolov8_detector
|
||
打包包: YOLOv8 (来自: YOLOv8/__init__.py)
|
||
打包模块: YOLOv8.yolov8_detector (来自: YOLOv8/yolov8_detector.py)
|
||
打包模块: YOLOv8.postprocess_numpy (来自: YOLOv8/postprocess_numpy.py)
|
||
添加文件失败 resources/yolov8_config.json: UNIQUE constraint failed: files.path
|
||
添加文件: resources/yolov8_config.json
|
||
文件大小: 0.01 MB
|
||
数据库文件: application.svfs
|
||
导出成功: resources/yolov8_config.json -> resources/yolov8_config.json
|
||
已保存到: resources/yolov8_config.json
|
||
已保存到: application.svfs
|
||
添加文件失败 resources/yolov8s_int8_1b.bmodel: UNIQUE constraint failed: files.path
|
||
添加文件: resources/yolov8s_int8_1b.bmodel
|
||
文件大小: 11.44 MB
|
||
数据库文件: application.svfs
|
||
导出成功: resources/yolov8s_int8_1b.bmodel -> resources/yolov8s_int8_1b.bmodel
|
||
已保存到: resources/yolov8s_int8_1b.bmodel
|
||
已保存到: application.svfs
|
||
添加模型文件: resources/yolov8s_int8_1b.bmodel
|
||
|
||
VFS内容:
|
||
FILE: config.json (313 bytes)
|
||
FILE: resources/yolov8_config.json (7178 bytes)
|
||
FILE: resources/yolov8s_int8_1b.bmodel (11993088 bytes)
|
||
|
||
复制文件到构建目录...
|
||
复制: public/main.py
|
||
复制: application.svfs
|
||
复制: plugins.mbank
|
||
|
||
构建成功完成! 输出目录: build
|
||
主程序: build/main.py
|
||
VFS文件: build/application.svfs
|
||
模块库: build/plugins.mbank
|
||
```
|
||
|
||
./build
|
||
```
|
||
部署目录/
|
||
├── main.py # 主程序（使用VFS和模块库）
|
||
├── application.svfs # 虚拟文件系统（存储配置文件和模型）
|
||
└── plugins.mbank # Python模块库
|
||
```
|
||
|
||
## 部署
|
||
|
||
方法一：构建 whl 并在生成环境中安装
|
||
|
||
```bash
|
||
uv build
|
||
ls dist
|
||
```
|
||
|
||
```text
|
||
(sophon-chakcy) linaro@bm1684:/data/sophon_chakcy$ uv build
|
||
Building source distribution...
|
||
Building wheel from source distribution...
|
||
Successfully built dist/sophon_chakcy-0.1.0.tar.gz
|
||
Successfully built dist/sophon_chakcy-0.1.0-py3-none-any.whl
|
||
(sophon-chakcy) linaro@bm1684:/data/sophon_chakcy$ ls dist
|
||
sophon_chakcy-0.1.0-py3-none-any.whl sophon_chakcy-0.1.0.tar.gz
|
||
(sophon-chakcy) linaro@bm1684:/data/sophon_chakcy$
|
||
```
|
||
|
||
方法二：复制 src/sophon_chakcy 到项目根路径
|