chakcy/sophon_ai_project
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

modify: sophon-sail 创建管理系统

Browse Source
This commit is contained in:
chakcy
2026-02-03 21:32:23 +08:00
parent b18bea8c5b
commit 0b32a1f70b
27 changed files with 4743 additions and 1132 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.gitignore
+5
View File
@@ -14,3 +14,8 @@ results
# resources # resources
resources resources
datasets
detection_results
*.svfs
*.mbank
output.md
README.md
+266 -43
View File
@@ -1,61 +1,284 @@
# 算能盒子开发 # Sophon Chakcy 插件管理系统使用文档
[TOC] ## 系统概述
## 盒子环境搭建 Sophon Chakcy 是一个基于 Sophon SAIL 的插件化目标检测框架，核心是动态插件管理系统，而非具体的检测算法。YOLOv8 仅作为预置的插件示例，系统支持任意检测算法的插件化扩展。
### 资源包 ### 核心特性
```
. - **动态插件加载**：运行时根据名称动态加载对应检测器
│ sdcard.tar # 刷机包 - **统一配置管理**：集中管理所有插件配置
│ uv-aarch64-unknown-linux-gnu.tar.gz # uv python 项目管理工具 - **资源虚拟化**：支持 SQLite VFS 打包模型和配置
- **模块化设计**：插件间互相独立，易于扩展和维护
## 核心架构
``text
sophon_chakcy/
├── get_detector.py # 核心插件管理器
├── __init__.py # 工具函数
└── build.py # 插件打包工具
``` ```
### 网络 ## 关键组件解析
1. 将盒子LAN口与计算机连接，WAN口与交换机/路由器连接 ### 1. 插件管理器 (get_detector.py)
2. 通过ssh协议与连接连接 192.168.150.1 地址操作盒子 ```python
def get_detector(name: str, is_vfs: bool = False):
"""核心插件加载函数
3. 若无交换机/路由器条件可使用电脑网络共享 Args:
name: 插件名称（在config.json中定义）
is_vfs: 是否从VFS加载资源
### python环境配置 Returns:
tuple: (检测器类, 配置字典)
在计算机上输入以下命令将 uv-aarch64-unknown-linux-gnu.tar.gz 上传到盒子的 /data 路径（linaro用户的密码为linaro） """
```sh
scp uv-aarch64-unknown-linux-gnu.tar.gz linaro@192.168.150.1:/data
``` ```
解压 uv-aarch64-unkown-linux-gnu.tar.gz 并配置环境 **工作原理：**
```sh 1. 根据插件名称查找 `config.json` 中的配置
tar -xzvf uv-aarch64-unknown-linux-gnu.tar.gz 2. 动态导入对应的 Python 模块
3. 加载插件配置文件
4. 返回检测器类和配置对象
# 将 uv 和 uvx 移动到 /usr/local/bin/（需要 sudo 权限） ### 2. 配置文件结构
sudo mv uv-aarch64-unknown-linux-gnu/uv /usr/local/bin/
sudo mv uv-aarch64-unknown-linux-gnu/uvx /usr/local/bin/
# 确保文件有可执行权限 ```json
sudo chmod +x /usr/local/bin/uv {
sudo chmod +x /usr/local/bin/uvx "model_map": {
"yolov8": {
# 验证是否能够执行uv命令 "name": "yolov8",
uv --version # uv 0.9.3 "module": "YOLOv8.yolov8_detector", # 模块导入路径
"config": "configs/yolov8_config.json" # 插件配置文件
}
}
}
``` ```
### 运行示例 ## 插件接口开发指南
```sh ### 插件接口规范
# 克隆示例项目
git clone http://124.71.68.6:3000/chakcy/sophon_ai_project.git 每个插件必须提供以下接口：
# 进入项目
cd sophon_ai_project #### 1. 配置类（必需）
# 初始化依赖
uv sync ```python
# 配置 sophon sdk 环境 from dataclasses import dataclass
soucre .venv/bin/activate from typing import Dict, Any
python -m dfss --install sail
# 运行代码 @dataclass
python3 main.py --input resources/datasets/test --bmodel resources/models/BM1688/yolov8s_int8_1b.bmodel --dev_id 0 --conf_thresh 0.25 --nms_thresh 0.7 class PluginConfig:
"""插件配置基类"""
bmodel_path: str
classes: list
@classmethod
def from_dict(cls, data: Dict[str, Any]) -> "PluginConfig":
"""从字典创建配置对象"""
return cls(**data)
```
#### 2. 检测器类（必需）
```python
class Detector:
"""插件检测器基类"""
def __init__(self, config_dict: Dict[str, Any], is_vfs: bool = False,
vfs_path: str = "./application.svfs", dev_id: int = 0,
conf_thresh: float = 0.25, nms_thresh: float = 0.7):
"""
初始化检测器
Args:
config_dict: 配置字典
is_vfs: 是否使用VFS
vfs_path: VFS文件路径
dev_id: 设备ID
conf_thresh: 置信度阈值
nms_thresh: NMS阈值
"""
self.config = PluginConfig.from_dict(config_dict)
# 其他初始化逻辑...
def detect_single_image(self, image_path: str, output_dir: str = "./results") -> list:
"""单张图像检测接口"""
pass
def detect_images_in_directory(self, input_dir: str, output_dir: str = "./results"):
"""批量图像检测接口"""
pass
```
### 创建新插件的完整流程
#### 步骤1：创建插件目录结构
```text
my_detector/
├── __init__.py
├── my_detector.py # 主检测器文件
├── preprocess.py # 预处理模块（可选）
├── postprocess.py # 后处理模块（可选）
└── utils.py # 工具函数（可选）
```
#### 步骤2：实现检测器类
```python
# my_detector/my_detector.py
import os
import numpy as np
from dataclasses import dataclass
from typing import List, Dict, Any
import sophon_chakcy.sail as sail
from .preprocess import Preprocessor
from .postprocess import Postprocessor
@dataclass
class MyDetectorConfig:
"""自定义检测器配置"""
bmodel_path: str
classes: List[str]
input_size: tuple = (640, 640)
mean: List[float] = None
std: List[float] = None
@classmethod
def from_dict(cls, data: Dict[str, Any]) -> "MyDetectorConfig":
return cls(**data)
class Detector:
"""自定义检测器实现"""
def __init__(self, config_dict: Dict[str, Any], is_vfs: bool = False,
vfs_path: str = "./application.svfs", dev_id: int = 0,
conf_thresh: float = 0.25, nms_thresh: float = 0.7):
# 解析配置
self.config = MyDetectorConfig.from_dict(config_dict)
# 加载模型
if is_vfs:
from sqlite_vfs.core import SQLiteVFS
vfs = SQLiteVFS(vfs_path)
bmodel_bytes = vfs.read_file(self.config.bmodel_path)
bmodel_size = vfs.get_file_info(self.config.bmodel_path)["file_size"]
self.net = sail.Engine(bmodel_bytes, bmodel_size, dev_id, sail.IOMode.SYSO)
else:
self.net = sail.Engine(self.config.bmodel_path, dev_id, sail.IOMode.SYSO)
# 初始化组件
self.preprocessor = Preprocessor(self.config)
self.postprocessor = Postprocessor(
conf_thresh=conf_thresh,
nms_thresh=nms_thresh,
classes=self.config.classes
)
# 其他初始化...
print(f"插件 '{self.__class__.__name__}' 初始化完成")
def detect_single_image(self, image_path: str, output_dir: str = "./results"):
"""实现单图检测逻辑"""
# 1. 读取图像
# 2. 预处理
# 3. 推理
# 4. 后处理
# 5. 返回结果
pass
def detect_images_in_directory(self, input_dir: str, output_dir: str = "./results"):
"""实现批量检测逻辑"""
for img_file in os.listdir(input_dir):
img_path = os.path.join(input_dir, img_file)
self.detect_single_image(img_path, output_dir)
```
#### 步骤3：创建插件配置文件
```json
// configs/my_detector_config.json
{
"bmodel_path": "models/my_model.bmodel",
"classes": ["cat", "dog", "person"],
"input_size": [640, 640],
"mean": [0.485, 0.456, 0.406],
"std": [0.229, 0.224, 0.225]
}
```
#### 步骤4：注册插件
修改 `config.json` 添加新插件：
```json
{
"model_map": {
"yolov8": {
"module": "YOLOv8.yolov8_detector",
"config": "configs/yolov8_config.json"
},
"my_detector": {
"module": "my_detector.my_detector",
"config": "configs/my_detector_config.json"
},
"faster_rcnn": {
"module": "faster_rcnn.detector",
"config": "configs/faster_rcnn_config.json"
}
}
}
```
#### 步骤5：测试插件
```bash
# 使用新插件
python main.py --name my_detector --input ./test.jpg
```
### 插件打包与部署
#### 1. 打包工具详解 (build.py)
构建工具执行以下任务：
1. **读取配置**：解析 `config.json` 中的所有插件配置
2. **打包模块**：将插件模块打包到 SQLite 模块库
3. **打包资源**：将模型和配置文件打包到 VFS
4. **生成部署包**：创建完整的可部署包
#### 2. 打包命令
```bash
# 基本打包
python -m sophon_chakcy.build
# 自定义输出路径
python -m sophon_chakcy.build --vfs_path my_app.svfs --mbank_path my_plugins.mbank
# 详细日志
python -m sophon_chakcy.build --verbose
```
#### 3. 部署结构
```text
部署目录/
├── main.py # 主程序（使用VFS和模块库）
├── application.svfs # 虚拟文件系统
│ ├── config.json
│ ├── configs/yolov8_config.json
│ ├── configs/my_detector_config.json
│ ├── models/yolov8n.bmodel
│ └── models/my_model.bmodel
└── plugins.mbank # Python模块库
├── YOLOv8
├── my_detector
└── faster_rcnn
``` ```
src/sophon_ai_project/__init__.py → YOLOv8/__init__.py
View File
src/sophon_ai_project/postprocess_numpy.py → YOLOv8/postprocess_numpy.py
View File
YOLOv8/yolov8_detector.py
+320
View File
@@ -0,0 +1,320 @@
# yolov8_inference_with_colors.py
import os
import numpy as np
import sophon_chakcy.sail as sail
from .postprocess_numpy import PostProcess
from sqlite_vfs.core import SQLiteVFS
from dataclasses import dataclass, field
@dataclass
class YOLOv8Config:
bmodel_path: str = field(
metadata={"description": "模型路径"}
)
classes: list = field(
metadata={"description": "检测类别"}
)
colors: list = field(
metadata={"description": "边框颜色"}
)
@classmethod
def from_dict(cls, data: dict) -> "YOLOv8Config":
return cls(**data)
class Detector:
"""完整的YOLOv8检测器，包含可视化"""
def __init__(self, config_dict: dict, is_vfs=False, vfs_path: str="./application.svfs", dev_id=0, conf_thresh = 0.25, nms_thresh = 0.7):
self.dev_id = dev_id
yolov8_config = YOLOv8Config.from_dict(config_dict)
# 加载bmodel
if is_vfs:
vfs = SQLiteVFS(vfs_path)
bmodel_bytes = vfs.read_file(yolov8_config.bmodel_path)
bmodel_size = vfs.get_file_info(yolov8_config.bmodel_path)["file_size"]
self.net = sail.Engine(bmodel_bytes, bmodel_size, dev_id, sail.IOMode.SYSO)
else:
self.net = sail.Engine(yolov8_config.bmodel_path, dev_id, sail.IOMode.SYSO)
print(f"加载模型: {yolov8_config.bmodel_path}")
self.handle = sail.Handle(dev_id)
self.bmcv = sail.Bmcv(self.handle)
self.graph_name = self.net.get_graph_names()[0]
# 获取输入信息
self.input_name = self.net.get_input_names(self.graph_name)[0]
self.input_shape = self.net.get_input_shape(self.graph_name, self.input_name)
self.input_dtype = self.net.get_input_dtype(self.graph_name, self.input_name)
self.input_scale = self.net.get_input_scale(self.graph_name, self.input_name)
# 获取img_dtype
self.img_dtype = self.bmcv.get_bm_image_data_format(self.input_dtype)
# 获取输出信息
self.output_names = self.net.get_output_names(self.graph_name)
self.output_tensors = {}
for output_name in self.output_names:
output_shape = self.net.get_output_shape(self.graph_name, output_name)
output_dtype = self.net.get_output_dtype(self.graph_name, output_name)
output = sail.Tensor(self.handle, output_shape, output_dtype, True, True)
self.output_tensors[output_name] = output
self.batch_size = self.input_shape[0]
self.net_h = self.input_shape[2]
self.net_w = self.input_shape[3]
# 预处理参数
self.ab = [x * self.input_scale / 255. for x in [1, 0, 1, 0, 1, 0]]
# 后处理参数
self.conf_thresh = conf_thresh
self.nms_thresh = nms_thresh
self.postprocess = PostProcess(
conf_thresh=self.conf_thresh,
nms_thresh=self.nms_thresh,
agnostic=False,
multi_label=False,
max_det=300
)
self.colors = yolov8_config.colors
self.classes = yolov8_config.classes
print(f"模型信息:")
print(f" 输入形状: {self.input_shape}")
print(f" 输入类型: {self.input_dtype}")
print(f" colors数量: {len(self.colors)}")
print(f" classes类别数: {len(self.classes)}")
def draw_detections(self, image, detections, save_path=None):
"""
绘制检测结果到图像
Args:
image: sail.BMImage 对象
detections: 检测结果列表，每个元素为 [x1, y1, x2, y2, confidence, class_id]
save_path: 保存路径，如果不为None则保存图像
Returns:
绘制后的图像
"""
if len(detections) == 0:
print("没有检测到任何物体")
return image
# 转换为BGR planar格式用于绘制
img_bgr_planar = self.bmcv.convert_format(image)
thickness = 2
print(f"\n绘制 {len(detections)} 个检测结果:")
print("-" * 80)
for i, det in enumerate(detections):
if len(det) >= 6:
x1, y1, x2, y2, conf, cls_id = det[:6]
# 转换为整数坐标
x1, y1, x2, y2 = int(x1), int(y1), int(x2), int(y2)
# 使用COLORS为不同类别分配颜色
# 注意：class_id是0-79，但COLORS列表有160个颜色
# 我们使用 (class_id + 1) % len(COLORS) 来确保不越界
color_idx = (int(cls_id) + 1) % len(self.colors)
color = self.colors[color_idx]
# 获取类别名称
class_name = self.classes[int(cls_id) + 1] if int(cls_id) + 1 < len(self.classes) else f"未知类别{int(cls_id)}"
# 打印检测信息
print(f"检测 {i+1:2d}: {class_name:15s} | 置信度: {conf:.4f} | 位置: [{x1:4d}, {y1:4d}, {x2:4d}, {y2:4d}] | 大小: {x2-x1:4d}x{y2-y1:4d} | 颜色: {color}")
# 绘制边界框
if (x2 - x1) > thickness * 2 and (y2 - y1) > thickness * 2:
self.bmcv.rectangle(img_bgr_planar, x1, y1,
(x2 - x1), (y2 - y1),
color, thickness)
print("-" * 80)
# 保存图像
if save_path:
self.bmcv.imwrite(save_path, img_bgr_planar)
print(f"\n检测结果已保存到: {save_path}")
return img_bgr_planar
def detect_single_image(self, image_path, output_dir="./detection_results"):
"""
检测单张图像
Args:
image_path: 图像路径
output_dir: 输出目录
Returns:
检测结果列表
"""
# 创建输出目录
os.makedirs(output_dir, exist_ok=True)
print(f"\n处理图像: {image_path}")
# 1. 解码图像
decoder = sail.Decoder(image_path, True, self.dev_id)
bmimg = sail.BMImage()
ret = decoder.read(self.handle, bmimg)
if ret != 0:
print(f"错误: 无法解码图像 {image_path}")
return []
orig_width = bmimg.width()
orig_height = bmimg.height()
print(f"图像大小: {orig_width}x{orig_height}")
# 2. 预处理
# 转换为RGB格式
rgb_img = sail.BMImage(self.handle, bmimg.height(), bmimg.width(),
sail.Format.FORMAT_RGB_PLANAR, sail.DATA_TYPE_EXT_1N_BYTE)
self.bmcv.convert_format(bmimg, rgb_img)
# 调整大小并填充
img_w = rgb_img.width()
img_h = rgb_img.height()
r = min(self.net_w / img_w, self.net_h / img_h)
tw = int(round(r * img_w))
th = int(round(r * img_h))
tx1 = int(round((self.net_w - tw) / 2))
ty1 = int(round((self.net_h - th) / 2))
# 创建Padding属性
attr = sail.PaddingAtrr()
attr.set_stx(tx1)
attr.set_sty(ty1)
attr.set_w(tw)
attr.set_h(th)
attr.set_r(114)
attr.set_g(114)
attr.set_b(114)
# 调整大小
resized_img = self.bmcv.crop_and_resize_padding(rgb_img, 0, 0, img_w, img_h,
self.net_w, self.net_h, attr)
# 归一化
preprocessed_img = sail.BMImage(self.handle, self.net_h, self.net_w,
sail.Format.FORMAT_RGB_PLANAR, self.img_dtype)
# 转换并归一化
self.bmcv.convert_to(resized_img, preprocessed_img,
((self.ab[0], self.ab[1]),
(self.ab[2], self.ab[3]),
(self.ab[4], self.ab[5])))
# 3. 准备输入张量
input_tensor = sail.Tensor(self.handle, self.input_shape, self.input_dtype, False, False)
self.bmcv.bm_image_to_tensor(preprocessed_img, input_tensor)
# 4. 执行推理
input_tensors = {self.input_name: input_tensor}
input_shapes = {self.input_name: self.input_shape}
self.net.process(self.graph_name, input_tensors, input_shapes, self.output_tensors)
# 5. 获取输出
outputs = {}
for name in self.output_names:
outputs[name] = self.output_tensors[name].asnumpy()[0]
# 6. 后处理
output_key = list(outputs.keys())[0]
pred = outputs[output_key]
# 添加batch维度
pred_with_batch = pred[np.newaxis, :, :] # (1, 8400, 84)
# 准备后处理参数
org_size_list = [(orig_width, orig_height)]
ratios_batch = [(r, r)]
txy_batch = [(tx1, ty1)]
# 执行后处理
detections = self.postprocess([pred_with_batch], org_size_list, ratios_batch, txy_batch)
if len(detections) > 0:
detections = detections[0] # 取第一个batch的结果
# 7. 过滤低置信度的检测
conf_threshold = 0.25
filtered_detections = []
for det in detections:
if len(det) >= 6:
x1, y1, x2, y2, conf, cls_id = det[:6]
if conf > conf_threshold:
filtered_detections.append([x1, y1, x2, y2, conf, cls_id])
print(f"原始检测数量: {len(detections)}")
print(f"过滤后数量 (置信度 > {conf_threshold}): {len(filtered_detections)}")
# 8. 绘制检测结果
if len(filtered_detections) > 0:
# 准备输出文件名
base_name = os.path.basename(image_path)
name_without_ext = os.path.splitext(base_name)[0]
output_path = os.path.join(output_dir, f"detected_{base_name}")
# 绘制检测框
self.draw_detections(bmimg, filtered_detections, output_path)
return filtered_detections
else:
print("没有检测到任何物体")
return []
def detect_images_in_directory(self, input_dir, output_dir="./detection_results"):
"""
检测目录中的所有图像
Args:
input_dir: 输入目录
output_dir: 输出目录
"""
if not os.path.exists(input_dir):
print(f"错误: 目录不存在 {input_dir}")
return
# 创建输出目录
os.makedirs(output_dir, exist_ok=True)
print(f"开始检测目录: {input_dir}")
print(f"结果保存到: {output_dir}")
# 统计信息
total_images = 0
total_detections = 0
# 遍历目录
for root, dirs, files in os.walk(input_dir):
for file in files:
image_path = os.path.join(root, file)
total_images += 1
print(f"\n{'='*80}")
print(f"处理图像 {total_images}: {file}")
# 检测单张图像
detections = self.detect_single_image(image_path, output_dir)
total_detections += len(detections)
print(f"\n{'='*80}")
print(f"检测完成!")
print(f"总处理图像数: {total_images}")
print(f"总检测物体数: {total_detections}")
print(f"平均每张图像检测数: {total_detections/max(total_images, 1):.2f}")
config.json
+15
View File
@@ -0,0 +1,15 @@
{
"file_system": {
"type": "vfs"
},
"vfs": {
"file_path": "./application.svfs"
},
"model_map": {
"yolo_example": {
"name": "yolo_example",
"module": "YOLOv8.yolov8_detector",
"config": "resources/yolov8_config.json"
}
}
}
docs/sail_api/1.基础函数.md
-75
View File
@@ -1,75 +0,0 @@
# sail 基础函数
## get_available_tpu_num
获取当前设备中可用的TPU数量
**接口形式：**
```python
def get_available_tpu_num() -> int
```
**返回值说明**:
返回当前设备中可用 TPU的数量
## set_print_flag
设置是否打印程序的计算耗时信息
**接口形式:**
```python
def set_print_flag(print_flag: bool) -> None
```
**参数说明**
- print_flag:bool
print_flag 为 True 时，打印程序的计算主要的耗时信息，否则不打印
## set_dump_io_flag
设置是否存储输入时间和输出时间。
**接口形式:**
```python
def set_dump_io_flag(dump_io_flag: bool) -> None
```
**参数说明**
- dump_io_flag: bool
dump_io_flag为True时，存储输入数据和输出数据，否则不存储。
## set_decoder_env
设置Decoder的环境变量，必须在Decoder构造前设置，否则使用默认值。
**接口形式:**
```python
def set_decoder_env(env_name: str, env_value: str) -> None
```
**参数说明:**
- env_name: str
选择设置Decoder的属性名称，可选的属性名称有:
- ‘refcounted_frames’ 设置为1时，解码出来的图像需要程序手动释放，为0时由Decoder自动释放。
- ‘extra_frame_buffer_num’ 设置Decoder的最大缓存帧数
- ‘rtsp_transport’ 设置RTSP采用的传输协议
- ‘stimeout’ 设置阻塞超时时间
- ‘rtsp_flags’ 设置RTSP是否自定义IO
- ‘buffer_size’ 设置缓存大小
- ‘max_delay’ 设置最大时延
- ‘probesize’ 解析文件时读取的最大字节数
- ‘analyzeduration’ 解析文件时读取的最大时长
- env_value: str
docs/sail_api/2.sail数据类型.md
-21
View File
@@ -1,21 +0,0 @@
# sail 数据类型
定义sophon环境中常用的数据类型
**接口形式**
```python
sail.Dtype.BM_FLOAT32
sail.Dtype.BM_INT8
sail.Dtype.BM_UINT8
sail.Dtype.BM_INT32
sail.Dtype.BM_UINT32
```
**参数说明**
- sail.Dtype.BM_FLOAT32 数据类型为float32
- sail.Dtype.BM_INT8 数据类型为int8
- sail.Dtype.BM_UINT8 数据类型为uint8
- sail.Dtype.BM_INT32 数据类型为int32
- sail.Dtype.BM_UINT32 数据类型为uint32
docs/sail_api/3.填充属性.md
-156
View File
@@ -1,156 +0,0 @@
# 填充属性
PaddingAtrr中存储了数据padding的各项属性，可通过配置PaddingAtrr进行数据填充
## \_\_init\_\_
初始化 PaddingAtrr
**接口形式：**
```python
def __init__(self)
def __init__(self, stx: int, sty: int, width: int, height: int, r: int, g: int, b: int)
```
**参数说明：**
- stx: int
原图像相对于目标图像在x方向上的偏移量
- sty: int
原图像相对于目标图像在y方向上的偏移量
- width: int
在 padding 的同时可对原图像进行 resize，width 为原图像 resize 后的宽，若不进行 resize，则 width 为原图像的宽
- height: int
在 padding 的同时对原图像进行 resize，height 为原图像 resize 后的高，若不进行 resize，则 height 为原图像的高
- r: int
padding 时在 R 通道上填充的像素值
- g: int
padding 时在 G 通道上填充的像素值
- b: int
padding 时在B通道上填充的像素值
## set_stx
设置原图像相对于目标图像在 x 方向上的偏移量
**接口形式**
```python
def set_stx(self, stx: int) -> None
```
**参数说明**
- stx: int
原图像相对于目标图像在x方向上的偏移量
## set_sty
设置原图像相对于目标图像在y方向上的偏移量
**接口形式:**
```python
def set_sty(self, sty: int) -> None
```
**参数说明:**
- sty: int
原图像相对于目标图像在y方向上的偏移量
## set_w
设置原图像resize后的width
**接口形式:**
```python
def set_w(self, width: int) -> None
```
**参数说明:**
- width: int
在padding的同时可对原图像进行resize，width为原图像resize后的宽，若不进行resize，则width为原图像的宽
## set_h
设置原图像resize后的height
**接口形式:**
```python
def set_h(self, height: int) -> None
```
**参数说明:**
- height: int
在padding的同时可对原图像进行resize，height为原图像resize后的高，若不进行resize，则height为原图像的高
## set_r
设置R通道上的padding值
**接口形式:**
```python
def set_r(self, r: int) -> None
```
**参数说明**
- r: int
R通道上的padding值
## set_g
设置G通道上的padding值
**接口形式:**
```python
def set_g(self, g: int) -> None
```
**参数说明**
- g: int
G通道上的padding值
## set_b
设置B通道上的padding值
**接口形式:**
```python
def set_b(self, b: int) -> None
```
**参数说明**
- b: int
B通道上的padding值
docs/sail_api/4.设备句柄.md
-51
View File
@@ -1,51 +0,0 @@
# 设备句柄
Handle是设备句柄的包装类，在程序中用于设备的标识。
## \_\_init\_\_
初始化 Handle
**接口形式：**
```python
def __init__(self, tpu_id: int)
```
**参数说明：**
- tpu_id: int
创建 Handle 使用的TPU的id号
## get_device_id
获取 Handle 中 TPU的id
**接口形式：**
```python
def get_device_id(self) -> int
```
**返回值说明：**
- tpu_id: int
Handle 中的 TPU 的 id 号
## get_sn
获取 Handle 中标识设备的序列码
**接口形式：**
```python
def get_sn(self) -> str
```
**返回值说明**
- serial_number: str
返回 Handle 中设备的序列码
docs/sail_api/5.resize插值策略.md
-17
View File
@@ -1,17 +0,0 @@
# resize 插值策略(sail.bmcv_resize_algorithm)
定义图像resize中常见的插值策略
**接口形式**
```python
sail.bmcv_resize_algorithm.BMCV_INTER_NEAREST
sail.bmcv_resize_algorithm.BMCV_INTER_LINEAR
sail.bmcv_resize_algorithm.BMCV_INTER_BICUBIC
```
**参数说明**
- sail.bmcv_resize_algorithm.BMCV_INTER_NEAREST 最近邻插值算法
- sail.bmcv_resize_algorithm.BMCV_INTER_LINEAR 双线性插值算法
- sail.bmcv_resize_algorithm.BMCV_INTER_BICUBIC 双三次插值算法
docs/sail_api/6.Tensor张量.md
-240
View File
@@ -1,240 +0,0 @@
# Tensor 张量
Tensor 是模型推理的输入输出类型，包含了数据信息，实现内容管理
## \_\_init\_\_
初始化 Tensor，并为 Tensor 分配内存
**接口形式1**
```python
def __init__(self, handle: Handle, data: np.arrat, own_sys_data=True)
```
**参数说明1**
- handle: sail.Handle
设备标识 Handle
- array_data: numpy.array
利用 numpy.array 类型初始化 Tensor，其数据类型可以是 np.float32，np.int8，np.uint8
- own_sys_data:bool
指示该 Tensor 是否拥有 system memory，如果为 False，则直接将数据复制到 device memory
**接口形式2**
```python
def __init__(self, handle: Handle, shape: tuple, dtype: Dtype, own_sys_data: bool, own_dev_data: bool)
```
**参数说明2**
- handle: sail.Handle
设备标识 Handle
- shape: tuple
设置 Tensor 的 shape
- dtype: sail.Dtype
Tensor 的数据类型
- own_sys_data: bool
指示 Tensor 是否拥有 system memory
- own_dev_data: bool
指示 Tensor 是否拥有 device memory
## shape
获取 Tensor 的 shape
**接口形式:**
```python
def shape(self) -> list :
```
**返回值说明:**
- tensor_shape : list
返回Tensor的shape的列表。
## asnumpy
获取Tensor中系统内存的数据，返回numpy.array类型。
**接口形式:**
```python
def asnumpy(self) -> numpy.array
def asnumpy(self, shape: tuple) -> numpy.array
```
**参数说明:**
- shape: tuple
可对Tensor中的数据reshape，返回形状为shape的numpy.array
**返回值说明**
返回Tensor中系统内存的数据，返回类型为numpy.array。
## update_data
更新Tensor中系统内存的数据
**接口形式:**
```python
def update_data(self, data: numpy.array) -> None
```
**参数说明:**
- data: numpy.array
更新的数据，数据size不能超过Tensor的size，Tensor的shape将保持不变。
## scale_from
先对data按比例缩放，再将数据更新到Tensor的系统内存。
**接口形式:**
```python
def scale_from(self, data: numpy.array, scale: float32)->None
```
**参数说明:**
- data: numpy.array
对data进行scale，再将数据更新到Tensor的系统内存。
- scale: float32
等比例缩放时的尺度。
## scale_to
先对Tensor进行等比例缩放，再将数据返回到系统内存。
**接口形式:**
```python
def scale_to(self, scale: float32)->numpy.array
def scale_to(self, scale: float32, shape: tuple)->numpy.array
```
**参数说明:**
- scale: float32
等比例缩放时的尺度。
- shape: tuple
数据返回前可进行reshape，返回shape形状的数据。
**返回值说明:**
- data: numpy.array
将处理后的数据返回至系统内存，返回numpy.array
## reshape
对Tensor进行reshape
**接口形式:**
```python
def reshape(self, shape: list)->None
```
**参数说明:**
- shape: list
设置期望得到的新shape。
# own_sys_data
查询该Tensor是否拥有系统内存的数据指针。
**接口形式:**
```python
def own_sys_data(self)->bool
```
**返回值说明:**
- judge_ret: bool
如果拥有系统内存的数据指针则返回True，否则False。
## own_dev_data
查询该Tensor是否拥有设备内存的数据
**接口形式:**
```python
def own_dev_data(self)->bool
```
**返回值说明:**
- judge_ret : bool
如果拥有设备内存中的数据则返回True，否则False。
## sync_s2d
将Tensor中的数据从系统内存拷贝到设备内存。
**接口形式:**
```python
def sync_s2d(self)->None
def sync_s2d(self, size)->None
```
**参数说明:**
- size: int
将特定size字节的数据从系统内存拷贝到设备内存。
## sync_d2s
将Tensor中的数据从设备内存拷贝到系统内存。
**接口形式:**
```python
def sync_d2s(self)->None
def sync_d2s(self, size: int)->None
```
**参数说明:**
- size: int
将特定size字节的数据从设备内存拷贝到系统内存。
lib/sophon_arm-3.10.7-py3-none-any.whl
BIN
View File
Binary file not shown.
main.py
+30 -5
View File
@@ -1,8 +1,33 @@
from sophon_ai_project.yolov8_bmcv import main import os
from sophon_ai_project.yolov8_bmcv import argsparser import argparse
from sophon_chakcy.get_detector import get_detector
def main():
parser = argparse.ArgumentParser(description="YOLOv8目标检测")
parser.add_argument('--name', type=str, required=True, help='算法名称')
parser.add_argument('--input', type=str, required=True, help='输入图像或目录路径')
parser.add_argument('--dev_id', type=int, default=0, help='设备ID')
parser.add_argument('--conf_thresh', type=float, default=0.25, help='置信度阈值')
parser.add_argument('--nms_thresh', type=float, default=0.7, help='NMS阈值')
args = parser.parse_args()
detector_class, model_config = get_detector(args.name)
# 初始化检测器
detector = detector_class(model_config, False, dev_id=args.dev_id, conf_thresh=args.conf_thresh, nms_thresh=args.nms_thresh)
# 根据输入类型进行处理
if os.path.isfile(args.input):
# 单张图像
detector.detect_single_image(args.input)
elif os.path.isdir(args.input):
# 图像目录
detector.detect_images_in_directory(args.input)
else:
print(f"错误: {args.input} 既不是文件也不是目录")
if __name__ == "__main__": if __name__ == "__main__":
args = argsparser() main()
main(args)
print('all done.')
public/main.py
+38
View File
@@ -0,0 +1,38 @@
import os
import argparse
from sophon_chakcy.get_detector import get_detector
from module_bank import PythonToSQLite
# 安装导入器
packer = PythonToSQLite("plugins.mbank")
finder = packer.install_importer()
def main():
parser = argparse.ArgumentParser(description="YOLOv8目标检测")
parser.add_argument('--name', type=str, required=True, help='算法名称')
parser.add_argument('--input', type=str, required=True, help='输入图像或目录路径')
parser.add_argument('--dev_id', type=int, default=0, help='设备ID')
parser.add_argument('--conf_thresh', type=float, default=0.25, help='置信度阈值')
parser.add_argument('--nms_thresh', type=float, default=0.7, help='NMS阈值')
args = parser.parse_args()
detector_class, model_config = get_detector(args.name, True)
# 初始化检测器
detector = detector_class(model_config, True, dev_id=args.dev_id, conf_thresh=args.conf_thresh, nms_thresh=args.nms_thresh)
# 根据输入类型进行处理
if os.path.isfile(args.input):
# 单张图像
detector.detect_single_image(args.input)
elif os.path.isdir(args.input):
# 图像目录
detector.detect_images_in_directory(args.input)
else:
print(f"错误: {args.input} 既不是文件也不是目录")
if __name__ == "__main__":
main()
public/requirements.txt
+4
View File
@@ -0,0 +1,4 @@
module-bank==0.1.1
numpy==2.2.6
opencv-python-headless==4.12.0.88
sqlite-vfs==0.1.0
pyproject.toml
+11 -2
View File
@@ -1,10 +1,16 @@
[build-system]
build-backend = "hatchling.build"
requires = ["hatchling"]
[project] [project]
dependencies = [ dependencies = [
"module-bank>=0.1.1",
"numpy>=1.24.4", "numpy>=1.24.4",
"opencv-python-headless>=4.12.0.88", "opencv-python-headless>=4.12.0.88",
"sqlite-vfs>=0.1.0",
] ]
description = "Add your description here" description = "Add your description here"
name = "sophon-ai-project" name = "sophon-chakcy"
readme = "README.md" readme = "README.md"
requires-python = ">=3.8" requires-python = ">=3.8"
version = "0.1.0" version = "0.1.0"
@@ -14,10 +20,13 @@ default = true
url = "https://pypi.tuna.tsinghua.edu.cn/simple" url = "https://pypi.tuna.tsinghua.edu.cn/simple"
[tool.hatch.build.targets.wheel] [tool.hatch.build.targets.wheel]
packages = ["src/sophon_ai_project"] packages = ["src/sophon_chakcy"]
[dependency-groups] [dependency-groups]
dev = [ dev = [
"dfss>=1.10.4", "dfss>=1.10.4",
"pip>=25.0.1", "pip>=25.0.1",
] ]
[project.scripts]
build = "sophon_chakcy.build:main"
requirements.txt
+4
View File
@@ -0,0 +1,4 @@
module-bank==0.1.1
numpy==2.2.6
opencv-python-headless==4.12.0.88
sqlite-vfs==0.1.0
scripts/code_content.py
+33
View File
@@ -0,0 +1,33 @@
import os
import sys
def generate_markdown_from_py_files(directory, output_file):
with open(output_file, "w", encoding="utf-8") as md_file:
for root, dirs, files in os.walk(directory):
# 排除 venv 目录
dirs[:] = [d for d in dirs if d != ".venv"]
dirs[:] = [d for d in dirs if d != ".vscode"]
dirs[:] = [d for d in dirs if d != "Scripts"]
dirs[:] = [d for d in dirs if d != "build"]
for file in files:
if (
file.endswith(".py")
or file.endswith(".rs")
or file.endswith(".toml")
):
file_path = os.path.join(root, file)
md_file.write(f"`{file_path}`\n")
md_file.write("```python\n")
with open(file_path, "r", encoding="utf-8") as py_file:
md_file.write(py_file.read())
md_file.write("\n```\n\n")
if __name__ == "__main__":
# 指定目录和输出文件名
target_directory = sys.argv[1] # 替换为你的目标目录
output_markdown_file = "output.md" # 输出的 Markdown 文件名
generate_markdown_from_py_files(target_directory, output_markdown_file)
print(f"Markdown 文件已生成：{output_markdown_file}")
src/sophon_ai_project/utils.py
-125
View File
@@ -1,125 +0,0 @@
#===----------------------------------------------------------------------===#
#
# Copyright (C) 2022 Sophgo Technologies Inc. All rights reserved.
#
# SOPHON-DEMO is licensed under the 2-Clause BSD License except for the
# third-party components.
#
#===----------------------------------------------------------------------===#
import os
import sys
import numpy as np
import cv2
COCO_CLASSES = ('background', 'person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus',
'train', 'truck', 'boat', 'traffic light', 'fire hydrant',
'stop sign', 'parking meter', 'bench', 'bird', 'cat', 'dog',
'horse', 'sheep', 'cow', 'elephant', 'bear', 'zebra', 'giraffe',
'backpack', 'umbrella', 'handbag', 'tie', 'suitcase', 'frisbee',
'skis', 'snowboard', 'sports ball', 'kite', 'baseball bat',
'baseball glove', 'skateboard', 'surfboard', 'tennis racket',
'bottle', 'wine glass', 'cup', 'fork', 'knife', 'spoon', 'bowl',
'banana', 'apple', 'sandwich', 'orange', 'broccoli', 'carrot',
'hot dog', 'pizza', 'donut', 'cake', 'chair', 'couch',
'potted plant', 'bed', 'dining table', 'toilet', 'tv', 'laptop',
'mouse', 'remote', 'keyboard', 'cell phone', 'microwave', 'oven',
'toaster', 'sink', 'refrigerator', 'book', 'clock', 'vase',
'scissors', 'teddy bear', 'hair drier', 'toothbrush')
COLORS = [
[56, 0, 255],
[226, 255, 0],
[0, 94, 255],
[0, 37, 255],
[0, 255, 94],
[255, 226, 0],
[0, 18, 255],
[255, 151, 0],
[170, 0, 255],
[0, 255, 56],
[255, 0, 75],
[0, 75, 255],
[0, 255, 169],
[255, 0, 207],
[75, 255, 0],
[207, 0, 255],
[37, 0, 255],
[0, 207, 255],
[94, 0, 255],
[0, 255, 113],
[255, 18, 0],
[255, 0, 56],
[18, 0, 255],
[0, 255, 226],
[170, 255, 0],
[255, 0, 245],
[151, 255, 0],
[132, 255, 0],
[75, 0, 255],
[151, 0, 255],
[0, 151, 255],
[132, 0, 255],
[0, 255, 245],
[255, 132, 0],
[226, 0, 255],
[255, 37, 0],
[207, 255, 0],
[0, 255, 207],
[94, 255, 0],
[0, 226, 255],
[56, 255, 0],
[255, 94, 0],
[255, 113, 0],
[0, 132, 255],
[255, 0, 132],
[255, 170, 0],
[255, 0, 188],
[113, 255, 0],
[245, 0, 255],
[113, 0, 255],
[255, 188, 0],
[0, 113, 255],
[255, 0, 0],
[0, 56, 255],
[255, 0, 113],
[0, 255, 188],
[255, 0, 94],
[255, 0, 18],
[18, 255, 0],
[0, 255, 132],
[0, 188, 255],
[0, 245, 255],
[0, 169, 255],
[37, 255, 0],
[255, 0, 151],
[188, 0, 255],
[0, 255, 37],
[0, 255, 0],
[255, 0, 170],
[255, 0, 37],
[255, 75, 0],
[0, 0, 255],
[255, 207, 0],
[255, 0, 226],
[255, 245, 0],
[188, 255, 0],
[0, 255, 18],
[0, 255, 75],
[0, 255, 151],
[255, 56, 0],
[245, 255, 0],
]
def is_img(file_name):
"""judge the file is available image or not
Args:
file_name (str): input file name
Returns:
(bool) : whether the file is available image
"""
fmt = os.path.splitext(file_name)[-1]
if isinstance(fmt, str) and fmt.lower() in ['.jpg','.png','.jpeg','.bmp','.jpeg','.webp']:
return True
else:
return False
src/sophon_ai_project/yolov8_bmcv.py
-393
View File
@@ -1,393 +0,0 @@
#===----------------------------------------------------------------------===#
#
# Copyright (C) 2022 Sophgo Technologies Inc. All rights reserved.
#
# SOPHON-DEMO is licensed under the 2-Clause BSD License except for the
# third-party components.
#
#===----------------------------------------------------------------------===#
import os
import time
import json
import argparse
import numpy as np
import sophon.sail as sail
from .postprocess_numpy import PostProcess
from .utils import COCO_CLASSES, COLORS
import logging
logging.basicConfig(level=logging.INFO)
class YOLOv8:
def __init__(self, args):
# load bmodel
self.net = sail.Engine(args.bmodel, args.dev_id, sail.IOMode.SYSO) # type: ignore
logging.debug("load {} success!".format(args.bmodel))
self.handle = sail.Handle(args.dev_id)
self.bmcv = sail.Bmcv(self.handle)
self.graph_name = self.net.get_graph_names()[0]
# get input
self.input_name = self.net.get_input_names(self.graph_name)[0]
self.input_dtype= self.net.get_input_dtype(self.graph_name, self.input_name)
self.img_dtype = self.bmcv.get_bm_image_data_format(self.input_dtype)
self.input_scale = self.net.get_input_scale(self.graph_name, self.input_name)
self.input_shape = self.net.get_input_shape(self.graph_name, self.input_name)
self.input_shapes = {self.input_name: self.input_shape}
# get output
self.output_names = self.net.get_output_names(self.graph_name)
self.output_tensors = {}
self.output_scales = {}
for output_name in self.output_names:
output_shape = self.net.get_output_shape(self.graph_name, output_name)
output_dtype = self.net.get_output_dtype(self.graph_name, output_name)
output_scale = self.net.get_output_scale(self.graph_name, output_name)
output = sail.Tensor(self.handle, output_shape, output_dtype, True, True) # type: ignore
self.output_tensors[output_name] = output
self.output_scales[output_name] = output_scale
if(output_shape[1] < output_shape[2]):
raise ValueError('Only support OPT model, please refer to the docs/YOLOv8_Export_Guide.md to export OPT model.')
# check batch size
self.batch_size = self.input_shape[0]
suppoort_batch_size = [1, 2, 3, 4, 8, 16, 32, 64, 128, 256]
if self.batch_size not in suppoort_batch_size:
raise ValueError('batch_size must be {} for bmcv, but got {}'.format(suppoort_batch_size, self.batch_size))
self.net_h = self.input_shape[2]
self.net_w = self.input_shape[3]
# init preprocess
self.use_resize_padding = True
self.use_vpp = False
self.ab = [x * self.input_scale / 255. for x in [1, 0, 1, 0, 1, 0]]
# init postprocess
self.conf_thresh = args.conf_thresh
self.nms_thresh = args.nms_thresh
self.agnostic = False
self.multi_label = False
self.max_det = 300
self.postprocess = PostProcess(
conf_thresh=self.conf_thresh,
nms_thresh=self.nms_thresh,
agnostic=self.agnostic,
multi_label=self.multi_label,
max_det=self.max_det,
)
# init time
self.preprocess_time = 0.0
self.inference_time = 0.0
self.postprocess_time = 0.0
def init(self):
self.preprocess_time = 0.0
self.inference_time = 0.0
self.postprocess_time = 0.0
def preprocess_bmcv(self, input_bmimg):
rgb_planar_img = sail.BMImage(self.handle, input_bmimg.height(), input_bmimg.width(),
sail.Format.FORMAT_RGB_PLANAR, sail.DATA_TYPE_EXT_1N_BYTE) # type: ignore
self.bmcv.convert_format(input_bmimg, rgb_planar_img) # type: ignore
resized_img_rgb, ratio, txy = self.resize_bmcv(rgb_planar_img)
preprocessed_bmimg = sail.BMImage(self.handle, self.net_h, self.net_w, sail.Format.FORMAT_RGB_PLANAR, self.img_dtype) # type: ignore
self.bmcv.convert_to(resized_img_rgb, preprocessed_bmimg, ((self.ab[0], self.ab[1]), \
(self.ab[2], self.ab[3]), \
(self.ab[4], self.ab[5]))) # type: ignore
return preprocessed_bmimg, ratio, txy
def resize_bmcv(self, bmimg):
"""
resize for single sail.BMImage
:param bmimg:
:return: a resize image of sail.BMImage
"""
img_w = bmimg.width()
img_h = bmimg.height()
if self.use_resize_padding:
r_w = self.net_w / img_w
r_h = self.net_h / img_h
r = min(r_w, r_h)
tw = int(round(r * img_w))
th = int(round(r * img_h))
tx1, ty1 = self.net_w - tw, self.net_h - th # wh padding
tx1 /= 2 # divide padding into 2 sides
ty1 /= 2
ratio = (r, r)
txy = (tx1, ty1)
attr = sail.PaddingAtrr() # type: ignore
attr.set_stx(int(round(tx1 - 0.1)))
attr.set_sty(int(round(ty1 - 0.1)))
attr.set_w(tw)
attr.set_h(th)
attr.set_r(114)
attr.set_g(114)
attr.set_b(114)
preprocess_fn = self.bmcv.vpp_crop_and_resize_padding if self.use_vpp else self.bmcv.crop_and_resize_padding
resized_img_rgb = preprocess_fn(bmimg, 0, 0, img_w, img_h, self.net_w, self.net_h, attr, sail.bmcv_resize_algorithm.BMCV_INTER_LINEAR) # type: ignore
else:
r_w = self.net_w / img_w
r_h = self.net_h / img_h
ratio = (r_w, r_h)
txy = (0, 0)
preprocess_fn = self.bmcv.vpp_resize if self.use_vpp else self.bmcv.resize
resized_img_rgb = preprocess_fn(bmimg, self.net_w, self.net_h)
return resized_img_rgb, ratio, txy
def predict(self, input_tensor, img_num):
"""
ensure output order: loc_data, conf_preds, mask_data, proto_data
Args:
input_tensor:
Returns:
"""
input_tensors = {self.input_name: input_tensor}
self.net.process(self.graph_name, input_tensors, self.input_shapes, self.output_tensors) # type: ignore
outputs_dict = {}
for name in self.output_names:
# outputs_dict[name] = self.output_tensors[name].asnumpy()[:img_num] * self.output_scales[name]
outputs_dict[name] = self.output_tensors[name].asnumpy()[:img_num]
# resort
out_keys = list(outputs_dict.keys())
ord = []
for n in self.output_names:
for i, k in enumerate(out_keys):
if n in k:
ord.append(i)
break
out = [outputs_dict[out_keys[i]] for i in ord]
return out
def __call__(self, bmimg_list):
img_num = len(bmimg_list)
ori_size_list = []
ratio_list = []
txy_list = []
if self.batch_size == 1:
ori_h, ori_w = bmimg_list[0].height(), bmimg_list[0].width()
ori_size_list.append((ori_w, ori_h))
start_time = time.time()
preprocessed_bmimg, ratio, txy = self.preprocess_bmcv(bmimg_list[0])
self.preprocess_time += time.time() - start_time
ratio_list.append(ratio)
txy_list.append(txy)
input_tensor = sail.Tensor(self.handle, self.input_shape, self.input_dtype, False, False) # type: ignore
self.bmcv.bm_image_to_tensor(preprocessed_bmimg, input_tensor) # type: ignore
else:
BMImageArray = eval('sail.BMImageArray{}D'.format(self.batch_size))
bmimgs = BMImageArray()
for i in range(img_num):
ori_h, ori_w = bmimg_list[i].height(), bmimg_list[i].width()
ori_size_list.append((ori_w, ori_h))
start_time = time.time()
preprocessed_bmimg, ratio, txy = self.preprocess_bmcv(bmimg_list[i])
self.preprocess_time += time.time() - start_time
ratio_list.append(ratio)
txy_list.append(txy)
bmimgs[i] = preprocessed_bmimg.data()
input_tensor = sail.Tensor(self.handle, self.input_shape, self.input_dtype, False, False) # type: ignore
self.bmcv.bm_image_to_tensor(bmimgs, input_tensor) # type: ignore
start_time = time.time()
outputs = self.predict(input_tensor, img_num)
self.inference_time += time.time() - start_time
start_time = time.time()
results = self.postprocess(outputs, ori_size_list, ratio_list, txy_list)
self.postprocess_time += time.time() - start_time
return results
def draw_bmcv(dev_id, image, boxes, output_img_dir, file_name, cn, masks=None, classes_ids=None, conf_scores=None, isvideo=False):
bmcv = sail.Bmcv(sail.Handle(dev_id))
img_bgr_planar = bmcv.convert_format(image)
thickness = 2
for idx in range(len(boxes)):
x1, y1, x2, y2 = boxes[idx, :].astype(np.int32).tolist()
if classes_ids is not None:
color = np.array(COLORS[int(classes_ids[idx]) + 1]).astype(np.uint8).tolist()
else:
color = (0, 0, 255)
if (x2 - x1) <= thickness * 2 or (y2 - y1) <= thickness * 2:
logging.info("width or height too small, this rect will not be drawed: (x1={},y1={},w={},h={})".format(x1, y1, x2-x1, y2-y1))
else:
bmcv.rectangle(img_bgr_planar, x1, y1, (x2 - x1), (y2 - y1), color, thickness) # type: ignore
# bmcv.putText(image, COCO_CLASSES[int(classes_ids[idx] + 1)], x1, y1, tuple(color),1.0,1)
logging.debug("class id={}, score={}, (x1={},y1={},w={},h={})".format(int(classes_ids[idx]), conf_scores[idx], x1, y1, x2-x1, y2-y1)) # type: ignore
if isvideo:
return img_bgr_planar
else:
bmcv.imwrite(os.path.join(output_img_dir, file_name), img_bgr_planar) # type: ignore
def main(args):
# check params
if not os.path.exists(args.input):
raise FileNotFoundError('{} is not existed.'.format(args.input))
if not os.path.exists(args.bmodel):
raise FileNotFoundError('{} is not existed.'.format(args.bmodel))
# creat save path
output_dir = "./results"
if not os.path.exists(output_dir):
os.mkdir(output_dir)
output_img_dir = os.path.join(output_dir, 'images')
if not os.path.exists(output_img_dir):
os.mkdir(output_img_dir)
yolov8 = YOLOv8(args)
batch_size = yolov8.batch_size
# warm up
# bmimg = sail.BMImage(yolov8.handle, 1080, 1920, sail.Format.FORMAT_YUV420P, sail.DATA_TYPE_EXT_1N_BYTE)
# for i in range(10):
# results = yolov8([bmimg])
yolov8.init()
decode_time = 0.0
# test images
if os.path.isdir(args.input):
bmimg_list = []
filename_list = []
results_list = []
cn = 0
yolov8_handle = sail.Handle(args.dev_id)
for root, dirs, filenames in os.walk(args.input):
for filename in filenames:
if os.path.splitext(filename)[-1].lower() not in ['.jpg','.png','.jpeg','.bmp','.webp']:
continue
img_file = os.path.join(root, filename)
cn += 1
logging.info("{}, img_file: {}".format(cn, img_file))
# decode
start_time = time.time()
decoder = sail.Decoder(img_file, True, args.dev_id)
bmimg = sail.BMImage() # type: ignore
ret = decoder.read(yolov8_handle, bmimg) # type: ignore
if ret != 0:
logging.error("{} decode failure.".format(img_file))
continue
decode_time += time.time() - start_time
bmimg_list.append(bmimg)
filename_list.append(filename)
if (len(bmimg_list) == batch_size or cn == len(filenames)) and len(bmimg_list):
# predict
results = yolov8(bmimg_list)
for i, filename in enumerate(filename_list):
det = results[i]
# save image
det_draw = det[det[:, -2] > 0.1]
draw_bmcv(args.dev_id,
bmimg_list[i],
det_draw[:,:4],
output_img_dir,
filename,
cn,
masks=None,
classes_ids=det_draw[:, -1],
conf_scores=det_draw[:, -2])
# save result
res_dict = dict()
res_dict['image_name'] = filename
res_dict['bboxes'] = []
for idx in range(det.shape[0]):
bbox_dict = dict()
x1, y1, x2, y2, score, category_id = det[idx]
bbox_dict['bbox'] = [float(round(x1, 3)), float(round(y1, 3)), float(round(x2 - x1,3)), float(round(y2 -y1, 3))]
bbox_dict['category_id'] = int(category_id)
bbox_dict['score'] = float(round(score,5))
res_dict['bboxes'].append(bbox_dict)
results_list.append(res_dict)
bmimg_list.clear()
filename_list.clear()
# save results
if args.input[-1] == '/':
args.input = args.input[:-1]
json_name = os.path.split(args.bmodel)[-1] + "_" + os.path.split(args.input)[-1] + "_bmcv" + "_python_result.json"
with open(os.path.join(output_dir, json_name), 'w') as jf:
# json.dump(results_list, jf)
json.dump(results_list, jf, indent=4, ensure_ascii=False)
logging.info("result saved in {}".format(os.path.join(output_dir, json_name)))
# test videos
else:
decoder = sail.Decoder(args.input, True, args.dev_id)
if not decoder.is_opened():
raise Exception("can not open the video")
enc_params = f"width=1920:height=1080:bitrate=2000:gop=32:gop_preset=2:framerate=25"
encoder = sail.Encoder("results/output.mp4", args.dev_id, 'h264_bm', 'NV12', enc_params, 10)
video_name = os.path.splitext(os.path.split(args.input)[1])[0]
cn = 0
frame_list = []
yolov8_handle = sail.Handle(args.dev_id)
end_flag = False
while not end_flag:
frame = sail.BMImage() # type: ignore
start_time = time.time()
ret = decoder.read(yolov8_handle, frame) # type: ignore
decode_time += time.time() - start_time
if ret: #differ from cv.
end_flag = True
else:
frame_list.append(frame)
if (len(frame_list) == batch_size or end_flag) and len(frame_list):
results = yolov8(frame_list)
for i, frame in enumerate(frame_list):
det = results[i]
cn += 1
logging.info("{}, det nums: {}".format(cn, det.shape[0]))
det_draw = det[det[:, -2] > 0.1]
drawed_bmimg = draw_bmcv(args.dev_id,
frame_list[i],
det_draw[:,:4],
output_img_dir,
video_name,
cn,
masks=None,
classes_ids=det_draw[:, -1],
conf_scores=det_draw[:, -2],
isvideo=True)
encoder.video_write(drawed_bmimg) # type: ignore
frame_list.clear()
decoder.release()
encoder.release()
logging.info("result saved in {}".format(output_img_dir))
# calculate speed
logging.info("------------------ Predict Time Info ----------------------")
decode_time = decode_time / cn
preprocess_time = yolov8.preprocess_time / cn
inference_time = yolov8.inference_time / cn
postprocess_time = yolov8.postprocess_time / cn
logging.info("decode_time(ms): {:.2f}".format(decode_time * 1000))
logging.info("preprocess_time(ms): {:.2f}".format(preprocess_time * 1000))
logging.info("inference_time(ms): {:.2f}".format(inference_time * 1000))
logging.info("postprocess_time(ms): {:.2f}".format(postprocess_time * 1000))
def argsparser():
parser = argparse.ArgumentParser(prog=__file__)
parser.add_argument('--input', type=str, default='../datasets/test', help='path of input')
parser.add_argument('--bmodel', type=str, default='../models/BM1684X/yolov8s_fp32_1b.bmodel', help='path of bmodel')
parser.add_argument('--dev_id', type=int, default=0, help='dev id')
parser.add_argument('--conf_thresh', type=float, default=0.25, help='confidence threshold')
parser.add_argument('--nms_thresh', type=float, default=0.7, help='nms threshold')
args = parser.parse_args()
return args
if __name__ == "__main__":
args = argsparser()
main(args)
print('all done.')
src/sophon_chakcy/__init__.py
+40
View File
@@ -0,0 +1,40 @@
"""sophon python module
"""
__all__ = []
try:
import numpy
except ImportError:
print('SAIL bindings requires "numpy" package.')
print('Install it via command:')
print(' pip3 install numpy')
raise
import os
from sqlite_vfs.core import SQLiteVFS
import json
def get_config(json_path: str, is_vfs: bool = False, vfs_path: str="./application.svfs"):
if is_vfs:
vfs = SQLiteVFS(vfs_path)
data = json.loads(vfs.read_file(json_path))
else:
with open(json_path, 'r') as file:
data = json.load(file)
return data
def is_img(file_name):
"""judge the file is available image or not
Args:
file_name (str): input file name
Returns:
(bool) : whether the file is available image
"""
fmt = os.path.splitext(file_name)[-1]
if isinstance(fmt, str) and fmt.lower() in ['.jpg','.png','.jpeg','.bmp','.jpeg','.webp']:
return True
else:
return False
src/sophon_chakcy/build.py
+183
View File
@@ -0,0 +1,183 @@
from sqlite_vfs.folder_packer import FolderPacker
from sqlite_vfs.core import SQLiteVFS
import json
from module_bank import PythonToSQLite
import os
import shutil
from pathlib import Path
import traceback
import sys
def main(vfs_path: str = "application.svfs", mbank_path: str = "plugins.mbank"):
"""
构建应用程序资源包和虚拟文件系统
Args:
vfs_path: 输出的VFS文件路径
mbank_path: 输出的模块库文件路径
"""
build_dir = r"build"
try:
# 清理旧的构建目录
if os.path.exists(build_dir):
print(f"清理旧的构建目录: {build_dir}")
shutil.rmtree(build_dir, ignore_errors=True)
# 创建构建目录
build_dir_path = Path(build_dir)
if not build_dir_path.exists():
build_dir_path.mkdir(parents=True, exist_ok=True)
print("开始构建...")
# 读取配置文件
try:
with open("config.json", 'r') as file:
config_data = json.load(file)
model_map = config_data["model_map"]
print("配置文件加载成功")
except FileNotFoundError:
print("错误: 找不到配置文件 config.json")
return
except json.JSONDecodeError as e:
print(f"错误: 配置文件格式错误 - {e}")
return
except KeyError:
print("错误: 配置文件中缺少 'model_map' 字段")
return
# 初始化VFS
try:
vfs = SQLiteVFS(vfs_path)
folder_packer = FolderPacker(vfs)
print(f"VFS初始化成功: {vfs_path}")
except Exception as e:
print(f"错误: 初始化VFS失败 - {e}")
return
# 初始化Python模块打包器
try:
packer = PythonToSQLite(mbank_path)
print(f"模块打包器初始化成功: {mbank_path}")
except Exception as e:
print(f"错误: 初始化模块打包器失败 - {e}")
return
# 添加配置文件到VFS
try:
folder_packer.add_file("config.json", "config.json")
print("配置文件已添加到VFS")
except Exception as e:
print(f"错误: 添加配置文件到VFS失败 - {e}")
return
# 处理每个模型配置
processed_models = 0
for model_name, model_value in model_map.items():
try:
config_file_path = model_value["config"]
module: str = model_value["module"]
# 打包Python模块
print(f"打包模块: {module}")
packer.pack_directory(module.split(".")[0])
# 添加模型配置文件到VFS
folder_packer.add_file(config_file_path, config_file_path)
# 读取模型配置文件获取bmodel路径
with open(config_file_path, "r") as file:
model_config = json.load(file)
bmodel_path = model_config["bmodel_path"]
# 添加bmodel文件到VFS
if os.path.exists(bmodel_path):
folder_packer.add_file(bmodel_path, bmodel_path)
print(f" 添加模型文件: {bmodel_path}")
else:
print(f" 警告: 模型文件不存在 {bmodel_path}")
processed_models += 1
except KeyError as e:
print(f"错误: 模型 '{model_name}' 配置缺少必要字段 - {e}")
continue
except FileNotFoundError as e:
print(f"错误: 找不到模型 '{model_name}' 的文件 - {e}")
continue
except json.JSONDecodeError as e:
print(f"错误: 模型 '{model_name}' 的配置文件格式错误 - {e}")
continue
except Exception as e:
print(f"错误: 处理模型 '{model_name}' 时发生未知错误 - {e}")
continue
if processed_models == 0:
print("警告: 没有成功处理任何模型")
# 显示VFS内容
print("\nVFS内容:")
try:
items = vfs.list_directory("/")
for item in items:
file_type = "DIR" if item["is_directory"] else "FILE"
size_info = f" ({item['file_size']} bytes)" if not item["is_directory"] else ""
print(f" {file_type}: {item['path']}{size_info}")
except Exception as e:
print(f"警告: 无法列出VFS内容 - {e}")
# 复制文件到构建目录
try:
print("\n复制文件到构建目录...")
# 复制主程序
main_py_source = "public/main.py"
if os.path.exists(main_py_source):
shutil.copy(main_py_source, build_dir)
print(f" 复制: {main_py_source}")
else:
print(f" 警告: 主程序文件不存在 {main_py_source}")
# 复制VFS文件
if os.path.exists(vfs_path):
shutil.copy(vfs_path, build_dir)
print(f" 复制: {vfs_path}")
else:
print(f" 警告: VFS文件不存在 {vfs_path}")
# 复制模块库文件
if os.path.exists(mbank_path):
shutil.copy(mbank_path, build_dir)
print(f" 复制: {mbank_path}")
else:
print(f" 警告: 模块库文件不存在 {mbank_path}")
except Exception as e:
print(f"错误: 复制文件失败 - {e}")
return
print(f"\n构建成功完成! 输出目录: {build_dir}")
print(f" 主程序: {os.path.join(build_dir, 'main.py')}")
print(f" VFS文件: {os.path.join(build_dir, os.path.basename(vfs_path))}")
print(f" 模块库: {os.path.join(build_dir, os.path.basename(mbank_path))}")
except Exception as e:
print(f"\n构建过程中发生严重错误: {e}")
print("错误详情:")
traceback.print_exc()
# 清理不完整的构建目录
if os.path.exists(build_dir):
print(f"\n清理不完整的构建目录: {build_dir}")
try:
shutil.rmtree(build_dir, ignore_errors=True)
except Exception as cleanup_error:
print(f"警告: 清理构建目录失败 - {cleanup_error}")
sys.exit(1)
if __name__ == "__main__":
main()
src/sophon_chakcy/get_detector.py
+11
View File
@@ -0,0 +1,11 @@
from . import get_config
import importlib
def get_detector(name: str, is_vfs: bool=False):
config = get_config(r"config.json", is_vfs)
model_map = config["model_map"]
model = importlib.import_module(model_map[name]["module"])
model_config = get_config(model_map[name]["config"], is_vfs)
detector_class = getattr(model, "Detector")
return detector_class, model_config
src/sophon_chakcy/sail.cpython-311-aarch64-linux-gnu.so
Executable
BIN
View File
Binary file not shown.
src/sophon_chakcy/sail.pyi
+3757
View File
File diff suppressed because it is too large Load Diff
uv.lock
Generated
+24 -2
View File
@@ -16,6 +16,15 @@ wheels = [
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/ed/66/388cee1df022233ef157347a7275f0e80cf297bfb344e7680001e5ae2acd/dfss-1.10.4-py2.py3-none-any.whl", hash = "sha256:37bf171350dac531c7185ea5789a6d4b0e63a7cc1852c909cd333e6a2efb0ef5", size = 8556566, upload-time = "2025-07-29T03:11:34.678Z" }, { url = "https://pypi.tuna.tsinghua.edu.cn/packages/ed/66/388cee1df022233ef157347a7275f0e80cf297bfb344e7680001e5ae2acd/dfss-1.10.4-py2.py3-none-any.whl", hash = "sha256:37bf171350dac531c7185ea5789a6d4b0e63a7cc1852c909cd333e6a2efb0ef5", size = 8556566, upload-time = "2025-07-29T03:11:34.678Z" },
] ]
[[package]]
name = "module-bank"
version = "0.1.1"
source = { registry = "https://pypi.tuna.tsinghua.edu.cn/simple" }
sdist = { url = "https://pypi.tuna.tsinghua.edu.cn/packages/73/52/1aeb99df074e827dce1337a9cf89d97ee88b636b1fe5d2db3366acf86dc3/module_bank-0.1.1.tar.gz", hash = "sha256:a168e600166d51bf38a3ee26247a878d65c16ed44109a919cc61ecfa4fe8b79e", size = 58296, upload-time = "2026-01-31T06:37:08.99Z" }
wheels = [
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/3e/ef/7bc3912ae9c8489b147e0392bc284ec66a0fb79974b76ff65145cf235ada/module_bank-0.1.1-py3-none-any.whl", hash = "sha256:dc4312b93f6c1cbecaf5e3b96d56f2af26326ebeb31bf148007556043846a106", size = 10867, upload-time = "2026-01-31T06:37:06.497Z" },
]
[[package]] [[package]]
name = "numpy" name = "numpy"
version = "1.24.4" version = "1.24.4"
@@ -219,14 +228,16 @@ wheels = [
] ]
[[package]] [[package]]
name = "sophon-ai-project" name = "sophon-chakcy"
version = "0.1.0" version = "0.1.0"
source = { virtual = "." } source = { editable = "." }
dependencies = [ dependencies = [
{ name = "module-bank" },
{ name = "numpy", version = "1.24.4", source = { registry = "https://pypi.tuna.tsinghua.edu.cn/simple" }, marker = "python_full_version < '3.9'" }, { name = "numpy", version = "1.24.4", source = { registry = "https://pypi.tuna.tsinghua.edu.cn/simple" }, marker = "python_full_version < '3.9'" },
{ name = "numpy", version = "2.0.2", source = { registry = "https://pypi.tuna.tsinghua.edu.cn/simple" }, marker = "python_full_version == '3.9.*'" }, { name = "numpy", version = "2.0.2", source = { registry = "https://pypi.tuna.tsinghua.edu.cn/simple" }, marker = "python_full_version == '3.9.*'" },
{ name = "numpy", version = "2.2.6", source = { registry = "https://pypi.tuna.tsinghua.edu.cn/simple" }, marker = "python_full_version >= '3.10'" }, { name = "numpy", version = "2.2.6", source = { registry = "https://pypi.tuna.tsinghua.edu.cn/simple" }, marker = "python_full_version >= '3.10'" },
{ name = "opencv-python-headless" }, { name = "opencv-python-headless" },
{ name = "sqlite-vfs" },
] ]
[package.dev-dependencies] [package.dev-dependencies]
@@ -238,8 +249,10 @@ dev = [
[package.metadata] [package.metadata]
requires-dist = [ requires-dist = [
{ name = "module-bank", specifier = ">=0.1.1" },
{ name = "numpy", specifier = ">=1.24.4" }, { name = "numpy", specifier = ">=1.24.4" },
{ name = "opencv-python-headless", specifier = ">=4.12.0.88" }, { name = "opencv-python-headless", specifier = ">=4.12.0.88" },
{ name = "sqlite-vfs", specifier = ">=0.1.0" },
] ]
[package.metadata.requires-dev] [package.metadata.requires-dev]
@@ -247,3 +260,12 @@ dev = [
{ name = "dfss", specifier = ">=1.10.4" }, { name = "dfss", specifier = ">=1.10.4" },
{ name = "pip", specifier = ">=25.0.1" }, { name = "pip", specifier = ">=25.0.1" },
] ]
[[package]]
name = "sqlite-vfs"
version = "0.1.0"
source = { registry = "https://pypi.tuna.tsinghua.edu.cn/simple" }
sdist = { url = "https://pypi.tuna.tsinghua.edu.cn/packages/70/ff/9cbb7f704d59f88ae048bb84a3de19b5d6dd0c1ae21af53396723af18d14/sqlite_vfs-0.1.0.tar.gz", hash = "sha256:cd6cf5cb7d60c1983d9befdc6c4cce8ab050fc4fd215e5e63ece1d2d28f4f9bb", size = 57435, upload-time = "2025-11-08T08:16:01.715Z" }
wheels = [
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/ef/26/a3f51ea9d10efe2c4f85cac78dcbd898b437d2331cf25d470f804e4498b7/sqlite_vfs-0.1.0-py3-none-any.whl", hash = "sha256:c6e27a35e504bb6360fb0d9083d79572367a903788b3b14aea5f1188130e9314", size = 10768, upload-time = "2025-11-08T08:15:18.588Z" },
]