feat: 按算法生成模板并支持从 Git LFS 下载模型

- init 新增 yolo26/resnet/arcface/lprnet/face-recognition 模板(多模型人脸检测+识别)
- 新增 embedding/text 输出类型与模板资源(COCO、字符表等)
- 新增 algpkg download-models:从 Gitea LFS 下载 models/BM1688 并做 SHA256 校验
- 补充单元测试与 README
This commit is contained in:
linaro
2026-10-17 20:33:00 +08:00
parent 897aa73bbf
commit 211341cbbb
7 changed files with 2220 additions and 110 deletions
+67 -2
View File
@@ -35,8 +35,12 @@ worker 侧 `workers/video/pyproject.toml` 以**路径依赖**指向本项目(`
# ① 生成脚手架(目录骨架 + manifest 模板 + 可运行示例 + 自测说明) # ① 生成脚手架(目录骨架 + manifest 模板 + 可运行示例 + 自测说明)
algpkg init /tmp/smoke-detect --id demo.smoke-detect --name 抽烟检测 --version 1.0.0 algpkg init /tmp/smoke-detect --id demo.smoke-detect --name 抽烟检测 --version 1.0.0
# 按算法生成(推荐):YOLO26 模板已内置完整 SAIL 前后处理与 COCO 80 类标签
algpkg init /tmp/yolo26-demo --id demo.yolo26 --name YOLO26检测 --algorithm yolo26
# ② 实现 /tmp/smoke-detect/<包内代码目录>/__init__.py 的 load() / infer() # ② 实现 /tmp/smoke-detect/<包内代码目录>/__init__.py 的 load() / infer()
# 模型放 /tmp/smoke-detect/models/,或打包时用 --model 指定 # yolo26 模板已实现,无需再写;模型放 /tmp/yolo26-demo/models/
# 或打包时用 --model 指定
# ③ 打包 + 自检(不需要设备) # ③ 打包 + 自检(不需要设备)
algpkg build /tmp/smoke-detect --out demo.smoke-detect-1.0.0.mbank algpkg build /tmp/smoke-detect --out demo.smoke-detect-1.0.0.mbank
@@ -44,6 +48,64 @@ algpkg inspect demo.smoke-detect-1.0.0.mbank # 清单/模型/快照/
algpkg verify demo.smoke-detect-1.0.0.mbank # 结构与模型校验 algpkg verify demo.smoke-detect-1.0.0.mbank # 结构与模型校验
``` ```
### 2.1 算法模板(`algpkg init --algorithm/--template`)
| 模板 | 生成内容 |
| ---- | -------- |
| `generic`(默认) | 目录骨架 + 占位 `load()/infer()`,自己按模型补前后处理 |
| `yolo26` | **可直接运行的 YOLO26 检测包**:`load()` 构造 SAIL 引擎;`infer()` 完成 letterbox 前处理 → 推理 → 置信度过滤/坐标反变换;manifest 预置 `yolo26s_fp16_1b` 模型、`conf/interval/draw` 参数与 COCO 80 类 labels |
| `resnet` | **可直接运行的 ResNet 图像分类包**:resize + ImageNet mean/std 归一化 → 推理 → softmax/top-1;分类结果用整帧框承载(score/类别),绘制与告警链路无需改动;manifest 预置 `resnet50_fp32_1b` 模型、`conf/interval/draw` 参数与 1000 类占位 labels |
| `arcface` | **ArcFace 人脸特征提取包**:RGB → resize 112×112 → 推理 → L2 归一化;输出 512 维 embedding 放在 `AlgoResult.meta["embedding"]`,`output.type=embedding`;manifest 预置 `arcface_resnet50_fp32_1b` 模型与 `interval` 参数 |
| `lprnet` | **LPRNet 车牌识别包**:BGR → resize 到模型输入 → `(x-127.5)/128` → CTC 贪心解码;车牌字符串放在 `AlgoResult.meta["text"]`,`output.type=text`;manifest 预置 `lprnet_fp32_1b` 模型、`interval` 参数与字符表 `assets/chars.txt` |
| `face-recognition` | **人脸检测 + 识别双模型包**:RetinaFace 检测人脸 → 按框外扩裁脸 → ArcFace 提取 512 维特征;`boxes` 为人脸框,`meta["embeddings"]` 与框逐项对应;manifest 同时声明 `face_detector` 与 `arcface` 两份模型 |
`--algorithm` 也接受 `yolov26`、`yolo26-detect`、`resnet50`、`classifier`、`arc-face`、
`face-embedding`、`lpr`、`license-plate`、`face`、`retinaface-arcface` 等别名;
`--kind` 缺省沿用模板(yolo26/face-recognition → `detector`,其余 → `classifier`),
显式给出可覆盖。模型放到模板提示的路径即可打包,不在源码目录时用
`--model <模型名>=<磁盘路径>` 指定实际文件:
```bash
algpkg init /tmp/yolo26-demo --id demo.yolo26 --name YOLO26检测 --algorithm yolo26
algpkg build /tmp/yolo26-demo --out demo.yolo26-1.0.0.mbank \
--model yolo26s_fp16_1b=/path/to/yolo26s_fp16_1b.bmodel
algpkg init /tmp/resnet-demo --id demo.resnet --name ResNet分类 --algorithm resnet
algpkg build /tmp/resnet-demo --out demo.resnet-1.0.0.mbank \
--model resnet50_fp32_1b=/path/to/resnet50_fp32_1b.bmodel
algpkg init /tmp/arcface-demo --id demo.arcface --name ArcFace特征 --algorithm arcface
algpkg build /tmp/arcface-demo --out demo.arcface-1.0.0.mbank \
--model arcface_resnet50_fp32_1b=/path/to/arcface_resnet50_fp32_1b.bmodel
algpkg init /tmp/lprnet-demo --id demo.lprnet --name 车牌识别 --algorithm lprnet
algpkg build /tmp/lprnet-demo --out demo.lprnet-1.0.0.mbank \
--model lprnet_fp32_1b=/path/to/lprnet_fp32_1b.bmodel
algpkg init /tmp/face-demo --id demo.face --name 人脸检测识别 --algorithm face-recognition
algpkg build /tmp/face-demo --out demo.face-1.0.0.mbank \
--model face_detector=/path/to/retinaface_mobilenet0.25_fp32_1b.bmodel \
--model arcface=/path/to/arcface_resnet50_fp32_1b.bmodel
```
### 2.2 下载演示模型(`algpkg download-models`)
仓库里的 5 个 BM1688 演示模型(YOLO26 / ResNet50 / ArcFace / LPRNet / RetinaFace,
共约 299 MB)通过 **Git LFS** 存放在 `models/BM1688/`。没有 `git-lfs`、或不想拉取
整个仓库时,可直接从 Gitea 的 LFS 端点下载(自动 SHA256 校验;已存在且正确的文件跳过):
```bash
algpkg download-models # 全部 5 个 → models/BM1688
algpkg download-models --model yolo26 --model resnet
algpkg download-models --dest /tmp/models --force # 自定义目录 / 强制重下
# 私有仓库或自定义地址:
algpkg download-models --url http://<gitea>/<owner>/<repo>.git --token <TOKEN>
```
默认仓库地址可用 `ALGPKG_REPO_URL` 覆盖,token 也可用 `GITEA_TOKEN` 环境变量提供。
模型名支持 `yolo26`、`resnet`、`arcface`、`lprnet`、`retinaface` 及常用别名
(`yolov26`、`resnet50`、`lpr`、`face-detector` 等)。模型需已 push 到远端 LFS。
## 3. 本地测试(不需要 worker) ## 3. 本地测试(不需要 worker)
```bash ```bash
@@ -144,10 +206,13 @@ algpkg-sdk/
├── src/algpkg_sdk/ ├── src/algpkg_sdk/
│ ├── api.py 接口契约(Algorithm / AlgoContext / AlgoResult)+ 别名注册 │ ├── api.py 接口契约(Algorithm / AlgoContext / AlgoResult)+ 别名注册
│ ├── manifest.py 清单字段与校验(models / params_schema / alarm / legacy…) │ ├── manifest.py 清单字段与校验(models / params_schema / alarm / legacy…)
│ ├── models.py download-models:从 Git LFS 拉取 models/BM1688 演示模型
│ ├── loader.py AlgoPackage:读 svfs、挂代码(试加载)、构造实例 │ ├── loader.py AlgoPackage:读 svfs、挂代码(试加载)、构造实例
│ ├── builder.py 打包(module_bank 打代码 + svfs 放模型/资源 + 快照=版本 + 签名) │ ├── builder.py 打包(module_bank 打代码 + svfs 放模型/资源 + 快照=版本 + 签名)
│ ├── testkit.py 本地测试台:check / run / compare(无需 worker) │ ├── testkit.py 本地测试台:check / run / compare(无需 worker)
│ └── cli.py algpkg build|inspect|verify|init|list|check|run|compare │ ├── templates.py init 算法模板(generic / yolo26 / resnet / arcface / lprnet / face-recognition)
│ └── cli.py algpkg build|inspect|verify|init|list|check|run|compare|download-models
├── models/BM1688/ Git LFS:5 个 BM1688 演示模型(可 download-models 拉取)
├── examples/person-count/ 完整示例:人员计数(自带实现,只依赖本 SDK) ├── examples/person-count/ 完整示例:人员计数(自带实现,只依赖本 SDK)
│ └── … │ └── …
└── tests/ pytest(无设备跑静态,有设备跑推理,自动跳过) └── tests/ pytest(无设备跑静态,有设备跑推理,自动跳过)
+9 -3
View File
@@ -36,8 +36,11 @@ SDK_MODULE_NAME = "video_algpkg_api"
#: 算法包类型(manifest.kind)——决定 UI 呈现与结果渲染方式。 #: 算法包类型(manifest.kind)——决定 UI 呈现与结果渲染方式。
KINDS = ("detector", "classifier", "counter", "tracker") KINDS = ("detector", "classifier", "counter", "tracker")
#: 结果类型(manifest.output.type)。 #: 结果类型(manifest.output.type)。`embedding` = 特征向量(如 ArcFace),
OUTPUT_TYPES = ("boxes", "count", "counter") #: 结果放在 :class:`AlgoResult` 的 ``meta["embedding"]``(列表);`text` = 文本识别
#: 结果(如 LPRNet 车牌号),放在 ``meta["text"]``;worker 的任务渲染仍以 boxes
#: 为主,embedding/text 供后续比对、检索或上层业务逻辑使用。
OUTPUT_TYPES = ("boxes", "count", "counter", "embedding", "text")
class AlgoError(RuntimeError): class AlgoError(RuntimeError):
@@ -51,7 +54,10 @@ class AlgoResult:
—— 绘制、告警、统计都基于它;无框的算法(纯计数)留 None; —— 绘制、告警、统计都基于它;无框的算法(纯计数)留 None;
- ``counts``:计数类算法的分项计数(如 ``{"person": 3}``;无分项可用 ``{"": n}``); - ``counts``:计数类算法的分项计数(如 ``{"person": 3}``;无分项可用 ``{"": n}``);
- ``value``:单值结果(如"当前画面人数"); - ``value``:单值结果(如"当前画面人数");
- ``labels``:类别名(下标与 boxes 的 cls 对应;缺省用包内 labels 或 COCO 表)。 - ``labels``:类别名(下标与 boxes 的 cls 对应;缺省用包内 labels 或 COCO 表);
- ``meta``:附加数据(如 ArcFace 的 ``{"embedding": [...], "dim": 512}``、
LPRNet 的 ``{"text": "京A12345"}``;worker 当前不参与绘制/告警,
供程序化调用与后续扩展)。
""" """
__slots__ = ("boxes", "counts", "value", "labels", "meta") __slots__ = ("boxes", "counts", "value", "labels", "meta")
+108 -104
View File
@@ -2,11 +2,12 @@
安装本项目后即可直接用(不需要 worker 仓库):: 安装本项目后即可直接用(不需要 worker 仓库)::
algpkg init <dir> --id my.algo --name 我的算法 # 脚手架 algpkg init <dir> --id my.algo --name 我的算法 [--algorithm ...]
algpkg build <src> --out my.algo-1.0.0.mbank [--model 名=路径] [--encrypt KEY] [--sign KEY] algpkg build <src> --out my.algo-1.0.0.mbank [--model 名=路径] [--encrypt KEY] [--sign KEY]
algpkg inspect <file.mbank> [--json] # 看清单/模型/快照/签名 algpkg inspect <file.mbank> [--json] # 看清单/模型/快照/签名
algpkg verify <file.mbank> [--signature-key KEY] # 结构+模型+(可选)验签 algpkg verify <file.mbank> [--signature-key KEY] # 结构+模型+(可选)验签
algpkg list <dir> algpkg list <dir>
algpkg download-models [--dest models/BM1688] [--model yolo26 ...] # Git LFS 拉模型
本地测试(不需要 worker;`run`/`compare` 需要设备):: 本地测试(不需要 worker;`run`/`compare` 需要设备)::
@@ -25,6 +26,7 @@ from pathlib import Path
from .builder import AlgoBuilder, BuildError, inspect_package, verify_package from .builder import AlgoBuilder, BuildError, inspect_package, verify_package
from .manifest import Manifest, code_package_for from .manifest import Manifest, code_package_for
from .templates import AlgorithmTemplate, get_template, template_summaries
# ====================================================================== 子命令 # ====================================================================== 子命令
@@ -115,7 +117,31 @@ def cmd_list(args: argparse.Namespace) -> int:
return 0 return 0
def cmd_download_models(args: argparse.Namespace) -> int:
from .models import DEFAULT_DEST, ModelDownloadError, download_models
dest = args.dest or DEFAULT_DEST
try:
results = download_models(dest, names=args.model, repo_url=args.url,
token=args.token, username=args.username,
force=args.force)
except (ModelDownloadError, ValueError) as exc:
print(f"✗ 模型下载失败: {exc}", file=sys.stderr)
return 1
downloaded = sum(1 for item in results if item["status"] == "downloaded")
skipped = len(results) - downloaded
print(f"✓ 模型目录: {Path(dest).resolve()}(下载 {downloaded},跳过 {skipped})")
return 0
def cmd_init(args: argparse.Namespace) -> int: def cmd_init(args: argparse.Namespace) -> int:
try:
template = get_template(args.algorithm)
except ValueError as exc:
print(f"✗ {exc}", file=sys.stderr)
for key, summary in template_summaries():
print(f" - {key}: {summary}", file=sys.stderr)
return 2
root = Path(args.dir) root = Path(args.dir)
manifest_path = root / "manifest.json" manifest_path = root / "manifest.json"
if manifest_path.exists() and not args.force: if manifest_path.exists() and not args.force:
@@ -126,119 +152,78 @@ def cmd_init(args: argparse.Namespace) -> int:
(root / code_package).mkdir(parents=True, exist_ok=True) (root / code_package).mkdir(parents=True, exist_ok=True)
(root / "assets").mkdir(parents=True, exist_ok=True) (root / "assets").mkdir(parents=True, exist_ok=True)
manifest_path.write_text(json.dumps(_scaffold_manifest(args, code_package), manifest_path.write_text(json.dumps(_scaffold_manifest(args, code_package, template),
ensure_ascii=False, indent=2) + "\n", encoding="utf-8") ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
(root / code_package / "__init__.py").write_text(_scaffold_code(args), encoding="utf-8") (root / code_package / "__init__.py").write_text(
labels = root / "assets" / "labels.txt" _scaffold_code(args, code_package, template), encoding="utf-8")
labels = root / template.labels_file
if not labels.exists(): if not labels.exists():
labels.write_text("# 每行一个类别名(第 0 行 = 类别 0);按需替换\nobject\n", encoding="utf-8") labels.parent.mkdir(parents=True, exist_ok=True)
labels.write_text(template.labels, encoding="utf-8")
readme = root / "README.md" readme = root / "README.md"
if not readme.exists(): if not readme.exists():
readme.write_text(_scaffold_readme(args, code_package), encoding="utf-8") readme.write_text(_scaffold_readme(args, code_package, template), encoding="utf-8")
print(f"✓ 已生成脚手架: {root}") model_specs = template.model_specs(args.id)
print(f" 1) 把模型放到 {root}/models/(或打包时用 --model 名=<路径> 指定)") print(f"✓ 已生成脚手架: {root}(模板 {template.key}: {template.label})")
print(f" 2) 实现 {code_package}/__init__.py 的 load/infer") if len(model_specs) == 1:
print(f" 3) 打包: workers/video/algpkg build {root} --out {args.id}-{args.version}.mbank") print(f" 1) 把模型放到 {root}/{model_specs[0]['file']}"
f"(或打包时用 --model {model_specs[0]['name']}=<路径> 指定)")
else:
print(" 1) 准备以下模型(放源码目录下,或用 --model 名=<路径> 指定):")
for spec in model_specs:
print(f" {root}/{spec['file']} (--model {spec['name']}=<路径>)")
if template.key == "generic":
print(f" 2) 实现 {code_package}/__init__.py 的 load/infer")
else:
print(f" 2) {code_package}/__init__.py 已按 {template.label} 初始化,按需微调")
print(f" 3) 打包: algpkg build {root} --out {args.id}-{args.version}.mbank")
return 0 return 0
# ====================================================================== 脚手架内容 # ====================================================================== 脚手架内容
def _scaffold_manifest(args: argparse.Namespace, code_package: str) -> dict: def _scaffold_manifest(args: argparse.Namespace, code_package: str,
return { template: AlgorithmTemplate | None = None) -> dict:
"manifest_format": 1, template = template or get_template(getattr(args, "algorithm", "generic"))
"id": args.id, return template.render_manifest(
"name": args.name, algo_id=args.id, name=args.name, version=args.version,
"version": args.version, kind=args.kind or template.kind, vendor=args.vendor,
"api_version": 1, code_package=code_package,
"kind": args.kind, )
"entry": "Algorithm",
"code_package": code_package,
"vendor": args.vendor or "",
"description": f"{args.name}(由 algpkg init 生成)",
"models": [{
"name": "default",
"file": f"models/{args.id}.bmodel",
"precision": "",
"resolution": [640, 640],
"batch": 1,
"default": True,
}],
"input": {"width": 640, "height": 640, "color": "rgb", "layout": "planar",
"mean": [0, 0, 0], "std": [255, 255, 255]},
"output": {"type": "boxes", "format": "N6", "classes_file": "assets/labels.txt"},
"params_schema": [
{"key": "conf", "type": "float", "default": 0.25, "min": 0.01, "max": 0.99,
"step": 0.01, "label": "置信度阈值", "desc": "低于该分数的目标被丢弃"},
{"key": "interval", "type": "float", "default": 0.2, "min": 0.02, "max": 60,
"step": 0.02, "label": "抽帧间隔(秒)", "desc": "两次推理之间的最小间隔"},
],
"defaults": {"conf": 0.25, "interval": 0.2},
"alarm": {"type_template": "{class}", "cooldown_key": "algo",
"need_snapshot": True, "need_record": True},
"requires": {"license_features": [], "devices": [], "frameworks": ["sail"]},
"ui": {"order": 100, "tags": [], "icon": "mdi-puzzle"},
"legacy": {},
}
def _scaffold_code(args: argparse.Namespace) -> str: def _scaffold_code(args: argparse.Namespace, code_package: str,
return f'''"""算法包实现:{args.name}({args.id})。 template: AlgorithmTemplate | None = None) -> str:
template = template or get_template(getattr(args, "algorithm", "generic"))
只依赖 `video_algpkg_api`(worker 提供的稳定接口)与设备框架(sophon.sail / numpy)。 return template.render_code(
本文件由 `algpkg init` 生成,改完即可打包: algo_id=args.id, name=args.name, version=args.version,
workers/video/algpkg build <本目录> --out {args.id}-{args.version}.mbank kind=args.kind or template.kind, code_package=code_package,
""" )
from __future__ import annotations
import numpy as np
import sophon.sail as sail
from video_algpkg_api import Algorithm as AlgorithmBase, AlgoResult
class Algorithm(AlgorithmBase): def _scaffold_readme(args: argparse.Namespace, code_package: str,
"""{args.name}。""" template: AlgorithmTemplate | None = None) -> str:
template = template or get_template(getattr(args, "algorithm", "generic"))
kind = "{args.kind}" models = template.model_specs(args.id)
if len(models) == 1:
#: 引擎句柄:load() 赋值、close() 释放。类级声明为非 Optional,infer() 里可直接用 step1 = (f"1. 把模型放到 `{models[0]['file']}`(或打包时 "
#: (不必判空);close() 置 None 与声明冲突,那处显式 `type: ignore`——运行时语义就是"丢引用"。 f"`--model {models[0]['name']}=<磁盘路径>` 指定);")
_net: sail.Engine else:
lines = "\n".join(f" - `{m['file']}`(`--model {m['name']}=<磁盘路径>`)"
def load(self) -> None: for m in models)
# 模型字节来自算法包自身(.mbank 内的 models/**),不落盘: step1 = f"1. 把模型放到源码目录下:\n{lines};"
data = self.ctx.model_bytes() if template.key == "generic":
size = len(data) step2 = (f"2. 实现 `{code_package}/__init__.py` 的 `load()`(加载模型字节)与 "
self.ctx.logger.info("加载模型 %s(%d 字节, conf=%s)", f"`infer(frame)`(单帧结果);")
self.ctx.model_name, size, self.ctx.param("conf")) else:
self._net = sail.Engine(data, size, self.ctx.dev_id, sail.IOMode.SYSO) step2 = (f"2. `{code_package}/__init__.py` 已按 **{template.label}** 完成初始化"
self._graph = self._net.get_graph_names()[0] f"(模型加载 + 前/后处理),一般只需按现场调整参数;")
# TODO: 按模型实际输入做前处理(可参考 worker 的 YOLO26._preprocess) notes = f"\n## 模板说明\n\n{template.notes}\n" if template.notes else ""
self._handle = sail.Handle(self.ctx.dev_id)
self._bmcv = sail.Bmcv(self._handle)
def infer(self, frame) -> AlgoResult:
"""单帧推理。返回 AlgoResult(boxes 为 (N,6):x1,y1,x2,y2,score,cls)。
TODO: 前处理 → self._net.process(...) → 后处理(NMS/反变换),
再 return AlgoResult(boxes=boxes, labels=[...])。
下面这行只做"链路可跑通"的占位(空结果),替换为真实实现。
"""
_ = frame
boxes = np.zeros((0, 6), dtype=np.float32)
return AlgoResult(boxes=boxes)
def close(self) -> None:
self._net = None # type: ignore[assignment]
'''
def _scaffold_readme(args: argparse.Namespace, code_package: str) -> str:
return f"""# {args.name}(算法包 {args.id}) return f"""# {args.name}(算法包 {args.id})
由 `algpkg init` 生成。目录结构: 由 `algpkg init` 生成,模板:**{template.key}**({template.summary})。
目录结构:
``` ```
manifest.json 清单(算法元数据 / 模型 / 参数表单 / 告警策略) manifest.json 清单(算法元数据 / 模型 / 参数表单 / 告警策略)
@@ -246,13 +231,13 @@ manifest.json 清单(算法元数据 / 模型 / 参数表单 /
models/ 随包模型(.bmodel;打包时进包,不单独分发) models/ 随包模型(.bmodel;打包时进包,不单独分发)
assets/ 可选资源(labels.txt 类别名等) assets/ 可选资源(labels.txt 类别名等)
``` ```
{notes}
## 开发步骤 ## 开发步骤
1. 把模型放到 `models/`(或打包时 `--model 名=<磁盘路径>` 指定); {step1}
2. 实现 `{code_package}/__init__.py` 的 `load()`(加载模型字节)与 `infer(frame)`(单帧结果); {step2}
3. 打包:`workers/video/algpkg build . --out {args.id}-{args.version}.mbank`; 3. 打包:`algpkg build . --out {args.id}-{args.version}.mbank`;
4. 检查:`workers/video/algpkg inspect {args.id}-{args.version}.mbank`; 4. 检查:`algpkg check {args.id}-{args.version}.mbank`(有设备时加 `--deep` 真加载);
5. 安装到设备:`POST /api/v1/video/algorithm/install`(算法管理页上传亦可), 5. 安装到设备:`POST /api/v1/video/algorithm/install`(算法管理页上传亦可),
随后在任务表单里选择本算法(参数表单由 `manifest.params_schema` 生成)。 随后在任务表单里选择本算法(参数表单由 `manifest.params_schema` 生成)。
@@ -387,6 +372,18 @@ def build_parser() -> argparse.ArgumentParser:
listing.add_argument("--encrypt", metavar="KEY") listing.add_argument("--encrypt", metavar="KEY")
listing.set_defaults(func=cmd_list) listing.set_defaults(func=cmd_list)
download = sub.add_parser("download-models",
help="从 Git LFS 下载 models/BM1688 下的演示模型")
download.add_argument("--dest", default="models/BM1688",
help="下载目录(默认 models/BM1688)")
download.add_argument("--model", action="append", metavar="NAME",
help="模型名/别名/文件名(可多次;缺省 all)")
download.add_argument("--url", help="Git 仓库 URL(默认 Gitea / ALGPKG_REPO_URL)")
download.add_argument("--token", help="Gitea token(默认读 GITEA_TOKEN)")
download.add_argument("--username", default="git", help="Basic 认证用户名(默认 git)")
download.add_argument("--force", action="store_true", help="已存在也重新下载")
download.set_defaults(func=cmd_download_models)
check = sub.add_parser("check", help="静态检查一个包(清单/结构/试加载;不需 worker)") check = sub.add_parser("check", help="静态检查一个包(清单/结构/试加载;不需 worker)")
check.add_argument("mbank") check.add_argument("mbank")
check.add_argument("--deep", action="store_true", help="再实例化入口类并加载默认模型(需设备)") check.add_argument("--deep", action="store_true", help="再实例化入口类并加载默认模型(需设备)")
@@ -428,7 +425,14 @@ def build_parser() -> argparse.ArgumentParser:
init.add_argument("--id", required=True, help="算法 id(小写字母/数字/._-)") init.add_argument("--id", required=True, help="算法 id(小写字母/数字/._-)")
init.add_argument("--name", required=True, help="显示名(如:抽烟检测)") init.add_argument("--name", required=True, help="显示名(如:抽烟检测)")
init.add_argument("--version", default="1.0.0") init.add_argument("--version", default="1.0.0")
init.add_argument("--kind", default="detector", choices=["detector", "classifier", "counter", "tracker"]) init.add_argument("--kind", default=None,
choices=["detector", "classifier", "counter", "tracker"],
help="算法类型(缺省由算法模板决定:yolo26/face-recognition → detector,其余 → classifier)")
init.add_argument("--algorithm", "--template", dest="algorithm", default="generic",
metavar="NAME",
help="算法模板:generic / yolo26 / resnet / arcface / lprnet / "
"face-recognition(RetinaFace+ArcFace 检测识别);"
"均含完整 SAIL 前后处理")
init.add_argument("--vendor", default="") init.add_argument("--vendor", default="")
init.add_argument("--code-package", default="", help="包内代码模块名(缺省由 id 推导)") init.add_argument("--code-package", default="", help="包内代码模块名(缺省由 id 推导)")
init.add_argument("--force", action="store_true") init.add_argument("--force", action="store_true")
+1 -1
View File
@@ -17,7 +17,7 @@
| `code_package` | 包代码的模块名(`modules/<code_package>/…`;缺省由 id 推导 `alg_<id>`) | | `code_package` | 包代码的模块名(`modules/<code_package>/…`;缺省由 id 推导 `alg_<id>`) |
| `models`* | `[{name, file, precision, resolution, batch, default}]`,含唯一 default | | `models`* | `[{name, file, precision, resolution, batch, default}]`,含唯一 default |
| `input` | `{width,height,color,layout,mean,std}`(前处理契约,展示用) | | `input` | `{width,height,color,layout,mean,std}`(前处理契约,展示用) |
| `output` | `{type: boxes|count|counter, format, classes_file}` | | `output` | `{type: boxes|count|counter|embedding|text, format, classes_file}` |
| `params_schema` | `[{key,type,default,min,max,step,label,desc,options}]`(任务参数表单来源) | | `params_schema` | `[{key,type,default,min,max,step,label,desc,options}]`(任务参数表单来源) |
| `defaults` | 默认参数(conf/nms/interval/draw 等,任务未显式给值时用) | | `defaults` | 默认参数(conf/nms/interval/draw 等,任务未显式给值时用) |
| `alarm` | `{type_template,cooldown_key,need_snapshot,need_record}`(告警生成策略) | | `alarm` | `{type_template,cooldown_key,need_snapshot,need_record}`(告警生成策略) |
+258
View File
@@ -0,0 +1,258 @@
"""演示模型下载:按 Git LFS 批量接口拉取 `models/BM1688/**`。
模型随本仓库走 Git LFS(见 `.gitattributes`),但 LFS 文件只在克隆时按需拉取;
本模块提供 ``algpkg download-models``,不依赖 ``git-lfs``、也不需要克隆仓库,
直接从 Gitea 的 LFS 端点下载并做 SHA256 校验。
默认仓库地址可用 ``--url`` 或环境变量 ``ALGPKG_REPO_URL`` 覆盖;
私有仓库可用 ``--token``(也读 ``GITEA_TOKEN``)配 ``--username``(默认 ``git``)。
"""
from __future__ import annotations
import base64
import hashlib
import json
import os
import urllib.error
import urllib.parse
import urllib.request
from dataclasses import dataclass
from pathlib import Path
from typing import Iterable
#: 模型随本仓库发布(Gitea);可用 --url / ALGPKG_REPO_URL 覆盖
DEFAULT_REPO_URL = os.environ.get(
"ALGPKG_REPO_URL", "http://124.71.68.6:3000/sophon-edge/algpkg-sdk.git")
#: 默认下载目录(相对当前工作目录)
DEFAULT_DEST = "models/BM1688"
class ModelDownloadError(RuntimeError):
"""模型下载/校验失败(消息面向"怎么修")。"""
@dataclass(frozen=True)
class ModelFile:
"""一个随包发布的模型(oid/size 即 Git LFS 指针里的 SHA256 与字节数)。"""
key: str
filename: str
oid: str
size: int
label: str
@property
def size_mb(self) -> float:
return self.size / 1048576.0
#: 本仓库 models/BM1688 下的演示模型(顺序 = 下载顺序)
MODEL_FILES: dict[str, ModelFile] = {
"yolo26": ModelFile(
"yolo26", "yolo26s_fp16_1b.bmodel",
"085a8e9fc8a72065cb8333ea86d27c61742755b0dd88d5766112e25f60256360",
24928256, "YOLO26 目标检测"),
"resnet": ModelFile(
"resnet", "resnet50_fp32_1b.bmodel",
"3c5131b6bb753c7cfaccf1be42d2a770156470dc1dc86147231b38d1728b0322",
102318696, "ResNet50 图像分类"),
"arcface": ModelFile(
"arcface", "arcface_resnet50_fp32_1b.bmodel",
"7427ad5cece4ac1ec21583a6a67a8aeb58b79e279f9126c027600dbca06282b6",
179355648, "ArcFace 人脸特征"),
"lprnet": ModelFile(
"lprnet", "lprnet_fp32_1b.bmodel",
"f97bd651db73d9c8378adb8d1a1d7bfcfc91e139a83c77c240bbf6ca14394e02",
4017640, "LPRNet 车牌识别"),
"retinaface": ModelFile(
"retinaface", "retinaface_mobilenet0.25_fp32_1b.bmodel",
"a7be883fc021f9c605fd356c9224a05aa5573ac530338cec72cbbf98722af03f",
2082912, "RetinaFace 人脸检测"),
}
#: 常用写法 → 模型 key(也接受完整文件名与 key 本身)
ALIASES: dict[str, str] = {
"yolov26": "yolo26",
"yolo": "yolo26",
"yolo26-detect": "yolo26",
"resnet50": "resnet",
"arc": "arcface",
"arc-face": "arcface",
"face-embedding": "arcface",
"lpr": "lprnet",
"license-plate": "lprnet",
"plate": "lprnet",
"face-detector": "retinaface",
"retina": "retinaface",
}
def resolve_models(names: Iterable[str] | None = None) -> list[ModelFile]:
"""把 `--model` 的名字/别名/文件名解析为模型列表(空/`all` = 全部)。"""
items = []
for raw in names or []:
for part in str(raw).replace(",", " ").split():
if part and part not in items:
items.append(part)
if not items or any(item.lower() == "all" for item in items):
return list(MODEL_FILES.values())
selected: list[ModelFile] = []
for name in items:
key = name.lower()
if key in MODEL_FILES:
spec = MODEL_FILES[key]
else:
by_file = next((m for m in MODEL_FILES.values() if m.filename == name), None)
alias = ALIASES.get(key)
spec = by_file or (MODEL_FILES.get(alias) if alias else None)
if spec is None:
available = "、".join(f"{m.key}({m.filename})" for m in MODEL_FILES.values())
raise ValueError(f"未知模型 {name!r};可选:{available}(或 all)")
if spec not in selected:
selected.append(spec)
return selected
def lfs_batch_url(repo_url: str) -> str:
"""仓库 URL → Git LFS batch API 地址。"""
base = str(repo_url or "").strip().rstrip("/")
if not base:
raise ValueError("仓库 URL 不能为空")
if base.endswith("/info/lfs"):
return base + "/objects/batch"
if not base.endswith(".git"):
base += ".git"
return base + "/info/lfs/objects/batch"
def _auth_headers(token: str | None, username: str | None) -> dict[str, str]:
if not token:
return {}
raw = f"{username or 'git'}:{token}".encode("utf-8")
return {"Authorization": "Basic " + base64.b64encode(raw).decode("ascii")}
def _sha256_file(path: Path, chunk: int = 1 << 20) -> str:
digest = hashlib.sha256()
with path.open("rb") as handle:
while block := handle.read(chunk):
digest.update(block)
return digest.hexdigest()
def _is_valid(path: Path, spec: ModelFile) -> bool:
return (path.is_file() and path.stat().st_size == spec.size
and _sha256_file(path) == spec.oid)
def _post_json(url: str, payload: dict, headers: dict[str, str],
timeout: float = 60.0) -> dict:
request = urllib.request.Request(
url, data=json.dumps(payload).encode("utf-8"), method="POST",
headers={"Accept": "application/vnd.git-lfs+json",
"Content-Type": "application/vnd.git-lfs+json", **headers})
try:
with urllib.request.urlopen(request, timeout=timeout) as response:
return json.loads(response.read().decode("utf-8"))
except urllib.error.HTTPError as exc:
detail = exc.read().decode("utf-8", "replace")[:500]
raise ModelDownloadError(
f"Git LFS 接口返回 HTTP {exc.code}: {detail or exc.reason}") from exc
except urllib.error.URLError as exc:
raise ModelDownloadError(f"无法连接 Git LFS 服务 {url}: {exc.reason}") from exc
def _download_file(href: str, target: Path, spec: ModelFile,
headers: dict[str, str], timeout: float = 300.0) -> None:
target.parent.mkdir(parents=True, exist_ok=True)
tmp = target.with_name(target.name + ".part")
request = urllib.request.Request(href, headers=headers)
digest = hashlib.sha256()
total = 0
try:
with urllib.request.urlopen(request, timeout=timeout) as response, \
tmp.open("wb") as handle:
while block := response.read(1 << 20):
handle.write(block)
digest.update(block)
total += len(block)
if total != spec.size:
raise ModelDownloadError(
f"{spec.filename} 大小不符:期望 {spec.size} 字节,收到 {total}")
if digest.hexdigest() != spec.oid:
raise ModelDownloadError(
f"{spec.filename} SHA256 校验失败(期望 {spec.oid[:12]}…,"
f"收到 {digest.hexdigest()[:12]}…)")
tmp.replace(target)
except urllib.error.HTTPError as exc:
raise ModelDownloadError(
f"下载 {spec.filename} 返回 HTTP {exc.code}: {exc.reason}") from exc
except urllib.error.URLError as exc:
raise ModelDownloadError(f"下载 {spec.filename} 失败: {exc.reason}") from exc
finally:
if tmp.exists():
tmp.unlink()
def download_models(dest: str | Path = DEFAULT_DEST, *,
names: Iterable[str] | None = None,
repo_url: str | None = None,
token: str | None = None,
username: str = "git",
force: bool = False,
log=print) -> list[dict]:
"""下载选中的模型到 `dest`;返回每个模型的状态(downloaded/skipped)。
已存在且 SHA256/大小正确的文件默认跳过;`force=True` 强制重新下载。
"""
selected = resolve_models(names)
target_dir = Path(dest)
target_dir.mkdir(parents=True, exist_ok=True)
token = token or os.environ.get("GITEA_TOKEN") or os.environ.get("ALGPKG_TOKEN")
results: list[dict] = []
missing: list[ModelFile] = []
for spec in selected:
target = target_dir / spec.filename
if not force and _is_valid(target, spec):
log(f"· {spec.filename} 已存在且校验通过,跳过")
results.append({"name": spec.key, "path": str(target),
"size": spec.size, "status": "skipped"})
else:
missing.append(spec)
if missing:
batch_url = lfs_batch_url(repo_url or DEFAULT_REPO_URL)
payload = {"operation": "download", "transfers": ["basic"],
"objects": [{"oid": spec.oid, "size": spec.size} for spec in missing]}
response = _post_json(batch_url, payload, _auth_headers(token, username))
objects = {str(item.get("oid")): item for item in response.get("objects") or []}
for index, spec in enumerate(missing, 1):
item = objects.get(spec.oid)
if item is None:
raise ModelDownloadError(
f"LFS 批量接口未返回 {spec.filename}(模型可能尚未 push 到远端)")
if item.get("error"):
error = item["error"]
raise ModelDownloadError(
f"{spec.filename} 下载被拒绝: {error.get('message') or error}")
action = (item.get("actions") or {}).get("download") or {}
href = action.get("href")
if not href:
raise ModelDownloadError(f"{spec.filename} 没有 download 链接")
href = urllib.parse.urljoin(
(repo_url or DEFAULT_REPO_URL).rstrip("/") + "/", href)
log(f"[{index}/{len(missing)}] 下载 {spec.filename}({spec.size_mb:.1f} MB)…")
_download_file(href, target_dir / spec.filename, spec,
action.get("header") or {})
log(f" ✓ {spec.filename} 完成(SHA256 校验通过)")
results.append({"name": spec.key, "path": str(target_dir / spec.filename),
"size": spec.size, "status": "downloaded"})
return results
__all__ = ["DEFAULT_REPO_URL", "DEFAULT_DEST", "MODEL_FILES", "ALIASES",
"ModelFile", "ModelDownloadError", "resolve_models", "lfs_batch_url",
"download_models"]
File diff suppressed because it is too large Load Diff
+436
View File
@@ -7,12 +7,18 @@
""" """
from __future__ import annotations from __future__ import annotations
import hashlib
import http.server
import importlib.util
import json import json
import subprocess import subprocess
import sys import sys
import tempfile import tempfile
import threading
import types
from pathlib import Path from pathlib import Path
import numpy as np
import pytest import pytest
from algpkg_sdk import (AlgoBuilder, AlgoError, AlgoPackage, AlgoPackageError, Manifest, from algpkg_sdk import (AlgoBuilder, AlgoError, AlgoPackage, AlgoPackageError, Manifest,
@@ -318,6 +324,436 @@ def test_cli_init_build_inspect_check(tmp_path: Path) -> None:
assert cli_main(["list", str(tmp_path)]) == 0 assert cli_main(["list", str(tmp_path)]) == 0
def test_cli_init_yolo26_template(tmp_path: Path) -> None:
"""`--algorithm yolo26` 生成可直接运行的检测包(不是 TODO 占位)。"""
src = tmp_path / "yolo.demo"
assert cli_main(["init", str(src), "--id", "yolo.demo", "--name", "YOLO26 示例",
"--algorithm", "yolo26"]) == 0
manifest = Manifest.parse(json.loads((src / "manifest.json").read_text()))
assert manifest.id == "yolo.demo" and manifest.kind == "detector"
assert manifest.default_model().name == "yolo26s_fp16_1b"
assert manifest.default_model().file == "models/yolo26s_fp16_1b.bmodel"
assert manifest.output["format"] == "N6"
code = (src / manifest.code_package / "__init__.py").read_text()
compile(code, "alg_yolo_demo/__init__.py", "exec") # 语法可编译(不 import 设备库)
assert "class Algorithm" in code and "_preprocess" in code and "_postprocess" in code
assert "TODO" not in code
labels = [line for line in (src / "assets" / "labels.txt").read_text().splitlines()
if line.strip()]
assert len(labels) == 80 and labels[0] == "person" and labels[-1] == "toothbrush"
model = tmp_path / "fake.bmodel"
model.write_bytes(b"FAKE-BMODEL" * 512)
out = tmp_path / "yolo.demo-1.0.0.mbank"
assert cli_main(["build", str(src), "--out", str(out),
"--model", f"yolo26s_fp16_1b={model}", "--quiet"]) == 0
assert cli_main(["verify", str(out), "--json"]) == 0
def test_cli_init_yolo26_alias_and_unknown_template(tmp_path: Path) -> None:
src = tmp_path / "alias.demo"
assert cli_main(["init", str(src), "--id", "alias.demo", "--name", "别名",
"--template", "yolov26"]) == 0
manifest = Manifest.parse(json.loads((src / "manifest.json").read_text()))
assert manifest.default_model().name == "yolo26s_fp16_1b"
bad = tmp_path / "bad.demo"
assert cli_main(["init", str(bad), "--id", "bad.demo", "--name", "未知",
"--algorithm", "nope"]) == 2
assert not (bad / "manifest.json").exists()
def test_cli_init_resnet_template(tmp_path: Path) -> None:
"""`--algorithm resnet` 生成 ResNet 分类包(ImageNet 归一化 + softmax + 整帧框)。"""
src = tmp_path / "resnet.demo"
assert cli_main(["init", str(src), "--id", "resnet.demo", "--name", "ResNet 示例",
"--algorithm", "resnet"]) == 0
manifest = Manifest.parse(json.loads((src / "manifest.json").read_text()))
assert manifest.id == "resnet.demo" and manifest.kind == "classifier"
assert manifest.default_model().name == "resnet50_fp32_1b"
assert manifest.default_model().file == "models/resnet50_fp32_1b.bmodel"
assert manifest.input["mean"] == [0.485, 0.456, 0.406]
assert manifest.schema_defaults()["conf"] == 0.1
code = (src / manifest.code_package / "__init__.py").read_text()
compile(code, "alg_resnet_demo/__init__.py", "exec")
assert "class Algorithm" in code and "_preprocess" in code
assert "_postprocess" in code and "yuv2bgr" in code
labels = [line for line in (src / "assets" / "labels.txt").read_text().splitlines()
if line.strip() and not line.startswith("#")]
assert len(labels) == 1000 and labels[0] == "class_0" and labels[-1] == "class_999"
model = tmp_path / "fake.bmodel"
model.write_bytes(b"FAKE-BMODEL" * 512)
out = tmp_path / "resnet.demo-1.0.0.mbank"
assert cli_main(["build", str(src), "--out", str(out),
"--model", f"resnet50_fp32_1b={model}", "--quiet"]) == 0
assert cli_main(["verify", str(out), "--json"]) == 0
def test_resnet_template_infer_with_fake_sail(tmp_path: Path, monkeypatch) -> None:
"""不依赖设备验证分类后处理:top-1 → 整帧框(score/cls)与阈值过滤。"""
src = tmp_path / "resnet.infer"
cli_main(["init", str(src), "--id", "resnet.infer", "--name", "推理示例",
"--algorithm", "resnet"])
code_path = src / "alg_resnet_infer" / "__init__.py"
# 生成代码只在方法体内使用 sail;空模块足以导入(不需要真实设备)
monkeypatch.setitem(sys.modules, "sophon", types.ModuleType("sophon"))
monkeypatch.setitem(sys.modules, "sophon.sail", types.ModuleType("sophon.sail"))
spec = importlib.util.spec_from_file_location("alg_resnet_infer_under_test", code_path)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
class Frame:
def width(self) -> int:
return 640
def height(self) -> int:
return 480
class Tensor:
def asnumpy(self):
return [[0.1, 3.0, 0.2]] # logits: top-1 = 类别 1
class Net:
def process(self, *args, **kwargs) -> None:
return None
class Bmcv:
def bm_image_to_tensor(self, *args, **kwargs) -> None:
return None
algo = module.Algorithm(ctx=None)
algo._conf = 0.5
algo._labels = ["cat", "dog", "bird"]
algo._output_scale = 1.0
algo._output_tensor = Tensor()
algo._output_tensors = {0: algo._output_tensor}
algo._input_tensor = object()
algo._input_name, algo._graph = "input", "graph"
algo._net = Net()
algo._bmcv = Bmcv()
algo._preprocess = lambda frame: object()
result = algo.infer(Frame())
assert result.boxes.shape == (1, 6)
assert (float(result.boxes[0, 0]), float(result.boxes[0, 2])) == (0.0, 639.0)
assert int(result.boxes[0, 5]) == 1
assert float(result.boxes[0, 4]) > 0.8
assert result.counts == {"dog": 1} and result.value == 1.0
assert result.meta["class_id"] == 1
algo._conf = 0.99 # 低于阈值 → 空结果,不上报告警
assert algo.infer(Frame()).boxes.shape == (0, 6)
def test_cli_init_arcface_template(tmp_path: Path) -> None:
"""`--algorithm arcface` 生成 ArcFace 特征包(embedding 输出、无类别表)。"""
src = tmp_path / "arcface.demo"
assert cli_main(["init", str(src), "--id", "arcface.demo", "--name", "ArcFace 示例",
"--algorithm", "arcface"]) == 0
manifest = Manifest.parse(json.loads((src / "manifest.json").read_text()))
assert manifest.id == "arcface.demo" and manifest.kind == "classifier"
assert manifest.default_model().name == "arcface_resnet50_fp32_1b"
assert manifest.default_model().file == "models/arcface_resnet50_fp32_1b.bmodel"
assert manifest.output["type"] == "embedding"
assert manifest.output["embedding_size"] == 512
assert manifest.schema_defaults()["interval"] == 0.5
code = (src / manifest.code_package / "__init__.py").read_text()
compile(code, "alg_arcface_demo/__init__.py", "exec")
assert "class Algorithm" in code and "_preprocess" in code
assert "meta" in code and "linalg.norm" in code
labels = [line for line in (src / "assets" / "labels.txt").read_text().splitlines()
if line.strip() and not line.startswith("#")]
assert labels == [] # 特征包不需要类别表
model = tmp_path / "fake.bmodel"
model.write_bytes(b"FAKE-BMODEL" * 512)
out = tmp_path / "arcface.demo-1.0.0.mbank"
assert cli_main(["build", str(src), "--out", str(out),
"--model", f"arcface_resnet50_fp32_1b={model}", "--quiet"]) == 0
assert cli_main(["verify", str(out), "--json"]) == 0
def test_arcface_template_infer_with_fake_sail(tmp_path: Path, monkeypatch) -> None:
"""不依赖设备验证特征提取:L2 归一化、512 维 embedding、face 计数。"""
src = tmp_path / "arcface.infer"
cli_main(["init", str(src), "--id", "arcface.infer", "--name", "特征示例",
"--algorithm", "arcface"])
code_path = src / "alg_arcface_infer" / "__init__.py"
monkeypatch.setitem(sys.modules, "sophon", types.ModuleType("sophon"))
monkeypatch.setitem(sys.modules, "sophon.sail", types.ModuleType("sophon.sail"))
spec = importlib.util.spec_from_file_location("alg_arcface_infer_under_test", code_path)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
class Frame:
def width(self) -> int:
return 112
def height(self) -> int:
return 112
class Tensor:
def asnumpy(self):
return [[3.0, 4.0] + [0.0] * 510] # 归一化后前两维 = 0.6, 0.8
class Net:
def process(self, *args, **kwargs) -> None:
return None
class Bmcv:
def bm_image_to_tensor(self, *args, **kwargs) -> None:
return None
algo = module.Algorithm(ctx=None)
algo._output_tensor = Tensor()
algo._input_tensor = object()
algo._input_name, algo._input_shapes = "input", {}
algo._output_name, algo._graph = "output", "graph"
algo._net, algo._bmcv = Net(), Bmcv()
algo._preprocess = lambda frame: object()
result = algo.infer(Frame())
assert result.boxes is None
assert result.counts == {"face": 1} and result.value == 1.0
assert result.meta["dim"] == 512
assert result.meta["embedding"][0] == pytest.approx(0.6)
assert result.meta["embedding"][1] == pytest.approx(0.8)
assert result.meta["norm"] == pytest.approx(1.0)
def test_cli_init_lprnet_template(tmp_path: Path) -> None:
"""`--algorithm lprnet` 生成车牌识别包(text 输出 + 字符表)。"""
src = tmp_path / "lprnet.demo"
assert cli_main(["init", str(src), "--id", "lprnet.demo", "--name", "车牌示例",
"--algorithm", "lprnet"]) == 0
manifest = Manifest.parse(json.loads((src / "manifest.json").read_text()))
assert manifest.id == "lprnet.demo" and manifest.kind == "classifier"
assert manifest.default_model().name == "lprnet_fp32_1b"
assert manifest.default_model().file == "models/lprnet_fp32_1b.bmodel"
assert manifest.default_model().resolution == [94, 24]
assert manifest.output["type"] == "text"
assert manifest.output["chars_file"] == "assets/chars.txt"
assert manifest.schema_defaults()["interval"] == 0.5
code = (src / manifest.code_package / "__init__.py").read_text()
compile(code, "alg_lprnet_demo/__init__.py", "exec")
assert "_preprocess" in code and "_postprocess" in code and "chars.txt" in code
chars = [line for line in (src / "assets" / "chars.txt").read_text().splitlines()
if line.strip() and not line.startswith("#")]
assert len(chars) == 68 and chars[0] == "京" and chars[-1] == "-"
model = tmp_path / "fake.bmodel"
model.write_bytes(b"FAKE-BMODEL" * 512)
out = tmp_path / "lprnet.demo-1.0.0.mbank"
assert cli_main(["build", str(src), "--out", str(out),
"--model", f"lprnet_fp32_1b={model}", "--quiet"]) == 0
assert cli_main(["verify", str(out), "--json"]) == 0
def test_lprnet_template_infer_with_fake_sail(tmp_path: Path, monkeypatch) -> None:
"""不依赖设备验证 CTC 贪心解码:去重、去 blank、空结果。"""
src = tmp_path / "lprnet.infer"
cli_main(["init", str(src), "--id", "lprnet.infer", "--name", "车牌推理",
"--algorithm", "lprnet"])
code_path = src / "alg_lprnet_infer" / "__init__.py"
monkeypatch.setitem(sys.modules, "sophon", types.ModuleType("sophon"))
monkeypatch.setitem(sys.modules, "sophon.sail", types.ModuleType("sophon.sail"))
spec = importlib.util.spec_from_file_location("alg_lprnet_infer_under_test", code_path)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
class Frame:
def width(self) -> int:
return 94
def height(self) -> int:
return 24
class Tensor:
def __init__(self) -> None:
self.logits = np.full((1, 4, 6), -10.0, dtype=np.float32)
def asnumpy(self):
return self.logits
class Net:
def process(self, *args, **kwargs) -> None:
return None
class Bmcv:
def bm_image_to_tensor(self, *args, **kwargs) -> None:
return None
tensor = Tensor()
algo = module.Algorithm(ctx=None)
algo._chars = ["京", "A", "1", "-"]
algo._blank = 3
algo._output_tensor = tensor
algo._input_tensor = object()
algo._input_name, algo._input_shapes = "input", {}
algo._output_name, algo._graph = "output", "graph"
algo._net, algo._bmcv = Net(), Bmcv()
algo._preprocess = lambda frame: object()
# 模型输出 [1, C=4, T=6]:0,0,3(blank),1,2,2 → CTC 去重后 "京A1"
for t, cls in enumerate([0, 0, 3, 1, 2, 2]):
tensor.logits[0, cls, t] = 10.0
result = algo.infer(Frame())
assert result.boxes is None
assert result.meta["text"] == "京A1"
assert result.meta["chars"] == [0, 1, 2]
assert result.counts == {"plate": 1} and result.value == 1.0
tensor.logits = np.full((1, 4, 6), -10.0, dtype=np.float32) # 全是 blank
tensor.logits[0, 3, :] = 10.0
empty = algo.infer(Frame())
assert empty.meta["text"] == "" and empty.meta["chars"] == []
assert empty.counts == {} and empty.value == 0.0
def test_cli_init_face_recognition_template(tmp_path: Path, monkeypatch) -> None:
"""`--algorithm face-recognition` 生成双模型包(检测框 + 512 维 embedding)。"""
src = tmp_path / "face.demo"
assert cli_main(["init", str(src), "--id", "face.demo", "--name", "人脸示例",
"--algorithm", "face-recognition"]) == 0
manifest = Manifest.parse(json.loads((src / "manifest.json").read_text()))
assert manifest.id == "face.demo" and manifest.kind == "detector"
assert [m.name for m in manifest.models] == ["face_detector", "arcface"]
assert manifest.default_model().name == "face_detector"
assert manifest.model("arcface").file == "models/arcface_resnet50_fp32_1b.bmodel"
assert manifest.output["embedding_size"] == 512
assert manifest.schema_defaults()["nms"] == 0.3
assert manifest.schema_defaults()["min_face"] == 16
code = (src / manifest.code_package / "__init__.py").read_text()
compile(code, "alg_face_demo/__init__.py", "exec")
assert "_priorbox" in code and "_detect" in code and "_embed" in code
assert 'model_bytes("arcface")' in code
labels = [line for line in (src / "assets" / "labels.txt").read_text().splitlines()
if line.strip() and not line.startswith("#")]
assert labels == ["face"]
det = tmp_path / "det.bmodel"
det.write_bytes(b"DET" * 2048)
arc = tmp_path / "arc.bmodel"
arc.write_bytes(b"ARC" * 2048)
out = tmp_path / "face.demo-1.0.0.mbank"
assert cli_main(["build", str(src), "--out", str(out),
"--model", f"face_detector={det}",
"--model", f"arcface={arc}", "--quiet"]) == 0
assert cli_main(["verify", str(out), "--json"]) == 0
# 不依赖设备验证先验框/解码/NMS 的集成(官方 RetinaFace 口径)
monkeypatch.setitem(sys.modules, "sophon", types.ModuleType("sophon"))
monkeypatch.setitem(sys.modules, "sophon.sail", types.ModuleType("sophon.sail"))
spec = importlib.util.spec_from_file_location(
"alg_face_detect_under_test", src / "alg_face_demo" / "__init__.py")
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
priors = module._priorbox(640, 640)
assert priors.shape == (16800, 4)
assert np.allclose(priors[0], [4 / 640, 4 / 640, 16 / 640, 16 / 640])
algo = module.Algorithm(ctx=None)
algo._conf, algo._nms, algo._max_faces = 0.5, 0.3, 20
algo._priors = priors
algo._det_w = algo._det_h = 640
loc = np.zeros((1, 16800, 4), dtype=np.float32)
conf = np.zeros((1, 16800, 2), dtype=np.float32)
conf[0, :, 1] = 0.01
conf[0, 0, 1] = 0.9
dets = algo._detect([loc, conf], 1.0, 640, 640)
assert dets.shape == (1, 6)
assert np.allclose(dets[0, :4], [0, 0, 12, 12])
assert dets[0, 4] == pytest.approx(0.9) and dets[0, 5] == 0
def test_model_download_specs_and_aliases() -> None:
from algpkg_sdk import models as model_store
assert [m.key for m in model_store.resolve_models(["all"])] == list(model_store.MODEL_FILES)
assert model_store.resolve_models(["yolov26"])[0].key == "yolo26"
assert model_store.resolve_models(["lpr"])[0].key == "lprnet"
assert model_store.lfs_batch_url("http://host/repo.git").endswith(
"/repo.git/info/lfs/objects/batch")
with pytest.raises(ValueError):
model_store.resolve_models(["nope"])
def test_cli_download_models_via_lfs(tmp_path: Path, monkeypatch, capsys) -> None:
"""本地假 LFS 服务验证下载/校验/跳过(不依赖外网与 git-lfs)。"""
from algpkg_sdk import models as model_store
content = b"FAKE-BMODEL-DOWNLOAD" * 4096
oid = hashlib.sha256(content).hexdigest()
spec = model_store.ModelFile("demo", "demo_model.bmodel", oid, len(content), "测试模型")
monkeypatch.setattr(model_store, "MODEL_FILES", {"demo": spec})
monkeypatch.setattr(model_store, "ALIASES", {"demo": "demo"})
monkeypatch.setenv("NO_PROXY", "127.0.0.1,localhost")
class Handler(http.server.BaseHTTPRequestHandler):
def do_POST(self) -> None: # LFS batch
length = int(self.headers.get("Content-Length") or 0)
payload = json.loads(self.rfile.read(length) or b"{}")
port = self.server.server_port
objects = [{
"oid": item["oid"], "size": item["size"],
"actions": {"download": {
"href": f"http://127.0.0.1:{port}/objects/{item['oid']}"}},
} for item in payload.get("objects", [])]
body = json.dumps({"objects": objects}).encode()
self.send_response(200)
self.send_header("Content-Type", "application/vnd.git-lfs+json")
self.send_header("Content-Length", str(len(body)))
self.end_headers()
self.wfile.write(body)
def do_GET(self) -> None: # LFS object
if self.path.startswith("/objects/"):
self.send_response(200)
self.send_header("Content-Length", str(len(content)))
self.end_headers()
self.wfile.write(content)
else:
self.send_error(404)
def log_message(self, *args) -> None:
return None
server = http.server.ThreadingHTTPServer(("127.0.0.1", 0), Handler)
thread = threading.Thread(target=server.serve_forever, daemon=True)
thread.start()
try:
dest = tmp_path / "models" / "BM1688"
url = f"http://127.0.0.1:{server.server_port}/sophon-edge/algpkg-sdk.git"
assert cli_main(["download-models", "--dest", str(dest),
"--url", url, "--model", "demo"]) == 0
assert (dest / "demo_model.bmodel").read_bytes() == content
assert "下载 1" in capsys.readouterr().out
assert cli_main(["download-models", "--dest", str(dest),
"--url", url, "--model", "demo"]) == 0
assert "跳过 1" in capsys.readouterr().out
assert (dest / "demo_model.bmodel").read_bytes() == content
finally:
server.shutdown()
server.server_close()
thread.join(timeout=5)
def test_cli_check_reports_static_results(tmp_path: Path) -> None: def test_cli_check_reports_static_results(tmp_path: Path) -> None:
"""check 在无设备时也要给出"清单/结构 OK、试加载跳过"的诚实结论。""" """check 在无设备时也要给出"清单/结构 OK、试加载跳过"的诚实结论。"""
src = tmp_path / "chk.demo" src = tmp_path / "chk.demo"