# tensorRT Web部署框架

**Repository Path**: altria1122/tensorRT-web

## Basic Information

- **Project Name**: tensorRT Web部署框架
- **Description**: No description available
- **Primary Language**: Unknown
- **License**: Not specified
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2025-07-18
- **Last Updated**: 2025-08-14

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# C++ TensorRT OCR Service

基于C++ TensorRT AI部署框架，针对原有API进行封装。

## 📝 项目说明

这是一个演示项目，展示如何使用C++和TensorRT构建高性能的AI推理服务中台。项目目标：

- 支持动态批次
- 利用TensorRT获得更好的推理性能和并发能力
- 提供Web API接口，方便其他服务调用（画饼中）

项目进度会及时与博客进行同步，对代码流程进行大致说明：

https://altria1122.github.io/2025/07/18/technology/202507/TensorRT%20&&%20Web%20C++/

## 🔧 当前状态


- ✅ TensorRT推理引擎集成
- ✅ yoloV8m 模型接入（有demo）
- 🚧 OCR模型接入 (mnist模型可用于测试)

## 🛠️ 技术栈

- **开发语言**: C++
- **推理引擎**: TensorRT 
- **构建工具**: CMake

## 📋 环境要求

```bash
- C++17 编译器
- CMake >= 4.0.3
- CUDA Toolkit
- TensorRT 
```

## 🚀 快速开始

如果是Windows系统，推荐使用wsl2安装Ubuntu，以及对应的开发环境，在Windows的Clion中可以选择wsl中的编译链。


## 当前进度

在yoloV8m模型下的8路推理结果如下，每一路约为17帧/s，float16,支持动态批次
```
=== Batch #4094 Performance ===
Batch size: 8

Time breakdown:
  Data preparation: 0 ms
  Model inference: 53 ms
  Result processing: 0 ms
  Total processing: 53 ms
  Per-frame inference: 6.625 ms

Batch composition:
  Stream 0 (1): 1 frames
  Stream 1 (2): 1 frames
  Stream 2 (3): 1 frames
  Stream 4 (5): 1 frames
  Stream 5 (6): 1 frames
  Stream 6 (7): 2 frames
  Stream 7 (8): 1 frames

Frame latency:
  Average wait time: 231 ms
  Max wait time: 283 ms

Per-stream statistics:
  Stream 0 (1):
    - Captured: 5704 (23.9664 fps)
    - Processed: 4029 (16.9286 fps)
    - Dropped: 1669 (29.2602%)
    - Time since last process: 0 ms
  Stream 1 (2):
    - Captured: 5516 (23.1765 fps)
    - Processed: 4186 (17.5882 fps)
    - Dropped: 1324 (24.0029%)
    - Time since last process: 0 ms
  Stream 2 (3):
    - Captured: 5547 (23.3067 fps)
    - Processed: 4203 (17.6597 fps)
    - Dropped: 1338 (24.1211%)
    - Time since last process: 0 ms
  Stream 3 (4):
    - Captured: 5420 (22.7731 fps)
    - Processed: 4131 (17.3571 fps)
    - Dropped: 1282 (23.6531%)
    - Time since last process: 53 ms
  Stream 4 (5):
    - Captured: 5527 (23.2227 fps)
    - Processed: 3987 (16.7521 fps)
    - Dropped: 1533 (27.7366%)
    - Time since last process: 0 ms
  Stream 5 (6):
    - Captured: 5427 (22.8025 fps)
    - Processed: 4129 (17.3487 fps)
    - Dropped: 1292 (23.8069%)
    - Time since last process: 0 ms
  Stream 6 (7):
    - Captured: 5350 (22.479 fps)
    - Processed: 3951 (16.6008 fps)
    - Dropped: 1393 (26.0374%)
    - Time since last process: 0 ms
  Stream 7 (8):
    - Captured: 5497 (23.0966 fps)
    - Processed: 4137 (17.3824 fps)
    - Dropped: 1354 (24.6316%)
    - Time since last process: 0 ms

Queue status: 50 frames waiting
```


## 🔄 后续改进

- **Web框架**: 从httplib.h迁移到更强大的框架， 比如：Drogon？
- **容器化部署**: 添加Docker支持
- **监控和日志**: 完善服务监控
- **API文档**: 添加Swagger/OpenAPI文档

---

**注意**: 这是一个开发中的演示项目，仅供学习和参考使用。