| | --- |
| | title: RTMO Checkpoint Tester |
| | emoji: 👀 |
| | colorFrom: pink |
| | colorTo: green |
| | sdk: gradio |
| | sdk_version: 5.27.0 |
| | app_file: app.py |
| | pinned: false |
| | license: apache-2.0 |
| | short_description: RTMO PyTorch Checkpoint Tester |
| | --- |
| | |
| | # RTMO PyTorch Checkpoint Tester |
| |
|
| | This HuggingFace Space provides a real-time 2D multi-person pose estimation demo using the RTMO model from OpenMMLab, accelerated with ZeroGPU. It supports both image and video inputs. |
| |
|
| | ## Features |
| |
|
| | - **Remote Checkpoint Selection**: Choose from multiple pre-trained variants (COCO, BODY7, CrowdPose, retrainable RTMO-s) via a dropdown. |
| | - **Custom Checkpoint Upload**: Upload your own `.pth` file; the application auto-detects RTMO-t/s/m/l variants. |
| | - **Image Input**: Upload images for single-frame pose estimation. |
| | - **Video Input**: Upload video files (e.g., `.mp4`, `.mov`, `.avi`, `.mkv`, `.webm`) to perform pose estimation on video sequences and view annotated outputs. |
| | - **Threshold Adjustment**: Fine-tune **Bounding Box Threshold** and **NMS Threshold** sliders to refine detections. |
| | - **Example Images**: Three license-free images with people are included for quick testing via the **Examples** panel. |
| | - **ZeroGPU Acceleration**: Utilizes the `@spaces.GPU()` decorator for GPU inference on HuggingFace Spaces. |
| |
|
| | ## Usage |
| |
|
| | 1. **Upload Image**: Drag-and-drop or select an image in the **Upload Image** component (or choose from **Examples**). |
| | 2. **Upload Video**: Drag-and-drop or select a video file in the **Upload Video** component. |
| | 3. **Select Remote Checkpoint**: Pick a preloaded variant from the dropdown menu. |
| | 4. **(Optional) Upload Your Own Checkpoint**: Provide a `.pth` file to override the remote selection; the model variant is detected automatically. |
| | 5. **Adjust Thresholds**: Set **Bounding Box Threshold** (`bbox_thr`) and **NMS Threshold** (`nms_thr`) to control confidence and suppression behavior. |
| | 6. **Run Inference**: Click **Run Inference**. |
| | 7. **View Results**: |
| | - For images, the annotated image will appear in the **Annotated Image** panel. |
| | - For videos, the annotated video will appear in the **Annotated Video** panel. |
| | The active checkpoint name will appear below. |
| |
|
| | ## Remote Checkpoints |
| |
|
| | The following variants are available out of the box: |
| |
|
| | - `rtmo-s_8xb32-600e_coco` |
| | - `rtmo-m_16xb16-600e_coco` |
| | - `rtmo-l_16xb16-600e_coco` |
| | - `rtmo-t_8xb32-600e_body7` |
| | - `rtmo-s_8xb32-600e_body7` |
| | - `rtmo-m_16xb16-600e_body7` |
| | - `rtmo-l_16xb16-600e_body7` |
| | - `rtmo-s_8xb32-700e_crowdpose` |
| | - `rtmo-m_16xb16-700e_crowdpose` |
| | - `rtmo-l_16xb16-700e_crowdpose` |
| | - `rtmo-s_coco_retrainable` (from Hugging Face) |
| |
|
| | ## Implementation Details |
| |
|
| | - **GPU Decorator**: `@spaces.GPU()` marks the `predict` function for GPU execution under ZeroGPU. |
| | - **Inference API**: Leverages `MMPoseInferencer` from MMPose with `pose2d`, `pose2d_weights`, and category `[0]` for person detection. |
| | - **Monkey-Patch**: Applies a regex patch to bypass `mmdet`’s MMCV version assertion for compatibility. |
| | - **Variant Detection**: Inspects `backbone.stem.conv.conv.weight` channels in the checkpoint to select the correct RTMO variant. |
| | - **Checkpoint Management**: Remote files are downloaded to `/tmp/{key}.pth` on demand; uploads use the provided local path. |
| | - **Image & Video Support**: The `predict` function automatically handles both image and video inputs, saving annotated frames or video to `/tmp/vis` and displaying them in the UI. |
| | - **Output**: Saves visualization images or videos to `/tmp/vis` and displays them in the UI panels. |
| |
|
| | ## Files |
| |
|
| | - **app.py**: Main Gradio application script. |
| | - **requirements.txt**: Python dependencies, including MMCV and MMPose. |
| | - **README.md**: This documentation file. |
| |
|
| |
|
