Computer Vision Common code

This code is designed to be shared across projects

Setup

Relative imports

If you are using pycharm, then add common as Sources Root (right click on common, then Mark Directory As > Sources Root).

It will enable the relative imports.

Requirements

Install the required python modules

python -m pip install -r requirements.txt

Dataset

Twitch

Idea

There are a lot of video posted by RoboMaster on their twitch channel.

But, there are 2 issues to convert it to an exploitable dataset:

• Videos are very long (a dozen of hours per video)
• Only short sequences (per chunks of ~5s) are from robot views

We found that it was possible to detect the HUD on the videos, to check if the image is a robot view or not.

Setup

ffmpeg

To split the video into frames, we use ffmpeg.

Note: If you get an error using ffmpeg, be sure that you installed the right python package (it should be ffmpeg-python, not ffmpeg or python-ffmpeg)

MAC

brew install ffmpeg

Windows

You should follow this tutorial.

Twitch Video Downloader

You have 2 options to download the videos:

• TwitchLeecher. Unfortunately, it is only available on Windows.
• 4k Video Downloader. It works on windows, Mac and Linux.

Procedure to process a video

1. First, go on the google sheet, and choose a video that nobody already did, and put your name in the 2nd column
2. Download the video, in 720p
3. Rename it using the video id on twitch, and place it in ../dataset/twitch/videos
4. Launch the python script ./research/scripts/extract_robots_views_from_video.py, with the video id as parameter (In Pycharm, Run > Edit Configurations..., then in parameters enter the id). You can put multiple video ids by separating them with spaces.

The frames will appear in the ../dataset/twitch/robots-views folder.