Motion capture of martial arts movements have been a major component in video games and film visual effects, typically involving cumbersome suits for the martial artists and an array of expensvie cameras.

What if all you needed was a single camera? Using a computer vision model trained to recognize a person’s body parts in 2D images and locate those body parts’ 3D locations, it can be possible to generate a 3D animation of a martial artist movements using only a single video capturing those moves.

Ideally, this means there’s no more need for motion capture suits.

After takingb both a robot perception course and a computer graphics course at NYU Tandon the same semester, each of which assigned a final project, I decided to combine the two: generate 3D human figures with computer vision libraries, then animate them in 3D using OpenGL.

Full 3D pose and animation pipeline code can be found on Github, based on rayat137’s Pose_3D code for 3D human pose generation.