HPE3D

Code for human pose estimation with VAE-GAN hybrid model.

Presentation Thesis

Human Pose Estimation - Literature Study

• Concetps
• 3D HPE
  • 2D-3D Lifting
• 2D HPE
• Datasets
• More Stuff

Concepts

• Reprojection Error Optimization - 3D human body model is deformed such that it satisfies a reprojection error
• Direct NN inference - estimate 3D poses directly from images or detected keypoints
• Kinematic Chain Space (KCS) - Projecting 3D human pose into KCS, a contraint is derived that is based on the assumption that the bone lengths are constant. This can be benificial as giving an additional feature matrix to the network, it doesnt have to learn joint length computation and angular constraints on its own KCS for monocular MoCap
• Non rigid structure from motion (NRSfM) - aims to obtain the varying 3D structure and camera motion from uncalibrated 2D point tracks NR-SFM video explanation
• Kinematic Feature updates (in 2D) - encode the kinematic structure via feature updates at coarse resolution RePose
• Real-time: Casading, Long short skip connections XNect
• Ordinal Ranking Ordinal+CVAE DRPose3D - Depth Ranking
• C-VAE Ordinal+CVAE

3D HPE

Camera Distance-aware Top-down Approach for 3D Multi-person Pose Estimation from a Single RGB Image

[Paper] [Code] [Code] [ICCV 2019] [image-2Dheatmap-3D, absolute pose, multi person, pose depth, cascade]

Unsupervised 3D Pose Estimation with Geometric Self-Supervision

[Paper] [CVPR 2019] [2D-3D, self supervised, discriminator, domain adaptation, temporal consistency]

PoseLifter: Absolute 3D human pose lifting network from a single noisy 2D human pose

[Paper] [2019,2020] [image-2D-3D, project w/ focal length, learn human size, absolute pose, noise, errors, cascade]

RepNet: Weakly Supervised Training of an Adversarial Reprojection Network for 3D Human Pose Estimation

[Paper] [Code] [CVPR 2019] [2D-3D, KCS, weakly supervised, adversial training, GAN]

Distill Knowledge from NRSfM for Weakly Supervised 3D Pose Learning

[Paper] [ICCV 2019] [Image features+2D-3D, NRSfM, teacher student, weakly supervised]

C3DPO: Canonical 3D Pose Networks for Non-Rigid Structure From Motion

[Paper] [Code] [ICCV 2019] [2D-3D, NRSFM, canonicalization network, self-supervised]

Unsupervised Adversarial Learning of 3D Human Pose from 2D Joint Locations

[Paper] [Code] [2018] [2D-3D, GAN, unsupervised]

Monocular 3D Human Pose Estimation by Generation and Ordinal Ranking

[Paper] [ICCV 2019] [2Dheatmap-3D, ambigious use of 3D GT, C-VAE, ordinal ranking and score, condition on 2d pose]

Occlusion-Aware Networks for 3D Human Pose Estimation in Video

[Paper] [ICCV 2019] [2Dheatmap-3D, reprojection+GT, adversarial, occlusion, video]

Using GT

2D HPE

RePose: Learning Deep Kinematic Priors for Fast Human Pose Estimation

[Paper] [2020] [kinematic priors]

Datasets

• Human3.6[2014] - 2D and 3D relative joint pose, multi view
• MPI-INF-3DHP[2017] - 2D and 3D relative joint pose, mutli view
• JTA Joint Track Auto[2018] - 2D, 3D, occulusion annotation, synthetic

More Stuff

• Learning to Reconstruct people
• Gyeongsik Moon - Seoul National University
• awesome-human-pose-estimation
• 3d-human-pose-estimation