Abstract

This paper proposes to enhance low resolution dynamic depth videos containing freely non–rigidly moving objects with a new dynamic multi–frame super–resolution algo-rithm. Existent methods are either limited to rigid objects, or restricted to global lateral motions discarding radial dis-placements. We address these shortcomings by accounting for non–rigid displacements in 3D. In addition to 2D opti-cal flow, we estimate the depth displacement, and simulta-neously correct the depth measurement by Kalman filtering. This concept is incorporated efficiently in a multi–frame super–resolution framework. It is formulated in a recursive manner that ensures an efficient deployment in real–time. Results show the overall improved performance of the pro-posed method as compared to alternative approaches, and specifically in handling relatively large 3D motions. Test examples range from a full moving human body to a highly dynamic facial video with varying expressions. 1.