3D spatial layout propagation in a video sequence