Motion Segmentation Using Global and
Local Sparse Subspace Optimization
Michael Ying Yang, Hanno Ackermann, Weiyao Lin, Sitong Feng, and Bodo Rosenhahn
Abstract
In this paper, we propose a new framework for segment-
ing feature-based moving objects under the affine subspace
model. Since the feature trajectories are high-dimensional
and contain the noise, we first apply the sparse
PCA
to repre-
sent the original trajectories with a low-dimensional global
subspace, which consists of the orthogonal sparse principal
vectors. Then, the local subspace separation is obtained
using automatically searching the sparse representation of
the nearest neighbors for each projected data. In order to
refine the local subspace estimation and deal with the miss-
ing data problem, we propose an error estimation function to
encourage the projected data that span a same local subspace
to be clustered together. Finally, the segmentation of differ-
ent motions is achieved through the spectral clustering on
an affinity matrix, which is constructed with both the error
estimation and the sparse neighbor optimization. We evalu-
ate our proposed framework by comparing it to other motion
segmentation algorithms. Our method achieves improved
performance on state-of-the-art benchmark datasets.
Introduction
Motion segmentation is an essential task for understanding
the dynamic scenes and other computer vision applications
[1].[2]. Particularly, motion segmentation aims to decompose
a video into different regions according to different moving
objects that tracked throughout the video. In case of feature
extraction for all the moving objects from the video, segmenta-
tion of different motions is equivalent to segment the extract-
ed feature trajectories into different clusters. One example of
feature-based motion segmentation is presented in Figure 1.
Generally, the algorithms of motion segmentation are classi-
fied into two categories [4]: affinity-based methods and sub-
space-based methods. The affinity-based methods focus on com-
puting the correspondences of each trajectory pair, whereas the
subspace-based approaches use multiple subspaces to model
the multiple moving objects in the video, and the segmentation
of different motions is accomplished through subspace cluster-
ing. Recently, some affinity-based methods [4] [5] are proposed
to cluster the trajectories with unlimited number of missing
data. However, the computational cost is very high. Whereas,
the subspace-based methods [6] [7] have been developed to
reconstruct the missing trajectories with their sparse representa-
tion. The drawback is that they are sensitive to the real video
which contains a large number of missing trajectories. Most of
the existing subspace-based methods still fall their robustness
for handling missing features. Thus, there is an intense demand
to explore a new subspace-base algorithm that can not only
segment multiple kinds of motions, but also handle the missing
and corrupted trajectories from the real video.
Contributions
We propose a new framework with subspace models for
segmenting different types of moving objects from a video
under the affine camera. We cast the motion segmentation as
a two stage subspace estimation: the global and local sub-
space estimation. Sparse
PCA
[8] is adopted for optimizing
the global subspace in order to defend the noise and outliers.
Meanwhile, we seek a sparse representation for the near-
est neighbors in the global subspace for each data point that
span a same local subspace. In order to solve the missing data
problem and refine the local subspace estimation, we build
the affinity graph for the spectral clustering with a novel error
estimation function. To the best of our knowledge, our frame-
work is the first one to simultaneously optimize the global
and local subspace with sparse representation.
The remaining sections are organized as follows. The
related works are discussed in the next Section, followed
by the basic subspace models for motion segmentation. The
proposed approach is described in detail followed by the ex-
perimental results are presented leading to the is conclusions.
Michael Ying Yang is with the University of Twente, ITC,
Hengelosestaat 99, Enschede, The Netherlands (michael.
).
Hanno Ackermann, Sitong Feng, and Bodo Rosenhahn are
with Leibniz University Hannover.
Weiyao Lin is with the Shanghai Jiao Tong University.
Photogrammetric Engineering & Remote Sensing
Vol. 83, No. 11, November 2017, pp. 769–778.
0099-1112/17/769–778
© 2017 American Society for Photogrammetry
and Remote Sensing
doi: 10.14358/PERS.83.10.769
Figure 1. Example results of the motion segmentation on the real traffic video
cars9.avi
from the Hopkins 155 dataset [3].
PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING
November 2017
769
