Pedestrian tracking and detection of crowd abnormal activity under dynamic and complex background using Intelligent Video Surveillance (IVS) system are beneficial for security in public places. This paper presents a pedestrian tracking method combing Histogram of Oriented Gradients (HOG) detection and particle filter. This method regards the particle filter as the tracking framework, identifies the target area according to the result of HOG detection and modifies particle sampling constantly. The authors' method can track pedestrians in dynamic backgrounds more accurately compared with the traditional particle filter algorithms. Meanwhile, a method to detect crowd abnormal activity is also proposed based on a model of crowd features using Mixture Of Gaussian (MOG).