Recent human-computer interaction studies focus on natural interface based on hand/finger movements by tracking body motion using various sensors. In this paper, the authors propose ‘Surface gesture’ recognition method using both 2D surface information and 3D spatial information. ‘Surface gesture’ is composed of continuous gestures using 2D and 3D coordinates in multi spaces. They use a touchscreen to obtain the 2D coordinate and depth sensors to obtain finger coordinates in 3D space. They decompose ‘Surface gesture’ into 3 particular steps and recognize ‘Surface gesture’ with tagging a label on each step.