In this paper, the authors are moving the mouse cursor on the screen without using any hardware which is used very often now-a-days i.e. mouse. They use the newly born technology named sixth sense technology for this purpose. They implement computer mouse movement through finger by image grabbing using sixth sense technology which gets processed in Matlab. By applying vision technology and controlling the mouse by natural hand gestures and color based sensing, they simulated the mouse system.