In this paper, the authors present the real-time implementation of security system using face detection and recognition using FPGA spartan 3AN evolution kit. The FPGA provides flexibility in both the software as well as hardware in programmability and run-time re-configurability performance. It performs the real time user authentication (i.e.) face detection and recognition using the skin depth based Principle Component Analysis (PCA) algorithm. According to the comparison result the FPGA spartan-3AN evaluation kit triggers certain actions.