Now-a-days, graphs play a very important role in the field of science and technology. There are some graph algorithms are fundamental to many disciplines and application areas. Large graphs are common in scientific and engineering applications consisting operation on millions of vertices and edges. To have faster execution of such operations parallel computation is very essential to reduce overall computation time. Today's Graphics Processing Units (GPUs) have high computation power and low price. This device can be treated as an array of Single Instruction Multiple Data (SIMD) processors using Compute Unified Device Architecture (CUDA) software interface by NVIDIA, becoming a new programming approach of the General Purpose computing on Graphics Processing Unit (GPGPU).