Dynamic Spectrum Access (DSA) scheme in Cognitive Radio (CR) can solve the current problem of scarce spectrum resource effectively, in which the unlicensed users (i.e. Second Users, SUs) can access the licensed spectrum in opportunistic ways without interference to the licensed users (i.e. Primary Users, PUs). However, SUs have to vacate the spectrum because of PUs coming, in this case the spectrum switch occurs, and it leads to the increasing of SUs' delay. In this paper, the authors proposed a Variable Service Rate (VSR) scheme with the switch buffer as to real-time traffic (such as VoIP, Video), in order to decrease the average switch delay of SUs and improve the other performance.