A novel decomposed transceiver design scheme is developed for multiuser non-regenerative MIMO relay downlink systems. With a decomposed relay structure, the whole system can be decomposed into a point-to-point and a point-to-multipoint system, thus the transceiver design can be individually preceded for the first and second hop. In the first hop, the transceiver is derived to minimize the Mean Square Error (MSE) of symbol estimations with equal MSEs of all data streams. While for the second hop, a modified Block Diagonalization (BD) and Geometric Mean Decomposition (GMD) based scheme is proposed to design the transceiver. Simulation results demonstrate the proposed method can effectively improve the BER performance.