In this paper, the authors introduce principles of software based on-line testing and monitoring of Multi-Processors System-on-Chip (MPSoC). The key idea is to have two applications running at the same time on the same hardware architecture: the Functional APPlication (FAPP) and a non-intrusive software/hardware Instrumentation APPlication (IAPP). The main goal of the instrumentation application is to detect and react to software or hardware malfunction. The two applications (FAPP is a MJPEG decoder) are mapped on a MPSoC integrating 10 processors. The cost of the on-line test and monitoring scheme is given in terms of performance overhead.