Business Process Management Systems (BPMS) are sets of tools to support the Business Process Management (BPM) life-cycle that are either offered by one vendor, or multiple vendors offer parts of a BPMS. Business Process Management Systems aim to support the Business Process Management paradigm and to ease legacy application integration. However, do such systems really meet real-world requirements? This paper introduces and discusses a set of criteria which are important for business process management systems and applies these criteria in comparing tools from three important vendors, namely IDS Scheer, Oracle and Intalio based on a real-world case study.