Data Centers

Data dumps are the dumps

I have recently spend a considerable amount of time on a project replacing an old mainframe system with a new web-based application.

I have recently spend a considerable amount of time on a project replacing an old mainframe system with a new Web-based application. Over the first few weeks the requirements grew from a few screens and reports to a staggering number of screens, reports, CSV and Excel dumps, etc. As we started talking to more and more users, we discovered that over the years they had been using data dumps into Excel, Access and other systems to try and run their business.

And what is worse, is that this "data dump" culture extended to their customers as well. They would send their customers huge CSV files for them to use for reporting and over the years, had customised these files to the point where a whole team of administrators would take the output and re-format it for each customer in Excel. Now, who deserves that job I wonder — sitting for two weeks out of each month re-formatting CSV files into nicely formatted Excel spreadsheets?

So in the end, with a 6-centimetre stack of requirements documents in hand, we decided to do what we should have done in the first place— start from scratch. So, forget about the mainframe processing and replacing every single mainframe report. Forget about the hundreds of Excel spreadsheets and data dumps that have been driving the business.

For developers, starting from a clean slate is sometimes not such a bad thing— and understanding the business is even better. If you can understand the business and the processing and information that is required to run the business, you can cut through years of "We have always done it this way" to get to a solution design that is both concise and elegant.

So there is a happy ending to this story— don't be afraid to wade in to the business that you are building an application around. And don't always take user requirements at face value either. Sometimes an application or process will grow like a vine, until it takes a life of it's own, covering up the real issues (and everything else as well). Don't be afraid to chop your way through the tangle and then or even sow a new seed in it's place. Only this time, don't let the systems or processes go wild— a little pruning now and then can save a lot of headache later!

Editor's Picks

Free Newsletters, In your Inbox