Leadership

Confessions from the PM of a chaotic project

If you've ever had the misfortune of managing a project that spiraled out of control, you'll relate to this story.

Regardless of past success, regardless of how much time and effort goes into a project, and no matter the talents of the team, some projects don't go as planned. And then, there are the projects that are even worse; the ones that simply go insane, plunging teams into chaos and leaving a line department that's unable to operate effectively.

I have the misfortune right now of being caught in the latter situation, and it's not a pleasant place to be for me or for any of the involved parties. Let me start by making it clear that I've been the project manager on this project and, although there are a multitude of reasons that got us to where we are, I've accepted responsibility for the situation and am working with all stakeholders to recover the project and get back on track. I've managed a lot of projects; this one happened to be the largest and has proven to be the one that got the best of me.

The beginning

As is the case with many projects, this particular project -- which is a conversion from one data system to another -- began with great intentions and a laundry list of business problems that had to be solved. A project team was formed, and I was appointed as the project manager by default. I can't go into all the reasons that I assumed the overall responsibility for the project except to say that it was felt that staff in the affected department did not have the knowledge necessary to accomplish the goals that were set out.

As a group, our project team put together a project plan -- which I approved -- that took into account the perceived quality of the data in the legacy system, the amount of time and support we expected to receive from our vendor, and the amount of time that each team member could devote to the project.

To say that I underestimated two key factors would be, in itself, a massive understatement. First, the quality of the data in the legacy system was far worse that we could have ever imagined. Second, the level of support and knowledge we received from the vendor was quite poor, at least at the beginning of the project. After we requested a new resource, we ended up with really good people, but by then, the project was doomed, but I just didn't know it.

From bad to worse

Because of internal delays -- the stakeholder department added projects that were not originally scheduled, which disrupted the schedule for this project -- we, along with the stakeholders, made the decision to postpone the original launch date. However, the initial damage had already been done, and, again, I just didn't realize it yet.

Fast forward to what we had intended to be our launch day. As per plan, we launched with a subset of the data that had to be migrated with the intent to continue to migrate the data that was remaining.

It didn't happen. I can't go into the confluence of events that led to a failure to meet the project goals, but we spent the next three months working night and day to get the data-quality issues resolved and to correct for the delays introduced due to the original vendor resourcing issues and some internal scheduling challenges in the stakeholder department that were originally unknown. Those were just two issues that raised their heads during this project. There were a whole lot of other things, too.

Now, months later, the stakeholder department is in limbo, and although the data-quality issues are close to being worked out, the time it will take to get everything corrected to a point where I feel comfortable is too far away. In this, I am likely erring on the side of caution when I look at the potential for data-quality problems, but, well, that's my job (I'm not a full-time project manager; this was just one duty among too many on my plate at the time).

It's really hard to go into someone's office -- a peer, a superior, a subordinate -- and admit defeat. However, that's exactly what I did. The project in its current form is a failure; again, this is not an easy admission to make and is potentially career threatening. The stakeholder department needs a system that allows them to meet basic operations. In its current form, the new system was not doing that, and it was going to be too long before we would get to that point.

So, we're going back. We're moving back to the original application, but only temporarily. The upside to this challenge is that many people have learned a lot, including:

  • The original data quality was far worse than ever expected. We need to spend more time on data quality this time around to make certain that we don't end up spinning our wheels again. The next time, during status update sessions, I will not hear "Well, we discovered another data issue that we missed during validation."
  • We cannot rely on tightly constrained vendor resources to help us when we have issues that arise.
  • The team needs to be able to focus less on the project and more on their day-to-day duties. We will extend the project time frame so that we don't overburden our staff.

I'm fortunate that I'm working with people that, while unhappy at the state of things, understand the why of where we are and continue to agree that a successful migration is in our long-term best interests. So, although we're moving back to the original system, we're going to once again work to migrate to our new system. This time, since we have a framework from the original migration that is actually quite valuable, we will focus our efforts on data validation in order to avoid some of the data issues that plagued the original migration. Over the past couple of months, the team has learned a lot that will assist them in this process. And, although I got to a point where I no longer had confidence in the outcome, I am highly confident in the new process we've put together to get us to ultimate success.

In speaking with another VP just today -- the VP in charge of the impacted division -- he said, "It sucks that we are where we are, but nobody's died, so there's that." The tone of his comment said it all. He hates this as much as I do (probably more), but realizes that things could be a whole lot worse. We actually have a fallback that, although it's not the prettiest, will get us operational, and he continues to believe that we're on to something good.

Perhaps the biggest lesson learned -- and I've shared with you just a few in this article; there are so, so many more -- is that there comes a time when, in the best interests of the organization, you have to admit defeat, failure, whatever you want to call it, no matter the potential personal price and take a step backward in order to allow the organization to ultimately move ahead.

About

Since 1994, Scott Lowe has been providing technology solutions to a variety of organizations. After spending 10 years in multiple CIO roles, Scott is now an independent consultant, blogger, author, owner of The 1610 Group, and a Senior IT Executive w...

19 comments
robinsdr
robinsdr

In a matrix organizations, the PM is severely limited by functional groups. That is why a company like Apple is so successful. A project is independent and autonomous and had one goal! David Robins CEO Binfire.com

Shadeburst
Shadeburst

I was a grunt (night shift operator) on a huge software migration project from Sybase to Oracle. The data was totally compromised and every back end procedure was suspect. But as an accountant by training I knew the importance of reality checks. Basic stuff like bank statements. So, during a few uneventful night shifts I started checking the data to reality. I'm not a rocket scientist nor a space cadet but I managed to pin the differences down to a few SQL "Select" statements that skipped over certain types of transactions. That wasn't my job, but I was bored and had nothing else to do on a smooth night shift except possibly surf the Net. Reality checks. Your testers are your first line of defense but in the end it takes someone at top level to check daily that the transaction table relates exactly to reality.

ricardo.limon
ricardo.limon

Benny I share your experience and I see it much like one I had a few years ago when I got bullied into leading a project with a hostile client, back then I took in the challenge assuming from the initial project overview work plan as a piece of cake, none the less my Service Manager already knew what the project implied and simply wanted to have one as an escape goat and what better than someone new to PM duties. None the less just like you stakeholders made changes to schedule, tasks, and architecture during development time taking advantage of my lack of experience. None the less this are the experiences that forge us to be more effective and to not take things for granted as easy as it might see. Benny I would just say keep on fighting and keep that morale up, you can do it.

Johny Morris
Johny Morris

It takes a lot of courage to step back from a the brink like that and publicly too. I salute you. To back up Scott's point and at the risk of being accused of being Job's comforter here, there is also my book Practical Data Migration - the first chapter of which has a fable in it of a project going wrong that reads spookily like your experience. Well maybe not so spookily given that it is, unfortunately a far from uncommon experience. As you say rigorous Data Quality assessment before we start based on a realistic understanding of how much our business colleagues can accomplish on top of their day jobs is key. PDM has methods in it for accomplishing this. I recommend to anyone new to Data Migration, or still scarred by the last time they were involved to do the reading. Best of luck to you next time around, I'm sure it will go fine. Johny

fsrscott
fsrscott

No Project Manager should ever embark on a data conversion project without having a copy of "Project Management for Data Conversion" by Charles Scott. If the methodologies as prescribed in the book were followed, much of the disruption and misunderstanding of the project could have been avoided.

blhelm
blhelm

1. Where were your Risk Identification and Mitigation Plans? Did it account for possible data integrity issues? If so, what was your plan for resolving missing or corrupted data? What probability and level of impact did you assign to it? Did that resonate with the stakeholders? Did they truly understand the impact of that risk? 2. Was the level of commitment from the vendor directly proportionate to the level of compensation? A vendor will not commit a full time resource if there is insufficient compensation. Did the stakeholders make an assumption that the vendor had something to gain by making an investment of some resource time? In my nearly 30 years of experience in the IT industry most failed projects come down to two undeniable facts; lack of commitment and failure to foresee the future. Risk Identification and Mitigation Planning is that crystal ball that gives you eyes into the future. For every assumption in your Project Charter and Statement Of Work there had better be an associated risk with a high probability and high level of impact assigned. The mitigation had better account for "worse case scenarios". Then, as those risk triggers are realized, you have the tools and plan to adjust stakeholder???s expectations. Don???t let the technical team try to convince you that the probability of any risk is so low that it isn???t worth spending time on an appropriate mitigation plan. Most of the time the resources that are the closest to the issue can???t see the forest through the trees. It???s most valuable to a project during the Project Planning Phase - so much so that PMI has considerable emphasis on it in the PMBOK and training. Lack of commitment is the Achilles Heel of many a project. So is under estimating the true level of effort. However, if proper project management tools are utilized to determine both the financial and schedule status then appropriate action can be taken to correct the course of a project that could be headed in the wrong direction. Of course this depends greatly on the quality of project data and metrics gathered. Again, if the project stakeholders do not consider the investment of time and money into appropriate data gathering then the PM is left to his/her own resourcefulness to get the job done. Unfortunately, there is little a PM can do if stakeholders do not place a high enough value on the involvement of critical resources such as vendors. Vendors cannot exist without fare compensation. Vendor resource availability and subsequent full time commitment to a client???s project is directly proportionate to the level of compensation. To think otherwise is just na??ve. So also is staffing a critical project with a PM that has little to no experience in Project Management. We have all desired to get experience with specific technologies or make our mark within a company by being responsible for a highly visible project. However, if that project???s failure could mean the difference between a paycheck and an unemployment check a PM needs to weigh the probable risks against the potential opportunities.

MikeGall
MikeGall

Time expectations have to be managed very well in my experience. Everyone thinks, oh those IT guys sit in their office for 40+ hours a week, so if this project takes 100 hrs of work it will be done in 2.5 hrs. It is even worse if you do a fair bit of software development or novel infrastructure. How long will this take? Umm, 10hrs +- 100/10 doesn't work very well with management but that is often the case if you might find a built in API call that will do what they want, find a vendor with a program ready to deploy, or have to build things from scratch with a technology you aren't familiar. Development [rojects by definition are novel and hard to manage. It is key that up front management know that you can only work on it a few hours a day because of operational requirements so they need to take the time and divide it by 2-4 not 8,or worse 10 (got to love management when they assume you'll continue to work OT indefinitely just because it makes their schedule look better).

fhrivers
fhrivers

Face it, you learn alot more by failing than succeeding. In fact, failure is just another stepping-stone to a successful project. A failure is only a bad thing when you can't admit that you failed, can't name at least one thing you would do differently next time or you can't explain what went wrong. If either of these (or all of them) are your problem, then you shouldn't be managing projects.

ferchr
ferchr

Great story....and it happenned to me. Honesty is the best way to deal with it. Also consider your accomplishments: 1) Team formation 2) what the team members has learnt individually 3) the TEAM now know the only way to fix it is using "in-house" talent....so invest in "talent development" 4) be objective. So add dates for an "objective review" of where you are in the project". This will help uncover future "surprises". :) good luck

himan0110
himan0110

It is nothing different. Everywhere it is the same situation. Underestimation of work, overcommitment, poor coordination, bad data in legacy, complex transformation, etc. I am in same shit by the way, but you taught me something good 'nodobdy is dead yet'.

markjstanley
markjstanley

What a well written tale of woe. I admire your openness Benny, it is an admirable trait and probably a key reason you kept your job. As the old adage goes, never work with children, animals or legacy data. I'm keen to know whether testing was missed out from this story to keep it short, or whether it was missed out from the project altogether. I'm trying to understand how you can have actually got to the go-live date and *then* discover more data quality issues. Isn't it typical to have proven the data migration process in a practice run, and then do it for real a little later?

lunchbeast
lunchbeast

to be working for a company where "...working with people that, while unhappy at the state of things, understand the why of where we are..." No matter what the circumstances, no matter how completely unforseeable, my company has always canned the PM in such situations. There always has to be a scapegoat, somebody to blame. I truly wish you the best of luck on your next effort.

marat-oz
marat-oz

I had a similar experience working on a big data integration project with the big players. There was much talk about data migration but not too much done about that. I realized many people involved into the project didn't understand all complexity of data migration and what was good in one case is not acceptable on other. Being honest and pointing to my management on wrong approach cost me a job, I was even threatened by one top block that this is the end of my career.

Englebert
Englebert

....too much is asked from with too few resources, time, budget and assuming that nothing will ever go wrong. You mention that you're not a FT Project leader and the staff are not assigned FT. Seems like a lack of Management commitment. At this point, gather a team and ask what could have been done differently ? Then plan this out well and execute with confidence. Because if you fail this time, you'll be out the door.

aureolin
aureolin

By being honest from the start you never have to go to someone and tell them that you were lying when you said everything was fine but you already knew the wheels were starting to come off. In fact, that's an easily overlooked turning point for a project. When the wheels first start to show signs of coming off, it's really tempting to tell yourself that 'We can get this under control' and not mention any issues to management. If you do that then you've started the first turn of a project death spiral: things are getting worse but you can't admit it because you'll be exposed as having covered up earlier problems. Best option: be honest and mention problems and issues early. If they are handled, your team can get bennies for fixing a tough situation; if things go out of control, management has already been primed that things are not going well.

gandrews
gandrews

This is a great article and highlights many issues and failures that PM's have experienced on projects. It is unfortunate that every failure was realized on a single project, but the response of the team, stakeholders and definitely the author show how difficulty can be overcome with a resilient and determined team.

Cindy60446
Cindy60446

Excellent article that truly exhibits high quality character traits: Honesty, Integrity and Communication. I admire and highly commend you for being true to yourself on this Project. In my Projet Management experience, the quality of the data is often over looked and/or not taken into serious consideration. Your very fortunate, in that, you have Stake Holders who understand and see the whole picture. Wishing you continued success.

yolanda.gerber
yolanda.gerber

I've yet to work on a perfect Project. I don't think there are such a thing and small things does go wrong. It is however, the attitude in how a person handles the adversities that makes a huge difference. I've luckily never yet experienced a doomed Project such as the one described in the article (i say with crossed fingers). I appreciate the honesty in the article and the valuable tips. Just hope I don't ever go through the same thing. Good Luck for a successful outcome.

bsgani
bsgani

Nice article and brings life to the ages old saying that Honesty will never let you down. Would love to hear back from benny on whether he was successful the second time.

Editor's Picks