As cities, states, and countries look to turn raw public data into refined fuel for economic growth, accountability and analytic insights, they’ll need to bring more technologists to the table who understand what startup veteran and former Chicago police officer Brett Goldstein does: sometimes, the most important work is often the least sexy.

Goldstein, who is currently a fellow at the University of Chicago, was appointed to be the City of Chicago’s first chief data officer (CDO) in 2011 by newly elected mayor Rahm Emanuel. In 2012, he started wearing another hat as Chicago’s CIO and Commissioner of the city’s Department of Innovation and Technology (DoIT).

During his two years in City Hall, Goldstein was a crucial part of how Chicago opened up its government. (The city also became one of the best exemplars of how open data may or may not relate to transparency and open government, as WBEZ reported in 2013.) Years before he donned a suit and walked into City Hall, however, Goldstein had traded the jeans he’d worn in the startup world at OpenTable for seven years for a blue police uniform.

After the terrorist attacks of September 11th planted a seed in his mind about serving and protecting his community, he attended the police academy and joined the police force as a beat cop in 2006. He put his technical skills to use starting up the department’s first predictive analytics department four years later. When he left the administration to become the University of Chicago’s inaugural fellow in Urban Science, Goldstein began advising the school of public policy on its Master of Science program in computational analysis.

When I started research into the growth and responsibilities of CDOs, interviewing Goldstein was a natural first step. Our conversation follows, lightly edited for clarity, along with video of his 2013 talk to the City Club of Chicago.

What are the core responsibilities of a CDO?

Brett Goldstein: My perfect model for a CDO is someone who operates in three areas, the first being open data. That’s a relatively obvious one, and it’s a relatively common thread for the public side.

The second one is performance management, where you have your standard key performance indicators and management by objectives and all of those typical things. I draw the analogy to “I have a 311 service. What are the metrics that I have on there?”

The third piece is the area of advanced data analytics and predictive data analytics. One of the things that worries me is that we don’t really see as many models which are covering the three. I think all three areas are critically important.

I assertively differentiate this from a CIO, even though I at one point held both roles. The CIO should be data-informed, but whether you’re in the public or private sector, you have a vast amount of things you’re going to cover. You have everything from messaging to infrastructure to data center to ERP. It’s somewhat unreasonable to ask for that level of expertise. You do still see it in Chicago, where the CIO has a lot of passion around the data portfolio, but they need to be able to have full coverage, and with that not the depth that you need for a CDO.

SEE: Watch this video of Tom Schenk, the current CDO for the City of Chicago, in which he talks about his work and

I have seen some change in the CDO role since I went in in 2011. It does worry me. If we go back to the three verticals that I think are important, I don’t see as much of all three. It seems that there’s a lot of effort within open data, which is great, and transparency is super important for all levels of government. We need to make sure — and I joke, since this is the name of the book we worked on, “Beyond Transparency” — that we’re getting beyond just data sets onto the Interwebs and “spreadsheets on the web.” What’s next? I would have hoped that as we got into year three, year four of this effort that we’d be advancing the model a bit. I think we’ve seen it in some cases, yet I’m not seeing it as much as I want.

A piece of that is also what is the reporting structure? We have some CDOs who report directly to the executive. We have others who report to a deputy of the executive, but it’s still a pretty prominent position. In other models, it’s nested within a department. I think that’s something that’s probably under-discussed and under-explored. It would be interesting to look at the efficacy of the CDO role when you position her or him directly under the executive versus in a nested department. I would adhere to the concept that a truly effective CDO needs to have wide and deep access to the data assets that are there. So, positioning can be somewhat critical.

What do you see people get wrong about the CDO role most often?

Brett Goldstein: I worry that with CDOs, we’ll get sucked into the sexy, front-facing deliverable too often. Certainly, it’s really rewarding when you build something that has a smart UI or a smart visualization and it’s citizen or resident-engaging. That is great, that is wonderful, but that should not be the entire portfolio. I think it’s easy to get trapped in that. I think it’s easy to get trapped into the number of data sets that you have released. Quantity is an interesting metric, but it’s certainly not a metric of success. As we have more and more CDOs, and I’m really referring to the public sector here, it’s important that you don’t get caught up with just the sexy piece or the quantity piece. At the same time, am I doing deep, meaningful work?

What should be a core role of a CDO within the government is to, one, inform policy. Policy has a qualitative aspect, it has a public policy mindset, but we need to introduce quantitative rigor for both tactical and strategic decisions. That needs to be core to what a CDO is doing, coupled with the themes that you and I have talked about before: making smarter decisions. We’ve spent many years talking about breaking down silos, understanding the relationships between different departments, and the different pieces that happen. A CDO should be driving that. She or he should be leading the way in how you create more efficient, smarter, proactive rather than reactive, services.

How should the CDO role evolve? Who’s ideal? Product managers? Lawyers? Technologists? How important is hands-on knowledge vs policy expertise? How close to the data do they need to be?

Brett Goldstein: I think if we have a CDO, the goal should be as the position evolves for it truly to be a C-Suite position. You don’t want to have a chief who’s not truly a chief. It needs to have that respect of the executive, the importance within the administration. With that, also comes the trust. One of the things I really enjoy talking about is the idea of the CDO being part of that policy discussion. When the CDO says “this policy is quantitatively weak” and lacks that empirical basis which should drive policy, that is a challenge that should be listened to. If the position does not hold that sort of weight, then is it really a CDO? That’s absolutely critical.

The second piece, and I tend to make the same comments about a CIO as a CDO, is that they need to have the knowledge to be able to know when things are right versus things are wrong. They need to have some depth in technology and within data, so that they can truly “talk the talk.” One of the things that I still reflect on happily is that when I, as a commissioner, would go out to a meetup. People would expect a “political commissioner type” and then I could “talk nerd.” I could speak the language.

One of my favorite things, when I was talking to a big data group, was when I started talking about OpenTable and data, and you saw everyone’s look change. There’s this level of respect. It also gives you this ability to call BS. When you’re working in government, you’re constantly barraged by proposals and ideas. You need to be able to rapidly vet them, know what is real, what is legitimate, what is sized correctly, and that’s important.

One of the things I question when someone comes into a room who’s responsible for this portfolio, and the first thing they say is, “Oh, I’m not technical.” Well, what do you want me to do with that? What if I were going into a financial analytics discussion, and I have no financial background? People need to have the background. The nice thing is that I think we are recognizing this, and certainly my lens right now is academia, and certainly at the University of Chicago and NYU we’re building programs to give people the foundation. The need is recognized.

We are going to get deep into these fields. Deep into open data, deep into data analytics, deep into prediction. You need to have leadership who has depth, who knows how to call what is real versus not real, and be respected by folks out there. You need some deeper skills to get there. I am not saying we need to have super technical candidates in all cases, because I want people in these roles who can straddle the worlds, that can go from understanding the innards to sitting down with the mayor or the president or whomever and saying “here’s how it ties to the policy” in a coherent, smart way.

If cities, states, and agencies don’t vest CDOs with sufficient clout, in terms of statutory authority and policy, are they setting them up for failure? Is legislative action needed?

Brett Goldstein: I think, and this may be naive of me, that it’s less about commodifying things through legislation. It’s more of the political decisions that are being made. One of the reasons it worked out for me and one of the reasons I joined the administration is the way that Mayor Emanuel structured it. I was senior, I reported to the mayor, I was initially in the Mayor’s Office, and I was a commissioner. I became a trusted voice within the administration. You know immediately, right then, based upon the topology that I laid out, how this would be structured, versus if I were nested multiple layers down in the reporting sequence. How could I have a strong voice if I were multiple layers down? I also would not be getting the exposure to the various policy initiatives that you need to comment on them.

With this being said, I think that there’s an evolution for new administrations. I think it would be challenging for a brand new executive in a major municipality to say, from day one, “yup, we’re just going to go with it.” There’s a bit of a learning curve.

One of the things I look back on — and it’s an open data example — is when we released all the crime data. That was a huge thing, and everyone was nervous about it, but we did it. The lesson learned was that everything was OK afterwards. You usually have a couple of these to build confidence from there. At the same time, it’s not just on the executive or the organization: it’s on a smart CDO to get at some initial wins. When I coach people now, I always talk about early wins. Not everything needs to be one of those insanely fun but complex, multi-variate machine learning equations to solve a problem. Sometimes you just need to do something that’s a little more basic. It’s basic, it’s smart, it helps, it makes [government] better, and you’re able to show the power of data.

I think placement is key. I think confidence in the person is key. That’s why who gets hired and who gets in and where they get placed is critical. It’s also making sure that they have the skillset to make smart choices and be able to have their own strong, independent view.

What’s the advice you’d give to CDOs? What’s the most important thing that they’ll need to know or do to be successful?

Brett Goldstein: The first is to develop your own view. When I sat down in the Emanuel administration and heard a presentation, I developed my own opinion on whether it was an analytic, a project, a side, any of those. You need to be able to independently assess and then figure out where my mind is at, versus what’s being presented.

Coupled with that is being able to accept when there are things that you don’t know. I have never been arrogant enough to think that I know all of the answers to everything. I have always had people who I can call and talk things through, and get advice. It’s actually remarkably analogous to the startup world.

When we did OpenTable, spending seven years to build it, we made a ton of mistakes. Now, the next generation of startups came along and say, “hey, we don’t want to make the same mistakes. What can we do?” Instead of being the arrogant startup that says I know everything and I’m not going to ask for help, when they come, I ask myself why would I want to have them make the same mistakes I did?

There are wonderfully pragmatic conversations that you can’t have on Twitter and you can’t have in the press, so that you can figure things out. I would strongly encourage the CDOs of today and tomorrow to leverage those of us from the past so that we can partner to make sure that they don’t make the same mistakes.