Built to Fail

The great truism that underlies the civic technology movement of the last several years is that governments face difficulty implementing technology, and they generally manage IT assets and projects very poorly.

It can be tempting to view this lack of technology acumen as a symptom of a larger disfunction. Governments are thought to be large, plodding bureaucracies that do lots of things poorly – technology management and implementation are but one of the many things that governments do not do well.

But the challenges facing governments as it relates to technology are quite specific – there are a handful of processes, all easily identifiable, that work against the efficient adoption of technology by governments. Understanding why governments struggle to implement new technology requires us to understand what these factors are and why they negatively impact technology adoption so specifically.

The challenges that governments face in adopting new technology are not symptomatic of a larger disfunction – when it comes to the efficient adoption of technology, governments are built to fail.

The Government Procurement Process

Over the past year, primarily as a result of the botched rollout of the federal Healthcare.gov website, there has been significant attention given to the aspects of the government procurement process that hinder efficient technology adoption.

The process clearly does not work well for governments – and many would argue that it does not work well for technology vendors either. Typical government procurement rules shut out many small firms that may be well positioned to execute on a government IT project but that lack the organizational endurance to make it through the lengthy selection process.

The process is complex, costly and – most importantly – slow. To illustrate the magnitude of the issue, consider that in the City of Philadelphia the period between a contract award (when a vendor gets selected to work on a project) and the final execution of that contract (when the work actually begins) can take an average of four months for some projects. Some technology solutions have shorter release cycles than that.

This makes the government procurement process particularly ill suited (in its current form) as a means for governments to acquire technology. The pace at which new technologies mature is much more rapid, and the glacial pace at which the government procurement process moves can lock governments into outdated solutions.

The most under-appreciated characteristic of the procurement process is that it’s current design is largely intentional. Governments imbue the process with requirements and other stipulations that they hope will lead to outcomes that are deemed desirable. Each of these requirements adds to the complexity of the process and the burden of firms that choose to respond to government RFPs.

For example, almost every government has purchasing requirements for minority- and women-owned businesses, and many have requirements that local companies receive preference over firms from outside the jurisdiction. The objective is to drive more government procurement dollars to minority- and women-owned businesses and to local businesses that create local jobs and pay local taxes.

There are also larger, overarching values embedded in the procurement process. For example, fairness and transparency are values that inform requirements like the public posting of bids and related materials, ample public notice of vendor meetings, and the clear specification of when and how bids must be submitted.

Risk aversion is another value that impacts the complexity and cost of the public procurement process. It is this value that informs requirements like performance bonds, vendor insurance, scrutiny of company financial statements, and requirements for financial reserves—all things that seek to reduce the risk assumed by governments from engaging with a company to provide a good or service. Each of these requirements can make the procurement process more complex and burdensome for bidders, particularly smaller companies.

These features of the procurement process were designed with a specific intent, and few people would argue with the laudable goals they seek to encourage. Yet, one of the side effects of these requirements is that they make the process slower, more complex, and harder for smaller and more nimble firms to participate in.

In other words, the procurement process works largely as it was built to work, but its design makes it a lousy tool for acquiring technology.

The Public Budgeting Process

Perhaps more fundamental than the procurement process, the process used by governments to deliberate and adopt spending plans is deeply flawed as it relates to encouraging innovation generally and adopting new technologies specifically. There are built-in disincentives in the public budgeting process for agencies to demonstrate large efficiency gains that result in the need to outlay fewer dollars.

Public agencies that have unused allocations at the end of a fiscal year – money that is appropriated but not spent – inevitably see those dollars redirected to another agency or policy priority in the next budget cycle. Mayors, governors, and legislators see this as an effective way to allocate finite resources between competing interests. If an agency doesn’t spend down their entire budget allocation, they must not need it and a reduction is justified. In other words, the budget process works as government administrators have designed it to work – to reallocate resources away from agencies that don’t need them to agencies that do.

However, most agencies see this outcome as unfavorable – a diminution of their mission and a loss for the clientele that they serve and are advocates for. It can be a powerful disincentive for agencies to use new technologies to create efficiencies that result in cost savings.

Employee Recruitment & Retention

Most government employees – at the federal, state and local levels – are hired into an agency through a merit-based civil service system. The development of civil service systems, which are supposed to grant job opportunities on the basis of merit, were a response to widespread political patronage that was common long ago.

Most civil service systems have fairly rigid job classifications and salary structures, meant to provide transparency into what specific positions are paid and to place limits on the ability to reward specific employees because of political preference or for other reasons. There are typically requirements for public job postings, and advertisement of openings for specified period. Applicants are typically required to demonstrate fitness for a position and may often be required to take a civil service exam to demonstrate minimum competency on a particular subject.

Not unlike the procurement system, we imbue certain values into the civil service system in the hopes of fostering outcomes deemed favorable. For example, in the City of Philadelphia the children of police and fire personnel are given preference in the civil service process. This is not meant to suggest that granting such applicants a preference is inherently a bad thing, but (as with the procurement process) using the civil service system as a vehicle for fostering outcomes can come at the cost of making the process more lengthy and complex for all applicants.

Public sector salaries and benefits generally lag behind the private sector, and altering pay structures and adding new job classifications in response to changes in the world of technology can be difficult. This is particularly troubling for those interested in recruiting top IT talent into government because, unlike with many other kinds of government job types, governments are in direct competition with the private sector for IT workers. A “system administrator” or a “web developer” or a “project manager” requires largely the same kinds of skills and experience inside of government or outside.

Like the procurement process and the budget process, the civil service system works as it was designed to work. Unfortunately, the way that it works makes it ill suited for attracting and retaining a highly skilled IT workforce.

The Cost of Building to Fail

What’s particularly interesting is that even if the policies that underpin each of these processes doesn’t change at all in the next several years, the problems that governments face in adopting new technology will get steadily worse. That’s because the pace of change in the world of technology continues to accelerate – as it does, the difference between the rate at which new technologies mature and the rate at which governments can make budget or purchasing decisions, or reclassify jobs and salaries in response will get larger.

The old Hippocratic maxim of “first, do no harm” will be insufficient in addressing this problem. We absolutely must do better, and we must do it soon.

How Do we Fix It?

It’s important for advocates of successful technology adoption in government to understand why governments face so many challenges in acquiring and implementing technology. The processes that support paying for and acquiring technology, and the process for hiring IT workers are designed in a way that make them a bad match for the dynamics of the technology industry.

Knowing why governments face challenges in implementing new technology is important if we’re going to find ways to address the problem. When we examine the range of different approaches being advocated to help governments more successfully adopt technology, we should evaluate their merits based on the degree to which they address one or more of the processes detailed above.

There are lots of other examples of efforts underway to help governments get better at implementing and using technology – too many to discuss fully here. But the ultimate success of each will be tied – I believe – to the degree to which they help address the challenges engineered into one of the three processes discussed above – procurement, budgeting and employee recruitment / retention.

Governments are not bad at adopting new technologies on accident. The processes that support the adoption of new technology were built to fail. Understanding this is the first step to fixing them.

Open Data and “Exoproduction”

I’ve been thinking about a way to describe what I have seen happening in the world of open data over the past few years, where outside developers create new applications and solutions built with government data to provide a service or transaction that might otherwise be provided by a government agency (or not provided at all). I’ve taken to describing this phenomenon as “exoproduction.”

Exoproduction of public services can be said to occur when all or part of a service typically provided by a government or a public sector entity occurs outside of, or exogenous to, the agency of government explicitly (or implicitly) charged with providing that service, or a component of it.

While governments do not directly provide services or information in scenarios where exoproduction occurs, they do make available the raw materials or components necessary for the service to be provided by a third party.

The clearest example of this phenomenon can be observed when looking at information on transit services. Transit authorities are charged with operating and maintaining public transit systems, and as part of their responsibilities they are charged with communicating to current and prospective transit riders the schedules for transit services, the fares for such services and even the current status of various transit assets (e.g., is the R3 train running on time?).

However, through deliberate programs designed to share the raw data on transit schedules and real time location information many transit agencies are encouraging the development of third-party applications and solutions that fill this role. A non-trivial and growing number of transit riders obtain information on transit services from a source that is outside the official purview or control of transit authorities.

There is an abundance of other examples where outside parties develop solutions on top of government provided or maintained data – to fill a role or address an issue that would logically fall under the official responsibilities of a government agency. In Philadelphia, there are a number of efforts underway to encourage the repurposing of vacant properties, even though official responsibility for this falls under the duties assigned to specific government agencies. These outside efforts are enabled by the deliberate release of property information by the City of Philadelphia.

In the past I’ve used the term “coproduction” to refer to this phenomenon, but that term has several decades worth of history behind it and more correctly refers to another scenario where outside actors impact how a government service is provided. Coproduction assumes that the direct recipients (or consumers) of a government service are involved in making a service more effective or delivered more efficiently.

For example, community policing services may be improved by interacting with residents in a neighborhood. Through these interactions, police may gain insights and information that help them develop better strategies for crime prevention and ultimately provide better service to the community.

Exoproduction differs from this because it frequently involves the development of solutions by those that are not the direct recipient of services. For example, the development of an application to find homeless shelters built by a group of young professional at a hackathon.

Government agencies have a variety of strategies for leveraging outside parties to provide direct services to citizens and to fulfill their missions. Most people are familiar with the example of governments using the standard procurement process to buy services from outside parties. Many of the technology solutions that are used by governments to provide services and information directly to citizens are built by (or with the help of) outside businesses.

Social service and mental health services may also be provided by outside parties like neighborhood-based agencies or church groups. Some cities have “business improvement districts” with special authorities established with quasi taxing authority to support things like trash pickup, graffiti removal and security. The list goes on.

However, each of these examples involves the direct oversight or control of government through a formal contract, or through enabling legislation. We need a new definition for what we see happening in the world of open data and civic technology, where the oversight or approval role of government is less direct.

In the more traditional examples above, governments typically provide money – through a formal contracting process – to outside parties to act as a proxy on behalf of government for the delivery of a service. With exoproduction, the currency used to drive service delivery is (typically) data.

Having said that, It should be noted that exoproduction may not only occur when software and technology solutions get built on top of government open data – there are some analog examples where exoproduction can take place. For example, food that is provided to support needy children may be provided through various government programs but is actually dispensed by private individuals outside of public facilities. There is a notable example of this from the Philadelphia area that also demonstrates some of the potential challenges with the shift to exoproduction.

A Delaware County woman who voluntarily distributes free food to children from her driveway has run afoul of officials in Chester Township who say her efforts violate zoning ordinances.

In this instance, a woman distributing food to needy children through a state-sponsored program administered by a local archdiocese ran afoul of local zoning ordinances. In a way, the issues encountered in this example might be viewed as analogous to those encountered by a software developer that scrapes data from public website in order to support a civic application. Actions supported (either directly or tacitly) by one arm of government might encounter resistance from another.

Questions remain about how exoproduction will ultimately impact the way that governments deliver services.

  • Will solutions developed via exoproduction augment or replace the service provided by government, or relieve governments of the responsibility of providing a service themselves?
  • How do we measure the impact or effectiveness of exoproduced solutions, and how does (or how should) this impact how government agencies with official responsibility for providing a service be evaluated?
  • How does (or how should) exoproduction impact the way we budget for or procure solutions through outside parties?

Ultimately, I believe that open data and the phenomenon of exoproduction (or whatever we decide to call it) have the potential to fundamentally change the way that public services are delivered. As governments continue to implement open data programs, it would be worth considering these questions and others that will impact how externally developed solutions are used to improve our communities.

It’s Not About Cheaper, It’s About Better

The Wall Street Journal recently featured an awesome story about civic hacking, focusing on the amazing work being done in the city of Chicago.

It’s great to see the efforts of civic hackers and open data advocates covered in the mainstream press, and the team in Chicago – those both inside and outside of city government – deserve every bit of praise they get for their tremendous efforts. But I did take issue with one point made in the article – interestingly, one made in the very first sentence:

Cash-strapped cities are turning to an unusual source to improve their online services on the cheap: helpful hackers, who use city data to create tools tracking everything from real-time subway delays to where to get a free flu shot near your home and information about a contentious school-closing plan.

I don’t think anyone would argue the the fiscal pressure faced by governments – particularly cities – hasn’t helped encourage officials to experiment with new ways to provide services and information, and helped highlight the need for technology innovation. However, I don’t think its accurate to say that what is happening is an effort by governments to get IT work done by outsiders “on the cheap.”

It’s not about building things cheaper, its about building them better.

Implicit in the idea of “government as a platform” and the factors that help to drive municipal open data programs is that the role of governments in the delivery of public services is changing – toward the role of data steward or API manager, and away from the more traditional role of “app builder.”

There are a number of reasons why the role of data steward is a better one for governments – most importantly, governments don’t typically make good bets on technology. They’re not set up to do it properly, and as a result its not uncommon to see governments invest in technology that quickly becomes out of date and difficult to manage. This problem is particularly acute in relation to web-based services and applications – which outside civic technologists are very good at building – the landscape for developing these kinds of applications changes far too rapidly for governments to realistically stay current.

Governments that focus on becoming data stewards are better able to break out of the cycle of investing in technology that quickly becomes out of date. It is these governments that are moving to release open data and deploy APIs to enable outside developers to build applications that can help deliver services and information to citizens.

However, this shift to the role of data steward doesn’t deobligate governments from investing in technology or skilled staff. It simply means that this investment can be focused within a role that governments are better structured to perform well. Developing the infrastructure and policies to support an open data program and API platform are not necessarily “cheaper” but they are a much better technology investment for governments to make.

In another sense, the move to the co-production of technology solutions with outside developers is also about building better applications as opposed to building cheaper ones.

It’s common for those behind civic technology projects to have personal investment in the issues being addressed by their solutions. These developers bring a different perspective to the problems that, in the past, may only have been addressed by governments – they are stakeholders in the success of our cities.

Engaging outside developers and marshaling the efforts of civic hackers to build new tools and new services to improve our communities is about enhancing the quality of solutions, and not lowering their cost.

It’s great to see the awareness of civic hacking and the open data efforts that fuel it getting coverage in the mainstream press. But we still have a ways to go on communicating the more fundamental changes to government this movement entails, and the real benefits we all stand to gain.

The Case For Making Procurement Harder

Momentum seems to be building around public sector procurement reform.

Governments are starting to experiment with new ideas and new approaches to procurement that hold the promise of streamlining the process for bidders and producing better outcomes for public sector purchasers. Startups are emerging with new tools to make the procurement process easier, and the issue is on the radar of public sector innovators.

But with all of the focus on improving the procurement process – making it more transparent, more streamlined and simpler for bidders and governments – I think there is a case to be made that in order to improve the public procurement process we have to make the process harder. Let me explain.

How can governments responsibly evaluate proposals for work when they are not able to effectively and clearly articulate what it is that they want?

One of the biggest problems with procurement is how Requests for Proposals (RFPs) get written. Having written and responded to state and local government RFPs, and having evaluated more vendor proposals than I care to remember, I have a pretty good sense of how most RFPs get developed.

It essentially boils down to this: copy + paste.

Most government RFPs begin as a template that is provided by a procurement or legal department. Much of the content in these templates contains all manner of byzantine legal language that includes specific things vendors must do, information on how and when and in what manner to respond, etc. Much of this language will not change from RFP to RFP, and most governments typically employ language that is tailored to their own local processes and requirements.

The heart of any RFP is what is usually referred to as the “scope of work” – the section of the document that is added to the initial template that describes the specific deliverable(s) that a vendor is to provide. This is the most important part of the procurement document because it describes the thing (or things) that governments want to purchase with taxpayer money.

Yet all to often, this core piece of the RFP document gets cobbled together by looking at other RFPs that have been issued for projects that are further along in the development cycle. If another RFP – so the logic goes – was issued, received responses and has moved into the work phase then it might have language that can be reused in a new RFP, particularly if both are for similar kinds of projects.

So, for example, if a government agency or department wants to build a web or mobile app, construct a data warehouse or obtain an analytics tool they simply need to go and find and RFP that was issued for a similar project and use that as the basis for building the scope of work for a new project. And while there is some value in identifying RFPs for similar types of projects to help ensure that language used across different projects is consistent, I think this practice is dangerous and leads to bad technology choices.

Far too often, the process of writing what is essentially the heart of an RFP is done by simply copying and pasting text from other documents that have progressed to a later stage in the process.

This means the author probably has very little idea what they are really asking for – the process requires no effort on the part of the requesting agency to clearly describe what it is that they hope to get from the RFP. How can governments responsibly evaluate proposals for work when they are not able to effectively and clearly articulate what it is that they want?

This is the part of the procurement process that needs to get harder – we must avoid the temptation to create shortcuts that gloss over this important function of accurately and clearly describing what it is a government agency hopes to get from engaging with an outside vendor.

Part of the reason that governments are lousy at describing what they want – particularly from technology vendors – is that there is a lack of people in government with hands on experience building or maintaining technology. There is a dearth of makers in government.

Until we find better ways to more broadly raise the technology IQ within governments – by bringing more hackers and makers inside government, or exposing those already inside to the hacker culture – we’ll continue to see copy + paste RFPs and poor technology selection by governments.

Experiments in GitHub Based Procurement

The City of Philadelphia is experimenting with some new ideas that we hope will change the way that city departments procure technology solutions. The “petrie dish” for some of the more interesting of these experiments is the social coding site GitHub.

The Background

GitHub logo

Philadelphia is looking for ways to partner more closely with local technology companies on small to mid-sized technology projects undertaken by city departments. One of the biggest challenges we face in creating these partnerships is our procurement process, which wasn’t designed in a way that is conducive to working with small companies and startups. This is not a new problem, and there are actually initiatives underway in Philadelphia to mitigate this challenge and create valuable partnerships with local startups.

One of the unique characteristics of Philadelphia’s procurement system is the process used for “Miscellaneous Purchase Orders” (MPOs). MPOs are are personal and professional services contracts valued at $30,000 or less for which city departments are required to obtain quotes from no less than three different vendors. Traditionally, the process of outreach to vendors for MPO projects has been decentralized and opaque. Department personnel would (and still do) reach out to vendors that they are personally aware of and obtain a quote for services.

This traditional practice is suboptimal for city departments because the pool of vendors from which quotes are obtained is limited. It is also not optimal for local vendors, many of whom may be qualified to provide the service desired but who may be unaware of the opportunity.

With the support of Philadelphia’s Deputy Mayor of Administration & Coordination and Managing Director, Rich Negrin, and at the direction of the Chief Innovation Officer, Adel Ebeid, we are experimenting with ways to change this practice for MPO projects (starting with technology projects), and we’re using GitHub to do it.

The Overview

$30,000 may not seem like a sizable budget for a big city project, but for some technology projects – specifically things like web site redesigns, mobile app development, etc. – this amount can go a long way, and provide ample funding for a small or mid sized effort. It’s probably not the right budget for larger companies but for small companies, or even freelance developers, this is nothing to sneeze at. It is these small companies that we want to create new partnerships with – today’s small startup is (potentially) tomorrow’s big company.

Philadelphia has a thriving technology community, and a robust network of developers and designers who would be ideal for many of the projects that city departments undertake as MPO projects. But until recently, there hasn’t been a mandate to rework how we conduct outreach for quotes on MPO projects. In addition, there hasn’t been the right concentration of knowledge within city government to think outside the box and try some new ways of doing old things.

The Managing Director and CIO have given the Innovation Management Team in the Office of Innovation and Technology latitude to try new things. A good chunk of this team is made up of former civic hackers, most of whom have collaborated on software projects with other developers in the past. When we got the green light to think of some new strategies, our collective interest settled almost immediately on GitHub.

The Experiment

We decided to try and use GitHub – from start to finish – as the mechanism for soliciting bids on a new technology project for the City of Philadelphia.

There are lots of reasons that we decided to use GitHub for this experiment. First, we wanted to utilize a platform that would resonate with technologists (duh, Github). Second, we wanted to have some insight into the quality of other work done by bidders – we expected that firms interested in bidding would already have GitHub accounts, allowing us to look at public repos to see what kind of solutions they were working on and how active they were in participating in other projects.

And finally, we were hopeful that by doing something different – something that potential vendors probably had not seen from a city government before – that we would encourage some “creative” responses.

To conduct our experiment, we needed the right project. The one we ended up choosing is called myPhillyRising – it involves the development of a mobile app to support the efforts of the PhillyRising Collaborative. PhillyRising is a “boots on the ground” effort that is run under the auspices of the Managing Director’s Office. It places city staff in some of the toughest neighborhoods in Philadelphia to support residents that want to bring together their neighbors and organize community events and address chronic neighborhood problems. The objective of the mobile app is to give both PhillyRising staff and residents in the neighborhoods they serve an easy to use tool for communicating, sharing information and organization events.

The first thing we wanted to do was to post the description of the myPhillyRising project publicly, to enable us to share it widely through various channels with our local technology community. We decided to use a simple GituHub Gist for this.

We asked interested parties that had questions to post them as a comment on this public Gist, allowing other interested parties to see both the question and our answers. For those that wanted to submit a response, we asked them to do one of two things:

  1. If they didn’t have a paid GitHub account, we asked that they create a private Gist containing their response and send us a link for review.
  2. For those with paid GitHub accounts, we asked that they create a new private repo containing their response and invite us to be collaborators on that repo.

While we anticipated that the process of using GitHub to solicit bids would be interesting, the response that we got from local vendors – and the nature of some of these responses – exceeded our expectations.

The Outcome

By the end of the submission process, we had received 9 high quality responses from local vendors – a number that far outstrips the number of responses received for similar MPO projects, and three times the number of responses required. This underscores the importance of publicly posting opportunities of this kind – if the city gives local technologists an opportunity to bid on work, local companies will respond.

In addition, several of the responses were highly creative – responding firms took advantage of the fact that the submission platform was a GitHub repo and made the most of the opportunity. One firm created a compelling webpage using GitHub pages to tell the story of Philadelphia residents who might use the new mobile app – they didn’t do this in addition to their standard response, this was their response. Another firm committed prototype code to their repo that let the evaluation team test drive how the proposed app might look and work on a mobile device.

Beyond the initial submissions from interested firms, we found that GitHub Issues were an effective means of interacting with respondents. We used this mechanism for followup questions, to request additional information and even to help us schedule in person meetings with finalists.

bug

question

Going into the final vendor selection, we expect to use the original GitHub repo that was set up to submit vendor responses as the mechanism for delivering the final working mobile application (and for all of the iterations and code reviews leading up to that point).

And ultimately, if we decide to release the code for the myPhillyRising app as an open source project, we can add the appropriate license and flip the winning vendor’s repo from private to public.

The Road Ahead

This experiment focused on a specific kind of city project. Even within the universe of MPO projects (which represents only part of the larger procurement picture), this effort was very narrowly focused on technology projects. This represents just a drop in the bucket of what the City of Philadelphia spends on technology projects.

That said, this exercise has given us valuable insights into ways that we can make the process of engaging with local technology companies easier and more efficient. The latitude provided by the MDO and CIO gave the project team room to try something that the city had never done before.

We are now expanding our efforts to reach out to local technology companies for small to mid-sized technology projects – aggregating new projects that have the right profile to a central website (a simple WordPress-based site) to make them easy to find and allow users to set up subscriptions for the kind of projects they are interested in. This will not be the last project to use GitHub as an integral part of the vendor selection process, and we are actively looking for ways to integrate GitHub into the processes used to interact with vendors on these projects.

We have much more work to do, but the steps that the City of Philadelphia is taking to engage more efficiently with small, nimble, local technology companies is really exciting.

It’s pretty cool to see GitHub being used as a part of these efforts.

The Year That Civic Hacking Changed Everything

Back in January, I predicted that 2012 could turn out to be the year of the civic startup.

And while I think that civic startups and other businesses built around open data and government innovation made great strides in 2012, more than anything else what happened this year demonstrated to me the real transformative power of open data and civic hacking.

The number of civic hacking events taking place across the country has never been higher. A partial list of events from this past year – just those events I was aware of or able to find out about – is here. I’m most certainly undercounting. And 2013 is sure to see even more events.

And yet, civic hacking is still not without its detractors – the usual group of professional contrarians, and those that continue to wring their hands over the longevity of civic apps. These critics typical make the same flawed observation – that a lot of civic applications (solutions developed at weekend hackathons or by the enthusiastic corps of volunteers that are surging in numbers in cities across the country) aren’t sustained “long term.” This argument always has been – and still is – pure nonsense.

Building Successful Civic Apps

A recent article in the New York Times detailing the new “app economy” being driven by developer friendly platforms like the iPhone and Android devices noted a similar phenomenon in the broader app ecosystem:

Despite the rumors of hordes of hip programmers starting million-dollar businesses from their kitchen tables, only a small minority of developers actually make a living by creating their own apps, according to surveys and experts.

Developing successful, sustainable apps of any kind is hard and only a small minority of them will ever be deemed “successful”. Why are we surprised then that the ecosystem for civic apps is also characterized by a small minority of successful apps.

This is the nature of app building.

But no one questions the benefits when companies like Apple, Google, Facebook and others deploy APIs and woo developers to help create the new app economy. Reasonable people can disagree about the magnitude of the real economic impact, but everyone should acknowledge that there has been some economic benefit from the rise of the app economy. Certainly those companies that have turned themselves into platforms for app builders are benefiting.

Governments can also reap benefits by pursuing actions that help implement the idea of government as a platform. Many cities across the country are working hard to transform themselves into the public sector equivalent of these app platform companies. And yet, the same tired arguments against civic hacking are brought up time and again. Why?

I’ve said it before and I’ll say it again – civic hacking is about more than just apps.

What Hath Civic Hacking Wrought?

It is impossible to imagine the open data movement as it exists today in cities across the country and around the world without civic hacking.

In cities like New York and Philadelphia, it was civic hackers that drove the eventual passage of formal open data policies and the adoption of structural changes that include the creation of positions like Chief Data Officer.

It was civic hackers – people that wanted to build useful applications for their cities – that first demonstrated that there was an appetite for public data outside of government. In both New York and Philadelphia, civic hackers scraped transit data from public websites to build apps like StationStops and iSepta and incurred the wrath of the transit agencies.

Both of these cities are now among the most progressive in the country when it comes to open data. The reason? Civic hacking.

Civic hackers more than any other group have demonstrated the need for formal open data policies in cities by showing people what can be built with open data. There is no doubt in my mind that cities like Chicago and Philadelphia have Chief Data Officers because of civic hacking, or that cities like San Francisco, New York and others will soon have them as well.

The idea that the work of civic hackers is largely trivial – a charge made by some critics – is laughable. I have personally been involved in more hackathons than I can remember, as a participant, sponsor, organizer, mentor and judge – in multiple cities. To a person, the people that take part in these events care deeply about their communities, and the vast majority are acutely aware of the problems their cities face.

Many of the participants in these events have already invested considerable time in pursuing the issues they choose to take up at a civic hacking event. And many others are busy professionals with little time to waste on a weekend event if they don’t believe that it will be of value.

Remember, some of the first civic hackers were transit riders that wanted a more convenient way to get information on transit options. Most other attendees at civic hacking events are similarly situated – they have direct exposure to a government service or a problem facing their own community that they want to help improve.

Almost a year ago I listed 5 things governments could do to nurture the civic app ecosystem – I stand by those suggestions, particularly my assertion that more open data is needed. The biggest impediment to fostering and sustaining more civic innovation through civic hacking is a lack of relevant open data to use to build apps.

Civic hacking events do not need better “guidance” from experts or input from the class of professional consultants cum contrarians – they need more open data from governments. Period.

There are a few notable examples of cities that are actively working to promote apps built by civic hackers, as a way to drive traffic to them and help sustain them going forward. The City of Chicago posted an app for finding flu shot locations built by a civic hacker on it’s website, and the transit authority in Philadelphia is promoting apps built at civic hacking events to riders on its regional rail service. We need more of this.

Not only are civic hackathons important for generating innovative new solutions, stretching the traditional notions of public service delivery, I believe they are an example of how we will deliver government services in the years ahead.

The Future of Government Service Delivery

I talked about this a bit in a previous post and I also presented this idea during an Ignite talk at a recent Government Technology event in New York (see video below). I think that we’re seeing a shift in how government services are being delivered. The change is small now, but we’re just at the beginning.

Open government data and civic hacking are the foundations on which this change will be built.

In a recent report from the Center for Technology in Government looking at the dynamics of opening government data, the authors observed:

“Open data initiatives disrupt government’s traditional role as ‘holder’ or ‘owner’ of the data. In thinking about open data governance, we need to re-think government’s role in relation to the entire set of new stakeholders. One possibility is to characterize government, as well as all other stakeholders as stewards [of data].”

I couldn’t agree more. It’s time for governments to stop self identifying as solution developers and start thinking of themselves as data stewards.

This requires governments to partner with outside developers, technologists and entrepreneurs to help build the applications that will provide governments services and deliver information to citizens.

Open data will be as disruptive to how governments provide public services in the next 20 years as the Internet has been in the last 20.

Civic hackathons are where this change started, and they are where we will get a glimpse of the road ahead.

Getting RHoKed in Philly

This past weekend, civic hackers gathered at the new Drexel ExCITe Center for the latest installment of the Random Hacks of Kindness hackathon in Philadelphia.

IMG_2849

In addition to a new location for this latest installment of RHoK, the event was organized by TechnicallyPhilly (in previous years, TechnicallyPhilly had been a collaborator and supporter of the event). The event was a huge success, with dozens of people turning out for the weekend to work on a great set of projects.

You can read TechnicallyPhilly’s coverage of the event here.

RHoK events in Philly always generate great projects like PhillySNAP and Sheltr, but I saw something really important and really exciting happening at this particular hackathon.

The teams that took the top two spots at the event both had public employees as active members – the first place team was led by an IT manager from the Philadelphia School District. The second place team had members from the City of Philadelphia’s Department of Licenses and Inspection and the Managing Director’s Office.

And the third place team developed an application that uses multiple open data sets published and maintained by government (transit information published by SEPTA and a list of SNAP retailers from the USDA).

In Philadelphia – maybe more so than other places – the idea that open data and hackathons are a key part of building the services and solutions that can help government work better has set in deeply. I can’t think of one other city where public employees are as directly engaged in the civic hacking culture as in the City of Brotherly Love.

This is a subject for a future blog post, but I think we’re at a pivotal point with government service delivery – at a place where a seismic shift in how governments deliver services and information to citizens is starting to occur. To me, it feels like we’re in a place like we were in the early 90’s, when the spread of the Internet forever changed they way the public sector provides services, and how citizens interact with their government.

A massive and transformative change is coming – open data and civic hacking are important components helping drive it. And Philadelphia is going to set the course for other cities to follow.

Stay tuned…

The App Economy and Government as a Platform

Most people in the civic technology world have heard of the concept of “government as a platform” – the term famously coined by Tim O’Reilly several years ago to describe the application of Web 2.0 concepts to government.

I talk to people in municipal government about this concept on an almost daily basis, and I think for the most part people inside government understand the general idea. They understand the potential benefits of turning open data into new and creative applications built by smart people outside government. Most have smart phones of one kind or another and consume apps through the various app marketplaces.

At a very high level, they get it. But most still have some concerns.

Many people have heard the story of how Apple helped create the “app economy” by opening up the iPhone to outside developers – it’s a powerful analogy for the idea of turing government into a “platform” for new apps trough open data and APIs.

But the parts of the story about the release of the first iPhone, and the subsequent creation of the app economy, that are less often heard are ones that I think are more relevant to what governments face when trying to create platforms for civic app development.

There is an interesting article in the New York Times this morning highlighting one of the these important points that is worth a careful reading for anyone that is passionate about government as a platform.

Specifically, it’s worth noting that executives at Apple were initially skeptical of the proposal to open the iPhone to outside developers. The risk inherent in abdicating control to developers that could build things not envisioned by Apple engineers was palpable.

Apple’s brilliant but mercurial chief executive, Steven P. Jobs, agreed to unlock the gates of the fledgling iPhone only after much internal argument, and he made sure that Apple would retain strict oversight of every app. In retrospect, it might have been the smartest decision ever made by a company that prides itself on creating the future.

This is probably the one factor I run up against most often when talking with government officials about the concept of government as a platform – the fear of losing control over what apps that use open data look like. Publishing data empowers outside developers – this power is shifted from government to people that build apps, and for many inside government that’s still kind of a scary concept.

For almost all of the time that the Internet has been a component in how governments deliver services and information to citizens, the government has been the one controlling the end user experience. The idea of government as a platform assumes a very different role for government – that of data steward or API manager, not as an app builder.

To a certain extent, the concerns that governments have about moving towards government as a platform are legitimate. If governments are to become more mature stewards of data and APIs, we need to think carefully about when and how users are “managed” through (for example) the issuance of API keys, and the language we use in things like “terms of use” statements.

But if the experience of Apple in opening up the iPhone, and the creation of the app economy that followed are any indication, the payoff for this transfer of control can be enormous for government.

Open Gov: What’s gone Right, What’s gone wrong

I had the pleasure of speaking on a panel at the recent MIT-Kinght Foundation Civic Media Conference in Boston.

MIT-Knight Civic Media Conference

The panel was chaired by Susan Crawford and included Chris Vein, Deputy United States Chief Technology Officer for Government Innovation, and Mike Norman from WeFunder.com.

The panel discussed what has worked well in the open government movement, and things that have not worked so well. Some good ideas for keeping the movement strong and thoughts on where it is going were offered.

A pretty good discussion I thought. I hope you do to.