Civic Innovations

Designing simple systems is one of the great challenges of Government 2.0. It means the end of grand, feature-filled programs, and their replacement by minimal services extensible by others.

— Tim O’Reilly, Open Government

The original idea of Government as a Platform is now almost a decade old. In the world of technology, that’s a long time.

In that time, people working inside and outside of government to implement this idea have learned a lot about what works well, and what does not. In addition, we’ve seen some significant changes in the world of technology over the past decade or so, and the way the we develop solutions (both in the world of civic tech, and outside of it) have changed fairly dramatically.

The power of the original idea for Government as a Platform continues to echo in the world of civic tech and open data. I have no doubt that it will for a long time to come.

But in 2015 what does Government as a Platform actually look like, and what should it look like going forward into the future? What are its component parts? How does it manifest in terms of actual infrastructure, both inside and outside government?

And, most importantly, who controls this infrastructure and has a say in how it is shaped and used.

The Architecture of Government as a Platform

The idea of Government as a Platform – originally built on the idea of open government data – has become almost synonymous with Application Programming Interfaces (APIs).

APIs are an important part of how modern software is developed, and have been for some time. Building APIs, and the accompanying tools to use them, are at the core of what most governments aspire to put in place to support the development of civic apps. A number of different federal agencies do this, as do large cities like New York, Chicago, Philadelphia and others.

For many governments, the centerpiece of their efforts to support civic apps is an open data portal. Open data portals provide a central location for governments to publish data so that it may be more easily obtained by third parties wishing to use it. In addition, these data portals almost always provide some sort of API access to data – so by publishing data to an open data portal, governments are creating APIs for developers to start using that data to build civic apps.

An overview of how Government as a Platform typically manifests can be seen in the graphic below.

In this scenario, governments make APIs available to third parties for building civic apps. Governments may build custom APIs for these third parties, or they may leverage the APIs that are baked into their open data portals. Alternatively, governments may choose to release the raw data that is used by third parties and civic tech actors to build their own APIs to power civic apps.

In general, this scenario sees governments acting as the stewards of data and APIs and empowering third parties to use these resources to build services which can be used by citizens to get information from government or to conduct transactions.

Third party civic app developers are consumers of data from government – either through a government managed API or by taking raw data from a government open data portal and creating their own APIs. The roles of data publisher and data consumer in this scenario are clearly defined.

There are a number of reasons why the role of data publisher is a better one for governments – governments have historically not made good bets on technology. They’re not set up to do it properly, and as a result it’s not uncommon to see governments invest in technology that quickly becomes out of date and difficult to manage. This problem is particularly acute in relation to web-based services and applications – which outside civic technologists are very good at building – because the landscape for developing these kinds of applications changes far too rapidly for governments to realistically stay current.

There are also benefits for third party civic app developers in this scenario. As more and more governments open their data and deploy open data portals or custom APIs, civic app developers can build solutions that benefit not just one jurisdiction, but many others that may face a similar issue or need.

There is strong evidence to suggest that this approach is working. More and more governments are launching data portals and releasing open data, and more and more groups of civic technology advocates are coalescing around these platforms to build new civic technology solutions.

But we have enough experience with this model now to critically evaluate how well it works, and to determine if it will help realize the full potential of the civic technology movement.

What We’ve Learned So Far

I can think of three key takeaways from what can be described as the “first phase” of Government as a Platform. There may be more, but I think these are among the most important.

The line between data producers & data consumers is fuzzy

The open data movement has its roots in the government transparency movement that came before it and gave it life. As a result, the traditional idea of “open government data” views governments as producers of data and outside actors (journalists, researchers, developers) as consumers of data.

But early on in the open data movement, it became apparent that governments themselves stand to benefit greatly from the process of releasing data in open formats. Opening data for external consumers means that anyone – inside or outside of government – can access it. The process of making open data available for external users makes it more discoverable and more valuable for internal data consumers.

Governments – big ones especially – are notoriously complex and stove-piped. Open data can help mitigate the impact of this organizational complexity, particularly if it is structured for easy use by outside developers as an API. As a result, governments themselves are becoming major consumers of the data they release on open data portals and through custom APIs.

Building APIs can be as challenging as building apps

For all the potential that open data and APIs may hold for governments the process of planning, developing, and maintaining big, expansive APIs can be a challenge.

The task of building and managing custom APIs often falls – for a variety of reasons – to a government’s central IT agency. Making one group within a government’s technology hierarchy responsible for custom API development and management can sometimes create issues. Proper stewardship of APIs often requires specialized technical knowledge and may also require specialized domain knowledge to understand the data that is being exposed through the API.

In addition, along with the new role of managing a production-grade API for external development, governments face the (still relatively new) challenge of building and managing developer communities around them. Concentrating responsibility for all of this in one particular group within the broader government IT community can create a bottleneck.

And while this reality can make the out of the box APIs built into open data platforms more attractive, this option is essentially a “one size fits all” answer for the needs of data users. Most open data portals are not built to incentivize data standardization because the end goal is simply publishing – making the data available. This often leaves the hard work of cleaning and standardising government data to external users, or layers a SQL-like interface on top of disparate data sets which can add complexity to civic apps and make them more brittle.

Building reusable civic apps is hard.

There are some notable examples of civic apps that have been deployed in multiple cities. This is encouraging to see, and validates some of the original vision for Government as a Platform, but often this is the exception to the rule. Reuse – it turns out – is really hard.

There is significant technical and administrative heterogeneity between governments and this has made efforts to share open source applications like CivicCommons and the earlier Government Open Code Collaborative much less successful than most proponents would have hoped.

It is becoming more broadly acknowledged in the world of technology that the once sacrosanct mantra of “build for reuse” might not actually make sense in every scenario. Building for reuse is difficult and expensive. Some argue that developers should engineer for replacement instead of reuse.

So knowing what we know now about the concept of Government as a Platform, how should we design the architecture of open data and civic technology for the future?

I am increasingly thinking that the answer – at least in part – is to think smaller.

What are Microservices?

[T]he microservice architectural style is an approach to developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms, often an HTTP resource API.

— Martin Fowler, Microservices

Microservices is a term used to describe a specific architectural style for building software applications. In essence, a microservices approach envisions the building of an application as the process of assembling a number of different small, independently running services – each distinct from each other.

I find this approach to be particularly interesting in light of some other recent observations on how we build software – specifically, the “no stack” approach:

“Some characteristics [of no stack apps] is that they use other platforms and APIs (or protocols, in the case of SMS) to cobble together a service and in doing so rely almost wholly on those platforms and APIs for every function of the business other than the one they can be the best at.

— Andy Weissman, The No Stack Startup

The term microservices is particularly “buzzy” at the moment, and some people claim is just the latest articulation of a much older concept in the world of technology. Nevertheless, I think the idea of microservices holds enormous potential for the way that we build civic applications and addresses some of the key issues that have been encountered in the first phase of Government as a Platform.

Government as a Platform in a Microservices World

So what does the infrastructure of civic tech look like if we apply the principles of the microservice architectural style? Here is one way I believe it can look.

The centerpiece of this approach is not a government open data portal, but rather a centralized hub for routing requests and controlling access to different microservices – essentially, a reverse proxy. Governments would manage this new civic tech infrastructure by managing access to different resources through this proxy.

Services would be compact and focused on doing one thing well. Because of this, the development and management of different services could be farmed out across the government IT footprint (and even out to the civic tech community). The central IT organization would no longer need to have concentrated expertise in both API management and domain knowledge in the data being exposed. Agencies outside the central IT agency would become full partners in building and maintaining the civic tech infrastructure.

Services can be created and consumed by both governments and third party civic app developers. Governments may deploy services for things like address verification and geocoding, while civic tech companies might deploy services to check zoning designations and to do things as simple as parsing an address.

An area that seems particularly well suited for a microservice approach in the local government space is property data. Data on properties can span multiple agencies within a government and across multiple governments (e.g., city, county and state government). In Philadelphia, almost half a dozen different agencies maintain data on properties – the Assessors Office, the Planning Commission, The Revenue Department, The Water Department and the Department of Licenses & Inspections. In addition, there are non-City entities like the Philadelphia Redevelopment Authority and utility companies like the Philadelphia Gas Works, Comcast and Verizon that all have information on properties in the city.

Each entity’s view of a property is through the lens of the service or oversight they are charged with. One property can have multiple representations in the systems of various different government offices and agencies.

Imagine a scenario where the Department of Licenses & Inspections wants to issue a building permit or rental license to a property owner – a service for property tax balance could help them ensure that permits aren’t issued to owners with outstanding property tax bills.

Suppose the Revenue Department wants to set up a payment plan for a property owner with an outstanding tax liability – a service for gas utility payment would let them check if the owner is in arrears on other payments.

Consider a non-profit that wants to encourage urban gardening – a healthy collection of services around property data would make it easy to check for properties that are optimally zoned, contain enough permeable square footage to support a garden, determine if there are liens agains the property that might prevent such a use, and even to determine who the owner is so they can conduct outreach.

Microservices can make all of these activities simpler and more efficient.

Government should only do what only government can do. If we’ve found a way of doing something that works, we should make it reusable and shareable instead of reinventing the wheel every time. This means building platforms and registers others can build upon, providing resources (like APIs) that others can use, and linking to the work of others. We should concentrate on the irreducible core.

— Government Digital Service, Design Principles

In instances where governments don’t build services themselves, or seek greater control than indirectly leveraging the efforts of civic technologists, they could use existing small dollar purchasing options to contract for the development of new microservices.

Citizen-facing solutions would be built by government agencies as well as third party civic app developers. Civic apps would become simpler and more replaceable. With a reduced emphasis on engineering for reuse, developers could focus more directly on the needs of the people using civic apps. Interfaces can be tailored more efficiently and more cheaply based on our understanding of user needs or to respond to new policy imperatives.

This approach would better leverage both the internal (inside government) and external (civic tech) communities to deploy new services and make data more useful. A reduced emphasis on a homogenous technology foundation could make it easier to adopt new technologies as they emerge and reduce the risk of large scale system failures.

I don’t know if this is what the future of civic tech looks like, but I do think its time we started talking about it.

Almost a decade on, a lot of work has been done to implement the idea of Government as a Platform, and we’ve learned a lot along the way. It’s time to put the hard won knowledge we’ve accumulated to date to work building the next phase of Government as a Platform.

Note – Eric Jackson, co founder of Democracy Apps, has written a thoughtful response to this post with some great ideas for implementing the concept discussed here. Worth checking out.