Opening Up the Inner Workings of New York City

On March 12, Mayor Michael Bloomberg signed the open data law. The law requires city agencies to make their data publicly available online over the next six years. Photo courtesy of NYC Open Data

What goes on behind closed doors is increasing available on your own desktop or mobile device when it comes to the inner workings of New York City government agencies.  In 2009, the city of New York created an open portal website through which agencies can release their data online for the public’s use. While still in its infancy, this open data movement is a a giant step for transparency advocates and democracy in general. One of the most fascinating impacts of the open data and open source (software code that’s available to the public to improve and reuse) movements has been the influx of new web tools, developed by private companies and nonprofits, that help people better engage with, and navigate, their city.

In March,  Mayor Michael Bloomberg signed a law mandating that all city agencies put their data online over the course of the next six years.

The Department of Information Technology and Telecommunications (DoITT), which oversees how new technologies are being used by other city agencies, began putting city data online in a Socrata site — technology created in Seattle — in 2011, and will enforce the city’s new requirements.

The benefits of open data can be seen in the work of the nonprofit company OpenPlans, which has been at the forefront of the open source movement in New York City. The products and services it creates using data and code from the MTA and other city agencies illuminate how New Yorkers might live in the near future, as the physical and digital versions of the city merge together.

Open Source Software and Transit

A screenshot of the MTA BusTime mobile app. OpenPlans created the software, which allows users to see where their next bus is. Image courtesy of OpenPlans.

Rather than craning their necks into traffic, people waiting for buses on Staten Island,  the B63 in Brooklyn and the 34th Street services in Manhattan can now glance at their cell phone to find out where the bus is.  Free to use, BusTime is a software built by OpenPlans, using the MTA’s open data standards and open source software to track buses locations.  “It’s a very different way for government to do things. Typically there’s some sort of closed system and there’s a vendor,” said Mark Gorton, OpenPlans’ founder and chairman.

By a closed system, Gorton is referring to the typical model governments use when contracting with tech companies. For example, when the Taxi and Limousine Commission wanted to install GPS systems in all the yellow cabs, the city agency contracted with a private company, Sense Networks, to install the hardware and manage the data.  This arrangement means that even if the city can now tell how fast traffic is moving, the data is owned by Sense Networks. If another developer dreams up a brilliant new way to use the taxi data, they’ll have to negotiate with Sense Networks and will probably pay it a great deal to access the data.

BusTime is different from the taxi model. OpenPlans has an agreement with the MTA and its project partner, Cambridge Systematics, that allows anyone access to the real-time bus data to improve or expand on the service, create new digital tools and allow other cities to create similar services.

“At MTA headquarters there’s a broad understanding and commitment to the power of government 2.0,” said Ernest Tollerson, the MTA’s director of environmental sustainability and compliance. “Over time I think you’ll find that advances in open government will lead to improvements in operations and maintenance.”

New Initiatives and Government Engagement

Mark Gorton founded OpenPlans in 1999. According to him, governments tend to adopt to new technology very slowly, but there are a few visionaries in New York City government who are helping to enact change. Photo courtesy of OpenPlans.

Gorton, an avid bicyclist and advocate for livable streets, founded OpenPlans in 1999 with the goal of getting governments to adopt open source software, with a specific focus on transportation reform. While there is a small, but vibrant open source community in New York City, OpenPlans is unique in that it’s a nonprofit. Gorton owns the hedge fund Tower Research Capital and founded LimeWire, a file sharing network that was shut down by the federal government in 2010. He uses his own money, as well as funding from other foundations and investors, to fund OpenPlans’ projects.

Besides collaborating with governments, the 63-member company launches a wide variety of independent initiatives, which ideally become self-funding over time. Past initiatives have included the sustainable transportation news websites, Streetsblog and Streetfilms; the education news website, Gotham Schools; the online marketplace for sharing mobile apps with government, Civic Commons; and OpenGeo, a geospatial tool that crowdsources online government data, allowing citizens to build applications around it.

This is a screenshot of OpenPlans' Shareabouts app. The software is in development, but will add users to add content to a map of New York City.

In terms of how a company like OpenPlans works with government, there are a few different routes.

In recent years, the MTA and the City of New York — in a project managed by DoITT — have both created contests in which private developers use city data to create new mobile apps, and the winners receive prize money. Because the apps are created for free, the city saves untold thousands of dollars by not having to pay someone to build the software, and start-ups  get free advertising and a check.  It’s not necessarily going to attract more entrenched developers, but scrappy developers flock to these contests.

City agencies also issue requests for proposals for new open source software, which was how the MTA connected with OpenPlans, or directly approach a company to build software.

But the most common occurrence, said Gorton, is for an open source software development company to approach the city with its own ideas.

“What we typically do with a software contract is we find the initial work internally by ourselves. We get the software project to a certain state where we can go to different government agencies where we can say, ‘look, we have this,’ and they say ‘do you want to get a contract to push the development forward and have us work on your project?'” Gorton explained.

Most commonly, however, government agencies will contract with a company that wants to use closed source data, according to Gorton. This option is often more lucrative in the short-term for a for-profit developer, because other companies will have to pay them if they want to use the data. Thus, closed source contracts can quickly attract the most reputable for-profit companies. It’s also just a matter of government bureaucracies being slow to change.

“It’s not like you can just have one sort of top-level announcement that makes this happen. You actually have to change every single software system in this city and how they’re procured and integrated,” said Gorton. “And what you find is that there are some people in government who are very dynamic and innovative, and are looking to do new things and make things work better, and are willing to fight the internal obstacles preventing things from happening. And then there are other people for whom it’s a lot easier to just go along with things the way they are and not innovate.”

Besides the MTA, which isn’t a city agency, the more forward thinking city agencies OpenPlans has worked with include the New York City Department of Transportation (DOT) and DoITT.

On the topic of closed or open systems and government vendors, Nicholas Sbordone, the director of external affairs for DoITT, commented, “I would say that if I’ve learned anything in my time with the city, it’s not always a one size fits all solution. I think where there are opportunities to use open source, we certainly try to do it,” said Sbordone. “But there’s not always going to be an open source solution out there the day you need it, when there’s a priority that needs to be met because there’s an exigent public need.”

In the last decade, the DOT has implemented a wide range of technological solutions, like parking sensors and a Tumblr account to track the city’s potholes. OpenPlans helped the DOT create open source websites for New York city neighborhoods, where community members can contribute to the neighborhood planning process and use city data to add and expand features, like maps and calendars. Currently, OpenPlans is working with the DOT to develop a GPS tracking system for the upcoming bikeshare program, and developing Shareabouts, a crowdsourced app that will allow users to add information to a map of the city.

OpenPlans has a contract with DoITT, which will use the code of OpenPlans’ geospatial mapping software, OpenGeo, to create their own new applications in-house, like an interactive map that shows which streets are closed at any given time.

Besides its work with OpenPlans, DoITT has been instrumental in connecting the city’s agencies to new technology and promoting open source solutions over the last decade.  DoITT worked to develop the city’s high speed internal broadband network; unite all the city’s agencies under the 311 call center; connect City Hall, the FDNY, NYPD and the Office of Emergency Management to real-time video from accident or crime scenes; automate meter readings across the city; and run the city with Redhat, an open source Linux operating system. (Redhat is the first open source company to make a billion dollars in revenue in a single year.) Currently, DoITT is working with the DOT to create an open source application to track bikes that are part of the bikeshare program, so that the city can see where new bike lanes are most needed.

When governments contract with private tech companies

One issue, as evidenced by the taxi GPS system, is that cash-strapped governments often think in the short-term when it comes to software contracts, either because they need quick dollars or they need to get things done as expediently as possible. That’s not necessarily good for open source opportunities, which tend to save taxpayers money in the longterm, or the public process.

A debate surrounding open source contracts, as well as other technologically oriented public-private contracts, concerns whether the city should pay companies (or get paid, in some cases) a flat rate or through a revenue sharing model. Neither of OpenPlans contracts with DoITT or the MTA are revenue shares.

“Some companies like the revenue-sharing model, but I have mixed thoughts about whether it works or not,” said Jeff Maki, principal of transportation projects for OpenPlans. In the BusTime contract, the MTA paid OpenPlans a flat, one-time rate. But in a revenue-sharing contract, which are very popular in city tech contracts, the retailer agrees to pay the supplier a certain amount for a product of service, plus a percentage of what the retailer makes over a certain amount of time. For example, in 2008, the City of Chicago entered a revenue sharing contract with Morgan Stanley to privatize its parking meters as a way to recover from the recession. Morgan Stanley paid the city over $1 billion for the contract — quick cash for Chicago — but Morgan Stanley anticipates coming away with over $11 billion over the next 75 years.

But that’s just the tip of the iceberg when it comes to growing concerns about software privatization.

Since 1996, the city’s spending on private contracts jumped from $5.7 billion to $10.5 billion per year, reported City Limits. Under Bloomberg, that spending has been directed heavily toward contracts with tech companies. There is notoriously weak oversight over the contracting process, especially with complex technology deals, which can sometimes result in waste and fraud, as in the infamous CityTime scandal that went nearly $700 million over budget.

Sbordone wishes the city’s technological achievements that have saved taxpayer money would receive the same attention as CityTime, but agreed that the city should be more mindful of its contracts. He also noted that the city is making an effort in that direction.

“On October 31 of last year, the deputy mayor [Cas Holloway] gave a testimony where he talked about a number of measures we’re going to put in place,” said Sbordone.

One of those measures recently moved forward in City Council. Councilmember Dan Garodnick introduced a bill designed to offer greater whistle-blower protections to employees of private companies working on city contracts.

Since OpenPlans occupies a unique position as a nonprofit, the risk of greed-induced corruption is reduced, but as the city offers more open data contracts to for-profit companies, the risk of corruption inevitably increases. At the same time, there are fears that if the code, used by the city but owned by a private company, crashes, then taxpayer dollars will be burned. Is there a solution to these problems?

“I think there’s two things that need to be done,” said Maki. “If the city published those things [costs associated with private contracts] in a shorter time frame, someone could have noticed this [CityTime] went millions over budget. If someone was poring over these numbers, some advocacy group could have found out, ‘hey, there’s a problem here.'”

As to the problem of a code crash, Maki says, “If code can be made open and in the public domain, even if that code goes down, you still have other options.”  Open source software allows a community of developers to fix a problem, or create alternative solutions, if the code itself is corrupted, so the taxpayer money spent on the technology isn’t wasted.

What kinds of developments might we see in a few years?

“We want to do more as fast as we can and within budget, and release more dynamic data that will give people more control over how they interact with our system. You’ll see TrainTime [like the subway countdown signs] on Metro-North,” said Tollerson of the MTA.

“I do think the tendency is more of these urban systems will have third parties interact with them. Taxis may have have an API [application program interface, which allows two computers to talk to each other] to allow people to pay for taxis with their phones,” said Maki of OpenPlans. “City operating systems may have an API, whereas before, APIs were things computers did.”

Generally speaking, it’s not clear what the possibilities for new open data will be until people see what’s released. But Andrew Nicklin, DoITT’s director of research and development, has some ideas.

“More start-ups and private tech organizations that use our data to grow will be one thing we’ll see,” said Nicklin, who added that new data will likely be capitalized upon by academics tinkering with new theories, and will foster more sharing of information between government agencies. “We’ve never really had an internal place where there’s a catalog of all the data that the city has. So in a large city like ours, one agency might not be aware that there’s another agency with data that’s useful to them.”