Bad metrics and PDF graveyards: why development needs open data
For too long the status quo has been to simply make knowledge available. But that's not enough, data must also be actionable. Nathaniel Manning reiterates the importance of open data. It is paramount to share data throughout the development sector, most importantly getting it into the hands of the technologists in the developing world. The sector would benefit tremendously by embracing a collaborative, open development model in which not only financial and human capital is shared with the developing world but also the rich value of information capital. Open data has a multiplier effect for development. The original work creates the benefit that it was specifically intended to do, and opening up the data produced from that work ignites entrepreneurship, creates jobs, and thus lifts people out of poverty in a local, sustainable way. Development organisations can bring their information, and with it the opportunity to build solutions, to local people who know their problems best. In this way the development community can shift from doing work for people in developing nations to working with people in developing nations. Places like the iHub in Nairobi, a tech startup, has grown in four years to have over 11,000 members, and 125 tech companies that have created over 750 jobs. These organisations have proven that there is a serious technical talent blossoming in the developing world. But it needs raw materials, like data and knowledge, to create technology magic.
The developers in Nairobi or Bangalore know their problems better than anyone. And if the lean startup trend – favouring experimentation over elaborate planning, customer feedback over intuition, and iterative design over traditional big design development – has taught us anything, it is that a successful startup is one that solves a real problem, and a company is successful when it understands the problems and needs of it's customers.
Companies like mFarm, a tech startup in Nairobi, have capitalised on intimately knowing a problem and solving that problem by putting data to use. It gathers agriculture market price data and sends daily prices to farmers via SMS and enables a marketplace for selling those products directly to buyers.
The iHub has built one of the first user experience labs in sub-Saharan Africa. It is also building a hardware lab called Gearbox. These shared services models are reducing the barriers to access equipment needed to build technology; and development organisations can do the same by opening up data.
For that the development community should adopt three key initiatives. First, open data does not only refer to transparency. PDF reports on an online report-archive are available knowledge, but they are not actionable data. We simply digitised the old analogue system, instead of redesigning the way we manage data using IT.
Open data refers specifically to machine-readable data – data that an entrepreneur, a programmer, can take and turn into something valuable. For instance, the US government released GIS and weather data, and the companies built directly from using that data contribute more than $100bn to the US gross domestic product.
The status quo in development reporting practices is built on the foundation of the PDF report. This is understandable. There are often numerous different documents used to make a single project report, including excel models, GIS shapefiles, and Photoshop charts. The ease of taking screenshots and putting it all into a PDF report, and sending it along via email is completely understandable. But this is like funding James Cameron to make Avatar, and then releasing it in a black and white flipbook. We are missing all the good stuff. This has to change.
In the US, chief technology officer Todd Park and USAid administrator Raj Shah have led this open data for collaborative development model within the government, with the White House issuing an executive order to release data in an open and machine-readable format.
Development organisation, both large implementers and funders, need to invest in the enterprise systems and tools needed to make systemised data collection, preparation and publication as fluid as the Japanese assembly line. Mobile devices and crowdsourcing tools like Ushahidi make the ability to collect data easier and cheaper, open source tools for cataloguing open data, like CKan are free and easy to deploy. Once the data is open, don't wait for a miracle, engage and collaborate with the technology communities in the developing world
"Give a man a fish, he will eat for a day. Teach a man to fish, he will eat for a lifetime." The equivalent of teaching someone to fish in the digital age is putting both the tools and the data into the hands of the people in developing nations who know their own problems.
The digital age offers the development community the ability to shift to a more collaborative approach that could yield higher impact returns. Integrate open data into all facets of organisation. Systemize data collection, preparation and publication across the organisation using digital tools. There is no longer an excuse for bad metrics, PDF graveyards, prohibitively expensive monitoring & evaluation, or closed datasets. Adopt a collaborate development model, gather your data and share it in machine-readable formats, encourage and co-operate with the entrepreneurs and technologists in the developing world, and stop the data divide.
Nathaniel Manning in San Francisco