Addressing the Data Deficit
In broad terms, the Global North is experiencing a “data deluge”, which provides many positive opportunities for socio-economic change. While the governance, access and quality of this data remain problematic (despite impressive technological advances), constraints on the amount of available data are not a concern. In the Global South, however, there is a relative “data deficit” because of significant constraints on the creation, collection and use of data.
Accurate, timely, disaggregated and accessible data are essential for governments to deliver services efficiently, fairly and transparently. As noted in a 2014 report by the Center for Global Development:
“Basic data like births and deaths, the size of the labor force, and the number of children in school are fundamental to governments’ ability to serve their countries to the fullest. And good data that are reliable and publicly available are a catalyst for democratic accountability. Data allow citizens to hold governments to their commitments. They allow governments and donors to allocate their resources in a way that maximizes the impact on people’s lives. And they allow us all to see the results.”4
One of the main challenges the poor face is being “under known”. They are often not referenced in databases that are generated from interactions with official organizations, as their interactions are infrequent. They may not even exist in government voter or birth registries, let alone in credit registries or other data sources. This “administrative invisibility” prevents them from gaining access to government services to which they are entitled and often prevents them from being offered life-enhancing services (e.g. credit) by the private sector as well.
A number of dimensions of the “gaps” contribute to the data deficit. The first is a “data gap”, which refers to data needed by the development community but that simply does not exist yet. The second is an “access gap”, which includes data that exist but that development actors and other stakeholders cannot access due to a lack of capacity, finances or agreements. The third is a “governance gap”, or the absence of legal, ethical and regulatory frameworks that enable and regulate the use of new data sources for international development.
Finally, there is a “usability gap”. Often, data are collected, made available and governed by clear rules but are not put to effective use because of a range of limitations affecting usability. For example, domain expertise to guide appropriate use of data and to enable users to locate data “needles” in data system “haystacks” is often missing. Additionally, the absence of communities of practice to serve as engines of learning, innovation and diffusion is a challenge. These gaps must be bridged to give decision-makers in every sector the tools to understand the impact of the policy and business decisions they make in a data-driven world.
Yet, despite these gaps, progress is being made. During the past 15 years (starting in 2000 with the UN’s Millennium Development Goals) major strides have been taken towards the monitoring and evaluation of core development goals within programme silos. But despite these gains, huge gaps remain that limit the ability of governments, donors, civil society and individuals to act and communicate fully and honestly in an integrated fashion.5 Addressing the data deficit can strengthen the dialogue between service providers and beneficiaries, taxpayers and governments, companies and employees and among the private sector, government and civil society regarding development issues.6
There are multiple dimensions to the data deficit challenge. They include resource constraints at the country level, misaligned incentives for accurate collection at the local level, competing agendas between outside donors and nation states, and the lack of incentives and agreements to encourage the sharing of private-sector data.7 Each of these dimensions presents a separate set of challenges which collectively can seem overwhelming. Nonetheless, there is no alternative to addressing these issues. Without access to data, ethical, fair and accountable governance is undermined.
Fortunately, in the new networked data landscape, sets of shared needs are emerging across stakeholder groups in both the digital North and South that can provide a basis for helping set the data-driven development agenda. With that said, constant attention must also be paid to those needs that are under-represented and for individuals who are currently “digitally invisible” and unaware of the potential impact of Big Data and its uses.
The “open data” movement presents a strategy to help address the data deficit challenge. By treating data as a common resource that can be freely used, shared and accessed by anyone, participants of the open data movement help inform global development efforts, strengthen civil engagement and improve donor decisions and policy-making. Incorporating open data sources into policy monitoring and evaluation processes enables governments to complement the mission-focused curating of relevant data sets that are accessible, interoperable and compatible with agreed-upon indicators. At the same time, it helps avoid costly and time-consuming data collection activities.
Likewise, improving access to private-sector data can serve to address the data deficit by augmenting and enhancing the statistical capacities of governments and development organizations. Many private-sector data streams have been identified as being directly relevant to governments and policy-makers. Location data, geospatial images from satellites, financial transaction data, logistics, supply chain details and the internal process data from enterprises are routinely identified as being of potentially high value in data development contexts. To date, the structures for sharing such proprietary data have not been standardized, so private data have been available only on an ad hoc basis.
To achieve the full potential of data-driven development, it will be necessary for all stakeholders – government, the private sector, development organizations, and the public – to work in coordination to bridge these gaps. However, successful collaboration has been elusive in the past. Reports on a wide range of development issues have concluded that public-private partnerships (PPPs) are only part of the answer. To be successful, stakeholders must each independently conclude that the sum is greater than the parts and that they can only achieve shared goals through collaboration.
The pervasive culture of not sharing data retards development. The private sector’s reluctance to share data is due largely to a utilitarian calculus of proprietary and competitive concerns that pervade market-based economies. They are not alone. Even within the UN system or among non-profits, a proprietary default position can make it difficult to get agencies to share programme data. Many concerns about sharing data are based on a lack of trust, a fear of incurring liabilities or a loss of institutional information control and arbitrage advantages (which create and maintain power differentials both within and between organizations).
Overcoming some of the issues that limit data sharing may require more direct forms of external compulsion in the form of improved oversight and enforcement, where such an authority is available (either by government regulation or enforceable stakeholder self-regulation). Table A (above) summarizes some of the private sector’s perceived risks and the suggestions for how they might be collaboratively mitigated.
The challenge for governments and the development community has been, and will continue to be, how to incentivize and/or compel the private sector to share certain data while ensuring legitimate and appropriate use. The perceived risks associated with sharing proprietary data are quite strong, mostly because they are not easily measured. The generic concerns are that sharing data costs time, money and personnel resources. It can impact competitive relationships, and misuse of the data can cause immense damage to a company’s reputation with a direct loss of earnings and market value.
Of all the various private actors holding unique and valuable data sets that could support sustainable development, the most widely recognized are mobile network operators. The precision, timeliness and depth of the behavioural insights that can be gleaned from these data streams make them extremely valuable and difficult to replicate by traditional data-collection procedures. While there are still many who are “digitally invisible”, mobile phone use is common among those living in even extreme poverty in many developing economies.8 Even without access to call content (which is typically prohibited), the granular data generated by the use of phones can provide richer information and understanding in the areas of health, finance, education and a variety of other needs.
Much has been written about the power of mobile data as a tool for gaining insights on individual behaviour.9 From a data deficit perspective, the big question is: why isn’t this data being used?
The reasons are a complex set of political, commercial, ethical, regulatory and reputational factors which reflect conflicts among multiple interests interacting in the ecosystem. These dimensions play out on multiple levels touching many varied stakeholders: business leaders, government policy-makers, donors, national security leaders, civil society, research organizations, shareholders and citizens. Each has their own incentives, priorities, real and perceived institutional constraints, and agendas. Few shared metrics and guidelines are available to enable “cross talk” among these stakeholders.
Right now we have a rare chance to take the dull world of data and make it headline news. As ONE’s Ebola tracker shows, bad data is killing people in West Africa right now. Bad data is helping corrupt leaders stay in power. Lack of data is making it hard for the poorest to hold leaders accountable. Data delivered by technology can’t alone guarantee transformation – but in the hands of actively engaged global citizens, a free media and responsible accountable governments, good data is the fuel of progress and a data revolution delivered in 2015 is the time to fill the tank.
Jamie Drummond, Co-Founder ONE
Some suggest that the hesitancy on the part of actors to remove barriers to data sharing is due to their discomfort with technology – older people have more limited understanding of advanced information and communications technology (ICT) capabilities, for instance, and this discomfort is deeply embedded in institutions. Also, as organizational representatives, people tend to act in conformity with often anachronistic policy constraints.
Achieving a balanced ecosystem – which is not to dismiss the strong incentives for incumbents to maintain power differentials and imbalance – is essential to realizing the full benefits of the data revolution. In some instances, this will require political will and leadership. More substantially, it will require the establishment of structures that can normalize risk and create unique risk reduction and leverage benefits. Addressing the challenges of the data deficit will require innovation in hybrid technology and policy architectures by the development community, private enterprises, governments, legal experts, ethicists and citizens. Stakeholders must enter new forms of dialogue and coordination (both informal and formal and across multiple sectors), and create new policy frameworks, supplemental institutional structures (such as public-private partnerships) and incentive structures.
Strengthening the Ability to Listen: The Question Box Use Case
As the UN Data Revolution report highlights, “Too many items that need to be known remain unknown.” One key concern is the growing number of “Data Invisibles” – individuals who are not counted within the formal or digital economy. These often include women, the elderly, children, migrants, indigenous populations and slum dwellers. Not being visible can mean that the ability to address domestic violence, the vulnerability of children, human trafficking and many other social concerns are absent or at best uninformed. During crisis events in particular, not listing individuals accurately in data records can have catastrophic consequences.
Question Box is an innovative ICT-for-development approach to addressing the data gap. Through the deployment of networks of free community call boxes in remote and rural areas, individuals can access information and services using their local language. Question Box networks are designed to attract specific subsets of a community through targeted service offerings. This approach promotes community-based reciprocity – information is exchanged in two directions – and local resilience, while producing unique data streams on “data invisible” populations.