McArdle, Gavin and Kitchin, Rob (2015) Improving the Veracity of Open and Real-Time Urban Data. The Programmable City Working Paper 13. Working Paper. Programmable City Working Paper, Maynooth University.

Preview

Download (499kB) | Preview

Abstract

Within the context of the smart city, data are an integral part of the digital economy and are used as input for decision making, policy formation, and to inform citizens, city managers and commercial organisations. Reflecting on our experience of developing real-world software applications which rely heavily on urban data, this article critically examines the veracity of such data (their authenticity and the extent to which they accurately (precision) and faithfully (fidelity, reliability) represent what they are meant to) and how they can be assessed in the absence of quality reports from data providers. While data quality needs to be considered at all aspects of the data lifecycle and in the development and use of applications, open data are often provided ‘as-is’ with no guarantees about their veracity, continuity or lineage (documentation that establishes provenance and fit for use). This allows data providers to share data with undocumented errors, absences, and biases. If left unchecked these data quality issues can propagate through multiple systems and lead to poor smart city applications and unreliable 'evidence-based' decisions. This leads to a danger that open government data portals will come to be seen as untrusted, unverified and uncurated data-dumps by users and critics. Drawing on our own experiences we highlight the process we used to detect and handle errors. This work highlights the necessary janitorial role carried out by data scientists and developers to ensure that data are cleaned, parsed, validated and transformed for use. This important process requires effort, knowledge, skill and time and is often hidden in the resulting application and is not shared with other data users. In this paper, we propose that rather than lose this knowledge, in the absence of data providers documenting them in metadata and user guides, data portals should provide a crowdsourcing mechanism to generate and record user observations and fixes for improving the quality of urban data and open government portals.

Item Type:	Monograph (Working Paper)
Additional Information:	NIRSA; National Centre for Regional and Spatial Analysis; Veracity; Open data; Real-Time; Urban Data; Smart City;
Academic Unit:	Faculty of Science and Engineering > Research Institutes > National Centre for Geocomputation, NCG Faculty of Social Sciences > Geography Faculty of Social Sciences > Research Institutes > National Institute for Regional and Spatial analysis, NIRSA
Item ID:	7237
Identification Number:	https://doi.org/10.2139/ssrn.2643430
Depositing User:	Prof. Rob Kitchin
Date Deposited:	15 Aug 2016 11:12
Publisher:	Programmable City Working Paper
Refereed:	Yes
Funders:	European Research Council Advanced Investigator Award, Science Foundation Ireland (SFI)
URI:	http://www.nuim.ie/progcity/ http://ssrn.com/abstract=2643430
Use Licence:	This item is available under a Creative Commons Attribution Non Commercial Share Alike Licence (CC BY-NC-SA). Details of this licence are available here

Repository Staff Only(login required)

Item control page

Downloads

Origin of downloads