Voyager: Data Discovery and Integration for Data Science

Abstract

Data discovery and integration have grown to become two important research fields in both academic and commercial domains, mainly fueled by the ever increasing availability of datasets that are stored by organisations without their conceptual meaning or relationships being explicitly known. These tasks can be carried out in different settings and for different purposes; here we focus on the collection of tasks performed by data scientists to acquire the knowledge needed when deciding what analyses to perform on client data. In this paper, we focus on support for three processes often encountered in practice by data scientists: data identification, data understanding and relationship discovery. We describe our practical experience with each of these processes and the means by which we assist data scientists in performing them. We have been informed by real–life use–cases in identifying the tasks carried out routinely by data scientists at Peak AI. The paper reports the design decisions made in the development of a system to support data discovery and integration, and reports on an evaluation that investigates both usability and task efficiency.<br/