Data pipelines are often a critical part of the infrastructure of interdisciplinary data science, that often goes beyond a simple connecting tool to have lasting impact on which and how questions are posed and answered. Unfortunately, those pipelines are severely challenged as soon as we move away from a “common task” model where a single prediction task is attempted in parallel by multiple teams using data that are made publicly available. Multiple application areas of data science have more complex requirements, that involve collaboration between entities with different data access, and an application task that may evolve with inputs from both side as limitations of formulation and analysis techniques affect the results. At the same time, pipelines are required to become more rigourous and trustworthy in ensuring that the concerns on data bias affecting high stake decisions are met with solid guarantee either local or global on its operation.

Over the course of three years the team of this project have experienced the everyday effort of interdisciplinary data science (from multiple domains: sociology, environmental science, psychology and public health, economics, media studies). At the same time, it creates tools and analyses aiming at documenting the opportunity of more complex data pipelines, on topic including but not limited to the theoretical limits of prediction, validation/invalidation of observations from limited sample. It also identified opportunities, when data is reuse among multiple parties, that fairness guarantees can be deployed faster using incentives.

In addition to research publications (published in multiple journals and conference, in multiple disciplines and computer science) that provided insightful knowledge, we focused on delivering these research outcomes in a sustainable, repeatable way - keeping with the times of open source movement, frameworks that address reproducibility crisis and development of scalable, sustainable communities of software development and usage. We here highlight the three powerful axes along which we generated key outcomes and highlight our unique achievements. 

Open source libraries: We have developed and delivered numerous “usable” open source libraries. These include, Ballet, MLBlocks, MLprimitives (both part of MLBazaar), Cardea. We highlight the word “usable” to mean that they have all followed the standard software engineering practices, multiple releases, continuous integration, documentation, testing. This creates a sustainable ecosystem beyond the timeframe of this project. Most of these libraries have active community usage and have gone on to solve impact problems in different domains. 

Empirical studies: The teams conducted numerous empirical studies both of collaborative data science (without the platforms) and of collaboration of data scientists and domain experts on the platforms. 

Closing the loop: Studies of how collaboration happens, solving a societal problem using data and developing platforms that are informed by the former two is a tough endeavor to balance. Often research covers one or two of the axes. Our collaboration resulted in tying together these three threads into a set of coherent outcomes that we hope are informative, usable in future development and progress in this direction. 

The project also offered multiple opportunities to broaden the base of leaders bringing data science and AI to multiple domains by organizing a summer school on computational social science for graduate students and a highschool program empowering diverse voices to join AI in its application to multiple fields.

Last Modified: 10/21/2022
Modified by: Augustin Chaintreau