Guiding Users through Diverse, Dispersed Data Ecosystems

Biological data is increasingly diverse in its nature and dispersed across many repositories. While debate often centres on the merits of federated versus centralised repositories, this can be a red herring: “federated” repositories involve distributed effort, and physically distributed data, but typically depend on a common schema and technology. Where no such unity exists, a first-parse approach to data integration can be achieved through the provision of an integrated search, whereby partners agree to supply basic meta data using agreed formats and vocabularies to a common search engine. This makes data findable and accessible (assuming that data would only advertised in search where the provider wishes to make it so), but does not directly address the problem of data (as opposed to meta-data) interoperability and thus re-usability. These latter problems require the commitment of domain experts to the development of richer data models, and their adoption by the wider community – and in effect, the transition to a full federated or centralised system based on this common schema.

In this talk, I will contrast traditional centralised and federated successfully dealing with common data types databases (e.g. the nucleotide sequence archives, InterPro), to a more lightweight approach we have taken in the transPLANT and EXCELERATE projects to make dispersed plant genotype and phenotype data searchable. I will explore the role of Application Programming Interfaces (APIs) in such a system and I will discuss the development of MIAPPE (a data standard for plant phenotyping data) as a basis for deeper integration in future.