ReFigure – Connecting Scientific Insights Across the Web

ReFigure was born from an idea that research outputs should be incremental, immediately connected to published findings and not confined to the websites of individual journals. For instance, there should be a way to rapidly curate a collection of related figures across publications based on similar experimental hypotheses, making it easier to identify whether the findings were reproducible. We also wanted to highlight new insights by connecting our own research findings, whether reproduced, negative or incremental new findings, to relevant, previously published research.

After spending many years at the bench together, we became increasingly frustrated that our daily and weekly scientific outputs, though concrete and valuable, had no place in traditional publications focused on the themes of “big”, “novel” and “complete stories”. The conventional publishing model is based on sharing results at the end of a major project. Yet in reality bench science is incremental, building on previous work with empirical evidence. While data and preprint repositories aim to disrupt the publication bottleneck of incremental outputs, they generally suffer from a lack of peer review mechanisms and poor discoverability of data. This results in much of this otherwise valuable incremental data being perceived as ‘noise’.

Illustrating the major disconnect between the actual timeline of scientific outputs and the conventional publishing cycle (Girija Goyal and James Akin)

Introducing ReFigure

In version one of ReFigure, we have built a simple Chrome extension that allows users to connect figures across both publishers and publications, as well as their own data. Our beta version supports research materials available through eLife, PLOS, PubMed Central (PMC) and Figshare. You can download the Chrome extension here.

//www.ncbi.nlm.nih.gov/pmc/ with the active ReFigure extension)ReFigure values scientific insight. The entries captured by ReFigure could link reproduced experiments, compile studies using a reagent of interest or highlight a novel connection between two studies. This includes uncovering connections between different studies that may only be evident to a daily practitioner in the field, or connecting related data across experiments, whether reproduced or negative. By facilitating both discoverability and curation, ReFigure helps researchers at all levels disseminate research data wherever it’s deposited to enhance impact and reuse. In the future, we want to build the capability of scientists creating and reviewing ReFigures collaboratively providing a crowdsourced version of peer-review.

On the four supported domains, the ReFigure extension further includes notifications to tell you if the article you are reading has been “ReFigured” by other users, helping you to discover other people’s insights. Our beta users liked learning that someone had “added to the edifice of knowledge” with a ReFigure and being able to navigate to these new insights easily. A reviewer described the experience as reading a “paper on steroids” as they could easily see related research findings.

ReFigures get published on ReFigure.org, allowing users to have a central access point for managing their own ReFigures. ReFigure leverages the availability of open-access research, showing the value of connecting research artefacts across domains, publishers and repositories. Important connections between discoveries can be made when there are no paywalls in the way. ReFigure.org is itself open access, improving the discoverability of incremental data and curated insights.

Exploring a ReFigure on ReFigure.org (Screenshot from ReFigure.org)

See ReFigure in action

We have demonstrated the utility of ReFigure with data from the Reproducibility Project: Cancer Biology (RPCB). The aim of the RPCB is to conduct and disseminate replication studies of a number of high-profile cancer papers. Trying to read the replication studies and comparing them to the original studies can be tedious; you might find yourself bouncing between browser tabs, trying to find out the replicated figure that corresponds to the original figure. On ReFigure, you will find the key figures side by side, alongside expert commentary from our team. By doing this comparative analysis, we were able to harvest important and independent unbiased insight. We welcome readers to create their own RPCB ReFigures. We explain how we made this ReFigure, and more, in this video:

The Zika and Ebola epidemics highlighted the need for rapid and trustable communication of the latest research. With the conventional publishing and current academic incentive structures, it would take years for Christa Osuna, a senior postdoctoral researcher, to publish the insights she gained whilst reading the literature in her field. Even these would only be publishable if they fit a narrative suitable for a traditional research article. Christa Osuna reviewed three single-patient case studies on Zika virus detection and created a ReFigure showing that when considered in combination, these studies suggest Zika infection is detectable longer in semen than in blood. Christa’s ReFigure was a valuable early resource suggesting that these findings might reflect a larger trend. To communicate this valuable insight, Christa did not have to write a lengthy review. Instead, by creating a ReFigure, she was able to collate the relevant figures across publications with a few clicks.

A traditional multi-patient study was conducted and published later by an independent group (de Laval and Matheus et. al, 2017) showing that Zika infection was detectable in semen in only a subset of male patients. The early ReFigure by Christa followed by the cohort study demonstrates the need for a balance between rapid communication of insights versus the formal process of preparing a multi-experiment/patient narrative for peer review through the traditional publishing process. In fact, ReFigure allows users to update their ReFigure with new data allowing Christa to keep her curated insights up-to-date.

Another use case for ReFigure is to collect figures related to a material or method of interest, particularly when you are designing your next experiment. Here’s one we made to see how well an anti-tau antibody performed in experiments performed across separate labs. To try it out for yourself, follow our instructions. You can include your own single-figure findings to improve the discoverability and hence impact of your own research, particularly data that doesn’t make it into a full research article.

As we designed ReFigure, we wanted to make sure that the tool would make it easy for researchers to use for the intended purpose: connecting research artefacts across different publisher websites. To stay close to prospective user’s needs and behaviours, we conducted surveys and held focus groups to test our assumptions and refine our designs. An early survey with 69 respondents identified a heightened awareness of interim research products as well as concerns about sharing early data. Multiple focus groups ranging from 2-15 participants provided insight that steered our development.

For example, we realised that questions about connections and reproducibility often come up as the reader is perusing a paper. It is at the time of reading, as these insights crystallize, that they should be shared and discoverable to others. We also learnt that a key element of any tool built for a diverse, non-technical user group is the design of the user interface. As representative non-technical scientists, we often run aground when trying a new tool with sometimes complex technical documentation. Much as the term “aliquot” might be cryptic to programmers, “parser” means nothing to most biologists. With these pieces of information in mind, we decided to realise our idea in the form of a browser extension – if you know how to read papers on the web, you know how to use ReFigure.

In publishing, like any industry, legacy systems are hard to change and can stifle innovation. The support for ReFigure from the eLife Innovation Initiative demonstrates that publishers can foster innovation. We created ReFigure using a fraction of the resources commonly available to even the smallest publishers.

Still, this is only the beginning. A major technical hurdle to making research communication tools that interact across publishers’ websites is the lack of any HTML standard for scientific manuscripts, even within a field. Thus, for ReFigure, figure and metadata capture had to be coded independently for PMC, eLife, PLOS and Figshare. Further, any changes to HTML code on the publisher site have to be monitored, and domain handlers updated, to guarantee a good user experience on ReFigure.

Going forward, we invite the community to help us build on ReFigure v1.0. Specifically, as ReFigure is released under the MIT open-source license, we invite the developer community to support the project by contributing domain handlers for additional publisher domains. This will help us broaden ReFigure’s reach in connecting scientific outputs.

We welcome contributions and feedback. You can find the project on GitHub and you can reach us on Twitter.

While formally supporting the development of ReFigure with the eLife Innovation Initiative, the team at eLife also advised us on the ReFigure UI/UX. We also thank Mozilla Science Lab whose inclusive gatherings taught us principles of open development, including the need to simplify our message.