Are Scientific Annotations Consistently Represented across Science Knowledge Graphs?

Scientific Knowledge Graphs (SKGs) are increasingly used to annotate and interlink research outputs. However, little is known about how consistently they annotate the same publication. This paper presents a comparative analysis of category annotations across four major SKGs (ORKG, OpenAlex, OpenAIRE, and Papers with Code) using a manually curated gold-standard dataset of 70 AI-related papers. We examine differences in annotation coverage, granularity, and semantic alignment, highlighting frequent inconsistencies such as label mismatches, overly generic terms, and coverage gaps. Our analysis reveals that manual curation offers high-quality but sparse annotations, while automated systems achieve broader coverage at the cost of precision. This work contributes insights into the reliability of SKG metadata and outlines pathways for improving interoperability and annotation practices.

Jenifer Tabita Ciuciu-Kiss, Daniel Garijo · Sci-K 2025 at ISWC, CEUR Workshop Proceedings 4065

Read PDF Related project DBLP record

This paper compares how scientific knowledge graphs annotate tasks and methods in shared AI publications. It identifies differences caused by classification schemas, editorial choices, and automated or manual curation.

Citation

Ciuciu-Kiss, J. T., & Garijo, D. (2025). Are Scientific Annotations Consistently Represented across Science Knowledge Graphs? Sci-K 2025, 14–28.

BibTeX

@inproceedings{ciuciukiss2025annotations,
  title={Are Scientific Annotations Consistently Represented across Science Knowledge Graphs?},
  author={Ciuciu-Kiss, Jenifer Tabita and Garijo, Daniel},
  booktitle={Proceedings of the 5th International Workshop on Scientific Knowledge},
  pages={14--28},
  year={2025}
}

Find citation updates on GitHub, Google Scholar, ORCID.