The paper introduces Visual Genome, the largest publicly available dataset of structured image annotations. It provides dense region descriptions, objects with bounding boxes, attributes, pairwise relationships, region graphs, scene graphs, and over 1.7 million question‑answer (QA) pairs across 108 K images. The authors detail the collection pipeline, present extensive statistics on objects, attributes, and relationships (including frequency tables and per‑category distributions), and describe a canonicalization process that maps all textual elements to WordNet synsets, achieving high precision and recall. Baseline experiments evaluate attribute prediction, relationship prediction, region description generation, and visual QA, revealing that attributes are more learnable than relationships, that region‑level captioning benefits from the dataset’s specificity, and that simple answer‑frequency baselines perform poorly on “why” and “where” questions. The paper also outlines future applications such as dense image captioning, semantic image retrieval, and using scene graphs as evaluation metrics, positioning Visual Genome as a comprehensive resource for advancing vision‑language research.
Topics
Visual Genome Dataset Overview: Scale, Structure, and Statistics
Introduces Visual Genome as the largest publicly available structured image dataset, detailing its components—objects with bounding boxes, attributes, pairwise relationships, region graphs, scene graphs, and 1.7 M question‑answer pairs across 108 K images. Provides extensive statistics such as frequency tables for objects, attributes, and relationships, per‑image averages of region and scene graphs, and highlights the distribution of spatial versus action‑oriented predicates.