فاين فيجن: فتح قوة البيانات المفتوحة لنماذج اللغة والرؤية

FineVision is a groundbreaking open dataset designed to enhance the training of Vision-Language Models (VLMs). Released by a team of researchers from Hugging Face, this dataset aggregates over 200 existing datasets, culminating in 24 million samples that include 17 million images and 89 million question-answer pairs. The total size reaches an impressive 5TB of meticulously processed data, aimed at fostering innovation in the rapidly evolving field of VLMs.

Key Features of FineVision

• Extensive Data Collection:

  • Over 200 datasets were sourced and unified.
  • The dataset includes 17M images, 89M question-answer turns, and 10B answer tokens.

• Quality Control:

  • All data was cleaned to remove duplicates and irrelevant content.
  • Each question-answer pair was rated across four qualitative metrics, ensuring high-quality training data.

• Performance Benchmarking:

  • Models trained on FineVision outperformed existing datasets across 11 common benchmarks.
  • FineVision demonstrated a 40.7% improvement over the Cauldron dataset, and even greater performance when comparing deduplicated versions.

Why FineVision Matters

• Accessibility:

  • As VLMs grow in complexity, the need for accessible training data has become critical. FineVision addresses this gap by providing a comprehensive and open dataset.

• Diversity and Scale:

  • The dataset not only offers a larger volume of data but also a diverse range of topics, including underrepresented domains like GUI-oriented data.

• Flexible Training Options:

  • Users can create custom training mixtures based on the dataset’s extensive subsets, allowing for tailored training experiences.

Data Processing and Rating

• Data Cleaning:

  • Samples exceeding 8192 tokens in length were removed, and images were resized to maintain quality.

• Rating System:

  • Each turn was evaluated using a pipeline of language and vision models across four axes: Formatting Quality, Relevance, Visual Dependency, and Image-Question Correspondence.

Experimental Insights

• Training Configuration:

  • The research utilized a 460M parameter VLM model, experimenting with various training setups to determine the best performance outcomes.

• Contamination Analysis:

  • FineVision showed the least contamination from test datasets compared to other baselines, ensuring the integrity of its training data.

• Diversity Metrics:

  • FineVision not only leads in size but also in diversity, outperforming other datasets in how evenly its data represents various visual features.

Conclusion

FineVision represents a significant advancement in the field of Vision-Language Models, providing a robust, high-quality, and open dataset that encourages further research and practical applications. By addressing the challenges of data accessibility and quality, FineVision sets a new standard for training VLMs, empowering developers and researchers to push the boundaries of what these models can achieve.

Source: Hugging Face