In much of the work in the field of machine learning and deep learning, a bottleneck exists in the dataloading phase itself. This is becoming increasingly recognised as an issue which needs to be solved.

Benzina aims to become a go-to tool for dataloading large datasets. Other tools exist, such as Dali. Yet Benzina concentrates itself on two aspects :

  • Highest level of performance for dataloading using GPU as loading device
  • Creation of a generalist storage format as a single file facilitating distribution of datasets and useful in the context of file system limits.

Further feature points

  • Generalist DNN framework methods provided to integrate Benzina to PyTorch and TensorFlow
  • Command line programs will be created to assist in Benzina - compatible datasets
  • API interface to interact with Benzina