DAPCOM Data Services, a spin-off of the University of Barcelona (UB) and the Polytechnic University of Catalonia (UPC), has developed FAPEC Archiver, a software tool that reduces the size of massive files by up to 50% more than current compressors.

The program was conceived as part of the European Space Agency’s (ESA) Gaia mission, which aims to create a three-dimensional map of the Milky Way. The Gaia group of the UB Institute of Cosmos Sciences plays an essential role in the data processing of this mission. FAPEC has been instrumental in compressing the mission’s second catalogue, which collects the position of 1.7 billion stars and includes data on the distance, motion, and colour of more than 1.3 billion stars in the Milky Way and nearby galaxies. Thanks to the software created by DAPCOM Data Services, the catalogue has reduced its size from the original 554 GB (in gzip format) to 471 GB.

FAPEC supports both binary and text files (such as application records and CSV files) with time series, multidimensional matrices, images with various colour bands, medical data (such as MRIs) and specific formats such as those mentioned above. In addition to offering a high data compression capacity, FAPEC has other advantages: high compression speed, data encryption, generation of statistics, compatibility with nearly all computing platforms, and ability to adapt to other specific types of data, such as scientific experiments, sensors, and cameras.

The versatility of this high-performance data compressor for specific applications allows it to be applied to very diverse fields, such as genomics (fastq files) and water column data gathered by seabed-mounted echo sounders, as described in a recent paper written in collaboration with the UB’s Marine Geosciences Consolidated Research Group and published in the IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing. Besides these terrestrial and marine applications, the program is being used on several Earth observation satellites.