Storage Capacity Enhancement of SSD-based Image Deduplication

As Cyber Physical Systems (CPSs), notably autonomous vehicles, generate increasing volumes of image-based data, efficient storage solutions become paramount. Leveraging high-density white-box SSDs and tailored data deduplication techniques presents a promising approach. However, conventional deduplication methods are ill-equipped to handle image file characteristics effectively, while existing image-specific solutions like imgDedup have limitations in format support and SSD integration. To address these challenges, we propose an SSDbased Approximate Image Deduplication (AID) system. SSD-AID comprises an image-customized deduplication, called AID, and an SSD-aware Image Consolidation. AID combines exact and approximate matching methods, along with optimized content compression, to enhance deduplication efficacy while integrating seamlessly with white-box SSDs. Furthermore, our SSD-aware Image Consolidation optimizes flash page utilization by grouping images based on access frequencies. Experimental results across multiple datasets demonstrate significant improvements in deduplication ratio and SSD page utilization. SSD-AID offers a comprehensive solution for efficient image data management in CPSs, addressing challenges in large-scale storage utilization and minimizing unused space.