Hybrid Encryption for Securing HDFS Data in Big Data Environments

Abstract In the previous era, Conventional encryption schemes that work on file were linked with a lot of processing overhead that negatively impacted performance in the Hadoop framework. But now, With the help of the MapReduce framework or parallel processing method, this paper suggests a unique solution for securing large data and boosting performance: A hybrid encryption technique that combines the Twofish and AES algorithms with the map-reduce framework inside the HDFS (Hadoop Distributed File System) environment. Different conventional methods exist like Twofish + RSA, and AES + Twofish as hybrid encryption that is not decent for dealing with large data, our solution drastically improves performance through parallel encryption by mapper-reducer processes and advances the security of HDFS storage data. In this paper, we use hybrid encryption using AES with Twofish because Twofish provides better data security, and AES is used for optimizing the speed of data encryption, and decryption. Empirical findings validate the suggested approach's effectiveness in protecting private information kept in HDFS and improving performance in terms of speed of encryption ≅ 2–3%, throughput ≅1–2%, and efficiency ≅ 1–2% parameters. This work enhances the performance and reduces the risks associated with unwanted access to important data assets by data security in HDFS-based systems.