A Novel Data Locality-Aware Scheduler for Improved Cloud Performance

Scheduling tasks closer to stored data can significantly reduce network traffic. By optimizing for data locality, tasks can be matched with their associated data on the same node, minimizing the need for data transfer. However, many existing schedulers overlook the balance between task placement, data transfer overhead, and bandwidth consumption, focusing only on locality. We present a novel Genetic Algorithm-based Data Locality Scheduler (GADLS), which aims to balance time consumption and network bandwidth while improving data locality and throughput. GADLS employs a genetic algorithm to model data-task placement as a chromosome, optimizing for configurations that maximize locality and minimize bandwidth use. It integrates a multi-objective fitness function, balancing data movement, network traffic, and task runtime, with adaptive mutation and crossover mechanisms to explore a broad range of placement options. Through this approach, GADLS achieves an improvement of 18% in data locality rate and a 27% increase in throughput, demonstrating its effectiveness in maximizing resource utilization and enhancing performance in distributed environments.