Benchmarking GPU Passthrough Performance on Docker for AI Cloud System

The use of artificial intelligence (AI), which depends only on CPU resources, tends to result in longer execution times or CPU time. Especially when handling large amounts or complex workloads. To overcome that issue, the use of a graphics processing unit (GPU) becomes a significant support. GPUs can significantly speed up AI inferences through their parallel architecture. One recent approach to integrating GPUs into an AI system is called GPU passthrough. Either natively (native environment), or through Docker environment. However, until recently, the efficiency and results between those methods have remained unexplored, particularly in local cloud environment.This study aimed to compare GPU performance between native and Docker environment using a 10.000 x 10.000 matrix multiplication workload with the TensorFlow frameworks. Execution time and GPU performance measured using the nvidia-smi tool. Data is recorded automatically in CSV format. The researcher used the NVIDIA CUDA environment to ensure full compatibility with GPU acceleration.The result demonstrated that GPU processing in native environment had faster average time, as in 1.52 seconds. In another case, GPU passthrough in docker environment demonstrated higher GPU utilization, as in 86.2% but had a longer execution time.These findings indicate that GPU overhead occurred in docker environment due to the containerization layer. On the contrary, the native environment resulted in shorter execution time, even though it did not maximize the GPU utilization. These results provide valuable basis data for technical decision-making in GPU-based AI deployment in a limited environment.