Evaluating GNN Inference on Edge Computers

Abstract

Graph Neural Networks (GNNs) are increasingly applied in fields like biomedical analysis, recommendation systems, and scientific computing. Deploying these models at the edge is attractive due to concerns about latency and privacy, but their feasibility on resource - limited hardware remains uncertain.In this work, we present, to our knowledge, the first systematic study of GNN inference on the NVIDIA Jetson AGX Orin, a popular embedded GPU platform. We evaluate three representative GNNs: GCN, GAT, and GraphSAGE on two different tasks: node classification on the PubMed citation network and graph classification on functional MRI from an OpenNeuro publicly available dataset.Using pretrained models, we measure inference performance under different conditions, including varying precision (FP32, FP16), batch sizes (1 and 8), and power profiles (MAXN 60W versus capped at 15 W). Metrics include latency distributions (p50/p95), throughput, peak memory usage, accuracy relative to FP32 baselines, and energy per inference calculated from tegrastats power logs.Our results show that FP16 delivers accuracy comparable to FP32 while reducing memory usage and offering modest throughput improvements. Power capping at 15 W produces mixed effects: fMRI workloads experience slight degradation while large - scale PubMed inference is significantly affected. Pareto analysis reveals different trade - off surfaces across the datasets. These preliminary findings show the potential and limitations of running GNN inference on edge devices and their embedded GPUs and provide guidance for practitioners deploying graph learning tasks in resource - constrained environments.