Abstract
Deploying machine learning across the edge–cloud continuum offers significant opportunities for dynamic configuration and coordination across different systems. Developers must decide what should run where, given privacy, capacity, resilience and energy constraints. They must then inspect hardware, select toolchains, generate deployment artefacts and validate results on physical devices. Each step demands specialist expertise, and the decisions depend on one another. This talk explores how an agent-based approach can be used to take on this coordination role, turning high-level user intent into configured, deployed and validated edge implementations. A prompt-driven infrastructure management, where LLMs connect user goals to system-level actions, is described. For instance, an agent interprets requests such as “minimise latency for video stream X” and generates resource-management plans that weigh edge constraints against cloud placement. LEMON [1] uses fine-tuned specialist LLMs to answer monitoring queries in Kubernetes-based microservice environments. An off-the-shelf coding agent that discovers connected FPGA-based devices, profiles their hardware resources and selects an implementation strategy is then outlined [2]. It can be used to generate and deploy across different hardware designs, and accept or refine it based on results measured on the device. The method was evaluated on a ZCU104 SoC FPGA and a PYNQ-Z1 with an HVAC energy-prediction workload. The agent explored several quantisation levels and two ways of generating the hardware. The talk closes by looking ahead: extending agents to mixed fleets of edge devices, and choosing deployment targets accurately, since FPGAs are not always the best choice. It also covers handling failures, interpreting user intent reliably, and adding long-term memory so that agents move from reactive assistance towards autonomous reasoning about operations across the computing continuum.
References:
[1] Götz, Markus, Baghban, Hojjat, Dazzi, Patrizio and Rana, Omer 2026. LEMON: LLM-enabled monitoring for microservices orchestration. IEEE Transactions on Services Computing 10.1109/tsc.2026.3679374
[2] Mohammed Mshragi, Ioan Petri, Othmane Belarbi, Charith Perera, and Omer Rana. 2026. Agent-based Deployment on FPGA-based Edge Devices. 16th International Conference on the Internet of Things (IOT 2026), November 17–20, 2026, Newcastle Upon Tyne, England, United Kingdom. ACM Press (to appear)