Abstract
Telemetry data, essential for operational intelligence and security, is produced in vast quantities by modern cloud data centers. In large-scale enterprise data centers operating hundreds of 100/400 Gbps links, this telemetry stream reaches 50 to 100 Terabytes per rack daily (exceeding 2 to 5 Petabytes per day facility-wide across millions of concurrent flows). This deluge of packets is processed by endpoint server CPUs in traditional storage architectures, resulting in a significant performance bottleneck that restricts visibility. Current methods ignore the network fabric’s computational potential for data orchestration, treating it as a passive collection of pipes. We introduce Synapse, a new switch-centric architecture that turns the network into an orchestrator for active storage. Synapse incorporates intelligent storage logic directly into a programmable switch’s data plane. Bypassing host CPUs entirely, the switch uses Remote Direct Memory Access (RDMA) over RDMA over Converged Ethernet (RoCE) to directly control data placement on a pool of distant Non-Volatile Memory Express (NVMe) SSDs. This design uses policy-driven Quality of Service to safeguard important data streams, autonomous data-plane resilience for microsecond-scale failover and rerouting, and zero-CPU-overhead in-fabric live indexing to make telemetry data query-ready upon ingress. Our analysis demonstrates that in-fabric live indexing accelerates diagnostic queries by up to 300x compared to unindexed scans, while eliminating the severe host CPU overhead (consuming over 90% of multiple dedicated host cores) incurred by software-based indexing baselines, minimizes host CPU overhead to almost zero, and achieves a linearly scalable storage throughput of over 730 Gbps. Microsecond-level failure rerouting is made possible by its data-plane-native resilience, which is five orders of magnitude quicker than conventional controller-based techniques. A new class of highly scalable, effective, and resilient network-native services is made possible by Synapse, which removes the main storage bottleneck in contemporary data centers by offloading the entire telemetry capture process to the network fabric.
Data availability
No datasets were generated or analyzed during the current study.
References
Makonyi K, Abrahamsson H, Henriksson D, Hock D, Kremling S, Lipp F, Salisbury J, Sandell J (2024) On the use of streaming telemetry data for network health monitoring and anomaly detection. Swedish National Computer Networking and Cloud Computing Workshop. SNCNW.
Putina A, Rossi D, Bifet A, Barth S, Pletcher D, Precup C, Nivaggioli P (2018) Telemetry-based stream-learning of bgp anomalies. In: Proceedings of the 2018 Workshop on Big Data Analytics and Machine Learning for Data Communication Networks. pp 15–20
Ahmadpanah SH, Mirabi M, Sahafi A, Erfani SH (2025) Fleximigrate: enhancing live container migration in heterogeneous computing environments. Clust Comput 28(13):847. https://doi.org/10.1007/s10586-025-05548-x
Zhu S, Lu J, Lyu B, Pan T, Zhang S, Sun X, Jia C, Cheng X, Kang D, Lv Y, Yang F, Xue X, Yang X, Wang Z, Yang J (2024) Proactive telemetry in large-scale multi-tenant cloud overlay networks. IEEE/ACM Trans Netw 32:3002–3017. https://doi.org/10.1109/TNET.2024.3381786
Tovarňák D, Racek M, Velan P (2021) Cloud native data platform for network telemetry and analytics. In: 17th International Conference on Network and Service Management (CNSM). pp 394–396. https://doi.org/10.23919/CNSM52442.2021.9615568
Pentaparthi SKR (2025) Observability for ai-enabled cloud-native networks: A unified framework integrating opentelemetry, cortexdb, loki, genai, and rag. Int J Sci Res Comput Sci Eng Inf Technol. https://doi.org/10.32628/cseit25112811
Bhosale P (2025) A comprehensive study of opentelemetry collector: Architecture, use cases, and performance. Int Sci J Eng Manag. https://doi.org/10.55041/isjem00020
Zhang J, Deng X (2021) Ldsqp: Scalable and efficient log data storage and query processing scheme for cloud data centers. In: Proceedings of the 5th International Conference on High Performance Compilation, Computing and Communications. https://doi.org/10.1145/3471274.3471276
Zawoad S, Dutta A, Hasan R (2016) Towards building forensics enabled cloud through secure logging-as-a-service. IEEE Trans Dependable Secure Comput 13:148–162. https://doi.org/10.1109/TDSC.2015.2482484
Simsek, G., Ergenç, D., Onur, E.: Efficient network monitoring via in-band telemetry. In: 2021 17th International Conference on the Design of Reliable Communication Networks (DRCN), 1–6 (2021) https://doi.org/10.1109/DRCN51631.2021.9477344
Kumar, H., Chakraborty, B., Sharma, S., Chawla, N., Mukhopadhyay, S.: Xmd: An expansive hardware-telemetry based malware detector to enhance endpoint detection. ArXiv abs/2206.12447 (2022) https://doi.org/10.48550/arXiv.2206.12447
Cerović D, Piccolo VD, Amamou A, Haddadou K, Pujolle G (2018) Fast packet processing: a survey. IEEE Commun Surv Tutor 20:3645–3676. https://doi.org/10.1109/COMST.2018.2851072
Hauser F, Häberle M, Merling D, Lindner S, Gurevich V, Zeiger F, Frank R, Menth M (2021) A survey on data plane programming with p4: fundamentals, advances, and applied research. J Netw Comput Appl 212:103561. https://doi.org/10.1016/j.jnca.2022.103561
Yazdinejad A, Parizi R, Bohlooli A, Dehghantanha A, Choo K (2020) A high-performance framework for a network programmable packet processor using p4 and fpga. J Netw Comput Appl 156:102564. https://doi.org/10.1016/j.jnca.2020.102564
Hussain M, Cho B (2024) Hptcollector: high-performance telemetry collector. Clust Comput 27:14729–14744. https://doi.org/10.1007/s10586-024-04650-w
Pradeep S, Sharma Y, Verma C, Sreeram G, Rao PH (2023) Express data processing on fpga: network interface cards for streamlined software inspection for packet processing. Appl Syst Innov. https://doi.org/10.3390/asi6010009
Zhang Q, Ansari N, Zhu Z (2023) Adaptive smartnic offloading for unleashing the performance of protocol-oblivious forwarding. IEEE Int Things J 10:7303–7314. https://doi.org/10.1109/JIOT.2022.3228796
Kfoury EF, Crichigno J, Bou-Harb E (2021) An exhaustive survey on p4 programmable data plane switches: taxonomy, applications, challenges, and future trends. IEEE Access 9:87094–87155. https://doi.org/10.1109/ACCESS.2021.3086704
Zhang X, Cui L, Tso FP, Jia W (2021) Pheavy: predicting heavy flows in the programmable data plane. IEEE Trans Netw Serv Manag 18:4353–4364. https://doi.org/10.1109/tnsm.2021.3094514
Johnson DS, London JM, Lea M, Durban J (2008) Continuous-time correlated random walk model for animal telemetry data. Ecology 89(5):1208–15. https://doi.org/10.1890/07-1032.1
Liu Z, Niu Z, Shu R, Cheng W, Yuan L, Nelson J, Ports DRK, Cheng P, Xiong Y (2025) Hyperdrive: direct network telemetry storage via programmable switches. IEEE Trans Cloud Comput 13(2):498–511. https://doi.org/10.1109/TCC.2025.3543477
Deri L, Cardigliano A, Fusco F (2013) 10 gbit line rate packet-to-disk using n2disk. In: 2013 IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS). IEEE, pp 441–446
Emmerich P, Pudelko M, Gallenmüller S, Carle G (2017) Flowscope: Efficient packet capture and storage in 100 gbit/s networks. In: 2017 IFIP Networking Conference (IFIP Networking) and Workshops. IEEE, pp 1–9
Zhu H (2020) Data Plane Development Kit (DPDK): A Software Optimization Guide to the User Space-Based Network Applications. CRC Press.
ntop: PF_RING: High-speed Packet Capture, Filtering and Analysis. http://www.ntop.org/products/pf_ring/ Accessed: 2025-07-18 (2025)
Zhu Y, Kang N, Cao J, Greenberg A, Lu G, Mahajan R, Maltz D, Yuan L, Zhang M, Zhao BY (2015) Packet-level telemetry in large datacenter networks. In: Proceedings of the 2015 ACM Conference on Special Interest Group on Data Communication. pp 479–491
Kim C, Sivaraman A, Katta N, Bas A, Dixit A, Wobker LJ (2015) -band network telemetry via programmable dataplanes, vol 15. pp 1–2 (ACM SIGCOMM)
Gupta A, Harrison R, Canini M, Feamster N, Rexford J, Willinger W (2018) Sonata: Query-driven streaming network telemetry. In: Proceedings of the 2018 Conference of the ACM Special Interest Group on Data Communication. pp 357–371
Liu Z, Manousis A, Vorsanger G, Sekar V, Braverman V (2016) One sketch to rule them all: Rethinking network flow monitoring with univmon. In: Proceedings of the 2016 ACM SIGCOMM Conference. pp 101–114
Langlet J, Ben Basat R, Oliaro G, Mitzenmacher M, Yu M, Antichi G (2023) Direct telemetry access. In: Proceedings of the ACM SIGCOMM 2023 Conference. pp 832–849
Mu, Y., Yao, K., Li, Y., Li, Z., Sun, T., Lu, L., He, J., Huang, M.: Sosp: A smartnic-based offloading framework for cloud storage pooling. In: Proceedings of the 2022 9th International Conference on Wireless Communication and Sensor Networks (2022) https://doi.org/10.1145/3514105.3514110
Kfoury EF, Choueiri S, Mazloum A, AlSabeh A, Gomez J, Crichigno J (2024) A comprehensive survey on smartnics: architectures, development models, applications, and research directions. IEEE Access 12:107297–107336. https://doi.org/10.1109/ACCESS.2024.3437203
Basu S, Nadig D (2024) Offloading nvme over fabrics (nvme-of) to smartnics on an at-scale distributed testbed. In: IEEE 10th International Conference on Network Softwarization (NetSoft). pp 316–318. https://doi.org/10.1109/NetSoft60951.2024.10588915
Kim, D., Liu, Z., Zhu, Y., Kim, C., Lee, J., Sekar, V., Seshan, S.: Tea: Enabling state-intensive network functions on programmable switches. In: Proceedings of the Annual Conference of the ACM Special Interest Group on Data Communication on the Applications, Technologies, Architectures, and Protocols for Computer Communication, pp. 90–106 (2020)
Zhu, H., Wang, T., Hong, Y., Ports, D.R., Sivaraman, A., Jin, X.: \(\{\)NetVRM\(\}\): Virtual register memory for programmable networks. In: 19th USENIX Symposium on Networked Systems Design and Implementation (NSDI 22), pp. 155–170 (2022)
NVM Express: NVMe-over-Fabrics Specification. https://nvmexpress.org/developers/nvme-of-specification/ . Accessed: 2025-04-05 (2019)
Funding
There is no funding for this research.
Author information
Authors and Affiliations
Contributions
Amir Sahafi was involved in conceptualization, supervision, and project administration. Seyed Hossein Erfani contributed to methodology, resources, and consulting. Seyed Hossein Ahmadpanah was involved in writing—original draft, writing—review and editing, data curation, formal analysis, investigation, validation, visualization, experimentation, and evaluation.
Corresponding author
Ethics declarations
Conflict of interest
The authors have no relevant financial or non-financial interests to disclose. The authors declare no conflict of interest.
Ethics approval
Not applicable.
Consent to participate
Not applicable.
Consent for publication
Not applicable.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Ahmadpanah, S.H., Sahafi, A. & Erfani, S.H. The network fabric as a storage orchestrator: a switch-centric architecture. J Supercomput 82, 699 (2026). https://doi.org/10.1007/s11227-026-08850-6
Received:
Accepted:
Published:
Version of record:
DOI: https://doi.org/10.1007/s11227-026-08850-6