Highlights: What are the main findings? Ensemble learning models achieve 99.8–99.9% attack detection accuracy on IoT network traffic with perfect ROC-AUC (100%) and inference times under 12 ms per 1000 flows, demonstrating performance comparable to the state of the art while uniquely providing complete computational benchmarks for real-time deployment assessment. Security audit reveals critical blind spot: man-in-the-middle attacks achieve only 78% F1-score despite 99% overall accuracy, demonstrating that aggregate metrics conceal systematic failures in rare but high-risk threat detection. What are the implications of the main findings? LightGBM provides optimal deployment balance (99.93% accuracy, 2.76 MB footprint, 10 ms latency), enabling edge-based IoT security monitoring without centralized infrastructure dependencies or specialized hardware requirements. Reproducible audit framework with transparent feature engineering, computational benchmarks, and complete artifacts enables credible security posture assessment and fair comparison across intrusion detection systems—addressing critical gaps in current IoT security research. Internet of Things deployments face escalating security threats, yet systematic methods for auditing the defensive posture of IoT device networks remain underdeveloped. Current intrusion detection evaluations focus on algorithmic accuracy while neglecting operational requirements—computational efficiency, reproducibility, and interpretable risk assessment—that security audits demand. This paper introduces a reproducible security audit framework for IoT device networks, demonstrated through systematic evaluation of four machine learning models (Random Forest, LightGBM, XGBoost, Logistic Regression) on the TON_IoT dataset containing nine attack categories targeting smart environments. Our audit methodology enforces strict feature hygiene by excluding identity-revealing attributes, benchmarks both threat detection capability and computational cost, and provides complete reproducibility artifacts including preprocessing pipelines and trained models. The framework evaluates security posture through dual lenses: binary classification (distinguishing compromised from legitimate traffic) and multiclass classification (attributing threats to specific attack types). Binary audit results show ensemble models achieve 99.8–99.9% accuracy with perfect ROC-AUC (100%) and sub-15 ms inference latency per 1000 flows, confirming reliable attack detection. Multiclass auditing reveals more nuanced findings: while overall accuracy reaches 99.0% with macro-F1 near 97%, rare attack types expose critical blind spots—man-in-the-middle threats achieve only 78% F1 despite representing serious security risks. LightGBM provides optimal audit performance, balancing 99.93% detection accuracy with 2.76 MB deployment footprint. We translate audit findings into actionable security recommendations (network segmentation, rate-limiting, TLS metadata collection) and compare against twenty published studies, demonstrating that our framework achieves competitive detection rates while uniquely delivering the transparency, efficiency metrics, and reproducibility required for credible security assessment of production IoT networks.
Security Audit of IoT Device Networks: A Reproducible Machine Learning Framework for Threat Detection and Performance Benchmarking
Kuznetsov O.
;
2025-01-01
Abstract
Highlights: What are the main findings? Ensemble learning models achieve 99.8–99.9% attack detection accuracy on IoT network traffic with perfect ROC-AUC (100%) and inference times under 12 ms per 1000 flows, demonstrating performance comparable to the state of the art while uniquely providing complete computational benchmarks for real-time deployment assessment. Security audit reveals critical blind spot: man-in-the-middle attacks achieve only 78% F1-score despite 99% overall accuracy, demonstrating that aggregate metrics conceal systematic failures in rare but high-risk threat detection. What are the implications of the main findings? LightGBM provides optimal deployment balance (99.93% accuracy, 2.76 MB footprint, 10 ms latency), enabling edge-based IoT security monitoring without centralized infrastructure dependencies or specialized hardware requirements. Reproducible audit framework with transparent feature engineering, computational benchmarks, and complete artifacts enables credible security posture assessment and fair comparison across intrusion detection systems—addressing critical gaps in current IoT security research. Internet of Things deployments face escalating security threats, yet systematic methods for auditing the defensive posture of IoT device networks remain underdeveloped. Current intrusion detection evaluations focus on algorithmic accuracy while neglecting operational requirements—computational efficiency, reproducibility, and interpretable risk assessment—that security audits demand. This paper introduces a reproducible security audit framework for IoT device networks, demonstrated through systematic evaluation of four machine learning models (Random Forest, LightGBM, XGBoost, Logistic Regression) on the TON_IoT dataset containing nine attack categories targeting smart environments. Our audit methodology enforces strict feature hygiene by excluding identity-revealing attributes, benchmarks both threat detection capability and computational cost, and provides complete reproducibility artifacts including preprocessing pipelines and trained models. The framework evaluates security posture through dual lenses: binary classification (distinguishing compromised from legitimate traffic) and multiclass classification (attributing threats to specific attack types). Binary audit results show ensemble models achieve 99.8–99.9% accuracy with perfect ROC-AUC (100%) and sub-15 ms inference latency per 1000 flows, confirming reliable attack detection. Multiclass auditing reveals more nuanced findings: while overall accuracy reaches 99.0% with macro-F1 near 97%, rare attack types expose critical blind spots—man-in-the-middle threats achieve only 78% F1 despite representing serious security risks. LightGBM provides optimal audit performance, balancing 99.93% detection accuracy with 2.76 MB deployment footprint. We translate audit findings into actionable security recommendations (network segmentation, rate-limiting, TLS metadata collection) and compare against twenty published studies, demonstrating that our framework achieves competitive detection rates while uniquely delivering the transparency, efficiency metrics, and reproducibility required for credible security assessment of production IoT networks.I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.
