Transport Thresholds and Network-Impairment Amplification Mechanisms in TLS 1.3 Post-Quantum Key Exchange
-- A Cross-Layer Experimental Study of ML-KEM, HQC, and Hybrid Schemes
DOI:
https://doi.org/10.54097/wyn6tw24Keywords:
TLS 1.3, Post-quantum Key Exchange, ML-KEM, HQC, TCP Segmentation and Network ImpairmentAbstract
Post-quantum key-encapsulation mechanisms substantially enlarge TLS 1.3 handshake material, but their end-to-end impact depends on the combined effects of TLS encapsulation, TCP segmentation, and network conditions. Using OpenSSL 3.0.21, liboqs 0.16.0, and oqs-provider 0.12.0-dev, this study conducts cross-layer experiments on X25519, ML-KEM-768, two implementation-specific HQC groups, hqc1 and hqc3 (not NIST security levels), and the corresponding hybrid schemes. It measures ClientHello, ServerHello, TLS record, and TCP carriage structures; locates segmentation-transition boundaries through adjacent-MTU scans; and completes 180,000 formal connection-establishment measurements plus 6,400 targeted review measurements in separate RTT, per-direction bandwidth, and independent per-direction loss experiments. The results show that ML-KEM is generally close to X25519 when there is no packet loss or material bandwidth bottleneck. At 1 Mbps, the latency increase of each scheme is broadly consistent with serialization estimates derived from bidirectional TCP payload. Packet loss first amplifies P95 and P99; under 10% independent loss in each direction, 30 of 18,000 measurements did not complete and verify the connection within 10 s. The higher median latency and long tail of HQC-L3 recur in both the formal batches and targeted review, whereas the single-batch anomaly of X25519-HQC-L1 does not. The study establishes a traceable evidence chain from cryptographic material, TLS encapsulation, and TCP segmentation to network impairment and connection-establishment performance, providing experimental support for MTU/MSS headroom assessment and post-quantum TLS deployment.
Downloads
References
[1] Shor, P. W. (1994). Algorithms for quantum computation: Discrete logarithms and factoring. In Proceedings of the 35th Annual Symposium on Foundations of Computer Science (pp. 124–134). https://doi.org/10.1109/SFCS.1994.365700. DOI: https://doi.org/10.1109/SFCS.1994.365700
[2] Rescorla, E. (2026). The transport layer security (TLS) protocol version 1.3. RFC 9846. https://doi.org/10.17487/RFC9846. DOI: https://doi.org/10.17487/RFC9846
[3] National Institute of Standards and Technology. (2024). Module-lattice-based key-encapsulation mechanism standard. FIPS 203. https://doi.org/10.6028/NIST.FIPS.203. DOI: https://doi.org/10.6028/NIST.FIPS.203
[4] Alagic, G., et al. (2025). Status report on the fourth round of the NIST post-quantum cryptography standardization process. NIST IR 8545. https://doi.org/10.6028/NIST.IR.8545. DOI: https://doi.org/10.6028/NIST.IR.8545
[5] Stebila, D., Fluhrer, S., & Gueron, S. (2026). Hybrid key exchange in TLS 1.3. RFC 9954. https://doi.org/10.17487/RFC9954. DOI: https://doi.org/10.17487/RFC9954
[6] Connolly, D. (2026, July 20). ML-KEM post-quantum key agreement for TLS 1.3. IETF Internet-Draft draft-ietf-tls-mlkem-09. Work in progress.
[7] Kwiatkowski, K., Kampanakis, P., Westerbaan, B. E., & Stebila, D. (2026, May). Post-quantum hybrid ECDHE-MLKEM key agreement for TLSv1.3. IETF Internet-Draft draft-ietf-tls-ecdhe-mlkem-05. Work in progress.
[8] Mosca, M., & Stebila, D. (2016). Post-quantum key exchange for the Internet and the Open Quantum Safe project. Cryptology ePrint Archive, Paper 2016/1017.
[9] Paquin, C., Stebila, D., & Tamvada, G. (2020). Benchmarking post-quantum cryptography in TLS. In Post-Quantum Cryptography (LNCS 12100, pp. 72–91). https://doi.org/10.1007/978-3-030-44223-1_5. DOI: https://doi.org/10.1007/978-3-030-44223-1_5
[10] Henrich, J., Heinemann, A., Wiesmaier, A., & Schmitt, N. (2023). Performance impact of PQC KEMs on TLS 1.3 under varying network characteristics. In Information Security (LNCS 14411, pp. 267–287). https://doi.org/10.1007/978-3-031-49187-0_14. DOI: https://doi.org/10.1007/978-3-031-49187-0_14
[11] Zheng, J., Zhu, H., Dong, Y., Song, Z., Zhang, Z., Yang, Y., & Zhao, Y. (2024). Faster post-quantum TLS 1.3 based on ML-KEM: Implementation and assessment. In Computer security - ESORICS 2024 (LNCS 14983, pp. 123–143). https://doi.org/10.1007/978-3-031-70890-9_7 DOI: https://doi.org/10.1007/978-3-031-70890-9_7
[12] Sosnowski, M., et al. (2023). The performance of post-quantum TLS 1.3. In Companion of the 19th International Conference on Emerging Networking Experiments and Technologies (CoNEXT Companion ’23) (pp. 19–27). https://doi.org/10.1145/3624354.3630585 DOI: https://doi.org/10.1145/3624354.3630585
[13] Kampanakis, P., & Childs-Klein, W. (2024). The impact of data-heavy, post-quantum TLS 1.3 on the time-to-last-byte of real-world connections. In Workshop on Measurements, Attacks, and Defenses for the Web (MADWeb 2024). https://doi.org/10.14722/madweb.2024.23010 DOI: https://doi.org/10.14722/madweb.2024.23010
[14] Henrich, J., Schmitt, N., Alnahawi, N., & Heinemann, A. (2026). A lot of data and added complexity. How does PQC affect the performance of my TLS connection? In Information security (LNCS 16186, pp. 107–128). https://doi.org/10.1007/978-3-032-08124-7_6 DOI: https://doi.org/10.1007/978-3-032-08124-7_6
[15] Souvatzidaki, K., & Limniotis, K. (2025). Post-quantum key exchange in TLS 1.3: Further analysis on performance of new cryptographic standards. Cryptography, 9(4), 73. https://doi.org/10.3390/cryptography9040073 DOI: https://doi.org/10.3390/cryptography9040073
[16] Rios, R., & Montenegro, J. A. (2026). Post-quantum cryptography: A multi-layer bottleneck analysis. International Journal of Information Security, 25, 89. https://doi.org/10.1007/s10207-026-01242-0 DOI: https://doi.org/10.1007/s10207-026-01242-0
[17] OpenSSL Project. (2026, June 9). OpenSSL 3.0.21. OpenSSL 3.0 LTS release.
[18] Open Quantum Safe Project. (2026). liboqs (Version 0.16.0, commit 5a1a854b).
[19] Open Quantum Safe Project. (2026). oqs-provider (Version 0.12.0-dev, commit 1670a8a).
[20] Hemminger, S. (2005). Network emulation with NetEm. OSDL.
[21] Eddy, W. (2022). Transmission control protocol (TCP). RFC 9293. DOI: https://doi.org/10.17487/RFC9293
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Frontiers in Computing and Intelligent Systems
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
