MambaFormer3D: A Hybrid Transformer-Mamba Network for 3D Brain Tumor Segmentation
DOI:
https://doi.org/10.54097/a5v4aj58Keywords:
Image Segmentation; Transformer; Mamba; Lightweight.Abstract
In recent years, Vision Transformers (ViTs) have demonstrated strong performance in 3D medical image segmentation, particularly in capturing global features and modeling long-range dependencies. However, the self-attention mechanism in ViTs faces significant computational challenges due to its quadratic complexity, especially when applied to three-dimensional medical images, resulting in a high number of parameters and increased computational overhead. This limitation hampers their deployment on resource-constrained devices. To address this challenge, this paper proposes a lightweight MambaFormer architecture, which leverages the efficient representation learning capability of the Mamba SSM and the global feature extraction strength of Transformers, achieving linear computational complexity. The synergy between the MALR and LMA modules ensures computational efficiency while significantly enhancing segmentation performance, particularly in brain tumor segmentation tasks. Experimental results demonstrate that MambaFormer3D exhibits superior performance in processing multi-modal BraTS data, showing great potential in computationally constrained environments.
Downloads
References
[1] Ramesh K K D, Kumar G K, Swapna K, et al. A review of medical image segmentation algorithms[J]. EAI Endorsed Transactions on Pervasive Health & Technology, 2021, 7(27).
[2] Abidin Z U, Naqvi R A, Haider A, et al. Recent deep learning-based brain tumor segmentation models using multi-modality magnetic resonance imaging: A prospective survey[J]. Frontiers in Bioengineering and Biotechnology, 2024, 12: 1392807.
[3] Zhang S, Edwards A, Wang S, et al. A prior knowledge based tumor and tumoral subregion segmentation tool for pediatric brain tumors[J]. arXiv preprint arXiv:2109.14775, 2021.
[4] Kaifi R. A review of recent advances in brain tumor diagnosis based on AI-based classification[J]. Diagnostics, 2023, 13(18): 3007.
[5] Malmer B, Henriksson R, Grönberg H. Different aetiology of familial low-grade and high-grade glioma? A nationwide cohort study of familial glioma[J]. Neuroepidemiology, 2002, 21(6): 279-286.
[6] Zhang W, Wu Y, Yang B, et al. Overview of multi-modal brain tumor mr image segmentation[C]//Healthcare. MDPI, 2021, 9(8): 1051.
[7] O'shea K, Nash R. An introduction to convolutional neural networks[J]. arXiv preprint arXiv:1511.08458, 2015.
[8] Ronneberger, Olaf et al. “U-Net: Convolutional Networks for Biomedical Image Segmentation.” ArXiv abs/1505.04597 (2015): n. pag.
[9] Milletari F, Navab N, Ahmadi S A. V-net: Fully convolutional neural networks for volumetric medical image segmentation[C]//2016 fourth international conference on 3D vision (3DV). Ieee, 2016: 565-571.
[10] Oktay O, Schlemper J, Folgoc L L, et al. Attention u-net: Learning where to look for the pancreas[J]. arXiv preprint arXiv:1804.03999, 2018.
[11] Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need[J]. Advances in neural information processing systems, 2017, 30.
[12] Gu A, Dao T. Mamba: Linear-time sequence modeling with selective state spaces[J]. arXiv preprint arXiv:2312.00752, 2023.
[13] Ma J, Li F, Wang B. U-mamba: Enhancing long-range dependency for biomedical image segmentation[J]. arXiv preprint arXiv:2401.04722, 2024.
[14] Ruan J, Li J, Xiang S. Vm-unet: Vision mamba unet for medical image segmentation [J]. arXiv preprint arXiv: 2402.02491, 2024.
[15] Hatamizadeh A, Tang Y, Nath V, et al. Unetr: Transformers for 3d medical image segmentation[C]//Proceedings of the IEEE/CVF winter conference on applications of computer vision. 2022: 574-584.
[16] Hatamizadeh A, Nath V, Tang Y, et al. Swin unetr: Swin transformers for semantic segmentation of brain tumors in mri images[C]//International MICCAI brainlesion workshop. Cham: Springer International Publishing, 2021: 272-284.
[17] Liu H, Huo G, Li Q, et al. Multiscale lightweight 3D segmentation algorithm with attention mechanism: Brain tumor image segmentation [J]. Expert Systems with Applications, 2023, 214: 119166.
[18] Wang, W., Chen, C., Ding, M., Yu, H., Zha, S., Li: Transbts: Multimodal brain tumor segmentation using transformer. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 109–119. Springer(2021)
Downloads
Published
Issue
Section
License
Copyright (c) 2025 Academic Journal of Science and Technology
This work is licensed under a Creative Commons Attribution 4.0 International License.
