Symbiotic Habitat: Multilayer Audio-Visual Generation and Graphic Notation Based on Ecological Soundscape Analysis
DOI:
https://doi.org/10.54097/xwcjh718Keywords:
Ecological Soundscape, Audio-Visual Generation, Graphic Notation, Acoustic Feature AnalysisAbstract
This paper offers Multilayer Ecological Soundscape Notation (MESN) to make the acoustic base of generative ecological soundscape graphics traceable. Environmental recordings from four Chongming habitats, Song recordings from 25 bird species, and human activity noises are evaluated independently using RMS Energy, Spectral Centroid, Onset Strength, and low/mid/high frequency-band energy. After source-wise normalization and temporal alignment, MESN maps acoustic information to graphical variables and temporal properties to graphic primitives on a shared notation timeline. Case studies indicate that identical mapping principles create diverse outputs while keeping correlation to substantial acoustic changes. Applied in Symbiotic Habitat, MESN provides a traceable mechanism for ecological soundscape notation.
Downloads
References
[1] Pijanowski, B. C., Villanueva Rivera, L. J., Dumyahn, S. L., & et al. (2011). Soundscape ecology: The science of sound in the landscape. BioScience, 61(3), 203 216. https://doi. org/ 10. 1525/ bio.2011.61.3.6.
[2] Sueur, J., & Farina, A. (2015). Ecoacoustics: The ecological investigation and interpretation of environmental sound. Biosemiotics, 8(3), 493 502. https://doi.org/10. 1007/ s1230 4 015 9248 x.
[3] Gasc, A., Francomano, D., Dunning, J. B., & Pijanowski, B. C. (2017). Future directions for soundscape ecology: The importance of ornithological contributions. The Auk: Ornithological Advances, 134(1), 215 228. https://doi.org/ 10. 1642/ AUK 16 124.1.
[4] Sethi, S. S., Bick, A., Ewers, R. M., & et al. (2023). Limits to the accurate and generalizable use of soundscapes to monitor biodiversity. Nature Ecology & Evolution, 7, 1373 1378. https://doi.org/10.1038/s41559 023 02148 z.
[5] Koch, T. M. I., Marks, E. S., & Roberts, T. F. (2025). A deep learning approach for the analysis of birdsong. eLife, 13, RP101111. https://doi.org/10.7554/eLife.101111.3.
[6] Shim, J. Y., Kim, J., & Kim, J. K. (2023). Audio to visual cross modal generation of birds. IEEE Access, 11, 27719 27729. https://doi.org/10.1109/ACCESS.2023.3257565.
[7] Kim, S. B., Senocak, A., Ha, H., & et al. (2023). Sound to visual scene generation by audio to visual latent alignment. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 6430 6440. https://doi.org/10. 1109/ CVPR52729.2023.00622.
[8] Finke, G. (2019). Partituren zum Lesen und Schauen: Bildlichkeit als Merkmal graphischer Notation. Zeitschrift der Gesellschaft für Musiktheorie, 16(1), 21 39. https://doi.org/ 10. 31751/ 1001.
[9] Hope, C., Vickery, L., Wyatt, A., & James, S. (2015). The Decibel ScorePlayer: A digital tool for reading graphic notation. Proceedings of the First International Conference on Technologies for Music Notation and Representation (TENOR 2015), 58 69.
[10] Karchkhadze, T., Shao, K., & Dubnov, S. (2024). Interpreting graphic notation with MusicLDM: An AI improvisation of Cornelius Cardew’s Treatise. Proceedings of the 2024 IEEE International Conference on Big Data (BigData), 3181 3190. https://doi.org/10.1109/BigData62323.2024.10825824.
[11] McFee, B., Raffel, C., Liang, D., & et al. (2015). librosa: Audio and music signal analysis in Python. Proceedings of the 14th Python in Science Conference, 18 24. https://doi.org/ 10. 25080 /Majora 7b98e3ed 003.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Frontiers in Computing and Intelligent Systems
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
