Application Progress of Computer Vision in the Research on Relationship Between Landscape and Health

CHEN Chongxian; LI Haiwei; HOU Yongqi; LIU Jingyi

doi:10.12409/j.fjyl.202207130405

Landscape Architecture > 2023 > 30(1): 30-37. > DOI: 10.12409/j.fjyl.202207130405

CHEN Chongxian, LI Haiwei, HOU Yongqi, LIU Jingyi. Application Progress of Computer Vision in the Research on Relationship Between Landscape and Health[J]. Landscape Architecture, 2023, 30(1): 30-37. DOI: 10.12409/j.fjyl.202207130405

Citation:

PDF (3117 KB)

Application Progress of Computer Vision in the Research on Relationship Between Landscape and Health

More Information

Author Bio:
CHEN Chongxian, Ph.D., is an associate professor in the School of Forestry and Landscape Architecture, South China Agricultural University, and a contributing editor of this journal. His research focuses on landscape planning and design, and theory of landscape architecture

LI Haiwei is a Ph.D. candidate in the School of Forestry and Landscape Architecture, South China Agricultural University. Her research focuses on landscape planning and design, and theory of landscape architecture

HOU Yongqi is a master student in the School of Forestry and Landscape Architecture, South China Agricultural University. Her research focuses on landscape planning and design, and theory of landscape architecture

LIU Jingyi, Ph.D., is a lecturer and master supervisor in the School of Forestry and Landscape Architecture, South China Agricultural University. His research focuses on landscape planning and design, and theory of landscape architecture
Corresponding author:
LIU Jingyi, Email: 380069609@qq.com
Received Date: July 12, 2022
Revised Date: December 04, 2022
Available Online: March 20, 2023
Issue Publish Date: January 09, 2023

Graphical Abstract

Abstract

Abstract

Objective
Although computer vision has been widely used in urban landscape research, the overall understanding of the application thereof in the research on the relationship between landscape and health is still insufficient. In view of this, this research summarizes current research trends to demonstrate where computer vision is mainly applied and the strengths and weaknesses thereof, so as to provide a reference for future research.
Methods
This research searches relevant literature included in CNKI databases and Web of Science core databases during the period from 2000 to 2021. The searching results cover both primary researches and systematic reviews, which are screened in a double-blind manner in line with the pre-defined inclusion criteria, with 20 Chinese and 90 English literature being finally identified. The research conducts a narrative synthesis of the literature identified to explore the current evidence for the application of computer vision in the research on the relationship between landscape and human health.
Results
1) In terms of technology application, semantic segmentation and image classification technologies are relatively mature and easy to operate, and are thus most widely used in the research on the relationship between landscape and health. At present, the semantic segmentation technology is mainly used to extract the proportion of landscape elements such as visible green index, sky visibility rate, enclosure degree, building and road, while the image classification technology is mainly used for streetscape image classification based on environmental perception. 2) In terms of application fields, previous researches mainly explore the association between landscape visual elements and human physiological, psychological and social activities based on the computer vision technology, which specifically includes the association between landscape elements and physical health involving such indicators as chronic non-communicable disease, infectious disease, air pollution, walking and cycling, and the association between landscape elements and mental health involving such indicators as happiness, depression, anxiety and environmental perception. For activities, existing researches mainly the target tracking and target detection technologies to investigate the connection between the spatiotemporal laws of crowd behavior and landscape elements. 3) Although most of the existing researches have the advantages of fast and convenient data acquisition, high operability and low cost, they still have defects in such aspects as algorithm and dataset. Although the computer vision technology may perform better than humans in processing simple images, the accuracy and stability thereof are prone to be affected by light and scale, and need to be further optimized for task processing in a complex environment. Meanwhile, due to the fact that most current researches only adopt a single algorithm, the landscape visual indicators extracted are relatively limited. Besides, in many researches, the data collected only describe the environment or health information at a certain moment, which is typically identified with mismatching in time series, and is different from the dynamic changes in the real world and people's health status. In addition, existing researches mainly adopt streetscape images from Tencent, Baidu or Google. However, these images are different from what is actually seen by pedestrians, and the street environment cannot represent the overall environmental characteristics of the city.
Conclusion
Although evidence shows the diversity of computer vision algorithms and applications in the research on the relationship between landscape and human health, there still exist some challenges in the understanding of computer vision algorithms, expansion of research scope, and collection of reliable datasets. More efforts are entailed to further improve computer vision algorithms, promote the integration between social sciences and computer vision, and collect data incorporating different spatiotemporal observations. This research shows that the computer vision technology needs to be further developed in such aspects as the optimization of target detection algorithms, integration into design and post-evaluation processes, expansion of research on the relationship between landscape and social health, and construction of dynamic spatio-temporal datasets.
- landscape architecture,
- computer vision,
- public health,
- deep learning,
- image data

FullText(HTML)

References (65)

References

[1]	WEICHENTHAL S, HATZOPOULOU M, BRAUER M. A Picture Tells a Thousand Exposures: Opportunities and Challenges of Deep Learning Image Analyses in Exposure Science and Environmental Epidemiology[J]. Environment International, 2019, 122: 3-10. doi: 10.1016/j.envint.2018.11.042
[2]	RZOTKIEWICZ A, PEARSON A L, DOUGHERTY B V, et al. Systematic Review of the Use of Google Street View in Health Research: Major Themes, Strengths, Weaknesses and Possibilities for Future Research[J]. Health and Place, 2018, 52: 240-246. doi: 10.1016/j.healthplace.2018.07.001
[3]	KANG Y H, ZHANG F, GAO S, et al. A Review of Urban Physical Environment Sensing Using Street View Imagery in Public Health Studies[J]. Annals of GIS, 2020, 26(3): 261-275. doi: 10.1080/19475683.2020.1791954
[4]	IBRAHIM M R, HAWORTH J, CHENG T. Understanding Cities with Machine Eyes: A Review of Deep Computer Vision in Urban Analytics[J]. Cities, 2020, 96: 102481. doi: 10.1016/j.cities.2019.102481
[5]	刘伦, 王辉. 城市研究中的计算机视觉应用进展与展望[J]. 城市规划, 2019, 43(1): 117-124. https://www.cnki.com.cn/Article/CJFDTOTAL-CSGH201901022.htm LIU L, WANG H. Application of Computer Vision in Urban Studies: Review and Prospect[J]. City Planning Review, 2019, 43(1): 117-124. https://www.cnki.com.cn/Article/CJFDTOTAL-CSGH201901022.htm
[6]	翟俊海, 赵文秀, 王熙照. 图像特征提取研究[J]. 河北大学学报(自然科学版), 2009, 29(1): 106-112. doi: 10.3969/j.issn.1000-1565.2009.01.024 ZHAI J H, ZHAO W X, WANG X Z. Research on the Image Feature Extraction[J]. Journal of Hebei University (Natural Science Edition), 2009, 29(1): 106-112. doi: 10.3969/j.issn.1000-1565.2009.01.024
[7]	黄凯奇, 任伟强, 谭铁牛. 图像物体分类与检测算法综述[J]. 计算机学报, 2014, 37(6): 1225-1240. https://www.cnki.com.cn/Article/CJFDTOTAL-JSJX201406001.htm HUANG K Q, REN W Q, TAN T N. A Review on lmage Object Classification and Detection[J]. Chinese Journal of Computers, 2014, 37(6): 1225-1240. https://www.cnki.com.cn/Article/CJFDTOTAL-JSJX201406001.htm
[8]	张慧, 王坤峰, 王飞跃. 深度学习在目标视觉检测中的应用进展与展望[J]. 自动化学报, 2017, 43(8): 1289-1305. doi: 10.16383/j.aas.2017.c160822 ZHANG H, WANG K F, WANG F Y. Application and Prospect of Deep Learning in Visual Object Detection[J]. Acta Automatica Sinica, 2017, 43(8): 1289-1305. doi: 10.16383/j.aas.2017.c160822
[9]	姜枫, 顾庆, 郝慧珍, 等. 基于内容的图像分割方法综述[J]. 软件学报, 2017, 28(1): 160-183. doi: 10.13328/j.cnki.jos.005136 JIANG F, GU Q, HE H Z, et al. Survey on Content-Based Image Segmentation Methods[J]. Journal of Software, 2017, 28(1): 160-183. doi: 10.13328/j.cnki.jos.005136
[10]	胡一可, 李晶. 基于旅游者和日常访问者人群行为的城市型景区"共处"空间研究[J]. 中国园林, 2019, 35(6): 61-66. https://www.cnki.com.cn/Article/CJFDTOTAL-ZGYL201906012.htm HU Y K, LI J. Research on the "Coexistence" Space of Urban Scenic Spots Based on the Behavior of Tourists and Daily Visitors[J]. Chinese Landscape Architecture, 2019, 35(6): 61-66. https://www.cnki.com.cn/Article/CJFDTOTAL-ZGYL201906012.htm
[11]	CARLSON J A, LIU B O, SALLIS J F, et al. Automated High-Frequency Observations of Physical Activity Using Computer Vision[J]. Medicine and Science in Sports and Exercise, 2020, 52(9): 2029-2036. doi: 10.1249/MSS.0000000000002341
[12]	YU D. Reprogramming Urban Block by Machine Creativity: How to Use Neural Networks as Generative Tools to Design Space[C]. Berlin: eCAADe, 2020.
[13]	YE X Y, DU J X, YE Y. MasterplanGAN: Facilitating the Smart Rendering of Urban Master Plans via Generative Adversarial Networks[J]. Environment and Planning B: Urban Analytics and City Science, 2021, 49(3): 689263772.
[14]	张丽英, 裴韬, 陈宜金, 等. 基于街景图像的城市环境评价研究综述[J]. 地球信息科学学报, 2019, 21(1): 46-58. https://www.cnki.com.cn/Article/CJFDTOTAL-DQXX201901007.htm ZHANG L Y, PEI T, CHEN Y J, et al. A Review of Urban Environmental Assessment Based on Street View Images[J]. Journal of Geo-information Science, 2019, 21(1): 46-58. https://www.cnki.com.cn/Article/CJFDTOTAL-DQXX201901007.htm
[15]	BIBAULT J, BASSENNE M, REN H, et al. Deep Learning Prediction of Cancer Prevalence from Satellite Imagery[J]. Cancers, 2020, 12(12): 3844. doi: 10.3390/cancers12123844
[16]	ORDONEZ V, BERG T L. Learning High-Level Judgments of Urban Perception[C]//FLEET D, PAJDLA T, SCHIELE B, et al. Computer Vision: ECCV 2014. Switzerland: Springer, Cham, 2014: 494-510.
[17]	WANG R Y, YUAN Y, LIU Y, et al. Using Street View Data and Machine Learning to Assess How Perception of Neighborhood Safety Influences Urban Residents' Mental Health[J]. Health and Place, 2019, 59: 102186. doi: 10.1016/j.healthplace.2019.102186
[18]	THOMPSON J, STEVENSON M, WIJNANDS J S, et al. A Global Analysis of Urban Design Types and Road Transport Injury: An Image Processing Study[J]. Lancet Planet Health, 2020, 4(1): e32-e42. doi: 10.1016/S2542-5196(19)30263-3
[19]	CLARK S N, ALLI A S, BRAUER M, et al. High-Resolution Spatiotemporal Measurement of Air and Environmental Noise Pollution in Sub-Saharan African Cities: Pathways to Equitable Health Cities Study Protocol for Accra, Ghana[J]. BMJ Open, 2020, 10(8): e35798.
[20]	VERMA D, JANA A, RAMAMRITHAM K. Predicting Human Perception of the Urban Environment in a Spatiotemporal Urban Setting Using Locally Acquired Street View Images and Audio Clips[J]. Building and Environment, 2020, 186: 107340. doi: 10.1016/j.buildenv.2020.107340
[21]	ZHANG F, ZHOU B L, LIU L, et al. Measuring Human Perceptions of a Large-Scale Urban Region Using Machine Learning[J]. Landscape and Urban Planning, 2018, 180: 148-160. doi: 10.1016/j.landurbplan.2018.08.020
[22]	ROSSETTI T, LOBEL H, ROCCO V, et al. Explaining Subjective Perceptions of Public Spaces as a Function of the Built Environment: A Massive Data Approach[J]. Landscape and Urban Planning, 2019, 181: 169-178. doi: 10.1016/j.landurbplan.2018.09.020
[23]	JAVANMARDI M, HUANG D, DWIVEDI P, et al. Analyzing Associations Between Chronic Disease Prevalence and Neighborhood Quality Through Google Street View Images[J]. IEEE Access, 2020, 8: 6407-6416. doi: 10.1109/ACCESS.2019.2960010
[24]	BOLME D S, BEVERIDGE J R, DRAPER B A, et al. Visual Object Tracking Using Adaptive Correlation Filters[C]//IEEE. 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2010: 2544-2550.
[25]	VOIGTLAENDER P, LUITEN J, TORR P H, et al. Siam R-CNN: Visual Tracking by Re-detection[C]//IEEE. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New York: IEEE, 2020: 6577-6587.
[26]	PANDEY A, WANG D. TCNN: Temporal Convolutional Neural Network for Real-Time Speech Enhancement in the Time Domain[C]//IEEE. ICASSP 2019: 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). New York: IEEE, 2019: 6875-6879.
[27]	CARLSON J, LIU B, SALLIS J, et al. Automated Ecological Assessment of Physical Activity: Advancing Direct Observation[J]. International Journal of Environmental Research and Public Health, 2017, 14(12): 1487. doi: 10.3390/ijerph14121487
[28]	曾明如, 熊嘉豪, 祝琴. 基于T-Fusion的TFP3D人体行为识别算法[J/OL]. 计算机集成制造系统, 2022: 1-13[2022-12-04]. http://kns.cnki.net/kcms/detail/11.5946.TP.20221027.1649.006.html. ZENG M R, XIONG J H, ZHU Q. TFP3D Human Behavior Recognition Algorithm Based on T-Fusion[J/OL]. Computer Integrated Manufacturing Systems, 2022: 1-13[2022-12-04]. http://kns.cnki.net/kcms/detail/11.5946.TP.20221027.1649.006.html.
[29]	位俊超, 陈春雨. 基于SAT-GCN的花样滑冰选手动作检测算法研究[J/OL]. 应用科技, 2022: 1-7[2022-12-05]. http://kns.cnki.net/kcms/detail/23.1191.U.20221101.1729.004.html. WEI J C, CHEN C Y. Research on Motion Detection Algorithm of Figure Skaters Based on Spatio-Temporal Graph Convolution Method[J/OL]. Applied Science and Technology, 2022: 1-7[2022-12-05]. http://kns.cnki.net/kcms/detail/23.1191.U.20221101.1729.004.html.
[30]	徐一峰. 生成对抗网络理论模型和应用综述[J]. 金华职业技术学院学报, 2017, 17(3): 81-88. doi: 10.3969/j.issn.1671-3699.2017.03.019 XU Y F. A Review of Generative Adversarial Network Theoretical Models and Applications[J]. Journal of Jinhua Polytechnic, 2017, 17(3): 81-88. doi: 10.3969/j.issn.1671-3699.2017.03.019
[31]	GE Y H, XU J S, ZHAO B N, et al. DALL-E for Detection: Language-Driven Context Image Synthesis for Object Detection[EB/OL]. (2022-06-20)[2022-07-12]. https://arxiv.org/abs/2206.09592.
[32]	KAPLAN S. The Restorative Benefits of Nature: Toward an Integrative Framework[J]. Journal of Environmental Psychology, 1995, 15(3): 169-182. doi: 10.1016/0272-4944(95)90001-2
[33]	NORDH H, HARTIG T, HAGERHALL C M, et al. Components of Small Urban Parks that Predict the Possibility for Restoration[J]. Urban Forestry and Urban Greening, 2009, 8(4): 225-235. doi: 10.1016/j.ufug.2009.06.003
[34]	LI D Y, DEAL B, ZHOU X L, et al. Moving Beyond the Neighborhood: Daily Exposure to Nature and Adolescents' Mood[J]. Landscape and Urban Planning, 2018, 173: 33-43. doi: 10.1016/j.landurbplan.2018.01.009
[35]	HELBICH M. Dynamic Urban Environmental Exposures on Depression and Suicide (NEEDS) in the Netherlands: A Protocol for a Cross-Sectional Smartphone Tracking Study and a Longitudinal Population Register Study[J]. BMJ Open, 2019, 9(8): e30075.
[36]	HELBICH M, POPPE R, OBERSKI D, et al. Can't See the Wood for the Trees? An Assessment of Street View and Satellite-Derived Greenness Measures in Relation to Mental Health[J]. Landscape and Urban Planning, 2021, 214: 104181. doi: 10.1016/j.landurbplan.2021.104181
[37]	SALESSES P, SCHECHTNER K, HIDALGO C A. The Collaborative Image of The City: Mapping the Inequality of Urban Perception[J]. PLoS One, 2013, 8(7): e684007.
[38]	DUBEY A, NAIK N, PARIKH D, et al. Deep Learning the City: Quantifying Urban Perception at a Global Scale[C]//LEIBE B, MATAS J, SEBE N, et al. Computer Vision: ECCV 2016. Switzerland: Springer, Cham, 2016.
[39]	TANG J, LONG Y. Measuring Visual Quality of Street Space and Its Temporal Variation: Methodology and Its Application in the Hutong Area in Beijing[J]. Landscape and Urban Planning, 2019, 191: 103436. doi: 10.1016/j.landurbplan.2018.09.015
[40]	KWON J, CHO G. An Examination of the Intersection Environment Associated With Perceived Crash Risk Among School-Aged Children: Using Street-Level Imagery and Computer Vision[J]. Accident Analysis & Prevention, 2020, 146: 105716.
[41]	MA X Y, MA C Y, WU C, et al. Measuring Human Perceptions of Streetscapes to Better Inform Urban Renewal: A Perspective of Scene Semantic Parsing[J]. Cities, 2021, 110: 103086. doi: 10.1016/j.cities.2020.103086
[42]	LIU Y Q, WANG R Y, LU Y, et al. Natural Outdoor Environment, Neighbourhood Social Cohesion and Mental Health: Using Multilevel Structural Equation Modelling, Streetscape and Remote-Sensing Metrics[J]. Urban Forestry and Urban Greening, 2020, 48: 126576. doi: 10.1016/j.ufug.2019.126576
[43]	HELBICH M, YAO Y, LIU Y, et al. Using Deep Learning to Examine Street View Green and Blue Spaces and Their Associations with Geriatric Depression in Beijing, China[J]. Environment International, 2019, 126: 107-117. doi: 10.1016/j.envint.2019.02.013
[44]	WEI H, HAUER R J, ZHAI X. The Relationship Between the Facial Expression of People in University Campus and Host-City Variables[J]. Applied Sciences, 2020, 10(4): 1474. doi: 10.3390/app10041474
[45]	PARSONS R. The Potential Influences of Environmental Perception on Human Health[J]. Journal of Environmental Psychology, 1991, 11(1): 1-23. doi: 10.1016/S0272-4944(05)80002-7
[46]	ZHANG F, ZU J Y, HU M Y, et al. Uncovering Inconspicuous Places Using Social Media Check-Ins and Street View Images[J]. Computers, Environment and Urban Systems, 2020, 81: 101478. doi: 10.1016/j.compenvurbsys.2020.101478
[47]	MITCHELL R, POPHAM F. Effect of Exposure to Natural Environment on Health Inequalities: An Observational Population Study[J]. Lancet, 2008, 372(9650): 1655-1660. doi: 10.1016/S0140-6736(08)61689-X
[48]	ULRICH R S. View Through a Window May Influence Recovery From Surgery[J]. Science, 1984, 224(4647): 420-421. doi: 10.1126/science.6143402
[49]	MAHARANA A, NSOESIE E O. Use of Deep Learning to Examine the Association of the Built Environment with Prevalence of Neighborhood Adult Obesity[J]. JAMA Network Open, 2018, 1(4): e181535. doi: 10.1001/jamanetworkopen.2018.1535
[50]	ZHENG T S, BERGIN M H, HU S J, et al. Estimating Ground-Level PM2.5 Using Micro-Satellite Images by a Convolutional Neural Network and Random Forest Approach[J]. Atmospheric Environment, 2020, 230: 117451. doi: 10.1016/j.atmosenv.2020.117451
[51]	WANG R, YANG B, YAO Y, et al. Residential Greenness, Air Pollution and Psychological Well-Being Among Urban Residents in Guangzhou, China[J]. Science of the Total Environment, 2020, 711: 134843. doi: 10.1016/j.scitotenv.2019.134843
[52]	ZHANG Y, CHEN N C, DU W Y, et al. Multi-Source Sensor Based Urban Habitat and Resident Health Sensing: A Case Study of Wuhan, China[J]. Building and Environment, 2021, 198: 107883. doi: 10.1016/j.buildenv.2021.107883
[53]	SUN Q C, MACLEOD T, BOTH A, et al. A Human-Centred Assessment Framework to Prioritise Heat Mitigation Efforts for Active Travel at City Scale[J]. Science of the Total Environment, 2021, 763: 143033. doi: 10.1016/j.scitotenv.2020.143033
[54]	RACHELE J N, WANG J, WIJNANDS J S, et al. Using Machine Learning to Examine Associations Between the Built Environment and Physical Function: A Feasibility Study[J]. Health and Place, 2021, 70: 102601. doi: 10.1016/j.healthplace.2021.102601
[55]	MENNIS J, LI X, MEENAR M, et al. Residential Greenspace and Urban Adolescent Substance Use: Exploring Interactive Effects with Peer Network Health, Sex, and Executive Function[J]. International Journal of Environmental Research and Public Health, 2021, 18(4): 1611. doi: 10.3390/ijerph18041611
[56]	WANG R Y, HELBICH M, YAO Y, et al. Urban Greenery and Mental Wellbeing in Adults: Cross-Sectional Mediation Analyses on Multiple Pathways Across Different Greenery Measures[J]. Environmental Research, 2019, 176: 108535. doi: 10.1016/j.envres.2019.108535
[57]	LU Y, SARKAR C, XIAO Y. The Effect of Street-Level Greenery on Walking Behavior: Evidence from Hong Kong[J]. Social Science & Medicine, 2018, 208: 41-49.
[58]	LI X, SANTI P, COURTNEY T K, et al. Investigating the Association Between Streetscapes and Human Walking Activities Using Google Street View and Human Trajectory Data[J]. Transactions in GIS, 2018, 22 (4): 1029-1044. doi: 10.1111/tgis.12472
[59]	LU Y, YANG Y Y, SUN G B, et al. Associations Between Overhead-View and Eye-Level Urban Greenness and Cycling Behaviors[J]. Cities, 2019, 88: 10-18.
[60]	LU Y. Using Google Street View to Investigate the Association between Street Greenery and Physical Activity[J]. Landscape and Urban Planning, 2019, 191: 103435.
[61]	WANG R Y, LIU Y, LU Y, et al. The Linkage Between the Perception of Neighbourhood and Physical Activity in Guangzhou, China: Using Street View Imagery with Deep Learning Techniques[J]. International Journal of Health Geographics, 2019, 18(1): 1-11.
[62]	CHEN L, LU Y, SHENG Q, et al. Estimating Pedestrian Volume Using Street View Images: A Large-Scale Validation Test[J]. Computers, Environment and Urban Systems, 2020, 81: 101481.
[63]	XING X Y, HUANG Z, CHENG X M, et al. Mapping Human Activity Volumes Through Remote Sensing Imagery[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2020, 13: 5652-5668.
[64]	LI M Y, LIU J X, LIN Y F, et al. Revitalizing Historic Districts: Identifying Built Environment Predictors for Street Vibrancy Based on Urban Sensor Data[J]. Cities, 2021, 117: 103305.
[65]	WANG M S, VERMEULEN F. Life Between Buildings from a Street View Image: What Do Big Data Analytics Reveal About Neighbourhood Organisational Vitality?[J]. Urban Studies, 2021, 58(15): 3118-3139.