International Review for Spatial Planning and Sustainable Development
Online ISSN : 2187-3666
ISSN-L : 2187-3666
Planning Strategies and Design Concepts
Seniors’ Outdoor Survey: Comparing Assessment Results and Interrater Reliability for Outdoor Environmental Spaces
A Case Study of Shuiqinggou Park, Qingdao City, China
Jing LuAlamah Misni
Author information
JOURNAL OPEN ACCESS FULL-TEXT HTML

2023 Volume 11 Issue 3 Pages 20-43

Details
Abstract

The residential outdoor environment may not always appropriately meet the needs of the elderly. Compared to other age groups, the elderly spend more time in their surrounding residential and outdoor spaces. This is due to their lack of ability physically and energetically. The purpose of this study was to test the applicability of the Seniors’ Outdoor Survey (SOS) tool, which is an environmental assessment instrument, in terms of its ratings and reliability based on different ages and genders. Labour Square in Shandong Province of China served as the site study location. Raters of different ages were appointed to increase the experiment's reliability; 20-40, 40-60, and 60-80 years old. Two trained raters of different genders were selected for each age group. Then, the raters were assigned to evaluate on a scale of 1 to 7 for 60 instrument items of outdoor environmental factors for the elderly. The results showed that the mean ratings among raters were similar, while the mean raters’ reliability was excellent and applicable to the elderly for all three groups. The mean interrater reliability (ICC) was 0.826. Moreover, the T-test's significance (p-value) was 0.869. Despite rating differences across age groups, the high ICC suggested that this instrument might be helpful in a wide range of environmental settings. Furthermore, the research findings can be used as a guide for the Chinese government and design institutes to apply the SOS tool to solve the environmental factors of the residential outdoor environment for the elderly.

Introduction

The residential environment plays an essential role in the subjective well-being of the elderly. As the residential environment of the elderly profoundly impacts their well-being, it should be carefully designed (Herbers and Mulder, 2017). It is essential to create an ecological vitality space to stimulate the elderly to be active. The Seniors’ Outdoor Survey (SOS) tool should be used to gauge whether they are participating in community activities, adjusting their emotions, and integrating into society.

According to the United Nations’ standard for an ageing society(United-Nations, 2018), China has been ageing since 2000 (Zhang, Y. and Jing, 2012). Even more surprising is that this "ageing" trend is proceeding at a galloping pace. In China, the study of residential outdoor landscapes suitable for ageing is still in its early stages, lacking a systematic and comprehensive theoretical support and a design practice basis. As an essential part of the residential area, the outdoor environment can promote the elderly’s contact with the natural environment, social interaction, as well as physical and mental health.

Benefits of nature and outdoor spaces

Access to nature and the outdoors brings multiple physical and psychological benefits, and older adults can benefit greatly from safe and practical activities (Helbich, Yao et al., 2019; Piercy, Troiano et al., 2018; Wang and MacMillan, 2013), such as improved mood and sense of well-being, better regulation of physical functioning, including greater longevity (Levinger, Panisset et al., 2021; Xie and Yuan, 2022). The benefits of outdoor activities are even more critical for the elderly. Compared to other age groups, the elderly is restricted to spending most of their time in their residential areas (Clarke and Nieuwenhuijsen, 2009). So, it is crucial to improve the residential outdoor environment for the elderly (Zhang, Z., Tang et al., 2022; Zheng, Akita et al., 2022). Evaluating whether the environment is suitable for the elderly is a critical study. No studies in recent years have systematically examined potential disparities in raters’ age and gender. This is important because many present scholars actively use the SOS instrument. Comparing the findings of the SOS instrument with different raters is a small step in this direction.

Importance of environmental assessment

Despite the potential benefits of contact with natural elements, the residential outdoor environment may only sometimes be appropriate to meet the needs of seniors. Physical environmental characteristics influence seniors' appreciation and usage of outdoor areas (Joseph, Zimring et al., 2006; Rantakokko, Mänty et al., 2009; Rodiek, Lee et al., 2014; Ross, 2014; Wolf and Housley, 2016). According to Chalfont and Rodiek (2005); Rodiek and Lee (2009), more services for seniors in outdoor spaces are required, and specific problems to be resolved in the design, planning, and maintenance conditions. To better understand how the elderly perceive and use outdoor spaces, several studies have assessed seniors' attitudes and use of various environmental features (Brüchert, Baumgart et al., 2022; Rodiek and Schwarz, 2006). Previous studies have explored differences in overall landscape design, perception, and use (Liang and Guan, 2023; Velarde, Fry et al., 2007; Yu, Guo et al., 2021), as well as employed various methods to evaluate the seniors in residential outdoor environments (Carlson, Sallis et al., 2012; Kemperman and Timmermans, 2014; Scopelliti and Giuliani, 2006; Wong, Yu et al., 2017). Very few validated instruments can produce valid and reliable assessments of the outdoor environment for seniors (Bardenhagen and Rodiek, 2015; Cutler, Kane et al., 2006; Rodiek, 2008). Hence, the Seniors’ Outdoor Survey (SOS) tool was recently developed to fill this gap and has demonstrated good psychometric properties (Bardenhagen and Rodiek, 2015; Rodiek, Nejati et al., 2016).

Comparison of different assessments

Due to the increasing global resources in improving the outdoor environment and the growing interest in the health of the elderly, SOS tools have been developed and are available for download at http://www.accesstonature.org/resources.html. The SOS tools have been adopted and utilised in several regions, including Italy, China, Japan, and Spain. Since its development was conducted in a range of long-term care settings in southeast Texas (Rodiek, Lee et al., 2014), the SOS tool has been used in Outdoor Spaces of Seniors Housing (Bardenhagen and Rodiek, 2015), and it has since been gradually popularised. Some scholars compared diverse settings with a similar climate: Milan, Italy, and Bryan/College Station, Texas, USA.

In Malaysian retirement homes for the elderly, Bardenhagen, Rodiek et al. (2018) and some scholars used SOS tools to evaluate outdoor facilities for elderly care (Hussain and Nayan, 2021). The items and domains of an SOS tool were compared with those discussed in 15 other published design recommendations in a systematic review conducted in Malaysia (Er and Shukor, 2016). However, the tool's reliability needed to be further explored.

In this study, the SOS tool was used to compare the reliability of different raters in evaluating the outdoor environments of advanced facilities in China. The use of SOS tools to assess the outdoor environments of older adults demonstrated a significant similarity to earlier studies. The difference lies in the choice of study areas. In addition, previous studies have not utilised SOS tools to compare the elderly’s outdoor spaces with different raters (including those of varying ages, cognition, and gender). Meanwhile, this study utilised the SOS tool by selecting raters of different ages, genders, and occupations in Qingdao, China for comparison. The study examined whether there were substantial differences between different raters or whether the characteristics of the available outdoor spaces for older adults in residential outdoor environments were somewhat cross-age, cross-gender, and universal.

Research Methodology

SOS tool and properties

According to Bardenhagen and Rodiek (2015); Bardenhagen, Rodiek et al. (2018), the SOS instrument can sensitively evaluate environmental characteristics in a study period. The SOS tool ultimately consists of five domains and 60 evaluable items: (1) access into nature (14 items), focusing on natural elements like plants and animals; (2) outdoor comfort and safety (15 items), emphasising ergonomics and landscape facilities; (3) walking and outdoor activities (14 items), mainly with encouraging and supporting physical movement; for physical activity; (4) indoor-outdoor connection (11 items), focusing on potential barriers and connection between indoor and outdoor space; and (5) connection with the world (6 items), concentrating on the effectiveness of communication between the site and the outdoor environment. The five domains are largely concerned with issues common to senior citizens, such as accessibility, environmental tolerance, and preference for nature.

On-site evaluation sheets were completed for each outdoor space, using a rating scale of 1 to 7 (1=worst, 7=best) for each function. In order to develop a numerical weighting based on resident preferences, behavioural outcomes, and expert opinions, Bardenhagen quantified the level of support for each item in 2017. In the end, a straightforward formula emerged which generated the following satisfactory results:

Item Rating * Item Weighting * 2 Multiplier + 60 Addend = Item Score.

This tool offers researchers automatic scoring and, eventually, a fixed format (Bardenhagen, Rodiek et al., 2018). The instrument can result in project scores ranging from 62.4 (the lowest score for the least essential item) to 116 (the highest score for the essential item). A medium-grade assessment of medium-importance items is expected to result in a score of 80, or roughly an equivalent to a low B-grade item; a medium-grade evaluation of medium-importance items would be expected to yield a score of 80, or approximately the equivalent of a low B-grade (Bardenhagen, Rodiek et al., 2018).

It not only fills an important gap in the assessment of the outdoor environment for the elderly but also comprehensively assesses the outdoor environment as an essential part of the residential outdoor environment without the need for particular expertise (Rodiek, Nejati et al., 2016). Using the SOS tool, the assessment can be conducted from a practical, convenient, and aesthetic perspective, providing researchers and designers with a simple and reliable way to assess and compare the potential of the residential outdoor environment to support the elderly.

The SOS tool does, however, apply mainly to older people, while its raters are not necessarily older people. In the development of the tool, there was no emphasis on factors such as age, occupation, or gender of the raters using the tool. This is because different raters may use this tool in different ways, hence there will be subtle differences in the survey results.

Affordances

The SOS tool tends to have some generality in terms of physical exercise and mental health and focuses mainly on the needs of the elderly, such as comfort and safety. The tool’s instructions allow raters to estimate the degree to which each environmental feature supports outdoor use by frail elderly residents of senior facilities in addition to assessing the intrinsic value of each feature. This emphasis on supporting or "affordability" features can be helpful in environmental assessments (Bardenhagen and Rodiek, 2015). Therefore, this study intended to provide a higher level of objectivity and reliability to the tool’s ratings by reducing the role of the rater’s personal preferences.

Although they may be scored differently by people of different ages, genders, and occupations, this study did not modify the tool. In addition to the primary purpose of using the tool to assess the ageing suitability of the outdoor environmental spaces in Qingdao residential outdoor environments, this study also tested whether different raters could effectively use the tool and whether the survey results were reliable.

Training process

Any number of raters can evaluate outdoor spaces in any role, ranging from administrators and activity directors to direct care staff and outdoor maintenance personnel (Bardenhagen and Rodiek, 2015). In some studies, two trainers were selected (Bardenhagen, Rodiek et al., 2018). In order to increase the validity of the study, this study selected six raters of different ages, genders, and occupations.

In the first step, three age groups of raters were selected: 20-40 years old, 40-60 years old, and 60-80 years old, and two individuals of different genders were selected for training from each age group. The trainers would randomly choose anyone, including activity users, maintainers, and professionals. The trainers did not need to have any particular expertise, but they should first carefully read and apply a brief description of the tool. The training was remarkably comparable across all age ranges. In the second step, the raters of each age group pre-tested other outdoor spaces using the SOS tool. To use the SOS tool for the test, they should understand not only the background of the space environment, but also the activities of the elderly, envisioning themselves in the position of the elderly. In the third step, the raters were given a brief training session that lasted around four hours, after which they pre-tested the SOS tool once again in an outdoor space outside the study. After the training, the researchers compared their results and discussed the differences in the understanding of the spaces and the tool. This was to resolve the differences and arrive at more accurate judgments.

The training process was the same as that of an earlier prototype of the tool, which was rated with 63 features of 152 outdoor spaces in a multi-area US study in 2007 (Rodiek and Lee, 2009).

Study site

This study was conducted at Labor Square, also known as Shuiqinggou Park, in Qingdao City of Shandong Province, China. Qingdao is in the east of China, in the southeastern part of Shandong Province. Qingdao is a coastal city about 700 kilometres from Beijing, i.e., more than eight hours of drive and about four hours of the train ride. It is close to the Yellow Sea, facing the Korean Peninsula across the sea, and is at the forefront of the China-Japan-ROK free trade area (Figure 1). Qingdao is in the north-temperate East Asian monsoon region, i.e., a warm temperate monsoon climate. The primary characteristics of the urban climate are "moist air, abundant rainfall, moderate temperature, four distinct seasons", very suitable for outdoor activities.

Figure 1. The key plan area of study was located in Qingdao, Shandong Province

Qingdao is a well-developed city. According to the seventh Chinese population census in 2020, Qingdao had a population of 10.07 million, ranking second in Shandong province and 16th in China. The ageing population structure was further deepened, with 2.04 million people aged 60 and above, accounting for 20.28 per cent. Labor Square is in the north district of Qingdao. It is adjacent to residential areas in the east, Siliu South Road in the west, Huaiyang Road in the south, and Kaiping Road in the north. Since 2005, the park covers an area of 21,000 square meters (Figure 2).

This area was selected because many senior citizens visit its outdoor environment for recreation. The study investigated new residential areas (built after 2000) within a 10-minute walk to Labor Square. There are about six new residential areas, mainly east and west of Labor Square. The rest are older residential areas (built around 1980), mainly to the east and west of Labor Square. There are about 30,000 people in the residential areas within a 10-minute walk to Labor Square, and the elderly population is about 6,000. This is shown in Table 1. About four to five hundred older people pass their time in Labor Square daily. Therefore, the residential outdoor environment should be designed better for the elderly.

Figure 2. Labor Square's surrounding environment. The green area is the labor square, surrounded by the new and old residential, and the public building

Table 1. Surrounding residential areas and the number of elderlies in Labor Square
Residential area Built years Floor area (m2) Building area (m2) Number of households Number of people Number of elderlies
Haiqing residential area 2012 32466 117006.5 1122 3480 630
Hongtai Jinyuan residential area 2015 45471 143688 986 3100 560
Qingshui Yuan residential area 2003 31000 86800 725 2160 400
Yalian new residential area 1998 20600 41200 360 980 180
Yalian apartment 2001 9450 11340 126 250 20
Yalian Villa residential area 2003 71190 106785 1017 2700 480
Shuiqing's new residential area 2012 26319 60008 577 1558 280
Old residential area In the 1980s 187851 565553 6461 19380 3490
Total 424347 1132381 11374 33608 6040

Outdoor spaces

Labor Square was divided into three zones based on its environmental characteristics (Figure 3). Space A is an open square covered with hard pavement. Space B is the centre zone/space consisting of a primary sculpture surrounded by pools, corridors, etc. The main feature of Space C is a tree array that combines corridors and sports fields to provide entertainment places for everyone. Although the three spaces are different, the number of older people who use the site on weekdays is similar. Space A has different outdoor activities in different periods, such as square dancing and square music festival. Space B is mainly for chatting, playing chess, and dancing. Space C is specifically for playing cards, sports, sitting, and other activities. In each space, the density of use by the elderly is different in different periods.

Figure 3. Labor Square is divided into three zone spaces: A, B, and C

These outdoor spaces were selected to be rated by observing the physical evidence of usage by the residents. These sites were also suitable and recommended in the SOS tool instructions based on methods previously described by Zeisel (2006). The main concern with the SOS tool was the use of outdoor spaces for the elderly. However, these spaces were originally designed for residents of all ages, which lacked an excellent opportunity for the elderly’s convenience. So, a complete redesign was needed to make them more suitable for the elderly, of all ages.

Data collection

The facility's outdoor space was evaluated in the early autumn, and the plants were luxuriant. During the scoring process, trained raters collected the data from each study site. Two seniors were selected among the volunteers who often conduct activities in the outdoor environment of the residential areas. They were both 68 years of age. They frequently engage in exercise and recreational activities together in Labor Square. Next were two 45-year-old middle-aged adults. One teaches environmental design courses at a university, while the other is mainly responsible for architectural and environmental designs at a design institute. The rest were two young students aged 21 who belonged to the same discipline (environmental design) and have taken professional courses together. Each rater took an average of more than 30 minutes to collect data from each outdoor space of Labor Square. They did not interfere with each other and scored individually.

Data entry and analysis

The data were generated on the SPSSPRO website ( https://www.spsspro.com/) to estimate their reliability statistics. The collected data were entered into a Microsoft Excel™ spreadsheet where each rater scored 60 sub-items of four significant domains for each space. From these values, the average was calculated for each of the three spatial scores, and the percentage difference was calculated against the total score.

Intraclass Correlation Coefficient (ICC) was used to assess interrater reliability. The raters stratified the process into a Microsoft Excel™ spreadsheet by listing their ratings for each of the three spaces (Table 2) and putting them on the SPSSPRO website. The intraclass Correlation Coefficient was selected to calculate the reliability of each score. ICC values ranged from 0 to 1, indicating the lowest to the highest possible protocol level. Although there is a lack of consensus in determining an acceptable ICC value, levels higher than 0.40 (P ≥0.40) have been proposed to be acceptable reliability levels on the SPSSPRO website. In some studies, the proposed acceptable reliability levels would be higher than 0.60 or 0.75 (Anastasi and Urbina, 1997; Portney and Watkins, 1993).

Table 2. Diagram of ICC calculation process
Space rater1 rater2 rater3 rater4 rater5
Space A Score1 Score2 …… …… ……
Space B Score3 …… …… …… ……
Space C …… …… …… …… ……

T-tests were used to check for statistically significant differences in raters’ scores between each item. The calculation process involved stratifying each item on a Microsoft Excel™ spreadsheet, counting the six raters’ ratings for each item (Table 3, and then putting them onto the SPSSPRO website to select the Wilcoxon Rank Sum Test. The one-sample Wilcoxon signed rank test was used to determine whether the p-value showed significance (P<0.05) to test the difference of the score variable of the raters among the items.

Table 3. Diagram of the T-Test calculation process
Rater Abundance of greenery

The diverse mix

of plants and trees

Easily reachable or raised plants Seating has pleasant views ……
rater1 Score1 Score2 …… …… ……
rater2 Score3 …… …… …… ……
rater3 …… …… …… …… ……

Results

The SOS tool was used to evaluate Access to Nature (14 items), Outdoor Comfort and Safety (15 items), Walking and Outdoor Activities (14 items), Indoor-Outdoor Connection (11 items), and Connection to the World (6 items). The results showed the tool's reliability and each area's environmental characteristics.

Table 4 compares the six raters' ratings of the five major domains (environmental characteristics) and the reliability among raters. Table 4 shows that the reliability of the average item score of the tool was relatively high among raters. In Space A, the highest reliability score was 0.958, the lowest was 0.557 (≥0.4), and the average was 0.871. In Space B, the highest reliability score was 0.982, the lowest was 0.505 (≥0.4), and the average was 0.872. In Space C, the highest reliability score was 0.976, the lowest was 0.432 (≥0.4), and the average was 0.847. The results showed that the SOS tool was reliable across different raters.

Table 4. The score of the raters, ICC, and comparisons for each item of outdoor space
Space

Raters

(Age - Gender)

Domains/Items Average Score

Access to Nature

(14 items)

Outdoor Comfort and

Safety

(15 items)

Walking and Outdoor Activities

(14 items)

Indoor-Outdoor Connection (11 items)

Connection to the

World

(6 items)

Space A 45 - Female 78.37 76.52 89.97 87.33 86.44 83.73
45 - Male 81.44 73.21 88.63 83.85 85.44 82.51
68 - Female 78.99 75.37 85.31 84.29 87.22 82.24
68 - Male 75.23 72.09 85.76 85.18 81.78 80.01
21 - Female 76.3 72.8 87.51 86.42 89.22 82.45
21 - Male 78.79 77.19 88.64 87.09 86.67 83.68
Average Score 78.19 74.53 87.64 85.69 86.13 82.44
Reliability (ICC) 0.945 0.941 0.956 0.958 0.557 0.871
Space B 45 - Female 85.89 77.84 90.19 88.36 89.78 86.41
45 - Male 85.2 78.54 88.96 88.31 90.00 86.20
68 - Female 82.79 78.87 85.64 87.87 86.44 84.32
68 - Male 82.84 76.29 87.86 90 81.89 83.78
21 - Female 84.83 79.39 87.89 92.25 84.22 85.72
21 - Male 86.71 80.46 89.06 89.95 84.67 86.17
Average Score 84.71 78.57 88.27 85.46 86.17 84.64
Reliability (ICC) 0.982 0.97 0.929 0.976 0.505 0.872
Space C 45 - Female 86.23 81.56 90.99 81.67 90.67 86.22
45 - Male 81.67 82.19 89.24 82.04 87.67 84.56
68 - Female 81.29 81.2 90.33 82.45 86.56 84.37
68 - Male 81.94 80.96 89.44 83.13 82.56 83.61
21 - Female 80.74 80.63 87.99 84.42 79.78 82.71
21 - Male 80.93 81.12 89.19 83.31 83.22 83.55
Average Score 82.13 81.28 89.53 82.84 85.08 84.17
Reliability (ICC) 0.972 0.976 0.896 0.957 0.432 0.847

Table 5 displays the scores and the reliability for each item (environmental characteristics) rated by the six raters, and the reliability of each item. Meanwhile, Table 6 continues to list the differences in the raters' scores, which were measured by the SPSSPRO software. Table 5 shows the scores, T-test, and ICC for the four major 60 items. From the T-test, the overall score was 0.869, and the ICC was 0.826, indicating that the reliability of the total score was relatively strong. For Access to Nature items, the T-test was 0.909 and the ICC was 0.942. Meanwhile, for Outdoor Comfort and Safety, the T-test was 0.732 and the ICC was 0.968. For Walking and Outdoor Activities, the T-test was 0.909 and the ICC was 0.667. Regarding Indoor-Outdoor Connection, the T-test was 0.864 and the ICC was 0971. As the T-test score for Connection to the World was 0.932, the ICC was 0.584. Even though the raters were of different age groups and genders, each T-test and ICC score were above the trustworthiness value (≥0.4), indicating the extensive application and reliability of the SOS tool.

Table 5. The scores by three age groups of raters on environmental characteristics
Domains/Items Raters
Concise Item Description 45 years old, female 45 years old, male 68 years old, female 68 years old, male 21 years old, female

21

years

old,

male

Overall Instrument

(5 domains)

85.82 84.94 84.16 83.08 84.31 85.13
Access to Nature 83.50 82.77 81.02 80.00 80.62 82.14
1 Abundance of greenery 96 98 96 92 92 96
2 The diverse mix of plants and trees 90.8 97.4 93 88.6 90.8 93
3 Abundant flowers and colour 93 90.8 95.2 88.6 90.8 88.6
4 Easily reachable plants 88 92 88 88 84 86
5 Seating has pleasant views 92 98 88 84 88 88
6 Hard boundaries screened by plants 96 96 81.6 84 91.2 93.6
7 Water features available 86 78 82 84 86 82
8 Features with movement 70.4 71.2 66.4 69.6 68.8 68.8
9 Amenities for pets 71.67 68.33 66.67 71.67 68.33 70
10 Amenities for birds and wildlife 72 73.33 74.67 73.33 70.67 77.33
11 Domesticated animals 71.2 69.6 69.6 68 68 74.4
12 The outdoor area is not noisy 82.67 78.67 80 78.67 80 80
13 Privacy from resident rooms 85.33 82.67 82.67 85.33 81.33 82.67
14 Private places to sit 70.4 64.8 68.8 67.2 68.8 69.6
Outdoor Comfort and Safety 78.64 77.98 78.48 76.45 77.61 79.92
1 Plenty of seating available 93.33 93.33 93.33 86.67 91.11 91.11
2 Choice of different seating types 88 80 88 82 80 84
3 Seats available in sun or shade 88.44 90.22 84.89 84.89 84.89 88.44
4 Some seating is easily movable 74.23 74.22 74.22 68.89 67.12 77.76
5 Seats will not tip over 88.44 84.89 90.22 86.67 90.22 90.22
6 Seating has arms and backs 86 82 80 86 84 84
7 Seats comfortably shaped 81.79 77.12 81.79 77.11 78.66 80.20
8 Seats do not get hot or cold 78.67 78.67 81.78 75.56 80.21 77.13
9 Some seats have cushions 67.78 64.68 67.78 66.22 66.22 67.78
10 Tables for coffee, food, etc. 66.22 72.44 66.22 66.22 67.78 70.88
11 Swings, gliders, rocking chairs 66.22 66.22 74 69.33 69.33 77.11
12 Restroom, drinking fountain 84 84 87.2 85.6 88.8 87.2
13 The outdoor area well maintained 77 79 76 79 76 79
14 Smoking areas well separated 72.8 77.6 66.4 68 74.4 76
15 Microclimate control 66.67 65.33 65.33 65.33 65.33 68
Walking and Outdoor Activities 90.38 88.94 87.16 87.84 87.80 88.96
1 Abundant walkways of different lengths 95.2 99.6 97.4 95.2 97.4 99.6
2 Round-trip walkways available 97.4 101.8 93 95.2 104 95.2
3 Interesting views from walkways 92 94 92 94 96 88
4 Paving level, easy for wheelchairs 101.8 101.8 99.6 99.6 101.8 97.4
5 Paving non-skid and non-glare 99.6 104 99.6 104 99.6 97.4
6 Walkways partly shaded 98 92 94 96 94 94
7 Handrails along some walkways 92 89.87 87.73 89.87 81.33 89.87
8 Frequent seating along walkways 86 82 88 82 70 90
9 Some walkway seating in the shade 90.6 85.2 90.6 83.4 78 81.6
10 Destinations to move towards 84.53 81.45 75.33 81.47 81.47 84.53
11 Places for social activities 89.87 88 82.4 86.13 86.13 88
12 Play areas for children 85 83.33 81.67 86.67 85 83.33
13 Places for recreation and exercise 82.67 74.67 74.67 72 82.67 82.67
14 The place for gardening, horticultural therapy 70.67 67.47 64.27 64.27 71.73 73.87
Indoor-Outdoor Connection 87.91 87.71 86.90 87.96 90.00 89
1 Outdoors visible from main indoor areas 94.53 87.13 87.13 99.47 99.47 94.53
2 Easily reached from indoor commons 105.33 97.33 94.67 102.67 97.33 105.33
3 Multiple ways to reach the outdoor area 76 94 86 94 86 86
4 Indoor transition space near doorway 84.67 84.67 87.13 87.13 92.07 89.6
5 Outdoor transition space near doorway 95.2 84.2 88.6 86.4 93 88.6
6 Doors unlocked during daytime 88 94 92 92 94 90
7 Doors open with minimal effort 90.6 87 85.2 83.4 88.8 85.2
8 Doors do not close too quickly 90.6 90.6 88.8 79.8 88.8 88.8
9 Automatic door available, easy to use - - - - - -
10 Can easily cross door threshold 76 80 81.3 80 81.3 82.67
11 Wide paved landing outside the doorway 78.13 78.13 78.13 74.73 79.27 79.27
Connection to the World (6 items) 88.67 87.29 87.26 83.15 85.52 85.63
1 The area is located near the main entry 92 92 90 90 90 92
2 View of vehicles arriving at the facility 90.22 88.44 86.67 81.33 84.89 84.89
3 Views of front-door activities 88.47 84.90 90.23 86.67 88.47 86.70
4 Views of off-site scenery 86.44 86.44 83.33 80.22 84.88 83.32
5 Views of nearby streets or traffic 89.56 88 88 80.22 86.43 86.44
6 Views of off-site buildings, activities 85.33 84 85.33 77.33 80 78.67

Note:

Download a computable form directly from the website ( http://www.accesstonature.org/resources.html) and fill out a scale of 1-7 to display different scores. Because the full score of each item was different, the full score of each item was weighted by 7 points to calculate the score.

Table 6. The summary score among raters on environmental characteristics; Percent Difference T-test; ICC, comparisons of outdoor spaces rated in Labor Square
Domains/Items Score
Concise Item Description Average Score Full Score Percent Difference

T-test Significance

(p-value)

Mean Reliability (ICC)
Overall Instrument (5 domains) 84.56 97.27 13.08% 0.869 0.826
Access to Nature 81.68 95.5 14.5% 0.909 0.942
1 Abundance of greenery 95 102 6.86% 0.486 0.674
2 The diverse mix of plants and trees 92.27 106.2 13.11% 0.666 0.813
3 Abundant flowers and colours 91.17 106.2 14.15% 0.931 0.917
4 Easily reachable plants 87.67 102 14.05% 0.954 0.728
5 Seating has pleasant views 89.67 102 12.09% 0.646 0.809
6 Hard boundaries screened by plants 90.4 110.4 18.12% 0.774 0.617
7 Water features available 83 102 18.63% 0.841 0.983
8 Features with movement 69.2 76.8 9.90% 0.709 0.824
9 Amenities for pets 69.44 95 26.9% 0.685 0.841
10 Amenities for birds and wildlife 73.56 88 16.4% 0.708 0.65
11 domesticated animals 70.13 93.6 25.07% 0.563 0.74
12 The outdoor area is not noisy 80.00 88 9.09% 1.000 0.568
13 Privacy from resident rooms 83.33 88 5.31% 1.000 0.537
14 Private places to sit 68.27 76.8 11.11% 0.774 0.72
Outdoor Comfort and Safety 78.18 94.82 17.55% 0.732 0.968
1 Plenty of seating available 91.48 106.7 14.26% 0.605 0.951
2 Choice of different seating types 83.67 102 17.97% 0.547 0.944
3 Seats available in sun or shade 86.96 97.3 10.63% 0.604 0.927
4 Some seating is easily movable 72.74 97.3 25.24% 0.509 0.86
5 Seats will not tip over 88.45 97.3 9.10% 1.000 0.875
6 Seating has arms and backs 83.67 102 17.97% 0.489 0.852
7 Seats comfortably shaped 79.44 92.7 14.3% 0.509 0.957
8 Seats do not get hot or cold 78.67 92.7 15.13% 0.819 0.847
9 Some seats have cushions 66.74 92.7 28% 0.454 0.459
10 Tables for coffee, food, etc. 68.29 92.7 26.33% 0.709 0.662
11 Swing, glider, rocking chairs 70.37 92.7 24.09% 0.863 0.836
12 Restroom, drinking fountain 86.13 93.6 7.98% 0.626 0.906
13 The outdoor area well maintained 77.67 81 4.11% 0.708 0.696
14 Smoking areas well separated 72.53 93.6 22.5% 0.863 0.617
15 Microclimate control 66 88 25% 0.927 0.689
Walking and Outdoor Activities 88.46 99.3 10.91% 0.909 0.667
1 Abundant walkways of different lengths 97.84 106.2 7.87% 0.840 0.66
2 Round-trip walkways available 97.77 106.2 7.94% 0.686 0.651
3 Interesting views from walkways 92.67 102 9.15% 0.731 0.605
4 Paving level, easy for wheelchairs 100.33 106.2 5.53% 0.562 0.766
5 Paving non-skid and non-glare 100.7 106.2 5.18% 0.431 0.632
6 Walkways partly shaded 94.67 102 7.19% 0.451 0.739
7 Handrails along some walkways 88.44 104.8 15.61% 0.585 0.81
8 Frequent seating along walkways 83 102 18.63% 0.761 0.822
9 Some walkway seating in the shade 84.9 97.8 13.19% 1.000 0.914
10 Destinations to move towards 81.96 92.2 11.11% 0.610 0.604
11 Places for social activities 86.76 99.2 12.54% 0.909 0.944
12 Play areas for children 84.17 95 11.4% 0.459 0.866
13 Places for recreation and exercise 78.22 88 11.11% 0.818 0.6
14 The place for gardening, horticultural therapy 68.71 82.4 16.61% 0.775 0.681
Indoor-Outdoor Connection 88.24 101.72 13.25% 0.864 0.971
1 Outdoors visible from main indoor areas 93.71 111.8 16.18% 0.625 0.702
2 Easily reached from indoor commons 100.44 116 13.41% 0.565 0.996
3 Multiple ways to reach the outdoor area 87 102 14.71% 0.965 0.63
4 Indoor transition space near doorway 87.55 111.8 21.70% 0.708 0.998
5 Outdoor transition space near doorway 89.33 106.2 15.88% 0.708 0.862
6 Doors unlocked during daytime 91.67 102 10.13% 0.707 0.892
7 Doors open with minimal effort 86.7 97.8 11.35% 0.507 0.857
8 Doors do not close too quickly 87.9 97.8 10.12% 0.795 0.747
9 Automatic door available, easy to use - 92.2 - - -
10 Can easily cross door threshold 80.21 88 8.85% 0.665 0.888
11 Wide paved landing outside the doorway 77.94 83.8 6.99% 0.751 0.836
Connection to the World (6 items) 86.25 95 9.21% 0.932 0.584
1 The area is located near the main entry 91 102 10.78% 0.864 0.874
2 View of vehicles arriving at the facility 86.07 97.3 11.54% 0.954 0.72
3 Views of front-door activities 87.75 97.3 9.82% 0.547 0.851
4 Views of off-site scenery 84.11 92.7 9.27% 0.864 0.696
5 Views of nearby streets or traffic 86.44 92.7 6.75% 0.886 0.592
6 Views of off-site buildings, activities 81.78 88 7.07% 0.864 0.409

Note:

1)The Percent Difference reflects the difference between each item and the full score. Percent Difference = (Full score - Average score)/Full score.

2)Because there are few automatic doors in the entrance hall of residential buildings in China, especially the old buildings around Labor Square, the score of this item was very low, and the value of the t-test could not be tested, so “Automatic door available, easy to use” item was omitted.

The trainers completed the scales of 1 to 7 according to the SOS items and obtained a specific score for each item. As total scores in Table 5 were not in percentage points after weighting (from a low of 62.4 to a high of 116), the trainers calculated the percentage difference for each item:

Per cent Difference = (Full score - Average score)/Full score

Therefore, the larger the percentage, the lower the satisfaction with this item. On the contrary, a smaller percentage means that the item is rated better. As there was no automatic door in the survey area, the six raters selected the lowest score. As the scores were consistent, the T-test Significance (p-value) and the Mean Reliability (ICC) could not be calculated, hence this item was disregarded. The SOS tool was widely and reliably used, as indicated by the scores the raters gave in the 7-point test. Although the raters of different ages and genders revealed different scores, The T-test Significance (p-value), the Mean Reliability (ICC), and the average value for the overall score of the space reflected that the SOS tool was suitable for the elderly outdoor environment assessment.

Discussion

Generally, the most striking similarity across the six raters was their overall ratings for the environmental characteristics of the space. The most notable differences among the raters were in scoring specific items and ratings. There was no clear preference for male and female ratings in different age groups. Nevertheless, the older group scored slightly lower than the other two groups. This could be because of factors such as the decline in physical fitness of the elderly, who have relatively high environmental requirements.

Comparing ratings by domains and items

The scores in Table 4 illustrate the environmental performance of ABC spaces. Space A obtained low scores in “Access to Nature” (78.19) and “Outdoor Comfort and Safety” (74.53). As seen from the scene photo in Figure 4, this space has many hard floors, which are relatively weak. It proves that the SOS score was reliable. In Space B, the score of “Outdoor Comfort and Safety” (78.57) was the lowest. In Figure 5, there are more green plants in Space B than in Space A; however, comfort and safety were still weak. In Space C, the score of “Outdoor Comfort and Safety” (81.28) was also the lowest. Figure 6 and Figure 7 show that the environment of the whole space is relatively comfortable. The three spaces had high scores in “Walking and Outdoor Activities” (14 items) and their average score was above 80 points, indicating that the whole space is comfortable. Nevertheless, compared to the first two spaces, the score was higher.

Figure 4. Space A

Figure 5. Space B

The ratings in Table 5 were recorded for each item and reflected the assessment of the whole park. Among the five domains, “Outdoor Comfort and Safety” had the lowest average score of 78.18. “Walking and Outdoor Activities” scored higher (88.46), and the other three items were also above 80 points. Therefore, the amenity and safety of the whole park should continue to be improved to meet the needs of the elderly in the residential outdoor environment.

Access to nature

This domain assessed whether nature-related elements are superior to man-made hard landscape elements. The average rating for the domain was similar among the six raters, ranging from 80 to 84. There was a percentage difference between “Privacy from resident rooms” and “Abundance of greenery”, even though lower. This means that the green spaces outside the residential area have enough plants and better privacy. The percentage values of the difference between “Amenities for pets” and “Can see domesticated animals” were relatively higher, indicating that the outdoor environment is less considered when it comes to pets and poultry. Nowadays, more and more people keep pets, especially the elderly, as pets relieve their loneliness. Therefore, pet facilities or spaces should be moderately considered in outdoor environments. In the context of Chinese culture, poultry is relatively rare in cities, hence this fact can be overlooked in designing. The other scores varied between 10 and 20%, indicating that the proximity to nature is both good and satisfactory.

Figure 6. Space C (afternoon)

Figure 7. Space C (evening)

Outdoor comfort and safety

As a reasonably general feature, the safety and comfort items in the field had the lowest average score among the four domains, with six scoring below 20%. The percentage for “Some seating easily movable percentage” was 25.24%, “Some seats have cushions percentage” was 28%, and “Tables for coffee, food, etc.” was 26.33%. The percentage of “Swings, gliders, rocking chairs” was 24.09%, “Smoking areas well separated” was 22.5%, and “Microclimate Control” was 25%.

In this space, the outdoor configuration is fixed facilities, and no seat cushion is configured. Some mobile seats are carried mainly by users or configured by active service personnel as these mobile facilities or cushions are inconvenient to manage and set up. Other arrangements, such as a swing, table, and smoking area, can be added later for maintenance and construction after consulting with the relevant departments. Improving microclimate control is one of the ecological environment constructions that can be perfected by contacting professionals. Except for the well-maintained outdoor area which had a high percentage of 4.11%, the other projects obtained scores between 9% and 20%. Therefore, the comfort and safety of the area still require more remarkable improvements.

Walking and outdoor activities

The domain focused on providing destination, safety, and comfort along the sidewalks. It also received similar ratings from the six raters, ranging from 87 to 90, with a difference in percentage value of 10.91%. It was the highest rating of the four domains. Most of the 14 items were at least 20 per cent, and the lowest one, “Frequent seating along with walkways”, had a percentage value of 18.63%. The survey discovered fewer seats on the walkways. “Place for Gardening and Horticultural Therapy” received a low score of 16.61%. Many scholars have studied the health effects of gardening in recent years, and the benefits are self-evident. Thus, the SOS tool can improve the residential outdoor environment. In Labor Square, seats should be added for the elderly to sit down and rest any time while performing their activities. At the same time, a better space can enhance the gardening site for the elderly to participate in gardening activities and construction.

Indoor-Outdoor Connection

The domain focused on the transition between indoor and outdoor spaces. On the east side of Labor Square, adjacent to commercial shops and a residential area, there are some indoor and outdoor spaces directly connected. On the south side of Labor Square, across the road, is an old residential area with some spaces connected with the square through the road. In this domain, automatic doors are used less in Chinese residential areas and the surrounding areas are old, so this needs to be addressed. Among other projects, the percentage of a wide-paved landing outside the doorway was the most negligible (6.99%). Meanwhile, the percentage of the indoor transition space near the doorways was the most significant (21.7%). Other projects were controlled within 10-20%.

All the entrance halls of the community have a flat and comprehensive platform which is convenient for people to travel, hence the scoring rate was high. The interior transition space near the door has a small transition space in the east community. However, there is almost no transition space in the south because it is an old community that has been built for a long time. This phenomenon would be much better in the new community. The indoor and outdoor transition spaces affect the elderly's travel. If there are convenient, observable, and smooth spaces, the elderly will have more travel opportunities.

Connection to the world

The domain assessed the extent to which outdoor spaces are in contact with the larger world beyond the facility's walls by observing what aspects of the nearby community can be viewed. The domain showed the smallest percentage of 9.21% of the six raters, indicating that it was the lowest of the four domains. The percentages for “Area are located near the main entry” was 10.78% and “View of vehicles arriving at facilities” was 11.54%. Only these two items were higher than 10%. It also shows that the whole area is relatively well-connected with the outside world. From the floor plan, the residential building is relatively independent of the outside world and its connection.

Comparing reliability across study sites

Table 4 exhibits the scores of the three spaces in the environment. The average score reliability of the six raters was more significant than 0.8, indicating that the overall score reliability was reliable. In ABC spaces, only the scores for “Connection to the World” (6 items) were relatively low, between 0.4 and 0.6. The “Connection with the External Environment” score can be further discussed. Access to Nature (14 items), Outdoor Comfort and Safety (15 items), Walking and Outdoor Activities (14 items), and Indoor-Outdoor Connection (11 items) had reliability scores greater than 0.8, which means it is very trustworthy. Table 4 highlights the reliability of the scores for the three spaces.

Table 5 and Table 6 compares the reliability of each item regarding the SOS tool. In the first domain, the average reliability of the score was 0.942, of which most values were higher than 0.6. Only “Privacy from resident rooms” and “Outdoor area is not noisy” obtained scores between 0.5 and 0.6, showing significant variabilities. There might be some disputes about the understanding of noise. Some raters believed that too much music in the space is noisy, while others believed that only outdoor vehicles and other sounds are noisy. Besides that, using music in the space is not problematic. In the second domain, the reliability of the average score was 0.968, most of which were higher than 0.6. Only the reliability of “Some seats have cushions” was 0.459. The personal cushions might have influenced the raters' judgment. In the third domain, the reliability of the average score was 0.667. Although the reliability of the average score was lower than those of the first two domains, the reliability of each item was higher than 0.6. In the fourth domain, the reliability of the average score was 0.971, most of which were higher than 0.6. In the fifth domain, the average reliability score was 0.584, hence the lowest among the five domains. Table 4 also shows that the reliability of this domain was not high.

Following the aforementioned rating, several raters discussed the rating process for the fifth domain in an effort to identify weaker reliability issues. They highlighted a big issue that revolved around the five directions of Labor Square. The varied viewpoints had a significant impact on it. For example, there is a road south of Labor Square where the raters could see the buildings or traffic even though they are not entirely adjacent. Therefore, some raters thought that the outdoor environment has a certain distance from the nearby buildings. As a result, the scores were low. Some reviewers, on the contrary, thought that the outdoor environment is across the road but convenient; henceforth, their scores were high. In this study, the original data were retained. The reliability of this score was relatively low as a score, but higher than 0.4.

Although Table 4Table 5 and Table 6 show the test reliability from different perspectives, the results were consistent. From the discussions with the raters, it came out that they were similar in their assessments of the environmental characteristics and agreed that the area was good.

Reliability differences between trainers

The differences in reliability among several raters might be related to their age, gender, academic background, and usage history. For instance, both elderlies had a career in architecture or design before they retired. Nevertheless, they knew the environment very well as they used it most often, i.e., all year round, from morning to night. In contrast, the other two middle-aged raters were engaged in architecture or environmental design-related occupations. Hence, their experience and academic rating of the community environment were relatively professional.

Meanwhile, the two young college students from the same school were majoring in environmental design. They studied together and possessed professional knowledge. During the assessment, they were asked to imagine themselves as older people using outdoor environmental facilities. At the same time, the raters of different genders were specifically selected for the assessment.

These differences in context led to different perceptions of how outdoor environments support older adults. Nevertheless, in the end, there was a slight variation in their assessments. In the first domain, the T-test score was 0.909. Only the “Abundance of greenery” was 0.486, and the rest were higher than that. The difference might be due to the definition of "large amount". In the second domain, the T-test score was 0.732. The different degrees between “Seating has arms and backs” (0.489) and “Some seats have cushions” (0.454) were the lowest. Like the reasons for the lower reliability evaluation, they were lower in the domain. The T-test score of the third domain was 0.909. The different degrees of “Paving non-skid and non-glare” (0.431) and “Walkways partly shaded” (0.451) were the lowest. The difference in ratings for "Pave the road without glare" could be influenced by the sun intensity, cloud cover, and time of day. In “Walkways partly shaded”, the definition of "partly" might result in the differences. In the fourth domain, the T-test score was 0.864. In the fifth domain, the T-test score was 0.932, and the difference value of each item was higher. The whole p-values were significant.

Innovations

Compared to previous studies, this study possessed three innovations: first, most previous studies were limited to the outdoor space around the elderly building. In contrast, this study used SOS tools for residential outdoor environment assessments. Second, there is a need for scholars in China to use SOS tools for residential outdoor environmental assessments, especially in Qingdao. Third, more studies are needed to compare the scores among different trainers. This study particularly made a horizontal comparison of different genders, ages, and occupations.

The results showed that the SOS tool is suitable for assessing the elderly’s residential outdoor environment. It also applies to the Chinese background. However, the individual item could be more suitable such as the setting of automatic doors, which does not affect the effectiveness of the overall score. At the same time, the scoring results of this study also revealed that different genders and occupations have little impact on the use of SOS tools, but the scoring data demonstrated that the scores of the elderly were slightly lower than those of the other two age groups. This indicates that although people have different perceptions of some environmental characteristics, their overall feelings toward the environment are similar.

Conclusion

In general, the assessment results of different ages, genders, and users proved that the SOS tool effectively evaluated the used space of the elderly in the residential outdoor environment. The development of the SOS tool was based on the use of accepted reliability standards, the consistency of ratings across multiple items in different spaces, and the high degree of confidence in rater reliability. Given the substantial background differences among the six raters (age, gender, academic, usage, etc.), it is surprising that many of the items received similar ratings. With more specialised training, many non-specialists can use the SOS tool to evaluate the outdoor environment after a simple training. This is especially true for a published weighting and scoring method, which is much simpler and easier to use. In fact, the test reliability from different perspectives showed consistency. The input from the raters was consistent in how they evaluated the environmental characteristics, and the T-test revealed that the values were significant. It can be concluded that the outdoor areas are excellent and practical for the elderly. Planners and designers can use the SOS tool to learn what features to consider when updating an existing outdoor space or developing new facilities.

The study has several limitations. First, it merely involved one site. Second is the inference made regarding the lower scores given by the older raters, as mentioned in the discussion. Next is the difference in population density and urbanisation level of the study sites. Therefore, several research sites should be selected for data collection, analysis, and verification in future studies. The area was chosen for the present study because it is in a dense urban residential area with many activities for the elderly. Therefore, future research can be conducted in areas with different population densities and several facilities, such as the rural environment.

Along with the original English version, the tool is now available online at www.accesstona.ture.org in Italian, Japanese, Chinese, and Spanish. Nevertheless, the assessment of the suitability for the residential outdoor environments can be further adjusted. For example, the indoor-related content can be reduced and more outdoor activity facilities for the elderly can also be added. This is to help more outdoor environments improve the quality of outdoor spaces by making them more suitable for the elderly.

Author Contributions

Introduction and conceptualisation, L.J. and A.M.; methodology, L.J.; software, L.J. and A.M.; investigation, L.J.; data curation, L.J.; writing—original draft preparation, L.J. and A.M.; writing—review and editing, L.J. and A.M.; supervision, A.M. All authors have read and agreed to the published version of the manuscript.

Ethics Declaration

The authors declare that they have no conflicts of interest regarding the publication of the paper.

Acknowledgments

The authors would like to thank the Management and Authority of Labor Square for permitting the involvement of the observation rater team, who greatly assisted in the research and sharing of information on elderly activities along the Labor Square.

References
 
© SPSD Press.

This article is licensed under a Creative Commons [Attribution-NonCommercial-NoDerivatives 4.0 International] license.
https://creativecommons.org/licenses/by-nc-nd/4.0/
feedback
Top