1 A Review of Sensors for Vacant Parking Spot Detection John Smith Faculty of Electrical, Computer and Software Engineering McGill University, SUMCOURS206 June 25, 2019 Abstract : Vehicles searching for parking can amount to 30% of traffic in certain areas, exacerbating congestion in urban environments among other environmental and productive detriments. For this reason, it is enticing for cities to develop smart parking systems that provide drivers with real-time vacant parking spot options. The first step in developing a smart parking system is the choice of sensors that implement the occupancy detection. This study reviews the state-of-the-art literature on using magnetometers, ultrasonic sensors and Computer Vision cameras as parking spot occupancy sensors, and evaluates their applicability to monitor the entire indoor and outdoor parking stock of a given city. We compare the feasibility of deploying each sensor type on a city-wide scale, based on the proposed criteria of accuracy, scalability, and efficiency. After careful analysis, this study suggests that presently the most feasible sensor choice is a combination of ultrasonic sensors to monitor indoor parking spots, and Computer Vision cameras to monitor outdoor parking spots. Acting in conjunction, these two sensor types enable cities to develop accurate, scalable, and efficient smart parking systems, minimizing future implementation costs while maximizing usage of existing infrastructure. 2 1 Introduction Drivers heuristically searching for parking amount to 30% of city traffic [1] [2], exacerbating congestion and carbon emissions in urban environments. This inefficient parking system wastes commuters’ time, results in lost productive and economic activity, and hinders city services. It is enticing for cities to develop smart parking systems providing drivers with real-time vacant parking spot options. The first step in developing a smart parking system is the choice of sensors implementing the occupancy detection. A typical city’s parking stock comprises indoor multi- story facilities, and outdoor street-side parking, complicating the approach. Much of the research conducted considers only usage of a unique sensor in a specific parking environment. Other papers survey various sensors, but offer no sensor comparison to propose feasible, city-wide sensor choices. The contributions of this study aim to bridge this gap. The functioning and choice of magnetometers, ultrasonic sensors and Computer Vision cameras as preferred sensors for parking occupancy detection is first explained. The criteria of accuracy, scalability and efficiency serve as basis for building a comprehensive comparison of the chosen sensors. This study suggests that presently the most feasible sensor choice combines ultrasonic sensors to monitor indoor parking spots, and Computer Vision cameras to monitor outdoor parking spots. 1.1 Considerations 1.1.1 Scope A centralized system to monitor the stock of parking spots implies sensors able to detect occupancy of each parking spot, and a communication protocol for the sensors to continuously relay the status of the parking spots to a centralized system. Although there are multiple sensor approaches, in practice, the abstract communication protocol is sensor invariant. Assuming each sensor can broadcast its data, there are multiple ways to transmit the data from every sensor into a pipeline that continuously outputs the status of all parking spots. For instance, recent works [3], [4] have successfully implemented street parking systems using wireless sensor networks of magnetometers. We will only conduct a feasibility study of the sensors, assuming later implementation of communication methods between the sensor and an abstract system that would exploit the data and disseminate the results. Moreover, this paper focuses on granular sensor networks – able to report the occupancy status of every parking spot monitored, not simply the occupancy rate of a collection of parking spots, as is already implemented in many parking lots by measuring vehicle flow at input and output gates. The sensor network must also be vehicle independent. Rather than use sensors in each vehicle detecting parking in its surroundings, the sensor is stationary, installed to monitor some parking spot(s) (depending on the sensor) and integrate into a network that communicates relevant data to drivers. A proposed method using “only those sensors already mounted on off -the- shelf vehicles” achieved “97.64 % precision” [5]. However, this method only allows cars to detect parking in their vicinity, which has marginal benefits over manually searching for parking. 3 1.1.2 Limitations Currently, all methods of detecting parking space occupancy require delimited spots. Although the delimitations have no effect on sensors, they ensure consistent placement of the vehicles. Since proximity sensors sample only a small region of each parking space, without delimitations, they could report false positives if vehicles occupied most of a parking spot except the specific area monitored by the sensor. Cameras likewise suffer in accuracy when the vehicles are inconsistently placed. The first operator step in most vision-based approaches is manually marking “t he limits of each park ing spot” [6] in the captured footage. If the parking spaces are not delimited, vision approaches are inapplicable since they are unable to estimate distances between detected vehicles. 2 Background Parking occupancy sensors are categorized into two classes: intrusive and non-intrusive sensors [7], presented in Sections 2.2 and 2.3 respectively. 2.2 Intrusive sensors Intrusive sensors are installed underneath road surfaces to monitor a single parking spot. We will study magnetoresistive sensors. Bes ides being “widely reviewed” [8], they are commercially available, developed by VehicleSense [9], and the sensor basis for multiple smart parking applications (AppyParking, ParkiFi [8]). 2.2.1 Magnetometer As seen in Fig. 1, magnetoresistive sensors detect the presence of vehicles by “measuring the change in the earth's magnetic field caused by the presence ” [10] of “ferromagnetic objects”, especially “massive components like engine, gearbox” [11]. The sensors are “placed under every parking space to know the occupancy ” [8]. They are placed beneath the road surface since they must be in “ close proximity to the vehicle ” [8]. Fig. 1 [11] . Simulation of the deformation of Earth’s magnetic field by a car 4 2.3 Non-Intrusive sensors Non-intrusive sensors are defined by their ease of installation. Of this sensor class, we will study ultrasonic and camera approaches. Ultrasonic sensors are chosen since they are commercially developed by industrial giants like ENCTECH, IDENTIPARK and SIEMENS [3], and employed for vehicle detection in the Baltimore-Washington Airport in the U.S. and Blagnac Airport in France [12]. Considerable research is also conducted on camera approaches due to advances in Computer Vision, and the expansion of Closed-circuit television (CCTV) networks under municipal control, which is why they are the third sensor solution studied. This study focuses on a vision-based solution that can be deployed on existing CCTV networks as implemented in [6]. 2.3.1 Ultrasonic sensor Ultrasonic sensors emit “ sound waves between 25 and 50 kHz ” [13] and detect objects based on energy reflected. They are “ mounted on the ceiling ” [8], above each parking spot [11]. The ultrasonic sensor estimates distances by measuring the time difference as ultrasonic waves reflect off objects. Zhou et al. use the time 𝑡 elapsed between ultrasonic pulses, and velocity 𝑐 of ultrasound propagation in air, to estimate the distance 𝐻 from sensor to target object [4] using (1): 𝐻 = 𝑐𝑡 2 The sensor determines base distance to the ground. When a car stops underneath the sensor, 𝐻 changes, and the sensor infers the spot became occupied. 2.3.2 Camera This sensor type encompasses the camera, microprocessor and software for video processing. A stationary camera, “installed overhead” [8] on street fixtures or buildings monitors the status of multiple parking spots within its field of view [14]. With increasing research being conducted, a “ vast variety of image processing algorithms ” [9], each leveraging various Computer Vision techniques, exploit camera footage for vehicle detection. Some vision-based approaches use “ segmentation ” [8] [9] to discern all vehicles each frame, w hile others rely on “temporal hysteresis” to ignore stationary cars as background and track moving vehicles to “toggl[e] the status of a parking spot ” [6]. The work in [6] introduces vacant parking spot detector exploiting the footage from existing, widely-deployed city CCTV networks, which is why it is chosen as the primary camera-based solution of this study. As shown in Fig. 2, this implementation is designed as “ background subtraction+object tracking+visual recognition pipeline ”. (1) 5 Fig. 2. [6] Flowchart of the motion detection with background subtraction algorithm 2.4 Criteria for comparison 2.4.1 Accuracy The solution that implements occupancy detection must be accurate. If the sensors read false positives, the benefits of knowing the location of available parking spots could be offset by having drivers navigate to actually occupied spots. Additionally, frequent false negatives are just as detrimental, further limiting parking options for drivers making the system unreliable. The vehicle detection accuracy of magnetometers is based on the experimental findings obtained from a large sample size over “six months” in [15]. For ultrasonic sensors, the test results from [16] were used, offering sufficiently average and realistic test conditions. For vision-based sensors, we employ the metric defined in [6]. This metric compares the status of each parking spot predicted by the vision model against the corresponding ground truth, which is the actual status of each parking spot as labelled in the testing footage. The status detection is correct if both status match in a given fra me. “ This idea is extrapolated to a certain span of frames, [yielding] the Average Detection Accuracy (ADA)” expressed in (2). 6 ADA = 1 K ∑ ∑ 𝑚𝑎𝑡𝑐ℎ(𝑉𝐵 𝑠,𝑓 , 𝐺𝑇 𝑠,𝑓 ) 𝑀 𝑀 𝑓=1 𝐾 𝑠=1 “where K is the number of parking spots in the parking site, M is the number of frames under analysis, [ 𝑉𝐵 𝑠,𝑓 ] is the status of spot 𝑠 in frame 𝑓 output by [the vision-based approach], 𝐺𝑇 𝑠,𝑓 is the status of spot 𝑠 in frame 𝑓 annotated in the ground truth, and 𝑚𝑎𝑡𝑐ℎ(𝑎, 𝑏) is a function that equals 1 if 𝑎 equals 𝑏 , and 0 otherwise” [6]. 2.4.2 Scalability With the advent of smart cities, this paper aims to propose a solution that is enticing to cities. To do this, it must be easily scalable; the sensors must be appropriate for deployment at every scale, from isolated parking lots to entire cities. Varying the scope of the sensors should not incur new challenges or considerations. This imposes that the sensor must be viable in multiple environments. Additionally, the scalability criterion comprises the time-to-operation metric, which indicates the delay from when the sensor is installed (in the case of vision-based sensors, from when the firmware is installed on existing CCTV sensors) to when the system begins operating. Finally, installation and maintenance, in cost and labor, are also considered. 2.4.3 Efficiency Efficiency is defined as the number of vehicles monitored by a single sensor, measured in vehicle(s)/sensor. The benefits of having an efficient sensor networks are self-evident, and make the smart parking solution enticing to cities. Higher efficiency means fewer sensors required to monitor the same number of parking spots, lowering purchase, installation, and maintenance costs. As with any network, fewer nodes also mean less troubleshooting errors, preventive redundancies, planning to optimize coverage, and communication infrastructure. 3 Analysis 3.1 Accuracy comparison To determine accuracy of the magnetometers, 82 magnetic nodes were deployed; after 6 months, the “accurac y rate [was] better than 98%” [15]. However, relatively high static magnetic fields, due to “ferromagnetic components inside the building” , can interfere with the sensors [11]. Magnetometers are unresponsive to pedestrians in the parking spot, preventing false negatives. Two ultrasonic sensor models (PING, HC-SR04) were tested in [16] on 89 vehicles. We interpret the findings of the test scenario shown in Fig. 3. (2) 7 Fig. 3. [16] Sensor location scenario 1 Table I tests the occupancy predictions of both ultrasonic sensors against the ground truth. TABLE I [16] Test results for ultrasonic sensor models in Test Scenario 1 at various distances above the vehicle If installed outdoors, they are sensitive to “environmental conditions” [8]. Likewise, ultrasonic sensors can distinguish between a vehicle and a pedestrian [8]. The ADA of the vision-based implementation by Marmol et al., was 0.988 [6]. A comparison of recent video-based parking detectors is presented in Table II. 8 TABLE II [6] Performance comparison between state-of-the-art techniques in vacant parking spot detection Overall, half the techniques report an ADA above 0.98, and all are above 0.9. The factors that lowered the ADA value were mostly “temporal misalignment”, wherein the “system detects a change in parking spot status a few frames earlier or later than annotated in the ground truth” [6], which poses no operational drawback. Another more critical source of detection errors was “the presence of severe shadows ... when a dark car is present” [6]; these false negatives hinder actual operation. Generally, all vision- based approaches have the drawback of “challenges with low- lighting conditions” [8]. The system in [6] is rob ust enough to determine “whether what has entered a particular parking spot is a car or a pedestrian”. 9 3.2 Scalability comparison Magnetometers are “expensive to install” [8] underground. They have a “battery life of a few years” and must be replaced when depleted, thus “maintenance costs [are] high” [8]. They are not sensitive to the environment [12]. Recent work used a “tri -axial magnetic sensor, located on the pavement” [17] rather than under. This does prove problematic as it is more susceptible to environmental and incidental wear. Magnetoresistive sensors are appropriate for parking spaces with or without ceilings [8] [11], improving scalability. They have negligible time-to-operation. Ultrasonic sensors have a low cost [8] and easy installation [13], but non-negligible maintenance when the deployment scale is large [8]. They “can only be used in [indoor] multi- story car parks”, requiring “a ceiling above” [11]. They are sensitive to “environmental conditions” [8] such as rain and snow [13]. These are significant drawbacks; ultrasonic sensors are not viable for monitoring the outdoor parking stock of a city. The vision-based approach in [6] uses existing CCTV cameras as sensors, thus requiring no additional installation or maintenance costs, making it easily scalable. Cameras are however inapt in indoor “facilities with a low ceiling”, since Idris et al. suggest the “camera mounting height ... required for optimum presence detection [is] about 50 to 60"” [9]. Cameras are more viable for outdoor use in private or public parking areas. Indeed, research by McCahill et al. determined that 41% of (primarily private) premises in London had CCTV systems, with an average of 4.1 cameras per system [18]. Additionally, the city of Beijing has a public CCTV network that covers “every corner” of the city [19], with the total amount of cameras near 470,000. One can reasonably expect that most of the outdoor parking spots (and the indoor parking spots with adequately high ceiling) of cities with sufficient CCTV coverage can be monitored by existing camera infrastructure. The implementation in [6] pre-emptively trains its recognition model with external footage database before deployment, lowering time-to-operation. 3.3 Efficiency comparison Magnetometers and ultrasonic sensors must be placed at “every parking space” [8] to detect vehicles, giving both an efficiency score of 1 vehicle/sensor. This is the tradeoff of proximity sensors; despite consistently high accuracy, low efficiency constrains its practical applicability. The factor limiting the efficiency of the camera sensor is not the visual recognition software, but rather the positioning of the camera dictating which parking spots are visible in the captured footage. The optimal height mentioned in Section 3.2 is likely not achieved by most existing cameras. Since the vision-based approach studied [6] is designed as a firmware solution installed on existing CCTV hardware, the positioning of each camera is predefined and cannot be modified to achieve greater sensor efficiency. Environmental factors specific to the parking lot under surveillance might also reduce efficiency as occlusion [7] [20] can prevent certain spots from being monitored. Despite sub-optimal mounting of some of the existing cameras, the efficiency of cameras in vision-based approaches remains order of magnitude greater than aforementioned proximity sensors. Marmol et al. used footage captured from a parking lot with 41 spaces to test their implementation [6]. A survey of the footage from other vision-based implementations suggests an efficiency range of 10-65 vehicles/sensor. 10 4 Conclusion All studied sensors offer high accuracy. Their respective drawbacks are shown in Table III. Although magnetometers are viable for indoor and outdoor parking spots, they are too expensive to feasibly deploy on a city-wide scale. Ultrasonic sensors are significantly less expensive, but are constrained to indoor deployment. The vision based-approach in [6] has no additional scalability costs. Although it is not viable for indoor parking facilities with low ceilings , it can feasibly monitor a city’s outdoor parking stock by making use of the extensive existing CCTV networks of many cities. The time-to-operation of all three types of sensors is too minimal to realistically delay operation. As seen in Table III, both proximity sensors have efficiency score of 1 vehicle/sensor, whereas cameras have significantly higher efficiency of 10-65 vehicles/sensor. The bottleneck for the camera efficiency is the number of parking spots typically visible in CCTV footage, not the visual recognition software. TABLE III Summary on accuracy, scalability and efficiency findings for each type sensor type studied The ultrasonic sensor is clearly more scalable than the magnetometer, with equivalent accuracy and efficiency, however it is constrained to indoor application. Furthermore, the advantages of the camera significantly outweighs its trivial drawbacks. It is significantly more efficient and scalable than its proximity counterparts, with comparable accuracy. It is however constrained to outdoor application. As such, this study suggests that the most feasible city-wide parking monitoring solution would leverage two distinct sensor types to account for differing parking environments. For indoor parking facilities, ultrasonic sensors are best suited. For outdoor parking lots and street-side parking, vision-based approaches exploiting existing city CCTV cameras are best suited. 5 Works Cited [1] IBM Corporation. (2011, June 25, 2019). IBM Global Parking Survey: Drivers Share Worldwide Parking Woes (IBM News room ed.) [Online]. Available: https://www- 03.ibm.com/press/us/en/pressrelease/35515.wss [2] D. Shoup, "Free parking or free markets," 2011. [3] A. Ahmed, M. Olaniyi, J. Kolo, and S. Oyetoro, "Developing smart car parking system using wireless sensor networks," in 2nd International Conference on Computing Research and Innovations, CoRI 2016, September 7, 2016 - September 9, 2016 , Ibadan, Nigeria, 2016, vol. 1755, pp. 201-206: CEUR-WS. 11 [4] H. Zhou and Z. Li, "An intelligent parking management system based on RS485 and RFID," in 8th International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery, CyberC 2016, October 13, 2016 - October 15, 2016 , Chengdu, China, 2016, pp. 355-359: Institute of Electrical and Electronics Engineers Inc. [5] S. Jae Kyu and J. Ho Gi, "A Universal Vacant Parking Slot Recognition System Using Sensors Mounted on Off-the-Shelf Vehicles," Sensors, vol. 18, no. 4, p. 1213 (21 pp.), 04/ 2018. [6] E. Marmol and X. Sevillano, "QuickSpot: a video analytics solution for on-street vacant parking spot detection," Multimedia Tools and Applications, vol. 75, no. 24, pp. 17711-43, 12/ 2016. [7] L. E. Y. Mimbela, "Summary of vehicle detection and surveillance technologies used in intelligent transportation systems," L. A. Klein, Ed., ed. [Washington, DC] :: Federal Highway Administration, Intelligent Transportation Systems Joint Program Office, 2000. [8] V. Paidi, H. Fleyeh, J. Hakansson, and R. G. Nyberg, "Smart parking sensors, technologies and applications for open parking lots: A review," IET Intelligent Transport Systems, vol. 12, no. 8, pp. 735-741, 2018. [9] M. Y. I. Idris, Y. Y. Leng, E. M. Tamil, N. M. Noor, and Z. Razak, "Car park system: A review of smart parking system and its technology," Information Technology Journal, vol. 8, no. 2, pp. 101-113, 2009. [10] Z. Zhang, X. Li, H. Yuan, and F. Yu, "A street parking system using wireless sensor networks," International Journal of Distributed Sensor Networks, vol. 2013, 2013. [11] J. Wolff, T. Heuer, G. Haibin, M. Weinmann, S. Voit, and U. Hartmann, "Parking monitor system based on magnetic field sensors," in 2006 IEEE Intelligent Transportation Systems Conference, 17-20 Sept. 2006 , Piscataway, NJ, USA, 2006, p. 5 pp.: IEEE. [12] C. J. P. D. Rodier and S. A. P. D. Shaheen, "Transit-based smart parking: Early field test results," in 12th World Congress on Intelligent Transport Systems 2005, November 6, 2005 - November 10, 2005 , San Francisco, CA, United states, 2009, vol. 6, pp. 3734-3742: Curran Associates Inc. [13] A. Kianpisheh, "Smart Parking System (SPS) Architecture Using Ultrasonic Detector," 2012. [14] H. Ichihashi, A. Notsu, K. Honda, T. Katada, and M. Fujiyoshi, "Vacant parking space detector for outdoor parking lot by using surveillance camera and FCM classifier," in 2009 IEEE International Conference on Fuzzy Systems , 2009, pp. 127-134: IEEE. [15] Z. Zusheng, T. Ming, and Y. Huaqiang, "A parking occupancy detection algorithm based on AMR sensor," IEEE Sensors Journal, vol. 15, no. 2, pp. 1261-9, 02/ 2015. [16] G. A. Mutiara, A. A. G. Agung, and R. Handayani, "Sensor comparation for smart parking system," in 2015 1st International Conference on Wireless and Telematics (ICWT), 17-18 Nov. 2015 , Piscataway, NJ, USA, 2015, p. 6 pp.: IEEE. [17] C. Trigona et al. , "Implementation and characterization of a smart parking system based on 3-axis magnetic sensors," in 2016 IEEE International Instrumentation and Measurement Technology Conference, I2MTC 2016, May 23, 2016 - May 26, 2016 , Taipei, Taiwan, 2016, vol. 2016-July, p. IEEE; IEEE Instrumentation and Measurement Society: Institute of Electrical and Electronics Engineers Inc. [18] M. McCahill and C. Norris, "CCTV in London," 2002. [19] V. Zeng. (2015, June 5, 2019). ‘Every corner’ of Beijing covered by surveillance cameras, state media proudly announce (Hong Kong Free Press ed.) [Online]. Available: https://www.hongkongfp.com/2015/10/05/every-corner-of-beijing-covered-by-surveillance- cameras-state-media-proudly-announce/ [20] K. C. Mouskos, M. Boilé, and N. A. Parker, "Technical solutions to overcrowded park and ride facilities," 2007. 12 (2590 words excluding Title Page, Tables, Figures, Equations, and Works Cited)