I-CVSSDM: IoT Enabled Computer Vision Safety System for Disaster Management

INTRODUCTION: Around the world, individuals experience flooding more frequently than any other natural calamity. OBJECTIVES: The motivation behind this research is to provide an Internet of Things (IoT)-based early warning assistive system to enable monitoring of water logging levels in flood-affected areas. Further, the SSD-MobiNET V2 model is used in the developed system to detect and classify the objects that prevail in the flood zone. METHODS: The developed research is validated in a real-time scenario. To enable this, a customized embedded module is designed and developed using the Raspberry Pi 4 model B processor. The module uses (i) a pi-camera to capture the objects and (ii) an ultrasonic sensor to measure the water level in the flood area. RESULTS: The measured data and detected objects are periodically ported to the cloud and stored in the cloud database to enable remote monitoring and further processing. CONCLUSION: Also, whenever the level of waterlogged exceeds the threshold, an alert is sent to the concerned authorities in the form of an SMS, a phone call, or an email.


Introduction
The subways present in urban areas with a rapid rise in population density, frequently experience extreme water logging during heavy rains.This may be due to the absence or lack of proper maintenance of storm water management systems or drainage systems.Problems related to flooding vary depending on their types and severity.Inland floods are caused by persistent rainfall and predicted slow-moving tropical cyclones [1].While Flash floods are mainly caused by (i) heavy rainfall and (ii) sudden overflows of water from dams and reservoirs, The severity caused by this is unpredictable.Similarly, river floods at severe stages are often life-threatening, causing complete flooding in lowlying areas and devastation of vegetation.Preventive measures are necessary to reduce uncertainties.
This research work focuses on providing a solution that can monitor water logging in subways in urban environments.Several studies have been carried out to mitigate the above issues.Most of the work involves the collection of data related to water levels logged in the flood area and sending alerts through mobile networks.Certain works under disaster management have adopted the use of technology in the evacuation process of people to shelters.The limitations observed from the above works are: (a) the P. Ramesh et al.
2 non-availability of communication networks for people in emergencies due to the natural calamities caused at the time of the disaster; (b) Network congestion due to a sudden rise in traffic (c) Lack of metadata, namely location, objects to be rescued, and severity of the situation related to the flood Hence, the existing water level monitoring system for disaster management systems needs an alternative approach to enhance its efficiency.The contribution claimed in this research is an enhancement in sensing accuracy achieved through the usage of cameras and ultrasonic sensors to detect water logging levels in flood zone areas.
The fundamental requirements to be met during floods are timely alerts indicating the precise level of the flood to both the public and disaster rescue teams.The metadata related to the public in life-threatening situations, indicating their presence and location, needs to be communicated to the rescue teams in order to save people in time.Even though floods account for a large number of disasters and cause damage to all kinds of life and sectors, an early warning system that focuses on prevention is the need of the hour.In this research, a low-power early warning system and computer vision system enabled with the IoT for flood monitoring are proposed.The data collection mechanism adopted in the proposed research can provide statistical information that can be used in the prediction of floods and enable subsequent preventive measures.
The Objective of the developed system is to alert the public about the increasing flood level.The system helps to monitor and detect objects, namely people, animals, and vehicles, prevailing in the flood zone.It sends the metadata about the disaster along with the location information to the disaster rescue teams via cloud, phone call, SMS, and email.The object detection algorithm is used to provide a vivid picture to end users.Convolutional Neural Networks (CNNs) have developed as a potent machine learning model in recent years [2].A deep learning model provides attainable weights as well as biases to items in a visual that enhance classification accuracy.The convolution operation is used to extract high-level characteristics, namely edges, from the considered input images.In addition, compared to previous image classification methods, they employ a simple pre-processing technique to improve the kernel.The pooling mechanism addresses the spatial interactions between different features; CNNs are more suitable for data processing related to images.
The developed system consists of a Raspberry Pi microcontroller, an ultrasonic sensor, a pi-camera, and a Wi-Fi module.It uses an ultrasonic sensor for detecting water level, pi-cameras to capture field images, and a Wi-Fi module for data communication.The Single Shot Detector (SSD) MobileNet-v2 model [3] is used to detect objects of interest from the captured image.This model is trained with Common Objects in Context (COCO) datasets.The model is converted into a Tensor Flow Lite model to reduce the computational complexity, thereby making it suitable to run on a Raspberry Pi board.IoT enablement is provided to the developed application using Adafruit Cloud, Push Bullet, and Twilio (the user interface).A prototype is created in a laboratory environment to validate the developed application.
Following this structure, the rest of the article will discuss: The literature review and new developments that address the gaps in the prior art are presented in Section II.The formulated approach is discussed in Section III.Findings from the implemented system are discussed in Section IV.At last, Section V concludes the paper with a futuristic perspective on the developed work.

Literature Survey
Xiaolong et al. [4] developed CLOTHO, a large-scale IoTbased disaster management cloud, to provide a solution for public evacuation from the affected area.The authors have designed a cloud computing platform with IoT using Artificial Potential Field (APF) to determine the evacuation direction of the public, thereby reducing the panic and chaos involved during evacuation.Data acquisition, cloud service, network transmission, and user access are the four key components of this system.The advantages of this method include a reduction in evacuation route length, time, and convergence rate.Through early prediction of disasters, it saves lives.The limitation of this method is the cause of network congestion due to a sudden surge in traffic intensity, which may affect the QoS.
In [5], the authors designed a continuous monitoring system to recognize swimming activities and provide early drowning alerts.The work has focused on two fundamental aspects: (i) Foreground silhouette extraction and (ii) behavioural recognition.The designed system consists of overhead cameras installed with overlapping coverage all around the swimming pool.This keeps the entire area under coverage, ensuring that no information is lost.The image processing algorithm adopted in the work has the capability to detect objects vividly even under occluded environmental conditions.
In [6], the authors created a novel automatic background removal algorithm for urban surveillance systems.To detect the presence of objects in the captured image, a novel grating filter is used.The original image is converted to a grey-scale image, and then the overall gradient of each pixel is computed.The significance of the work is the automatic recognition of both moving and non-moving objects.However, the approach created fails to support lowresolution images.
In [7], the authors developed a sensor gadget to monitor urban flash floods and traffic congestion simultaneously.The gadget uses a combination of ultrasonic range finders and passive infrared temperature sensors to measure rainfall, water presence, and water level with a high degree of precision.The use of non-linear regressions and fuzzy-based decision tree models enhances prediction accuracy.However, the limitation observed is that the training phase is time-consuming as the processor used in the work has low processing capability.
Lan et al. [8] explored the significance of FeatherCNN on ARM architecture.The authors have used Tensor-General Matrix Multiply (GEMM), which is a high-performance linear algebra primitive that supports efficient SIMD (single instruction, multiple data) processing and memory movement minimization, to speed up the processing of Convolutional layers.Further, to enhance the performance, advanced memory blocking and packing technologies are used.
Suppakhun et al. [9] developed an alert system to prevent casualties during floods.The system is created to assess and communicate information and notifications by installing sensors in flood-prone locations.In addition [10] to the data collected from these sensors, it also receives critical data from the Meteorological Department and the Royal Irrigation Department for further processing.The limitation observed in the system is the connectivity issue faced by the Wi-Fi module.
In [11], the authors investigated the functionalities of Mobile networks in the applications of object detection, fine grain classification, and large-scale geo-localization.RMSprop and asynchronous gradient descent, Mobile Nets, were employed in the training of object detection on COCO data models in Tensor Flow, similar to Inception v3.As tiny models have a lesser problem with overfitting than bigger models, less regularization and data augmentation strategies were applied.The performance of these models was compared in terms of scalability, computational ability, and accuracy.Further, the selection of hyper-parameters suiting the application problem constraints was identified.
In [12], the authors provide an efficient approach for spectrum normalization for depth-wise separable convolutions with low computational and memory requirements.Image categorization using MobileNetV2 is also shown to be successful.The specific structure of depthwise separable convolutions increases the computational time when compared to other methods without adding additional memory overhead.In a realistic training scenario, each epoch takes only 2% additional training time.
From the above literature survey, it is inferred that the limitations of a real-time monitoring system designed for flood management include traffic congestion, ineffective measurement of flood level, computational capabilities, and issues related to connectivity.In this research, an attempt has been made to address issues pertaining to flood level measurement.

Proposed I-CVSSDM
To address the above limitation, an IoT-enabled computer vision safety system for Disaster Management (I-CVSSDM) is developed in this research.This system uses ultrasonic sensors to measure the water level in the flood zone, along with a camera module.The approach adopted in water level measurement is more accurate and reliable when compared to the traditional image processing approach mentioned in the literature.Further, it is a low-power, cost-effective solution with ease of installation that is also maintenancefree.The use of cameras for detecting the objects present in the flood area and the early warning provided to the concerned disaster management team reduce casualties.The developed I-CVSSDM system is detailed in this section.

System Model and Architecture
Figure 1 shows the system architecture of the developed I-CVSSDM system.Two sensors are used to obtain data from the flooded area: (a) a waterproof ultrasonic sensor module (JSN-SR04t) [13] and (b) a pi-camera with a resolution of 5 Megapixels (MP) [14].The water level data from JSN-SR04t and the output of object detection are analysed to monitor the flooded area.
Let WC represent the capacity of the waterlogging area.Let WL represent the level of water logging that has to be monitored by the developed system.This parameter has three different conditions, which are quantified by the parameter Ci.Let C1 represent the minimum allowable limit of water logging.C2 represents the safety threshold level of water logging at which the object detection module present in the developed system initiated Let C3 be the level indicating the criticality of water logging where the necessary alert generation is initiated.Let SR be the sensing range of the ultrasonic sensor used to detect the water logging.The accuracy of the object detection depends on the object acquisition model and the object detection module.As mentioned earlier, a camera with specifications Cs and a resolution of IR is used.SSD The MobiNET V2 object detection algorithm is used for object recognition and classification.Further, two sets of metrics are used to analyse the performance of the I-CVSSDM system.
Mean Average precision (γ) is the metric used to determine the accuracy of the object detection module, and (τ) is the time taken to detect the object in the flooded area.The developed work involves remote monitoring, and the delay time (δ) involved in delivery of the detected information is used as a metric to evaluate the developed system under different conditions.
Figure 2 illustrates the modules involved in the development of the I-CVSSDM system.The system consists of three parts: Data Acquisition, Data processing, and Event Communication.These parts are detailed in the following sections:

Data Acquisition
Real-time detection of flash floods is difficult because the sensors used in these applications must have a long lifetime in order to measure water levels in all flow conditions.They must also be able to self-monitor.Hence, in this research, JSN-SR04t, a waterproof ultrasonic sensor module, is used to measure the water level.It can provide a 25-450 cm noncontact distance sensing function with a ranging accuracy of up to 2mm.The module includes a 2.5-metre wire-encased waterproof probe that is suitable for wet and harsh measurement environments, attached to a transceiver at the end.
The module contains in-built circuitry that calculates the time taken for the ultrasonic wave to return and holds the echo pin high for that amount of time.The time interval between sound wave transmission and reception is recorded, and the distance between the sensor and the water surface is calculated using the following formula: D = S*(t/2) (1) D is the sensor-to-water surface distance, S is the sound wave speed, and t is the duration in seconds.The time is split in half to show round-trip time.
The camera used to monitor the objects prevailing in the flood is a Pi camera [14], which is portable and lightweight with low power usage.The camera has a resolution of 5 MP and 1080p (30 fps), 720p (60 fps), and 960p (45 fps) video recording capabilities.It can be used for real-time surveillance applications whenever a smaller payload is demanded.The Pi Camera can be used to capture images as well as videos.It is connected directly to the Raspberry Pi Board using a 2-lane MIPI Camera Serial Interface port interface.The Pi Camera Board [15] is tiny, with a dimension of 25mm x 20mm x 9mm and weighing 3.5g.The image and video quality are superior to those of the USB webcam.
When the water level in a flood-prone area increases, the distance of the water level is computed from the data provided by the sensor module.The level of water is calculated by subtracting the distance from the ground level.The calculated water level is constantly updated in the Adafruit IO cloud for further monitoring.

Cloud Enablement Unit
Raspberry Pi 4 model B is a small, credit card-sized, powerful computer board that functions similar to a personal computer plugged in with a monitor, keyboard, mouse, and a power supply.It is efficient and affordable.This module is used to process the collected sensor information and send the processed data to the User Interface applications.Data processing has two modules: (a) the water level monitoring module and (b) the object detection module.The distance to the water surface is obtained from the ultrasonic sensor module.The water level is computed by subtracting this distance data from the ground level.The computed water level is continuously monitored and periodically uploaded to the Adafruit_IO cloud from the Raspberry Pi using the Wi-Fi communication module.

Water Level Monitoring Module
The developed system workflow is depicted in the Figure 3 and it would be monitoring when the water level rises above C2.The developed system makes early warning alerts when object detection is found beyond the C3.Further, the system stops monitoring whenever it is interrupted or activated.

Object Detection Module
Object detection is a computer vision technique that recognizes, classifies, and locates instances of previously trained objects in an image or video.It entails enclosing these objects in a tight bounding box and associating the appropriate object category with each bounding box.In order to produce meaningful results, object detection algorithms employ machine learning or deep learning techniques.The algorithm assigns a confidence level, or the probability that a particular object belongs to a particular category, to every object detected.Only objects with high confidence are bound and labelled.
The process flow for the object detection method employed by the developed I-CVSDDM system is depicted in Figure 4.The Pi camera, once turned ON, starts streaming video with a resolution of 640 x 480 and a frame rate of 30 fps.The frames from the video streaming are pre-processed to convert their colour to RGB and for resizing.Then a pixel-to-numpy array conversion is done with the converted frames.Then the array, along with the pre-trained SSD (Single Shot Multi-Box Detector) Mobile Net V2 model, is fed to the interpreter to perform object detection.The score of the interpreter's output is determined.It indicates the probability of the class of the detected object.The coordinates of the bounding box for each of the detected objects are then determined.Upon analysis of score, the detected objects having a score greater than the preset minimum threshold are bounded by boxes and labelled with their score.
The screen capture of the frames with detected objects is periodically uploaded to the Adafruit IO cloud, and this information is sent to the respective disaster management teams in case of emergencies.The object detection stops when the water level drops below C2 or when the system is interrupted.
The SSD MobileNet V2 model is trained on the MSCOCO dataset.In this research, Tensor Flow Lite is used to run this model.It is capable of detecting and identifying many different common objects, such as people, vehicles, animals, and so on.SSD MobileNet V2 is a dynamic object detection model designed for mobile and embedded vision applications.Where M indicates the number of objects detected and identified.

Performance Parameters
Mean Average Precision (mAP) (γ) [16] is one of the common metrics used to analyse the performance of object detection models.It is the sum of all average prediction (AP) values across all classes and categories.The performance of an object detection model is analysed using its loss functions.Let ZPkl = {1, 0} be a match indicator for the kth default box to the lth ground truth box of category p. (

Event Communication
The water level is fed to the cloud periodically, along with snap shots of objects detected in the flood water.When the water level crosses a prefixed safety threshold (in this case, 80 cm), alert notifications along with detected data are made available to the concerned emergency services and rescue teams via electronic mail (email), phone calls, Short Message Service (SMS), and push-bullet notifications.The User Interface applications, namely Adafruit_IO cloud, email, push bullet, and Twilio for SMS and automated voice calls, are used in this research.
Adafruit_IO [17] is a cloud service that primarily serves to store, retrieve, and display data over the internet.Adafruit_IO is capable of handling and visualizing multiple data feeds.Feeds contain both data uploaded and metadata that indicates the date and time of updating.Dashboards are a feature integrated into Adafruit_IO that provides visualization of data in the form of a chart, graph, gauge, log, etc. Adafruit_IO keeps the data protected from unauthorized access by assigning a key to each user.Only the user with this key can upload, change, or visualize data in their respective Adafruit_IO feed or dashboard.
In the developed I-CVSDDM system, the MQTT (Message Queuing Telemetry Transport) [18] protocol is used to send water level and detected objects to the Adafruit_IO feed.The image block provided by the Adafruit_IO supports only base64-encoded images of a size less than 10 KB.Hence, the screen capture is resized and converted into a base64-encoded image, which is then sent to the feed using the MQTT protocol.MQTT is an excellent option for sending a high volume of sensor messages while working with IoT devices since they have limited power supply, bandwidth, or packet size restrictions and are connected to the internet via Wi-Fi or Cellular.

Alert Notification
The gathered information, along with an alert message, is sent through email, SMS, and automated voice calls to the concerned disaster management departments and rescue teams.E-mail is sent using SMTP (Simple Mail Transfer Protocol) [19], and HTTP (Hypertext Transfer Protocol) [20] is used to send alert notifications to Push Bullet.

System Parameters
Table 1 illustrates the system parameters used to validate the developed I-CVSSDM system.The total threshold of water level that can be measured is 0-450 cm in the real-time implementation; however, it is between 0-38 cm in the developed experimental setup as the maximum height of the tank is 38 cm.The condition under which an object is detected is also illustrated.

Experimental Setup
Figure 5 depicts the experimental setup utilised to evaluate the I-CVSSDM system, and a 38-cm-high fish tank is used for modelling the waterlogging area in real time.The ultrasonic sensor transceiver is fixed at 65cm above the surface of the fish tank, and the safety threshold is set to 10 and 15 cm.The embedded module in the setup consists of a Raspberry Pi 4 model B board, a 5 MP pi-camera, a power bank of 20000 mAH to power up the raspberry pi, a JSN-SR04t ultrasonic sensor, a LED, and a buzzer.In this setup, when the water level rises to C2, the pi camera is turned ON and performs object detection, and when the water level crosses C3, alert notifications along with the data collected are sent through email, push notification, phone call, and SMS.

Performance Analysis
The observed outputs from the experimental setup are shown and briefed in this section.The outcome of the analysis (γ, δ, and τ) is presented in Figure 6 to 8.
Figure 6 demonstrates that as fps rises in γ, the mean average accuracy rises along with it.

Figure 6. Analysis of fps vs. Precision
Figure 7 shows the event condition in relation to the actual delay time; as the condition value becomes close to the threshold value, a delay is achieved in order to capture the event.Object detection time will increase when more objects have been identified inside the frame rate, as seen in Figure 8 after the performance of the fps has been evaluated.An alert displaying "water level is greater than 15cm" being delivered to the push bullet with a link to the Adafruit_IO cloud dashboard is shown in Figure 11.
Figure 12 exhibits the alert notification sent by mail.In the subject line of the email, location and time are specified.The alert message, the link to monitor the Adafruit_IO cloud dashboard, and a snapshot of the frame with detected objects are attached to the body of the email.The observations from the experimental setup are detailed in this section.The confidence scores for each object at different time intervals are observed from the experimental setup.The confidence score is the probability of how confident the system is when an object is associated with a particular class.

Figure 13. Alert notification received via SMS and Call
The confidence score for the model objects used in the experimental setup was found to be between 60% and 75%.It is also observed that the confidence level of the detected objects drops for a short period of time when the objects are on mobility.When performing the SSD MobileNet V2 model on the COCO dataset, the mAP is found to vary between 50 and 76.8%.
Figure 14 displays the outcomes of testing the SSD MobileNet V2 Object Detection module using online realtime flood photos.

Conclusion and Future Work
In this research, the I-CVSSDM system was developed to provide an early warning alert to the disaster management team about the water logging and objects affected in areas under heavy floods.The developed system can be implemented in subways, dam reservoirs, and streets with a high risk of flooding.The SSD MobileNet V2 Object Detection Mechanism is used for object detection.Initially, the accuracy of this mechanism is tested with the COCO dataset and, subsequently, with real-time data taken from the internet.The performance of the developed system is validated in terms of γ, δ and τ.A maximum delay of 12 seconds is observed; a reduction can be realized with ensured connectivity.Around 50 to 76.8% of mAP is achieved for Experimental setups, while in real time it achieves only 55 to 68%.This can be further enhanced by increasing the training dataset.

Figure 1 .
Figure 1.System architecture of the developed I-CVSSDM System

Figure 2 .
Figure 2. Modules Involved in the developed I-CVSSDM System.

Figure 3 .
Figure 3. Water Level Monitoring Module of the developed I-CVSSDM System.

Figure 4 .
Figure 4. Object Detection Mechanism used in the developed I-CVSSDM System.
) The weighted sum of the localization loss and the confidence loss yields the total objective loss function: L (a, b, c, d) = 1/N (Lconf (a, b) + α Lloc (a, c, d) (3) N denotes the number of matching boxes.When N = 0, no loss is experienced between the predicted box (c) and ground truth box (d) parameters; the localization loss is a smooth L1 loss.Between the projected bounding box adjustment and the reality values, the localization loss is a smooth L1 loss.The weight term N = 1 by cross-validation, and the confidence loss is the SoftMax loss over multiple class confidences (c).Lconf (a, b) = -∑k pos Zpkl log (cpi) -∑k Neg log (c0i) (4) EAI Endorsed Transactions on Internet of Things | Volume 10 | 2024 | P. Ramesh et al.

Figure 5 .
Figure 5. Experimental setup of the developed I-CVSSDM System.

Figure 7 .Figure 8 .
Figure 7. Analysis of Event condition with delay time

Figure 14 .
Figure 14.Object detection performed on real-time flood images

Table 1 .
Description of the parameters