Received: 17 April 2023 | Revised: 9 June 2023 | Accepted: 12 June 2023 | Published online: 27 June 2023
Artificial Intelligence and Applications
RESEARCH ARTICLE 2023, Vol. 00(00) 1–6
DOI: 10.47852/bonviewAIA3202975
An AI-Based Early Fire Detection System
Utilizing HD Cameras and Real-Time
Image Analysis
Leendert Remmelzwaal1,*
1
Department of Electrical Engineering, University of Cape Town, South Africa
Abstract: Wildfires pose a significant threat to human lives, property, and the environment. Rapid response during a fire’s early stages is critical to
minimizing damage and danger. Traditional wildfire detection methods often rely on reports from bystanders, leading to delays in response times
and the possibility of fires growing out of control. In this paper, ask the question: “Can AI object detection improve wildfire detection and response
times?”. We present an innovative early fire detection system that leverages state-of-the-art hardware, artificial intelligence (AI)-powered object
detection, and seamless integration with emergency services to significantly improve wildfire detection and response times. Our system employs
high-definition panoramic cameras, solar-powered energy sources, and a sophisticated communication infrastructure to monitor vast landscapes in
real time. The AI model at the core of the system analyzes images captured by the cameras every 60 s, identifying early smoke patterns indicative
of fires and promptly notifying the fire department. We detail the system architecture, AI model framework, training process, and results obtained
during testing and validation. The system demonstrates its effectiveness in detecting and reporting fires, reducing response times, and improving
emergency services coordination. We have demonstrated that AI object detection can be an invaluable tool in the ongoing battle against wildfires,
ultimately saving lives, property, and the environment.
Keywords: wildfire detection, artificial intelligence, object detection, panoramic cameras, solar-powered system
1. Introduction demonstrated their potential to revolutionize various industries,
including fire detection and response. CNNs have achieved
Wildfires pose a significant threat to human lives, property, and exceptional performance in object classification (He et al., 2016;
the environment. The rapid response during a fire’s early stages is Krizhevsky et al., 2012; Simonyan and Zisserman, 2015; Szegedy
critical in determining the level of damage and danger it can cause. et al., 2017), object detection (Bochkovskiy et al., 2020; Li et al.,
Traditional wildfire detection methods often rely on reports from 2022; Lin et al., 2017; Liu et al., 2016), and object segmentation
bystanders, leading to delays in response times and the possibility of (Chen et al., 2018; He et al., 2017; Long et al., 2015; Long et al.,
fires growing out of control. On average, Australia experiences 2015; Ronneberger et al., 2015) on established image datasets and
around 50,000–60,000 bushfires annually, with estimated damage have found applications in autonomous driving, robotics, and
varying significantly depending on the severity and location of the fires. video surveillance. Among these tasks, object detection is
A recent example of the devastating impact of wildfires can be particularly relevant for fire detection, as it involves identifying
seen in the Margaret River bushfires in Western Australia in 2011. the location and category of objects within an image, such as
These fires ravaged the area, destroying more than 30 homes and smoke or flames.
forcing over 200 residents to flee to safety. Smoke from the In this paper, we present an innovative early fire detection
inferno engulfed the region, creating hazardous conditions and system that utilizes state-of-the-art hardware, AI-powered object
complicating firefighting efforts. The annual cost of bushfire detection, and seamless integration with emergency services to
damages in Australia is around AUD 1.6 billion, but this figure significantly improve wildfire detection and response times.
can be much higher in years with particularly severe fires, such as Our system combines high-definition panoramic cameras,
the 2019–2020 Australian bushfire season, which resulted in solar-powered energy sources, and a sophisticated communication
damages estimated to exceed AUD 10 billion. Given the infrastructure to monitor vast landscapes in real time. The AI
potentially disastrous consequences of wildfires, there is a model at the core of the system analyzes images captured by the
pressing need for more advanced and efficient detection methods cameras every 60 s, identifying early smoke patterns indicative of
to enable a faster and more effective response. fires and promptly notifying the fire department (see Figure 1).
Recent advancements in artificial intelligence (AI), specifically This approach ensures rapid response and coordination,
convolutional neural networks (CNNs) (Sun et al., 2021), have minimizing the potential damage caused by wildfires.
This paper is organized as follows: Section 2 details the system
*Corresponding author: Leendert Remmelzwaal, Department of Electrical architecture, including the camera system, solar power system,
Engineering, University of Cape Town, South Africa. Email:
[email protected] communication infrastructure, and public interface. Section 3
© The Author(s) 2023. Published by BON VIEW PUBLISHING PTE. LTD. This is an open access article under the CC BY License (https://creativecommons.org/
licenses/by/4.0/).
01
Artificial Intelligence and Applications Vol. 00 Iss. 00 2023
Figure 1 even in low visibility conditions, such as in the presence of smoke
Detection of bushfire or fog (Bouguettaya et al., 2022).
Machine learning-based detection: With the advancements in
machine learning and computer vision, various models have been
proposed for fire detection using features learned by algorithms.
Bouguettaya et al. (2022) discuss the effectiveness of deep learning-
based computer vision algorithms for early wildfire detection from
unmanned aerial vehicles (Bouguettaya et al., 2022). Akagic and Buza
(2022) present a lightweight wildfire image classification method using
deep CNNs (Akagic & Buza, 2022). Wang et al. (2019) explore forest
fire image recognition based on CNNs (Wang et al., 2019).
For the current project, access to thermal imaging was
unavailable, with only panoramic cameras at our disposal. However,
we were not constrained to edge processing and had access to cloud
computing power. Consequently, we opted to utilize the machine
learning-based detection approach for our fire detection system.
This research paper introduces an AI model capable of real-time
processing of panoramic images. Notably, we employ cutting-edge
object detection techniques, which distinguish our methodology
from previous studies, such as the work conducted by Bouguettaya
et al. (2022), which utilizes image tessellation and object
classification. Our approach, centered around object detection,
demonstrates higher efficiency due to its independence from the
need for image tessellation.
3. Research methodology
The proposed early fire detection system consists of a
sophisticated hardware setup and a communication infrastructure
that seamlessly integrates with the AI model and fire department
outlines the AI model architecture, focusing on the object detection resources. This section outlines the key components of the
model, dataset creation, and training process. Section 4 presents the system’s architecture and their roles in ensuring effective and
results obtained during the testing and validation of the system, efficient fire detection and response.
demonstrating its efficacy in detecting and reporting fires, reducing
response times, and improving emergency services coordination. 3.1. Camera system
Finally, Section 5 discusses potential future work and enhancements
to further improve the system’s performance and capabilities. The entire hardware solution comprises three cameras
In summary, this paper introduces a cutting-edge early fire and a solar power unit, mounted on a pole (see Figure 2). Two
detection system that combines high-definition cameras and high-definition 180-degree cameras work together to create a
AI-driven image analysis to revolutionize wildfire monitoring and comprehensive 360-degree panoramic view of the monitored
response. By providing accurate, real-time information to emergency landscape. A single high-definition pan–tilt–zoom (PTZ) camera
services, the system has the potential to significantly reduce the is employed for detailed fire imaging and investigation. This
damage caused by wildfires and protect human life and property. PTZ camera can be remotely controlled from the control tower
or by the AI system, allowing for adjustments in pan (x-axis),
2. Review of early fire detection approaches tilt (y-axis), and zoom as needed. The integration of a feedback
loop between the software and the PTZ camera ensures optimal
In recent years, the application of cameras for early fire detection imaging and analysis.
has garnered significant attention, with several methodologies
emerging as the most prevalent. The following literature review 3.2. Solar power system
explores these methods and the research that underpins them:
Flame detection: One of the widely used techniques for fire The system utilizes state-of-the-art solar panels, a solar
detection involves the identification of unique flame features, such controller, and solar batteries with smart charging capabilities to
as flickering behavior, color, and shape. Celik (2010) ensure continuous operation in any environment (see Figure 2).
demonstrates that flame detection algorithms typically rely on The solar power system eliminates the need for external power
image processing techniques to extract these features and supplies, making it suitable for remote deployment. The batteries
recognize the presence of fire in captured images (Celik, 2010). have a life expectancy of 5 years, reducing the need for frequent
Smoke detection: Another common approach to fire detection is maintenance and replacement.
the analysis of images for specific smoke characteristics, including
color, texture, and motion. Töreyin et al. (2006) reveal that by 3.3. Communication infrastructure
detecting and tracking the movement of smoke, these algorithms
can provide early warning of a developing fire (Töreyin et al., 2006). A reliable 4G connection is incorporated into the system,
Thermal imaging: Thermal cameras detect heat and create enabling 24/7 real-time internet connectivity for live streaming of
visual representations of temperature variations in the scene. By images from the cameras to a cloud server. This high-speed
identifying temperature anomalies, this method can detect fires connection also facilitates real-time feedback to the PTZ
02
Artificial Intelligence and Applications Vol. 00 Iss. 00 2023
Figure 2 to view real-time images from the camera system and receive
Camera and solar panel hardware information on detected fires, fostering awareness and community
engagement in wildfire monitoring and prevention.
In summary, the system architecture consists of an advanced
camera system, a solar power system, a communication infrastructure,
and a user-friendly public interface. These components work together
to create a robust and reliable early fire detection system that can be
deployed in various environments, providing real-time data and
images for effective fire detection and response.
4. AI model architecture
The AI component of the early fire detection system plays a
crucial role in analyzing images captured by the camera system and
identifying smoke patterns indicative of fires. This section describes
the architecture of the AI model, including the methodology, object
detection model, dataset creation, and training process.
4.1. Methodology
In this paper, we present a methodology for the development of
an AI component for early bushfire detection. The AI model is
designed to analyze 360-degree panoramic images, identify
distinctive smoke trails and signs of fire, utilize object detection to
recognize smoke or fire, track the movement of smoke over time,
and map the direction of movement with known wind direction to
correlate the direction of growth.
To process the images, the AI system employs a camera system
that captures 360-degree panoramic images at a rate of one image per
Figure 3 minute. These high-resolution images provide comprehensive
System architecture coverage of the monitored area, enabling the early detection of
emerging fires. The primary objective of the AI model is to
identify distinctive smoke trails and signs of fire within the
captured images, which is achieved by analyzing the visual
features of the images, such as color, texture, and shape, to
pinpoint potential fire-related patterns.
The AI model leverages advanced object detection techniques to
accurately identify smoke or fire within the images. This involves
training the model on a comprehensive dataset containing annotated
images of smoke and fire, allowing it to learn the unique
characteristics of these phenomena and distinguish them from other
objects in the scene. In order to determine the direction of
movement, rate of growth, and size of a detected fire, the AI model
tracks the movement of smoke over time. By comparing
consecutive images, the system can analyze the changing patterns of
smoke and infer valuable information about the progression of the fire.
Lastly, the AI model maps the direction of smoke movement
with known wind direction data to correlate the direction of
growth of the fire. This information can be used to predict the
likely path of the fire, facilitating more efficient and effective
firefighting efforts. By integrating these methodologies, the
AI-based early fire detection system can rapidly and accurately
detect emerging bushfires, ultimately assisting in the timely
mitigation of potential damages and loss of life.
controller, ensuring efficient communication between the hardware
components and the AI system. High-speed video links provide 4.2. Object detection model
seamless transmission of images to the AI system and the fire
department for rapid analysis and response (see Figure 3). The system incorporates a multitude of intricate steps,
including pre-processing, various AI detection and classification
3.4. Public access and interface models, and post-processing. This discussion provides a
glimpse into one specific object detection model utilized within
The software for the fire detection system is accessible to the the pipeline. However, other components of the system are
public through the website bushfire.ai. This platform allows users proprietary and cannot be disclosed.
03
Artificial Intelligence and Applications Vol. 00 Iss. 00 2023
Among the AI object detection models employed in this system, In summary, the AI model architecture combines a powerful object
one is founded on the YOLOv5 architecture, an advanced deep learning detection model with a diverse dataset, data augmentation techniques, and
model recognized for its exceptional accuracy and efficiency in an optimized training process. This comprehensive approach enables the
identifying objects within images (Bochkovskiy et al., 2020; Jocher, early fire detection system to effectively identify and track the movement
2022; Redmon et al., 2016; Thuan, 2021). Utilizing the PyTorch of smoke over time, enhancing the overall performance and utility of the
framework, the YOLOv5 model is configured to process input system in wildfire monitoring and response.
images with dimensions of 640 × 640 pixels, ensuring high-
resolution analysis for precise fire detection. 4.6. Possible shortcomings
4.3. Dataset creation Within the domain of camera systems, the orientation of the
cameras, in conjunction with the application of Zoom functionality
To train the AI model, a dataset of 20,000 images was created and the presence of image distortion, holds the potential to exert an
using panoramic cameras deployed across the United States and influence on the outcomes of AI detection. To ensure the adaptability
Australia. The images were collected over a 12-month period to and reusability of trained AI models, it is imperative to undertake the
account for variations in seasons and weather conditions, ensuring process of normalizing and deskewing raw images prior to their
a diverse and representative dataset. The dataset was split into a submission for AI inference. By implementing these measures, the
70% training set and a 30% validation set to evaluate the model’s effectiveness of AI detection can be enhanced, allowing for the
performance during the training process. preservation and continued utilization of trained AI models.
The dataset includes images with and without fire, and the
model is trained to recognize both scenarios. Images were 5. Results
annotated by wildfire experts. Images without annotations,
i.e., those without visible fires, are not ignored during training. This The performance of the early fire detection system’s AI model
approach informs the model about the absence of fires, enhancing was evaluated based on its ability to accurately identify and report
its ability to differentiate between fire and non-fire conditions. fires in their early stages. This section presents the key results
obtained during the testing and validation process, demonstrating the
4.4. Data augmentation system’s efficacy and its potential to enhance fire response efforts.
The AI model achieved a mAP score ([email protected]:0.95) of 0.04 and
To improve the model’s robustness and generalization an accuracy of 93.5% during the training process. These results indicate
capabilities, data augmentation techniques were applied during that the model is highly effective at detecting smoke patterns indicative
the training process. These techniques include horizontal flipping, of fires within the input images (see Figure 4). The trained AI model was
scaling, brightness adjustments, hue changes, and saturation able to identify early-stage fires with high precision, ensuring prompt
modifications. Vertical flipping was excluded from the augmentation notification of the fire department and facilitating rapid response efforts.
process, as it may introduce unrealistic scenarios for wildfire detection. In addition to the detection performance, several anecdotal
benefits have emerged from the implementation of the system:
4.5. Model training Reduced response time: Anecdotal evidence suggests that the
proposed system, which continuously monitors large areas with
The AI model was trained for 679 epochs, achieving a mean high-resolution cameras and employs the AI model for real-time
average precision (mAP) score ([email protected]:0.95) of 0.04 and an image analysis, has significantly reduced response times compared
accuracy of 93.5% (see Table 1). The training process was halted at to traditional fire reporting methods. This improvement in
679 epochs because the model’s loss did not improve further, and response time has the potential to greatly reduce the scale of
additional training epochs led to overfitting and reduced generalization. wildfires and the associated damage to human life and property.
During the training process, the AI model learned to detect Web portal for first responders: The early fire detection system
early smoke patterns from fires by analyzing the input images. features a real-time dashboard tailored for first responders,
It also learned to control the PTZ camera through the feedback integrating AI processing and facilitating communication and
loop, adjusting the camera’s pan, tilt, and zoom settings to coordination among emergency services. This web portal not only
better capture and analyze fires.
Figure 4
Example detection of a fire
Table 1
Fine tuning parameters
Parameter Value
Framework PyTorch
Model Similar to YOLOv5
Input size 640 × 640 pixels
Dataset size 20,000 images
Data augmentations Horizontal flip, scale,
brightness, hue, saturation
Number of epochs 679
Accuracy 93.5%
Loss Not improved after 679 epochs
mAP score 0.041 ([email protected]:0.95)
Train/Val split 70% / 30%
04
Artificial Intelligence and Applications Vol. 00 Iss. 00 2023
notifies the relevant fire department upon detecting a potential fire To ensure reliable and real-time communication between the
but also provides high-resolution images of the fire location, system, emergency services, and the public, the communication
satellite map data with a weather overlay, and real-time infrastructure could be further improved. This might include the
information about fire department aircraft, which enhances implementation of a dedicated communication network for
situational awareness and decision-making. emergency services, the use of edge computing to reduce latency
Public engagement and awareness: The system promotes and improve data processing, or the development of a
community engagement and awareness in wildfire monitoring and decentralized communication protocol to enhance system resilience.
prevention by making the software accessible to the public In conclusion, there are numerous opportunities for future work
through the https://bushfire.ai website. This platform allows users to improve the early fire detection system’s performance and
to view real-time images from the camera system and receive capabilities. By addressing these areas, the system could become
information on detected fires, fostering a sense of ownership and an invaluable tool in the ongoing battle against wildfires,
responsibility among community members. ultimately saving lives, property, and the environment.
These anecdotal benefits underscore the overall value of the early
fire detection system in not only detecting and reporting fires but also Acknowledgments
improving response times, coordination of emergency services, and
public engagement. The system’s performance highlights its potential The AI modeling and training tools utilized in this project were
to significantly mitigate the impact of wildfires on human life, made available by https://firststep.ai/. Public access to the AI fire
property, and the environment. detection tool is available at https://bushfire.ai/.
6. Future work Conflicts of Interest
While the current implementation of the early fire detection The authors declare that they have no conflicts of interest to this
system has shown promising results, there are several areas for work.
future research and development to further enhance its
capabilities, effectiveness, and robustness. In this section, we References
discuss potential future work that could lead to improvements in
fire detection, response times, and overall system performance. Akagic, A., & Buza, E. (2022). LW-FIRE: a lightweight wildfire
The AI model could be improved by exploring other state-of-the- image classification with a deep convolutional neural
art object detection algorithms, such as Faster R-CNN, Single network. Applied Sciences, 12(5), 2646.
Shot MultiBox Detector (SSD), or EfficientDet. Additionally, Bochkovskiy, A., Wang, C. Y., & Liao, H. Y. M. (2020).
incorporating transfer learning from pre-trained models on large-scale YOLOv4: Optimal speed and accuracy of object detection.
image datasets like ImageNet could potentially boost performance arXiv preprint:2004.10934. https://arxiv.org/abs/2004.10934.
and reduce training time. Moreover, employing techniques such as Bouguettaya, A., Zarzour, H., Taberkit, A. M., & Kechida, A.
model assembling or employing a multi-stage detection pipeline (2022). A review on early wildfire detection from unmanned
could help increase the overall accuracy and reduce false positives. aerial vehicles using deep learning-based computer vision
To further improve the AI model’s generalization and algorithms. Signal Processing, 190, 108309.
robustness, the training dataset could be expanded to include Celik, T. (2010). Fast and efficient method for fire detection using
images from diverse geographical regions, climates, and image processing. ETRI Journal, 32(6), 881–890.
vegetation types. This would allow the model to better adapt to Chen, L. C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille,
varying environmental conditions and fire behavior patterns. A. L. (2018). Deeplab: semantic image segmentation with
Furthermore, incorporating synthetic data generated through deep convolutional nets, atrous convolution, and fully
computer graphics or data augmentation techniques could help connected CRFs. IEEE Transactions on Pattern Analysis
increase the dataset’s size and diversity. and Machine Intelligence, 40(4), 834–848.
Integrating multispectral or hyper spectral imaging into the camera He, K., Gkioxari, G., Dollár, P., & Girshick, R. (2017). Mask
system could provide additional information about fires, such as R-CNN. In Proceedings of the IEEE International
temperature, chemical composition, and combustion stages. This Conference on Computer Vision, 2980–2988.
additional data could be used to improve fire detection accuracy and He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for
provide more detailed information to emergency services, enabling image recognition. In Proceedings of the IEEE Conference on
them to better assess the situation and allocate resources accordingly. Computer Vision and Pattern Recognition, 770–778.
Incorporating real-time weather data into the system’s decision- Jocher, G. (2022). ultralytics/yolov5: v7. 0-YOLOv5 SOTA realtime
making process could help improve the accuracy of fire detection and instance segmentation. https://doi.org/10.5281/zenodo,7347926.
prediction. Factors such as wind speed, humidity, and temperature Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). Imagenet
can significantly influence fire behavior and spread. By classification with deep convolutional neural networks. Advances
considering these factors, the system could potentially anticipate in Neural Information Processing Systems, 1097–1105.
fire growth patterns and provide more accurate alerts and Li, X., Liang, J., Li, S., Shen, S., & Liu, L. (2022). Neural architecture
recommendations to emergency services. search for object detection: a survey. IEEE Transactions on
Developing a fire spread prediction model based on factors such Neural Networks and Learning Systems, 33(1), 73–89.
as terrain, vegetation, and weather conditions could help emergency Lin, T. Y., Dollár, P., Girshick, R., He, K., Hariharan, B., & Belongie,
services better anticipate the evolution of a fire and plan their S. (2017). Feature pyramid networks for object detection.
response accordingly. By providing an estimation of the fire’s Artificial Intelligence and Applications, 31(1), 211–220.
future behavior, this prediction model could enable more effective Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C. Y., &
resource allocation and response strategies. Berg, A. C. (2016). SSD: single shot multibox detector.
05
Artificial Intelligence and Applications Vol. 00 Iss. 00 2023
In European Conference on Computer Vision, 21–37. Cham: review. Journal of Intelligent Manufacturing, 32(3),
Springer. 629–656.
Long, J., Shelhamer, E., & Darrell, T. (2015). Fully convolutional Szegedy, C., Ioffe, S., Vanhoucke, V., & Alemi, A. A. (2017).
networks for semantic segmentation. In Proceedings of the Inception-v4, inception-ResNet and the impact of residual
IEEE Conference on Computer Vision and Pattern connections on learning. In Proceedings of the AAAI
Recognition, 3431–3440. Conference on Artificial Intelligence, 4278–4284.
Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016). You Thuan, D. (2021). Evolution of YOLO algorithm and YOLOVv5:
only look once: unified, real-time object detection. In the state-of-the-art object detection algorithm.
Proceedings of the IEEE Conference on Computer Vision Töreyin, B. U., Dedeoğlu, Y., Güdükbay, U., & Cetin, A. E. (2006).
and Pattern Recognition, 779–788. Computer vision based method for real-time fire and flame
Ronneberger, O., Fischer, P., & Brox, T. (2015). U-net: detection. Pattern Recognition Letters, 27(1), 49–58.
convolutional networks for biomedical image segmentation. Wang, Y., Dang, L., & Ren, J. (2019). Forest fire image
In International Conference on Medical Image Computing recognition based on convolutional neural network.
and Computer-Assisted Intervention, 234–241. Journal of Algorithms & Computational Technology, 13,
Simonyan, K., & Zisserman, A. (2015). Very deep convolutional 1748302619887689.
networks for large-scale image recognition. In International
Conference on Learning Representations. How to Cite: Remmelzwaal, L. (2023). An AI-Based Early Fire Detection System
Utilizing HD Cameras and Real-Time Image Analysis. Artificial Intelligence and
Sun, J., Wu, J., Wu, C., Zhang, L., Zhang, X., & Wang, Y. (2021).
Applications https://doi.org/10.47852/bonviewAIA3202975
Object detection in smart manufacturing: a comprehensive
06
