Introduction: Mastering todays challenges in firefighters’ training with smart glasses

How to react to unknown situations? It is either curiosity or fear. In contrary, professional firefighters are trained to react accordingly, to protect human lives from harm and goods from damage.

The degree of incidents varies from saving people from a burning house or burning electrical car to dealing with hazardous substances. Especially hybrid or electrical cars and buses are new on the roads and require special treatments in an incident situation. As they are currently rather expensive and therefore difficult to implement in modern firefighters training further solutions are appropriate to test.

The transnational EU-Project “FIGHTARs” enriches current firefighters training with immersive technologies, like Augmented Reality. This enables incident commanders to train, in real-time, on specific 3D model operations like cutting the car body open to rescue passengers or extinguishing fires. The project consortium consists of four professional firefighters training organization from Czech Republic (SPSCH Pardubice), Estonia (Estonian Academy of Security Sciences), Lithuania (Fire Fighters Training School) and Slovakia (University of Zilina). They are accompanied by experts from Germany (SBG Dresden) and Cyprus (SCP SERV), for the integration of immersive technology in training.

During the two years runtime of the project, from 11/20-10/22, the project partners will develop, test, evaluate and transfer suitable Augmented Reality and 360° video training scenarios for professionals and students training. These scenarios and more offers will be integrated in the planned immersive training centres for modern firefighters training in the four professional training institutions.

What is Augmented Reality (AR)?

The real and the digital world overlap every day more and more. The use of smartphones and tablets enriches our daily communication at work and in the private life. The application of interactive 3D objects (= holograms) provides the floor for integrating them in professional guidance and in the explanation of non-visible processes. To see holograms, specialised technology, like smart glasses (Augmented reality glasses or AR glasses), smartphones or tablets are necessary. The use of smart glasses allows a hand-free use.

Currently no or less experiences to apply AR in classroom or live training of firefighters.

Augmented reality is at its best when providing technical and procedural knowledge like assisting in real-time when e. g. dealing with hazardous substances. The so-called AR remote support enables the expert to provide through audio-visual support by voice and by holograms, in a kind of livestream, professional guidance to the firefighters in training. The necessary hard- and software is shown below, when working e. g. with Microsoft products.

Example of AR remote training in chemistry: https://www.youtube.com/watch?v=pdKW-hF20jI

When training for technical more complex or for more dangerous situations interactive AR digital twins or interactive digital copies of real objects, e. g. electrical cars are suitable. Normally no tailor-made solutions are available. So, they must be programmed by a software programmer.

Survey results

You might ask: Is Augmented Reality (AR) needed in firefighter training? See what more than 60 firefighters said about it in April 2021 (online survey).

• EXISTING EXPERIENCE: Only 7 of them had previous knowledge about Augmented Reality in comparison to around 30 firefighters, which already had tried out Virtual Reality glasses (simulating dangerous environments).

• PREFERRED LEARNING SCENARIOS: Augmented Reality enriched trainings were seen useful by around 80% for application like extinguishing a burning vehicle and when dealing with hazardous substances. Around 70% see potential to use AR in rescue a person scenario in contrast to 60% for training how to cut a crashed car.

Meet the firefighters (Part 1)

Are you curious, what motivates firefighters to be a firefighter and what find they interesting when thinking about Augmented Reality use in classroom and live training? We ask four of them.

1. What is your name and where do you work?

My name is Stella Polikarpus and I work as senior lecturer in the Estonian Academy of Security Sciences (EASS).

2. Why you became a firefighter (in the first place)?

In the year 2000 I started to study in EASS to become fire engineer. In entrance exam to EASS I said: “I want to learn this field, because 18 years I have lived for myself, now I want to start to help others.” They accepted me into school and after that I had to start to live out that promise, I gave on interview. My whole career is marked with Estonian Rescue board moto: “To honour God and to protect fellowman”. We had four years curriculum and we started off basic firefighting training. So, after my second year in summer I could start to work as firefighter in Tartu fire station. Most of my career I have worked as trainer, not as firefighter

3. On what are you proud as a firefighter?

I am most proud when my students’ years after graduation give me feedback that they really learned something in my classes. In Estonia to be able to work as female firefighter is achievement itself as it is very man-centred field and we have always had less than 1% female firefighters at stations. I had to prove myself a lot to be accepted member of team

4. Have you ever worked before with Augmented Reality or Virtual Reality? If yes, how?

I have not worked with Augmented Reality before the TARGET or FIGHTARs projects. However, I have years of experiences training with Virtual Reality (VR). I can build scenarios in VR myself. We have made guidelines for others to build assessment scenarios. I train trainers and assessors to use VR. I study VR based situational awareness assessments

5. Where would you like to use AR in your work?

I mostly want to use AR to train firefighter teams to do good 360° and build their situational awareness and making safe response plans in incidents with hazardous materials. Because there it is very important to create realistic, authentic, complicated but same time totally safe training environment. All physical activities like putting on chemical suit and decontamination have very many technical details that cannot be trained in VR but could be trained using AR. AR would make hazardous materials training much more meaningful for students.

6. What do you think how the firefighters’ profession and training will look in five years from now?

I have been in rescue service a bit more than 20 years and learned that to create sustainable innovation in training programs 5 years is too short time. It takes 10 years to change the way of thinking and training of firefighters and I believe also other fields are the same. The innovation of technology is of course very fast and there 5 years is long time. However, that these innovations would be useable, affordable, easy to use etc it takes more than 5 years for trainers to get it working as part of curriculum. In Estonia no incident commander is working anymore who has not have virtual reality-based training experiences. To achieve that for all country we needed more than 10 years. AR based training of firefighters will be normal in Estonia by the year 2031!

Pedagogical guidelines

Augmented Reality use in firefighters training is new ground. We drafted first ideas on which technologies to use and which scenarios does make sense (and much more).

Let us know your thoughts to: support@fight-ar.com

FIGHTARs_PEDAGOGICAL GUIDELINES (DRAFT)

International meeting of firefighters

From 14. and 15.10.2021 firefighters’ training organisation from the:

• Czech Republic (SPSCH Pardubice),
• Estonia (Estonian Academy of Security Sciences),
• Lithuania (Fire Fighters Training School) and the
• Slovak Republic (University of Zilina)

met in Pardubice with IT experts from Cyprus (SCP School of Certified Professionals) and with media-pedagogical experts from Germany (Saxon Training Company from Dresden) to discuss the use of Augmented Reality in classroom and live training.

The result of the meeting is a structured process for developing, testing, and evaluating interactive visualisation for dealing with crashed electrical cars (e. g. rescue person, hazardous substances, cutting car body).

Augmented Reality and live training

To make the digital world visible requires normally programming effort, especially for multiple training occasions. There is one exception. The application of a ready-made app on smart glasses. With that app it is possible to enrich a live stream with holograms. These holograms are in the shape of e. g. an arrow.

The use of it enables audio-visual support from an incident commander to a firefighter. The provision of step-by-step and especially technical information suits the best. The expert can support his or her words with geolocated holograms in the field of view of the smart glass user, e. g. Microsoft HoloLens 2 (see below). For that relevant soft- and hardware is needed. Microsoft offers for that the Remote assist app on a subscription base.

Training event with decision-makers of firefighters’ schools

From 08.-11.02.2022 the project invited directors and departments heads of the involved organisations to meet in Vilnius / Lithuania for a 4 days strategic meeting on the strategic and operational options about Augmented Reality in firefighters training (focus: classroom-training).

During the meeting at the premises of the Fire Fighters Training School ideas and approaches on the structural integration of digital-media enriched training by modern technologies, like Augmented Reality (AR), were presented and discusses. It showed, that the degree of modern digital media use among the four firefighters schools from Estonia, Lithuania, Slovakia, and Czech Republic differ. The Estonian organisation has years of track record to training firefighters and incident commanders with Virtual Reality. The use of Augmented Reality will help to provide in upfront by using visualisation on an object to provide e.g. technical details.

The Lithuanian organisation found the use of live-support by Augmented Reality Remote Training interesting. Possibilities to use it in training are currently evaluated.

The meeting showed, that the integration of AR needs a problem-based appraoch and the involvement of decision-makers is necessary to ensure resources as well as the support digital transformation. AR can be become a habit, rather than an exception.

Meet the firefighters (part 2)

1. What is your name and where do you work? 

I am Tomas Šarauskas and I work in the Firefighters` training school of Lithuania         .

2. Why you became a firefighter (in the first place)?

To become a firefighter, I was encouraged by my father. He has been working as a shift commander in my hometown for over 20 years. I have spent a lot of time with the fire brigade since I was a child. That probably fascinated me.

3. On what are you proud as a firefighter?

I’m glad I managed to get a job at school and see and learn so many things. Working anywhere else would take a lot of time to gain so much knowledge.

4. Have you ever worked before with Augmented Reality or Virtual Reality? If yes, how? 

No, I have not.

5. Where would you like to use AR in your work? 

As an instructor I would like AR to be used with very dangerous things thus reducing the risk during activities. as well as by modeling situations with AR, the listener can be provided with the necessary information and task performance instructions in real-time.

6. What do you think how the firefighter’s profession and training will look in five years from now? 

I think the firefighting profession is inert, so it won’t change very strongly. Of course, there will be new equipment, new technology that will make the job easier. Under the current situation, training should move to distance or training with small groups of firefighters. In both cases, the AR can help to illustrate situations and potential hazards.

International meeting of firefighters in Cyprus

From 13. and 14.12.2021 firefighters’ training organisation from the:

• Czech Republic (SPSCH Pardubice),
• Estonia (Estonian Academy of Security Sciences),
• Lithuania (Fire Fighters Training School) and the
• Slovak Republic (University of Zilina)

met in Limassol / Cyprus at the premises of SCP School of Certified Professionals. The media-pedagogical experts from the German training provider SBG Dresden were virtually present.

During the two days hybrid meeting the Augmented Reality (AR) App was discussed, which will include e. g. scenarios on first aid and hazardous substances. The App will be connected to online learning modules, which are presented in the Online Learning Management System (LMS) of the Fight-AR.com-Webpage. The LMS and the AR app will also include training cases.

Sneak peak to the Augmented Reality FIGHTARs app

The FIGHTARs Augmented Reality App for the smart glass Microsoft HoloLens 2 involves 4 different training possibilites or scenarios in classroom training such as: first aid, hazardous substances, electrical car: deactivation of the battery and cutting the car body open.

Deactivation of the car battery

Put to the acid test: FIGHTARs app tested in live training

Practitioners tested the latest version of the FIGHTARs Augmented Reality App. The four training cases on first aid, hazardous substances, car chassis, stabiliation and cutting, and deactivation of electrical cars’ battery were put to the acid test by firefighters from Estonia, the Czech Republic, Lithuania, and Slovakia during classroom and live training in Tallinn, Estonia, from 06.-09.09.2022.

The feedback from practitioners showed that the chosen scenarios are suitable. The environmental and technical setup must be always taken into account (see below).

The input obtained, were applied for a further update of the FIGHTARs app scenarios. For exampleE. g. more avatars for the first aid scenarios e. g. for training a triage case will be created. These avatars are so integrative that they overlap Cardiopulmonary resuscitation (CPR) training dummies.

Real life size 3D Holograms allow the hologram overlay with a passenger car, so we can create scenarios for practical skills training.

Demo-day: International conference at the University of Zilina (Slovak Republic)

On 06.10.2022 more than 50 participants from the Slovak Ministry of interior, Firefighters from the Žilina Fire brigade, firefighters’ trainers, researchers, and students witnessed the presentation of the updated firefighters’ app and the updated pedagogical guidelines. The event took place at the premises of the University of Žilina, Slovakia. The full day programme also involved hands-on sessions on the FIGHTARs app training cases on three spots. The feedback from the practitioners was enthusiastic.

Further try out sessions will be provided to the interested public in Czech Republic, Lithuania, and in Estonia until April 2023. Follow the news and invitations to these events.

We welcome everyone to make those first steps with AR

First Aid in action

Group Picture: Project team with firefighters from Slovakia

Transnational project meeting in Slovakia

Around the international conference, the fourth transnational project meeting took place on 05. and 07.10.2022 in Žilina. During the joint meeting the partners discussed the progress made with the FIGHTARs Augmented Reality App, the pedagogical guidelines, and the next steps to enrich the learning management systems (LMS) with further content, which will be then easily accessible either in the FIGHTARs app or on the FIGHTARs webpage.

Partners reviewed further future steps “What could be happening after the project will finish?”. It is always in our minds while developing an app that could be integrated in the training or expanding firefighters / commanders’ digital skills, knowledge about immersive technology possibilities in trainings.

FIGHTARs pedagogical guidelines: UPDATED

he latest version of the guidelines is available. Inputs from the testing in Estonia are integrated. New sections include the Dos and Don’ts when using the Microsoft HoloLens 2 in the training ground as well as hints for trainers to implement the FIGHTARs app in a pedagogical-sound way in the training.

http://fight-ar.com/wp-content/uploads/2022/10/FIGHTARs_PEDAGOGICAL-GUIDELINES_FINAL.pdf

Check this document if you want to learn basics of Augmented Reality technology and how it can be implemented in fire fighters’ training. Join us!

FIGHTARs Augmented Reality app: UPDATED

The video teaser providesed you with updated content of the app. More here: https://www.youtube.com/watch?v=P06kY8xbPfg

Meet the firefighters

1. What is your name and where do you work?    

My name is Jan Meloun. I am working at SPSCH Pardubice as an instructor of practical training.

2. Why did you become a firefighter (in the first place)?

I never worked as a firefighter, but I have long experience saving someone’s life as a former member of the mountain rescue service in the Great Britain.

3. What are you proud of as a firefighter? 

I like to help people; I like varied work and physically demanding work and I am proud that I can do all this and save someone’s life.

4. Have you ever worked before with Augmented Reality or Virtual Reality? If yes, how? 

No, this is the first time that I am using this technology.

5. Where would you like to use AR in your work? 

After I spoke with some of my colleagues, I think one of the many options is preparation of new students with varying simulated situations, which I will be able prepare using this technology.

6. What do you think how the firefighter’s profession and training will look in five years from now? 

I think this technology of AR will be part of normal life in the work of any firefighter. Same as we used to never think that we can use GSM signals to help save someone’s life.

Scientific publication about FIGHTARS