Context:
The Belgian Special Forces often need to intervene in unknown environments. Sometimes this requires entering buildings or ships for a first time, which are only known from plans. This procedure brings constraints in terms of safety and needs to be performed and repeated ahead. The aim of this research is to develop a VR simulation for the Belgian Defense to train soldiers to perform multiple scenarios on a 3D reconstruction of a building or ship.
The Special Forces are currently repeating different scenarios in a physical shoothouse for their training. In this environment they can perform hostage and inspection situations. The drawback of such a traditional shoothouse is that they cannot change their layout easily, which makes the trainees too comfortable and used to the same area.
Description:
The aim of this project is to create a Virtual Reality Shoothouse. Based on a 2D floorplan or a 3D plan of a building or a boat, an algorithm that features Artificial Intelligence and Image processing will recreate a 3D simulation of it. In this environment, the trainees will perform their approach using VR headsets while being watched and supervised by an overseer who can see the decision making of his troops. With this project we ensure a more versatile environment for inspection procedures where Special Forces can perform any intervention in virtually millions of different layouts without getting too comfortable with it.
Algorithm:
The algorithm uses multiple frameworks and techniques to fulfil its duty. Object recognition using deep learning and OpenCV are the library used to achieve the task. We have developed an automated framework to turn floor plans into VR environments that feature walls, doors, windows, and large pieces of furniture. Our system allows to build VR-ready environments that depict the geometry of entire floors with the locations of doors and windows, critical for tactical intervention teams.
Android & iOS Application:
We developed an Android and iOS application. The aim here is that our partners can test in real-time the advancement of our algorithm. The back-end server is regularly upgraded which ease the testing process. The application allows you to take pictures from your phone or pictures already stored and transform it into a 3D file that you can see on your phone and then download if needed.
Virtual Reality Ready Rifles:
These rifles are airsoft replica and are connected to the simulation via a Base Station and Bluetooth to ensure their reactivity in the VR world. Two models are currently available: The FN F2000 and the FN-SCAR-H. Pistols such as the FN5-7 are also in development to fulfill future needs.
Publications
2023
- Fréville, T.; Hamesse, C.; Pairet, B. and Haelterman, R. (2023). Object Detection in Floor Plans for Automated VR Environment Generation.In Proceedings of the 18th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications – Volume 5: VISAPP, ISBN 978-989-758-634-7, ISSN 2184-4321, pages 480-486. [PDF]
2021
- T. Fréville, C. Hamesse, B. Pairet, R. Lahouli and R. Haelterman, “From Floor Plans to Virtual Reality,” 2021 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR), 2021, pp. 129-133, doi: 10.1109/AIVR52153.2021.000. [PDF]
