Editor’s Blog – Armed forces explore AR for animals

Augmented reality (AR) for animals is being considered by the US Army for dogs and dog handlers in hazardous situations, writes Caroline Hayes

Seattle-based Command Sight has developed an AR headset for dogs. It is being tested by the US Army with the aim of distancing handlers from hazardous operations while giving the dog directions with an insight into what the dog is seeing and surrounded by. 

The project was funded by the Small Business Innovation Research programme and managed by the Army Research Office. 

The AR headset is worn by the dog which is trained to react to specific visual cues that are displayed on the googles. The handler can see everything that the dog sees, such as obstacles as well as dangers, and can issue commands to the dog through the glasses.

The dog uses the AR to receive cues and commands, whereas a human wears an AR headset for interaction. For humans, AR and virtual reality (VR) is used to recreate real-life scenarios for training and in the field where soldiers can access advanced information systems while moving. Soldiers also use AR googles to display overlays for target identification and assistance, view and path finding and information gathering in the field.

For the canine AR system, however, a camera streams the environment to a display which can be viewed by the handler. This allows the handler to identify an area, or particular object for the dog to investigate more closely. A visual cue, such as a light beam or illumination, is relayed to the dog via the headset to highlight the object or area of interest.  

Command Sight’s founder and CEO, A J Peper created the prototype using his rottweiler, Mater as a test model. He said that the initial research results were “extremely promising”. “[Mater’s] ability to generalise from other training to working through the AR goggles has been incredible. We still have a way to go from a basic science and development perspective,” he added. 

When searching for explosives, the handler is typically close to the dog, guiding with hand signals.

In typical man-dog explorations, the handler directs the dog with hand signals, so the handler has to be within sight. Laser pointers are also used, as visual cues for the dog to follow, but even these require the handler to remain close to the dog as it works. It also generates a light source which can be a safety issue, points out the USA Army.

In some instances, a camera and walkie-talkie are strapped to the dog’s back using a harness. This allows extra distance between the dog and its handler but the dog can be confused by verbal-only commands, for example running around a staircase inside of going up the stairs.

The AR headset uses googles by Rex Specs which are already used by military working dogs to protect them against weather conditions or dust and particles near helicopter landings and take-offs.

Military canines are already familiar with Rex Specs’ or “doggles”.

Dr Stephen Lee, a senior scientist at the Army Research Office, explained that putting the camera in the googles means the handler can see what the dog sees. Additionally, the close fitting headset means that the camera does not bounce around like it does when it is on the back of an active dog.

The project is in the second of three phases, which is to develop a wireless version – at the moment, the wired version means that a dog lead is still used. The next round of funding will be used to build prototypes for Navy Special Forces’ working dogs. 

Each dogs undergoes a 3D scan for the fit of the headset and to plan the optimal placing of optics and electrical components for each one. 

Benefits of the AR headset is that the handler can stay at a remote distance and out of the line of sight. The dog can operate without detection and with specific direction. Command Sight the system increases the dog’s confidence with the task. “Ultimately, these additional benefits mean increasing mission capabilities and soldier and canine survivability,” said Command Sight.

Humans in the army have been using AR/VR headsets for training and information. (Picture: Microsoft HoloLens)

In 2019, the US army partnered with Microsoft HoloLens to supply 60,000 VR headsets capable of AR information inlays. The $20 billion project was paused in 2021.