Public:Graphical User Interface

From Wiki Furtherium

Abstract

→redirect Further information: Public:ARHUDFM Manifesto, Public:Applications, Public:ARHUDFM Features Summary

Augmented Reality Heald-Up Display Fullface Mask (ARHUDFM) is a complex device that is not easy to understand at a first look, because there are little relevant examples to compare. Therefore, we suggest that you first read the information on the Public:ARHUDFM Manifesto page at the link above.

This page describes the structure and contents of the GUI (Graphical User Interface) used in the ARHUDFM device.

For a brief description of the functions of each application, please see the Public:Applications page.

UI example figure 1.1

Click on the picture to zoom in

Interface description

Appearance

Left Center Right
LMMP Left main menu fading pads SBP Status bar fading pads RSBP Right submenu fading pads
LQMP Left quick menu fading pads CLW Central left window RQMP Right quick menu fading pads
LFSP Left features sidebar fading pads CRW Central right window RFSP Right features sidebar fading pads
LUW Left Upper window CWJP Central window joystick pads RUW Right Upper window
LTAW Left Top A window CWBP Central window buttons pads RTAW Right Top A window
LTBW Left Top B window RTBW Right Top B window
LBAW Left Bottom A window RBAW Right Bottom A window
LBBW Left Bottom B window RBBW Right Bottom B window

Features

The interface is designed for the screen of the ARHUDFM device with FOV 105°h (stereo overlap) and 33°v, has a resolution of 3840px in width and 1200 px in hight.

Normal human stereoscopic binocular vision is 180° horizontally, up 55°, down 60°.[1] If the area of ​​peripheral vision important for situational awareness is excluded, a comfortable field of view, especially if a person wears optic corrective glasses, will be 105-115° horizontally, 20° up and 25° down. Since the airway obturator of the mask from below is an obstacle to vision, the field of view of a person wearing a mask from below is less than usual (however, pilots of combat aircraft have sufficient visibility). Horizontally, the mask user has a complete view of the entire field of view, including the peripheral part, which is important.

The visible field is projected by the LED beams of a DLP projector offering high contrast and better color reproduction with high response.

Unlike the harmful effects of the blue light spectrum (light colors contain a lot of blue) emitted by the LED screen of a monitor or mobile device, which causes fatigue, headaches and sleep problems when exposed for a long time, the reflected light of a DLP projector (and other types of projectors also) does not have such disadvantages. In addition, in the ARHUDFM device, the principle of displaying the interface resembles the night mode (dark background, light graphic symbols). These conditions for the user create the possibility of round-the-clock work without stress.

The constant optical focus is set to an apparent distance of 1900 mm (75 in) from the user's eyes, which prevents conflict of vergence and accommodation so that the user's eyes do not get tired during long-term use. No virtual visual objects have a different image depth, just as if the user were to observe the display screen at a distance of 1900 mm while observing the rest of the environment with the focus oriented to infinity. This statement is based on numerous studies and experiments of different laboratories with a large number of respondents.

The tension of the muscles of the eye due to the close distance to the object in question causes fatigue, including being one of the causes of the development of myopia. Neck muscle tension and stoop due to the need to focus on a fixed part of field of view for a long time is also an unfavorable factor. This applies to all cases of human work with the screens of laptops, mobile devices, reading paper books and documents.

To imagine the actual visible edges (not the focal length) of the screen, sit in front of a 65" or 1440mm (57.3 inches x 32.9 inches) TV screen at a distance of 22" or 550mm from your eyes. You can clearly see the right and left sides of the screen by turning your eyes. You don't need to use the peripheral part of the vision at the edges, it remains completely open due to the wide field of view of the device's visor. Only in the lower part, when using the airway obturator of the device, the field of view of the user is limited to about 20°, as is the case with any respirator or when wearing most models of glasses. Glasses tend to shrink the effective field of view to somewhere between 90 and 115 degrees horizontally and 35-45 degrees vertically, depending on the width and hight of the glasses and their proximity to the eyes.

We've added these comments because most people don't have experience with these kinds of AR devices yet. Microsoft IVAS (Hololens 2) uses holographic optics, thick lenses and FOV 43°h 28°v, Magic Leap has similar technology to Microsoft and FOV 45°h 55°v. Apple Vision Pro is a VR device (opaque LED display) like the Meta (Oculus) Quest - FOV 90°h 98°v[2], HTC Vive, SONY Playstation, etc. None of these technologies or devices are like ARHUDFM and when used in tactical tasks carry many significant risks. More about it here.

The main feature of Augmented Reality devices is that a person can only see lighter areas on a darker screen background. The nature of vision is that a person sees not objects, but the light reflected from them. Those, can see a high-contrast image in a darkened room (Microsoft uses a tinted visor) or at night, but see almost nothing against a bright sky, on a sunny day, with strong reflections from snow or building walls, when looking at the sun. A person will not be able to see dark areas on a light background in principle, since this contradicts the physics of light (dark areas are the absence of light). The ARHUDFM solves this problem by using "smart windows" - the interface contains 10 large and 56 small fading pads that instantly change their transparency from 0% (opaque black) to 100% (fully transparent) depending on the ambient light or the mode selected by the user.

Principles

Since the user interface of the ARHUDFM device has more functions than is usually used by users of computers and mobile devices, and the device is such that it can be used for a long time in tactical conditions, the following principles are observed:

  1. As small screen area as possible should obstruct the view of the actual visual environment at all times
  2. The user must maintain the minimum necessary view of the visual environment for orientation in space
  3. If any element of the interface is not directly needed by the user at the moment, it should not be displayed on the screen
  4. The user should be able to change the interface configuration very quickly
  5. The user must be able to simultaneously clearly distinguish between interface elements and the visual environment at any brightness.
  6. Interface elements must not be visible from the outside of the device or create light emission to the outside
  7. The user's face must not be illuminated by reflected radiation, must not be visible from the outside at the night
  8. The audio environment should not have an attenuation greater than 10 dB SPL, except when using the noise reduction features

Interface controls

ARHUDFM multimedia location

Right hand hardware controls

  • Joystick. Cursor movement, drone and robot control
  • Device on/off button
  • Headphone mode control button (see Status bar)
  • Button "- >". Decrease, right arrow, PgDown.
  • Button "< +". Increase, left arrow, PgUp.
  • Button C. Click, left control button
  • Button D. Context menu, Shift, right control button

Left hand hardware controls

  • Headlamp (Spotlight) on / off button
  • Headset connector for handheld portable / manpack radio (also ground / vehicular radio headset cable)
  • Loudspeaker on / off and control button
  • Button "< -". Decrease, left arrow, PgDown
  • Button "+ >". Increase, right arrow, PgUp
  • Button A. Click, left control button, headset PTT button for radio
  • Button B. Context menu, Shift, right control button, Photo, other assigned functions
  • Buttons A+B simultaneously start / stop video record

Software controls

  • Voice assistant
    • Machine learning (AI) algorithms are used to recognize the meaning of user commands, not necessarily the exact pronunciation of the voice command, including the ability to adapt to the voice and pronunciation of the user
  • Gesture control
    • The stereo camera captures the image of the hands and recognizes the user's gestures (more details on the Hand Tracking page)
    • Gesture library constantly updated, Machine Learning allows the user to create custom gestures
    • Gestures can indicate not only the state, but also the dynamics, for example, smoothly change the volume level, brightness, etc.
    • Capturing and recognizing fingers allows to use software interface controls, including scrollbars, buttons (and a virtual keyboard if necessary), swipe sideways and up or down, scale and rotate screens inside a window, hide and move windows

Perception hardware

The advantage of this wearable and inexpensive device for tactical use by almost every military person is the ability to perceive light, sound and radio waves, inaccessible to humans.

Cameras

  • Image capture with a three-sensor front stereo camera. One objective lens transmits the image to the separation prisms and then to 3 sensors: DSLR IR-cut 4K/8K/12K UHD, SenSWIR, HDR
  • Separate camera in the range of LWIR (thermal image vision)
  • Separate rear view camera with extended visible light and SWIR
  • Folded optical lenses design
  • Built-in Speed ​​Booster reduces focal length and increases aperture for individual sensors
  • Built-in teleconverter, on the contrary, increases the focal length for other sensors, increasing the image of distant objects
  • IR filter laser protection lens coating for 700-1080 nm (against systems for remote detection of optical and optoelectronic devices)
  • Infrared and laser light sources are not used in this device, because this demasks the user


Microphones

  • Built-in MEMS (Micro-Electro-Mechanical-Systems) microphone (1) in mask obturator airway to transmit the user's voice
    • If the lower part of the mask is not used, a boom microphone (2) with a windproof pad on the flexible goosneck is connected
  • 19 external MEMS cardioid microphones, "flat/dome" acoustic arrays allow to perceive sound waves in a wide range of frequencies, including those inaudible to humans, from very low to very high, used in Computer Audition modes (CAC)
  • The super-cardioiod and ultra-cardioid microphones with a narrow directivity pattern on the front of the mask allow to accurately determine the source of the sound wave
  • 2 insulating layers, airway obturator and seal around face make the user's voice almost inaudible from the outside

Radio antennas

  • Built-in transmit-receive antennas in the ranges of the near radius of communications (Bluetooth, Wi-Fi, LTE, 5G)
  • Integrated manpack antennas for communications in the VHF, UHF, UHF SAT, MUOS, L/S bands
  • Integrated manpack antennas for analyzing and listening to the frequency ranges 2 MHz - 8 GHz
  • Not every user needs a separate antenna or a set of several antennas with different characteristics, using the P2P network within the group, packets of messages and received data can be exchanged and routed
  • Integrated passive radar antenna with active phased array (installed in NVG mount)
  • Integrated handheld portable short-range active metal re-radiation radar antenna
  • Integrated receiving antenna for radar warning system
  • Integrated antenna for remote control of a drone or robot
  • Integrated antenna for close-range remote control of an active phased array radar (portable or vehicle-mounted)
  • Software-defined (reconfigurable) antenna (installed in NVG mount and the other two via the picatinny rails on the sides of the helmet)
  • The software-defined antenna is capable of combining the functions of most integrated antennas


The specific is that when using a separate optical lens for each sensor on the left and right, there is a problem of parasitic multi-angle, which will have to be constantly leveled by software, which affects the accuracy of binocular stereo vision. If in a smartphone this is not critical, and there may be 4-8 cameras with their own optics, then for tactical tasks this is of great difficulty. In this case, on the left and right, it is necessary to accurately hold the horizontal distance between the lines of sight for each pair of sensors and position them vertically, and then programmatically compensate for the vertical shift in order to reproduce a single optical axis for the left and right channels. In addition, the design of the device should allow this to be done without difficulty, including taking into account the angle of the field of view of the optics and solving the problem of shadowing.

Menu items description

→redirect Further information: Public:Applications, Public:Voice User Interface

The menu has 7 functional sections:

  • Status bar (SB)
  • Left main menu (LMM)
  • Right submenu (RSB)
  • Left quick menu (LQM)
  • Right quick menu (RQM)
  • Left features sidebar (LFS)
  • Right features sidebar (RFS)

The principle of managing menu items of some sections is the same, and some are different.

Status bar (SB)

The status bar is located in the top middle of the visible area and contains all the main and frequently used information about the use of device functions and applications.

A total of 12 items for quick selection and tracking are available in the status bar, each of which contains sub-elements responsible for specific function presets. Preset settings are carried out not through the status bar, but in the settings of services or applications. The status bar is always displayed and is not hidden. Each element is located in a separate field (fading pad).







The sub-elements of the status bar are controlled by clicking (joystick, gesture control) or by voice (the result corresponds to the click). Default states:

  • Hover state: 4 corner segments of the field frame (in the above example in the "Radio Detection modes" field with the "IFF" function)
  • Click state: frame on the border of the field (animation effect, twice 100 ms), with each click sequenced display of list elements (presets)
  • Selected (long click >2 s): vertically positioned rectangle in the lower left corner
  • Not selected (long click >2 s): no vertical rectangle in the lower left corner

Differences in voice control:

  • The command to the voice assistant contains a description of the desired action and state, so it is faster
  • The voice assistant understands different interpretations of actions and states, so there is no need to memorize voice commands

SB indication and change of display

Indication Designation Field Change of display / Short description
Time / Date 1.
  • Selected / Not selected: not applicable
 Battery 1.
  • Not selected: Battery Saver (BSVE) disabled - displays levels 40-60-80-100% - azure color
  • Selected: BSVE enabled - <40% - yellow color, <10% - red color
 Water 1.
  • Not selected: Water Saver (WSVE) disabled - displays levels (12 bars) >5/12 - azure color
  • Selected: WSVE enabled - <5/12 - yellow color, <2/12 - red color
 Spotlight 1.
  • Not selected: disabled - displays the intensity level of the last use (5 divs: 20-40-60-80-100%) - azure color
  • Selected: enabled - displays the current intensity level - yellow color
    • in flickering mode - displayed with flickering
    • in strobe mode - red
 Brightness modes 2. States are the default. Selecting a mode changes the brightness and fading of the fields (fading pads and all windows at the same time, it acts as sunglasses). For some apps, the app window fading settings are adjusted individually in the app for greater contrast, these settings take priority over the default settings
Windows modes 3. States are the default. The user can quickly and flexibly manage the window configuration for each window individually, see below Windows items description
Camera modes 4. States are the default. Depending on weather conditions and time of day, indoor or outdoor user activity, a quick preset selection is needed for effective surveillance, detection, and task execution
Audio modes 5. States are the default. Allows the user to quickly turn the loudspeaker on or off (as well as the buttons on the left headphones ear cup). Adjust the loudspeaker volume and monitor its status

Also allows to quickly adjust the mode and volume of the headphones (as well as the buttons on the right headphones ear cup). At least one mode and one headphone volume level are always on

Computer Vision modes 6. States are the default. Presets are available to the user for quick selection depending on tactical tasks. Presets can be turned on several at the same time and separately, which makes it possible to optimize the used computing resources and power consumption
Computer Audition modes 7. States are the default. A person hears sound as several frequencies (frequency carriers) at the same time and different frequencies at different volume levels. Presets are available to the user for quick selection depending on tactical tasks. Presets can be turned on several at the same time and separately, which makes it possible to optimize the used computing resources and power consumption
Radio Detection modes 8. States are the default. Any mission is associated with the tasks of SIGINT, identification of friend or foe, detection of drones and robots, anti-personnel and anti-tank mines, trip wires, IEDs, and counter-radar protection for each military person without exception. Presets are available to the user for quick selection depending on tactical tasks. Presets can be turned on several at the same time
Firing Assistance modes 9. States are the default. This mode helps to conduct accurately aimed fire using all types of sights: Red Dot, telescopic sight, and digital sight and is also compatible integration with the remote control of the robotic turret
Stealth modes 10. States are the default. This mode is necessary to limit visibility by means of enemy SIGINT
Vitals Body Tracking modes 11. Vitals Body Sensors Control (VBS)
Networks modes 12. Networks (NET)

SB Presets

Indication Field name Preset Preset name Actions
Time / Date, Battery, Water, Spotlights modes na Time / Date
  • each click alternates the display of the current time and date, nothing else
na Battery
  • each click, the display of the indication of the battery and the water supply of the drinking system alternates
  • in the Selected state, launches the Battery Saver (BSVE) application, which allows to optimize power consumption
  • at <40%, the BSVE application starts automatically, at <10%, only the minimum necessary functions and applications function
na Water
  • each click, the display of the indication of the battery and the water supply of the drinking system alternates
  • when in the Selected state, launches the Water Saver (WSVE) application, which allows to hydrate on a schedule (text and voice user notifications)
  • at level <5/12, the WSVE application starts automatically, at <2/12 the hydration mode changes (interval increases, monitoring of vital signs indicators is turned on, a message is sent to the cloud system)
na Spotlight
  • alternative to a long click - a long press of the button on the left headphones ear cup turns on, and when you click again, turns off the flashlight
  • when click (or +/- button on the left headphones ear cup simultaneously with the flashlight control button) switches the light intensity (5 options)
  • when simultaneously pressing the B button (on the left headphones ear cup) + the flashlight control button (on the left headphones ear cup) with one hand, it turns on the flickering mode (SOS)
  • when simultaneously pressing with both hands the D button (on the right headphones ear cup) + the flashlight control button (on the left headphones ear cup) turns on the strobe mode
  • after turning off, when turned on again, it turns on the constant light mode and the intensity of the last use
  • when the full invisibility mode is on, the user will not be able to turn on the flashlight
  • in battery saving mode, the intensity will be reduced automatically or automatic shutdown will occur
  • fine adjustments are made in the LED Spotlight settings (SPOT) app
 Brightness modes

OR condition when selecting presets (selecting one preset cancels the selection of the previously selected preset)

AUTO Auto
  • the light sensor detects the ambient light level and uses the settings of one of the following modes
  • fading windows 100% in all full-color projections
CLR Clear day fading pads 50%, fading windows 50%, projector beam 100%
CLD Cloudy fading pads 25%, fading windows 25%, projector beam 75%
TWL Twilight fading pads 0%, fading windows 0%, projector beam 75%
DRK Dark night fading pads 0%, fading windows 0%, projector beam 50%
NLTS Night lights fading pads 50%, fading windows 25%, projector beam 75%
DZLD Dazzling day fading pads 75%, fading windows 75%, projector beam 100%
Windows modes

AND / OR condition when selecting presets (selecting one preset in one case cancels, and in the other case does not cancel the selection of the previously selected preset)

DFLT Default displays all active windows according to the configured application and feature priorities; DFLT - deselects any previously selected preset
OFF Turn off all windows turns off the display of all windows; OFF - deselects any previously selected preset
CLW Central left window is On, central right window is Off if both central windows (left and right) were used by default, disables the central right window, all functions remain available in the central left window; CLW cancels CRW and vice versa
CRW Central right window is On, central left window is Off as in the previous case, on the contrary, disables the central left window, improving the view of the environment; CRW cancels CLW and vice versa
BWON Bottom windows only turns off the upper windows (except LUW and RUW) on the left and right to improve the view ("knight's narrow visor effect"), transfers higher priority windows to the lower area, closes lower priority windows, does not turn off the central windows; BWON cancels LWON and RWON
LWON Left windows only turns off all windows on the right (except RUW) ("left eye effect"), transfers higher priority windows to the left area, closes lower priority windows, does not turn off central windows; LWON cancels RWON and BWON
RWON Right windows only turns off all windows on the left (except LUW) ("right eye effect"), transfers higher priority windows to the right area, closes lower priority windows, does not turn off central windows; RWON cancels LWON and BWON
Cameras modes

AND / OR condition when selecting presets (selecting one preset in one case cancels, and in the other case does not cancel the selection of the previously selected preset)

FHD Frontal HD cam uses visible light range (400-780 nm) and dual sensor capabilities, high resolution and wide dynamic range under strong contrast lighting
RHD Rear HD cam turn the rear view camera on and off
SWIR Night / Sky / Fog / Sea vision mode uses a wider short-wavelength infrared light range (780-1700 nm), which is used to observe the reflection of ambient light in this range by objects located on the surface of the water, in fog, in smoke, in clouds, at dusk and at night
LWIR Thermal image vision mode uses long-wavelength invisible infrared light (7.5-14 µm) emitted by objects warmer in relation to the environment, mainly at night
SLV Silhouette vision mode turns on the transformation of the fill into a color outline mode that is well distinguishable in relation to the background for contrasting objects
MXV1 Mixed vision mode Type 1: FHD + SWIR allows to use the capabilities of three different sensors in different conditions day and night; 4K UHD + HDR + SWIR
MXV2 MXV Type 2: SWIR + LWIR allows to use the capabilities of two different sensors in different conditions day and night; SWIR + LWIR
MXV3 MXV Type 3: FHD + SWIR + LWIR allows to use the capabilities of four different sensors in different conditions day and night; 4K UHD + HDR + SWIR + LWIR
MXV4 MXV Type 4: FHD + SWIR + LWIR (silhouette mode) allows to use the capabilities of four different sensors in different conditions day and night; 4K UHD + HDR + SWIR + LWIR (silhouette mode)
Audio modes

AND / OR condition when selecting presets (selecting one preset in one case cancels, and in the other case does not cancel the selection of the previously selected preset)

LSPK Loudspeaker On/Off
  • alternative to a long click - a long press of the Loudspeaker control button on the left headphones ear cup turns on, and when you long click again, turns off the loudspeaker
  • the volume is changed with one hand with the +/- buttons on the left headphones ear cup at the same time as the loudspeaker power button (when it is turned on)
  • after turning off when turning on again sets the volume of the last use
  • in full stealth mode, the user will not be able to turn on the loudspeaker
L Left On, right Off
  • alternative to a long click - a long press of the Headphones control button on the right headphones ear cup switches only between L, R, LR modes; L cancels R, LR
  • the soundproof flap in the left headphones ear cup is closed
  • the volume is changed by selecting presets 25, 50, 75, 100 or using the +/- buttons on the right headphones ear cup
  • after switching when selected again sets the volume of the last use
R Right On, left Off
  • see above; R overrides L, LR
  • the soundproof flap in the right headphones ear cup is closed
LR Left On, right On
  • see above; LR cancels L, R
  • the soundproof flaps in both headphones ear cups are open
25 25% sound level changes the headphone volume; 25 cancels 50, 75, 100
50 50% sound level changes the headphone volume; 50 cancels 25, 75, 100
75 75% sound level changes the headphone volume; 75 cancels 25, 50, 100
100 100% sound level changes the headphone volume; 100 cancels 25, 50, 75
Computer Vision[3][4] modes

AND condition when selecting presets (selecting one preset in one case does not cancel the selection of a previously selected preset)

OD Object detection
  • OD based on a trained neural network allows to detect people, equipment, drones, artificial structures from different angles and at different distances, highlighting them on the screen, measuring distance, azimuth, elevation, height above the ground, speed, course destination and geolocation, as well as displaying them on the navigation grid
  • launches the CVC app in the CRW window by default at the same time as the last used front camera preset (FHD, SWIR, LWIR, SLV, MXV1-4)
  • if necessary, prompts the user to select another front camera preset, as well as launch Computer Audition and Radio Detection presets to refine detection results
  • uses local pattern library only
  • can be run simultaneously with MD, FD, FR, OCR, HSTD presets
MD Motion detection
  • MD helps to detect moving objects, people and evehicles ignoring the flights of birds and insects, tand he movement of clouds. MD mode is recommended to be used in conjunction with Radio Frequency Drone Detection (RFDD) and Computer Audition Detect noises of aircraft motor (DAM) preset
  • uses local pattern library only
  • can be run simultaneously with OD, FD, FR, OCR, HSTD presets
FD Facial detection
  • FD incl. using LWIR camera sensor and trained neural network helps to detect people's faces
  • uses local pattern library only
  • can be run simultaneously with OD, MD, FR, OCR, HSTD presets
FR Facial recognition
  • FR using a neural network in the presence of an exchange with the server allows to recognize people's faces for identification (part of the database can be stored on a local drive)
  • always executed at the same time as the FD preset
  • can be run simultaneously with OD, MD, OCR, HSTD presets
OCR Optical character recognition
  • OCR allows to recognize text and other graphic characters for subsequent translation and documentation
  • uses local pattern library only
  • can be run simultaneously with presets OD, MD, FD, HSTD
HSTD Heat and smoke trace detection
  • HSTD helps detect aircraft and missile trajectories, artillery and mortar fire from closed masked positions
  • uses local pattern library only
  • can be run simultaneously with presets OD, MD, FD, FR
 Computer Audition modes

AND condition when selecting presets (selecting one preset in one case does not cancel the selection of a previously selected preset)

DAM Detect noises of aircraft motor
  • DAM, unlike DHUM, works only with patterns of aerodynamic, electromechanical and mechanical noise, using a trained neural network, isolating and amplifying sounds and noises that are most likely associated with the operation of drones, helicopters, turboprops and jet aircraft
  • this preset allows to focus on likely threats from the air and determine the direction of the source of sound waves
  • DAM is recommended to be used in conjunction with Radio Frequency Drone Detection (RFDD) and Computer Vision Motion Detect (MD)
  • uses local pattern library only
  • can be played simultaneously with ENAT, ECHO, DHUM, STHR presets
ENAT Exclude sounds of nature
  • ENAT sequentially selects the sounds and noises of nature (wind, rustle of leaves, birdsong, water sound, etc.) in several frequency ranges with the help of filters, then using a trained neural network to select patterns for final filtering
  • this preset allows to focus on the sounds and noises associated with human activity
  • uses local pattern library only
  • can be played simultaneously with presets DAM, ECHO, DHUM, STHR
ECHO Eliminate echo
  • ECHO, using built-in algorithms, allows to detect primary and reflected sound waves in order to correct the direction of the source of sound waves when other presets are working
  • prompts the user when to enable this preset if one of the other presets is already enabled
  • uses local pattern library only
  • can be played simultaneously with DAM, ENAT, DHUM, STHR presets
DHUM Detect noises of human activity
  • unlike ENAT, DHUM works in the opposite direction, using a trained neural network, to isolate and amplify sounds and noises that are most likely associated with human activity (tire friction, footstep sounds, stone friction, radio speaker sound, engine sounds, electromechanical noise, coughing, sneezing, speech, etc.)
  • this preset highlights these patterns of audio frequencies, amplifies them, while attenuating the sound level of the environment
  • this preset allows to focus on the sounds and noises associated with human activities, and determine the direction of the source of sound waves
  • this preset also uses the method of acoustic location and determining the coordinates of the enemy battery based on the sound of firing its guns (or mortars, or rockets)[5]
  • uses local pattern library only
  • can be played simultaneously with DAM, ENAT, ECHO, STHR presets
STHR Limit sound threshold
  • STHR assists the user in variable high ambient sound pressure conditions by dynamically reducing the maximum level by ∆ 80 dB SPL to 15-85 dB SPL
  • STHR is synchronized with the electromechanical sound dampers in the headphone design
  • can be played simultaneously with DAM, ENAT, ECHO, DHUM preset
GFL Gunfire Locator is a system that detects and conveys the location of gunfire or other weapon fire using acoustic[2], vibration, optical[3], or potentially other types of sensors, as well as a combination of such sensors. These systems are used by law enforcement, security, military, government offices, schools and businesses to identify the source and, in some cases, the direction of gunfire and/or the type of weapon fired.
Radio detection modes

AND condition when selecting presets (selecting one preset in one case does not cancel the selection of a previously selected preset)

IFF Identification Friend-or-Foe
  • IFF uses a transponder (transmitter-responder) Identification Friend-or-foe system, which in response to an encrypted interrogation signal received on one frequency, sends an encrypted message packet on another frequency containing the confirmation "Friend", Id, callsign and position
  • when used simultaneously with one of the "Stealth" modes presets, IFF will send a response message at different frequencies, in different networks, and with different power (for a limited distance) or will not send a message at all
  • a transponder with a specified interval at certain frequencies periodically generates and sends interrogation signals to an indefinite circle of people for registration and display on the navigation grid of one of the screens of user positions, with confirmation "Friend"
  • interrogation signals sent via an omnidirectional antenna or a directional antenna (the omnidirectional pattern can be limited by a reflective array or a software-defined antenna can be used)
RDF Radio Direction Finding[6]
  • RDF applies the radio direction finding of received radio frequencies that are identified as potentially threatening (radar, EW&SIGINT, ECM systems) or identified as the source of an RF carrier (analog or digital signal) for communication
  • using triangulation in collaboration with a similar device of another user in the P2P network (at a short distance with it) determines the direction and distance to the radio signal source
  • without the help of another user's device or software-defined antenna determines only the direction (course) to the radio signal source
  • commonly used after running the SDR Scan (SDRS) app to first identify the characteristics of the radio signal and not unmask itself unnecessarily
  • in accordance with the current rules and restrictions, the mission can initiate a task for the above IFF app if the interrogation signal is not sent to identify the source of the signal by other methods or with the help of other persons
  • in conditions of distortion or jamming of radio signals of navigation satellite systems, it is used for radio navigation using terrestrial radio beacons operating in other frequency bands, including FM, TV and mobile radio broadcasting bands
RFDD Radio Frequency Drone Detection[7]
  • RFDD enables the mode of radio frequency detector used to detect the presence of RF waves in physical transmission mediums
  • the system uses these RF Detectors to detect drones and drone pilots (land, waterborne and aerial drones)
  • RFDD also uses AI to identify the type of threat by comparing different frequency patterns
  • the system can easily distinguish drones from common RF signals by using learned patterns and can identify almost all types of threats as well as the location of the drone pilot
  • can use a compact omnidirectional mobile manpack antenna, a directional software-defined antenna, or a longer-range mobile portable antenna
  • has a range of 9 kHz - 8 GHz and a detection distance of up to 3 miles (conical or dome range)
  • this function displays on the navigation grid and / or on the map of the terrain of the interface screens the positions of the identified RPA (UAV) and its operator
  • RFDD mode - a special case of the RDF function, it is recommended to use it in conjunction with the Computer Vision Motion detection (MD) and Computer Audition Detect noises of aircraft motor (DAM) modes
EODD Radio Frequency Explosive Ordnance Disposal Detection
  • EODD - uses an external handheld Metall Re-radiation Radar (MET3R) device, via a Bluetooth or Wi-Fi Direct connection
RWR Radar Warning Receiver[8]
  • RWR - turns on the mode of detecting exposure from air defense radars and counter-battery radar[9], displays on the screen in a window with a navigation grid and/or a map the geolocation of these radars
  • RWR mode is indispensable when performing tactical tasks of mortar and artillery fire
 Firing assistance modes

OR condition when selecting presets (selecting one preset cancels the selection of the previously selected preset)

GF Fire on ground fixed targets
  • captures and tracks up to 10 targets simultaneously
  • for each target calculates the ballistic trajectory at a distance based on 20 parameters
  • displays 16 ballistic calculation reference data (display can be turned off)
  • the user sees in the interface and can make manual changes (input field) to the parameters of the ballistic calculation
  • displays the aiming point taking into account corrections on the reticle relative to the crosshair with a scale in the form of digital data (Eunit and Wunit) MOA/MIL (true minute of angle/milliradian)
  • separately displays the numerical values ​​of the corrections: Elevation and Windage when using Red Dot Sights or Holographic Sights (both eyes open), the user visually compares the position of the aiming dot and corrections relative to the crosshair on the screen (point of impact) and the contour of the target - with the contour in the sight window and the sight dot (usually 2-5 MOA), such shifting the sight dot (line of sight) in a way to hit the target at the crosshair point as on the screen of the device
  • when integrated with a digital sight, it immediately displays on the screen in the form of a diamond the sighting line of the digital sight, which must be aligned with the displayed aiming point
  • evaluates the inaccuracies of the ballistic calculation in case of a miss and offers the user corrections (takes into account aiming inaccuracies only for a digital sight)
GM Fire on ground moving targets
  • additionally calculates the angular velocity for the uniform movement of the target, the measurement of the distance to the target, the course of the target and indicates corrective lead corrections
AM Fire on aerial moving targets
  • additionally calculates the angular velocity, change in azimuth and elevation, determines the nearest probable trajectory with the designation of corrective lead corrections
MF Mountain fire on fixed targets (prospective function)
  • additionally takes into account more complex parameters for calculating a ballistic trajectory in mountainous and desert areas based on an assessment of the landscape, temperatures, an assessment of ascending air currents and the mirage effect, shot history
MM Mountain fire on moving targets (prospective function)
  • as above plus additionally calculates angular velocity and denotes lead corrections
 Stealth modes

OR condition when selecting presets (selecting one preset cancels the selection of the previously selected preset)

IFF IFF visible only
  • all radio transmissions (including outgoing IFF interrogation signals) are blocked, with the exception of encrypted response messages of the Identification Friend-or-foe system transponder, including masking the transmission of a response message in the background noise frequency range of the broadcasting band (FM, TV) and using more long pauses in response intervals to interrogation signals (other radio strategies are possible to remain unnoticed)
  • this mode enables the IFF mode of the same name in the Radio Detection modes section, if it has not been enabled
SD Short distance visible only
  • only short range networks such as Bluetooth, Wi-Fi Direct (P2P) and UHF are used with limited transmission power over short distances
FULL Zero emission
  • all radio transmissions are blocked, only reception (completely passive mode), IFF mode is automatically turned off (and vice versa, when changing Stealth mode to another, IFF mode is turned on again)
  • in full stealth mode, the user will not be able to turn on the flashlight and loudspeaker
OFF OFF
  • does not restrict radio transmission
 Vitals body tracking modes

OR condition when selecting presets (selecting one preset cancels the selection of the previously selected preset)

WRK Workout mode
  • Vitals body data-driven monitoring including: HR bpm (ECG)[10] - assessment of rhythm and heart rate; NIBP mmHg (ART) - blood pressure; TEMP - body temperature; StO₂ % - non-invasive method for monitoring the level of oxygen saturation (saturation); ETCO₂ % - (capnometry) to measure the level of carboxyhemoglobin (COHb) and methemoglobin (MetHb); RESP bpm (RR) - Respiratory Rate; HYDRL - determines the amount and frequency of hydration
  • displays tips for the user in case of many unwanted risks: bradycardia, tachycardia, arrhythmia, hypothermia, hyperthermia, hypoxemia, hyperventilation, hemorrhagic shock, etc.
  • when a dangerous level of indicators occurs, displays a message to the user and sends an SOS message to the analytical center
  • keeps a log of monitoring states
TRN Training mode
  • additionally takes into account the duration of the session of training exercises and the possibility of live fire training
  • applies a programmatically set training mode for the use of automatic tourniquets
TACT Tactical mode
  • in addition to the above, if a dangerous level of acute massive blood loss occurs in one of the 8 sites protected by automatic tourniquets, it controls the "Care Under Fire" task for stop of acute bleeding and start emergency automatic fluid resuscitation
  • if there is a risk of ventricular fibrillation, it automatically starts the defibrillation procedure under ECG control (cardio resuscitation)
  • if necessary, start automatic pulmonary resuscitation using pneumatic pumps built into the design of air filters
OFF OFF
  • does not monitor
 Networks modes /

Com status AND

Signal levels Network 1, 2, 3, 4
  • displays a scale with the bandwidth level of each connected network
  • displays the status of receiving and transmitting packets
  • displays the packet transmission prohibition
  • long click launches NET application

Signal levels, upload / download transmission status

online
back soon
don't disturb
offline

Left main menu (LMM) and Right submenu (RSB)

→redirect Further information: Public:Applications, Public:Voice User Interface

Left Center Right
LMMP Left main menu fading pads SBP Status bar fading pads RSBP Right submenu fading pads
LQMP Left quick menu fading pads CLW Central left window RQMP Right quick menu fading pads
LFSP Left features sidebar fading pads CRW Central right window RFSP Right features sidebar fading pads
LUW Left Upper window CWJP Central window joystick pads RUW Right Upper window
LTAW Left Top A window CWBP Central window buttons pads RTAW Right Top A window
LTBW Left Top B window RTBW Right Top B window
LBAW Left Bottom A window RBAW Right Bottom A window
LBBW Left Bottom B window RBBW Right Bottom B window

The main menu (LMM) is located at the top left of the visible area and contains a list of sections.

Only 7 elements for quick selection in one row. Use the navigation arrows on the right to select the next row. The main menu is only displayed on hover or on a voice call. When inactive, hides after 15 seconds. Each element is located in a separate field (fading pad). Only the fields that display the section are shown.

The right submenu (RSB) is at the top right and expands the second lower level of the main menu items. Only 7 elements in one row. The arrows on the left are used for navigation. The right submenu is displayed only when one of the main menu items on the left (LMM) is active or when the submenu item is directly called by voice. When inactive, hides simultaneously with the main menu after 15 seconds. Each element is located in a separate field (fading pad). Only the fields that display the section are shown.

When an app is opened using this menu, it is displayed on the screen. If the user closes the app, it does not stop its work, but continues to work in the background, at the same time, an element with the name of this application appears in the Left quick menu (LQM) or Right quick menu (RQM) menu field to quickly return to the screen. See the sections Left quick menu (LQM) and Right quick menu (RQM) below for how to completely terminate the application.

LMM v1.0
LMM v1.0






The sub-elements of the main menu and sub-menu are controlled by clicking (joystick, gesture control) or by voice. States:

  • Hover state: 4 corner field frame segments
  • Click state: frame on the field border (animation effect, twice 100 ms)
  • Selected (long click >2 s): horizontally placed two rectangles to the right and left of the caption
  • Not selected (long click >2 s): there are no horizontally placed two rectangles to the right and left of the caption

Differences in voice control:

  • The command to the voice assistant contains a description of the desired action and state, so it is faster
  • The voice assistant understands different interpretations of actions and states, so there is no need to memorize voice commands
  • During a voice call, not the entire menu is shown, but the corresponding items of the main menu and submenu
  • The user can voice the display of the entire menu
LMM RSB Action Notice
MSG

(Messenger)

IMSG (instant messaging system)
  • displays the IMSG application in a window
  • an IMSG application usually does not require the use of a fading window
CHAT (multichannel chat)
  • displays the CHAT application in a window
  • displays in the RFS (right features sidebar) field below the icons of several recently used chats (one-to-one and group)
  • if there are unviewed messages marked as important, the chat icons in the RFS field are displayed in red (automatic sorting of such chats)
  • if there are messages that require a response, the chat icons in the RFS field are displayed in yellow (automatic sorting of such chats)
 LMM for all channels, RSB and RFS for the corresponding channel:
  • A counter of notifications of messages requiring a response is displayed at the top (yellow color)
  • Below is the number of unread messages (azure color)
MAIL (eMail)
  • displays the MAIL application in a separate window
SDR2 (SDR 2-way)
  • displays the SDR2 application in a separate window
CRPT (cryptologic control)
  • launches the CRPT application in a separate window
PTTH (PTT headset)
  • uses the built-in headset function for a handheld portable or manpack radio connected with a 4-pin cable (1-ch) or 5-pin cable (2-ch); for the PTT ("push-to-talk") function, use the A button on the left headphones ear cup
  • allows you to use a programmatic way, where instead of pressing, a voice command is used
TSK (Tasks)
  • OVDU (Overdue)
  • ONGO (Ongoing)
  • ASST (Assisting)
  • SBME (Set by me)
  • FLWG (Following)
  • CMNT (Comments)
  • DONE (Done)
 The task management system allows you to plan and control the deadline, accuracy, results, and efficiency of tasks. Contains a description, checklist, deadline, actual completion time and elapsed time (including auto-tracking of completion and accounting of elapsed time), notification of the current status, geolocation at the time of completion, task completion confirmation artifacts, attachments, comments
  • In the upper right corner, a counter of notifications about overdue tasks and with an approaching deadline is displayed
  • The number of tasks for today is displayed in the lower left corner
CAL (Calendar)
  • DAY (Day)
  • WEEK (Week)
  • MNTH (Month)
  • YEAR (Year)
 The event calendar allows you to plan and coordinate events in advance that require certain time regulations: collective online and offline communications, individual classes and trainings, meetings, training seminars, TDY (temporary duty) trips, and deployments
  • In the upper right corner, a counter of notifications of upcoming events is displayed
  • The number of events for today is displayed in the lower left corner
WGR (Workgroups)
  • TAC (Tactical)
  • ISR
  • FSUP (Fire support)
  • FAIR (Fast air)
  • MAS (Medical assistance)
  • EVAC (Evacuation)
  • LOG (Logistic)
 The submenu contains filters for groups.

You can select a member or group in the appropriate section to communicate, share data, send or request a position. CDI element (Cross-Domain Interaction)

  • The number of available group members is displayed in the upper right corner
  • The number of active workgroups the user is a member of is displayed in the lower left corner
MAP

(Maps & Navigation)

CMPS (Compass)
  • displays the CMPS application in the LUW window
  • shows azimuth and elevation in degrees
NAV (Navigation)
  • displays the NAV application in a separate window
  • allows using GPS or RDF (radio beacon triangulation) to determine the location on a local map
  • allows you to calculate distances, travel time, perform route optimization
PATH (Path tracking)
  • displays the PATH application in a separate window
  • allows you to track the distance traveled and marks points on the map with the time
  • integrated with the TSK application for automatic execution of tasks related to changing positions
PLAN (Mission planning)
  • displays the PLAN application in a separate window
  • allows to plan the mission in detail in terms of tasks (route, time, resources, targets, threats, communications, ISR tasks, logistics, necessary reports)
  • integrated with BMS (Battlefield Management System) and TSK app
  • integrated with L-STE (Live-Synthetic Training Environment)
  • significantly optimizes time and resources, improves mission efficiency
ANLS (Mission analysis)
  • displays the ANLS application in a separate window
  • allows you to analyze the performance of the mission in real time, taking into account the deviation from the mission plan by different domains and divisions
  • integrated with BMS (Battlefield Management System) and TSK app
  • integrated with L-STE (Live-Synthetic Training Environment)
  • significantly reduces risk and increases mission consistency
SINT

(SIGINT)

SDRS (SDR Scan)
  • displays the SDRS application in a separate window
  • allows you to analyze radio frequency 2D / 3D spectrum and waterfall diagram in the range of 2 kHz - 6 GHz with different modulations
  • allows you to listen to unencrypted analog and digital radio broadcasts
  • allows you to analyze patterns of radio frequencies, incl. masked in bands not used for communications
  • allows you to analyze the safest and most unloaded frequencies for communication, transponder (IFF), remote control (RC)
RDF (Radio Direction Finding Control)
  • displays the RDF app in an overlap mode in the IFF / NAV app window or in a separate window
  • allows you to perform positioning in the conditions of a distorted GPS signal through triangulation by radio beacons
  • allows you to determine the direction and distance to the source of the radio signal of interest, detected using the SDRS application
PCSR (Passive covert radar control)
  • displays the PCSR app in an overlap mode in the IFF / NAV app window or in a separate window
  • allows using a passive program-controlled antenna to receive reflected radio signals in the FM range to detect aircraft and equipment, while not detecting itself (without using the radiation of the radio signal of its own radar)
RWRC (Radar warning receiver control)
  • displays the RWRC app in an overlap mode in the IFF / NAV app window or in a separate window
  • allows using a passive antenna to detect the radiation of directional radars and determine the course in relation to them
  • integrated with the IMSG and TSK app for fast response to threats
APAR (Active phased array radar control)
  • displays the APAR app in an overlap mode in the IFF / NAV app window or in a separate window
  • allows remote control of an active radar at a safe distance
RFDD (RF Drone Detection Control)
  • displays the RFDD app in an overlap mode in the IFF / NAV app window or in a separate window
  • allows using a passive antenna to detect a radio signal between the drone and its operator, determine the distance
EODD (RF EOD Detection Control)
  • displays the EODD app in an overlap mode in the IFF / NAV app window or in a separate window
  • allows using a portable wearable metal re-radiation radar (MET3R) to detect anti-tank mines, metal anti-personnel mines (including those with remote control), IEDs and trip wires
  • integrate with CVC application for additional use of Computer Vision, especially against plastic anti-personnel mines
RC

(Remote Control)

RPAC (Drone RC)
  • displays the RPAC app in a separate window
  • allows to control different models of drones using the built-in controller (land, waterborne and aerial drones)
  • allows to use the tilt of the head as one of the manipulators (instead of the joystick) and the joystick on the right headphones ear cup to control
  • allows to use one joystick to control two manipulators
  • allows to use Hand Tracking to control software joysticks and buttons
  • allows to use a voice assistant TESS and AI (Artificial Intelligence) technology for hands-free control
  • allows to use built-in navigation without using a separate front-facing camera for navigation
RBRC (Robot RC)
  • displays the RBRC application in a separate window
  • allows to remotely control the robot, similar to controlling a drone
FTRC (Fire turret RC)
  • displays the FTRC app in a separate window
  • allows to remote control of the small arms turret
UVRC (Unmanned vehicle RC)
  • displays the UVRC app in a separate window
  • allows to remote control a vehicle, similar to controlling a drone and a robot
WIKI (Wiki)
  • MAN (Manuals)
  • RPT (Reports)
  • ART (Articles)
  • UPD (Updates)
  • TUT (Tutorial)
 Wiki systems are proven to be effective in creating and sharing the knowledge that everyone needs on an almost daily basis. This system allows for to a collection of best practices and improves the process of learning and training, as well as refreshing knowledge from time to time even from experienced users. All sorts of information about the technical arrangement and maintenance of equipment, the organization of processes at military bases, schedules, analysis materials of military experts, etc. Something that will help you become more efficient. Of course, distributed access rights, information protection, and confidentiality are in accordance with current protocols
  • The issues of security, protection against unauthorized access, confidentiality and ensuring uninterrupted access of users to the network, we intend to detail in a separate article.
  • Most likely, this process will be transferred to the developers of one of the DoD agencies (CYBERCOM) and the NSA as part of a deployment of the system (I am sure that these issues will not be within our competence)
MMD (Multimedia) REC (multimedia recorder)

PLAY (multimedia player)

  • launches the REC/PLAY app as the tab of the MMD app group window (tabs: REC, PLAY, VM, TRSL) or in a separate window
VM (Virtual mentor incl. video chat)
  • launches the VM app as the tab of the MMD app group window (tabs: REC, PLAY, VM, TRSL) or in a separate window
  • video conferencing system that allows remote data streaming with video, voice, text and file sharing
  • allows remote mentoring assistance in the context of medical care, incl. Care Under Fire, repair and maintenance of equipment, experienced mentors online to observe the training process and help correct mistakes
TRSL (Translater)
  • starts the TRSL application in the background without opening a window
  • allows, based on the AI small language model, to translate the text, taking into account the context and specialized terms
  • integrated with STT (speech-to-text) services and CV (Computer Vision) system to generate source material for translation
FILE (File explorer)
  • FAV (Favorites)
  • FLR (Folders)
  • CLD (Clouds)
  • TAG (Tags)
  • launches the FILE application in a separate window
  • the specified filters allow to quickly and easily find the files you need
  • additionally applies full text search, search by date, time, data format, size, author, etc.
PROC (Processing Control) CVC (Computer Vision Control)
  • launches the CVC app in an overlap mode in the CAM app window
  • allows to configure of presets and additional widgets for simultaneous use with other apps in the same window (overlapping) and use the streaming data of these apps
  • applies AI methods
  • allows to flexibly configure the rules for using your own, cloud and peer-to-peer (P2P network) computing resources
  • allows to configure methods and conditions for optimizing loads in different cases
CAC (Computer Audition Control)
  • launches the CAC app in an overlap mode in the CAM / IFF / NAV apps window or in a separate window
  • allows to customize presets and AI algorithms
  • allows load optimization for different apps
IFF (IFF Control)
  • launches the IFF application in a separate window
  • allows using transponder data of other users and generalized data of other detection systems (SDRS, RDF, CVC, CAC, PCSR, RWRC, APAR, RFDD, satellite data, and data of other users) in the navigation mesh mode
  • allows to automatically and manually mark the statuses of detected people, vehicles and objects
  • applies AI methods
STM (Stealth Modes Control)
  • launches the STM application in a separate window
  • allows you to configure presets for different stealth modes for the radio
FA (Firing Assistance Control)
  • launches the FA app in an overlap mode in the CAM app window
  • allows to execute real-time of ballistic calculations for small arms and mortar fire
  • allows to use all types of sighting devices
  • allows to configure presets and a widget to work with other applications in the same window at the same time and use the streaming data of these apps (drone / robot / turret cams view)
  • applies AI methods
VBS (Vitals Body Sensors Control)
  • launches the VBS app in a background ormode in a separate window
  • allows to set up presets and integration with sensors
  • allows accurate monitoring of vital signs in any position and on the move
  • based on Machine Learning algorithms and medical scientific data, it assesses the presence of dangerous and critical conditions, in the event of which it sends messages and independently controls medical devices (automatic tourniquets, difibrillator as cardio resuscitation and after support, fluid resuscitation, pulmonary resuscitation)
  • applies AI methods
ANTC (Antennas Control)
  • launches the ANTC app in a separate window
  • based on the built-in VNA antenna analyzer, determines the impedance (impedance factor), SWR (standing wave ratio for matching with the feeder path), radiation pattern, gain, and other characteristics of retractable and software-defined antenna for tuning in a specific radio session
  • advises the user on antenna tuning recommendations or controls the software-defined antenna itself
DVC (Integrated devices)
  • SGHT (Digital sights driver settings)
  • DRON (Drones and bots driver settings )
  • RADR (Radars driver settings)
  • SENS (Other Sensors driver settings)
  • launches the ANTC app in a separate window
  • allows to configure integration with external devices
  • allows to download new drivers and test functionality directly on the device to find errors
  • allows to roll back (undo) to the previous driver in specific specified cases
NET (Networks)
  • VHF
  • UHF
  • HF
  • 3/4/5G
  • WLAN
  • BT
  • P2P
  • launches a NET app in a separate window
  • allows to connect and disconnect for wireless networks, analyze network bandwidth, set network selection priorities for each type of data and for each app
  • allows real-time analysis of data exchange across networks
  • allows to configure rules for P2P networks, participants, resource allocation for multi-party computing, rules for user view exchange
SYS (System) CAM (Cameras Control)
  • launches the CAM app in a separate window or in multiple windows at the same time
  • allows to set up presets and integration with other apps
  • allows to perform digital zoom, interpolation to improve image quality and edge detection, use filters, deep processing of images and real-time video, mix channels and signals from 4 different sensors
  • dynamically detect contours and highlight them on the screen
  • calibrate and adjust the stereo camera for accurate distance measurement
  • integrated with other applications to use external cameras, Computer Vision technologies, Fire Assistant
DISP (Display Control)
  • launches the DISP app in a separate window
  • allows to customize presets and work with different apps
FPAD (Fading Pads Control)
  • launches the FPAD app in a separate window
  • allows ou to customize presets and work with different apps
MMC (Multimedia Control)
  • launches the MMC app in a separate window
  • allows to customize presets and work with different applications for multimedia devices: headphones (2), embedded microphones (11), handheld radio via built-in headset and voice software headset, loudspeaker (1)
 11 embedded microphones:
  • 6 omnidirectional (built-in airway obturator, boom microphone with a windproof pad on the flexible goosneck, front, left and right headphones ear cups, back)
  • 2 super-cardioid (front, -35 / +35° angle)
  • 3 ultra-cardioid (front, -10 /0 / +10° angle)
TESS (Voice assistant)
  • launches the TESS app in a separate window for configuration and management
  • this application is a small language model with the possibility of learning on the dataset of a specific user and many users of this platform
  • applies AI methods
HT (Hand tracking system)
  • launches the HT app in a separate window for configuration and management
  • this application is based on Computer Vision technology for recognition of the user's hands in the field of view of a stereo camera
  • has the ability to learn new gestures on the dataset of a particular user and many users of this platform
  • applies AI methods
STT (Speech-to-text settings)
  • launches the STT app in a separate configuration and management window
  • has the ability to adapt to different pronunciations
  • integrated with the TESS application for perception of text in the context of semantic categories
  • applies AI methods
VOVR (Voiceover settings)
  • launches the VOVR app in a separate window
  • allows you to speak text and commands in multiple voices
INP (Joystick and buttons settings)
  • launches the INP app in a separate window
  • allows you to configure and calibrate physical and software input devices: buttons and joysticks
GNSS (GNSS)
  • launches the GNSS app in a separate window
  • allows you to set up communication with geospatial positioning satellites (GPS, Galileo, QZSS), track positioning errors, issue messages to the user
ACC (Accounts & Sync)
  • launches the ACC app in a separate window
  • manages the user profile and authorizes it to access cloud data on the network and within the peer-to-peer network
SEC (Security)
  • launches the SEC app in a separate window
  • allows to configure: password, emergency alert, emergency SOS, lock & unlock
  • must comply with the requirements of security, data protection, established by the protocols of defense agencies and information security agencies
ADM (Admin only)
  • launches the ADM app in a separate window
  • provides access to administrators, incl. remote access, for error control, correction and user assistance, as well as for maintenance and updates
SRV (Services)
  • launches the SRV app in a separate window
  • allows administrators to configure the operation of software services and embedded devices: Time and Date, Notifications, LED Spotlight settings (SPOT), Battery saver (BSVE), Water saver (WSVE), Accelerometer, Gyroscope, Hall sensor, Barometer, Thermometer, Ambient light sensor, Humidity sensor, Gas sensor (CO, NO2), Radiation sensor
  • launches scheduled calibration, automatically sends a report to the analytical system

Left quick menu (LQM)

Displays hides (within a frame) and open app windows on the left half of the screen. To completely stop the app, you need to perform a long click and press confirmation in the notification window.

Right quick menu (RQM)

Displays hides (within a frame) and open app windows on the right half of the screen. To completely stop the application, you need to perform a long click and press confirmation in the notification window.

Left features sidebar (LFS)

Displays:

  • functional buttons of some active apps in the left half of the screen, when there are a lot of them and it is undesirable to reduce the visible area of ​​the application window
  • list of priority tasks for the TSK application
  • list of recently viewed articles for the WIKI app
  • list of peer-to-peer leads for P2P application and their real-time characteristics
  • priority channel list for WGR application

Right features sidebar (RFS)

Displays:

  • function buttons of some active apps in the right half of the screen, when there are a lot of them and it is undesirable to reduce the visible area of ​​​​the application window (for example, SDRS)
  • CHAT app dialog list and dialog filters
  • list of recent commands via the IMSG app
  • list of taken photos and videos for the PLAY application
  • list of video conferencing participants for VM application

Virtual control tools

Displays:

  • Software (duplicated and synchronized with hardware) joysticks and buttons for controlling a drone, robot, turret, unmanned vehicles
  • Software buttons for controlling cameras of drones, robots and unmanned vehicles
  • Joysticks and buttons to control the movement of positions on the map and mark waypoints
  • Multimedia control and navigation buttons

Windows items description

→redirect Further information: Public:Applications

Fading windows and pads


Pads-100
Pads-100


Left Center Right
LMMP Left main menu fading pads SBP Status bar fading pads RSBP Right submenu fading pads
LQMP Left quick menu fading pads CLW Central left window RQMP Right quick menu fading pads
LFSP Left features sidebar fading pads CRW Central right window RFSP Right features sidebar fading pads
LUW Left Upper window CWJP Central window joystick pads RUW Right Upper window
LTAW Left Top A window CWBP Central window buttons pads RTAW Right Top A window
LTBW Left Top B window RTBW Right Top B window
LBAW Left Bottom A window RBAW Right Bottom A window
LBBW Left Bottom B window RBBW Right Bottom B window

As it was already written at the very beginning of the description of the user interface, fading windows and pads are used in the device for contrast display. Otherwise, on a light and bright background of the environment, it will be difficult for the user to see images with contrast. Fading windows and pads, depending on the brightness modes, have different degrees of transparency. And in the case of bright light, they additionally perform the function of a sun screen.

App windows are placed in a larger and better viewable area. There are 12 independent windows in total (8 large 1.78:1, 2 large 1.20:1, 2 small 19.86:1), each containing control elements when the window is clicked or voice selected (selected state). A non-selected window is a passive window that only displays app graphics, window control elements can be hidden, app control elements can be hidden. Windows of different apps have different display priority, which is configured in the system settings for each application. Depending on the selected preset in the Status bar (SB), the configurations of the displayed windows can be quickly and flexibly changed by the user.

(!) Insert gallery with pictures of examples



Window subelements are controlled by clicking (joystick, gesture control) or by voice. States:

  • Hover state: thick dotted line along field border
  • Click state: frame on the field border (animation effect, twice 100 ms)
  • Selected (long click >2 s): sparse dotted line along field border
  • Not selected (long click >2 s): no dotted line

Differences in voice control:

  • The command to the voice assistant contains a description of the desired action and state, so it is faster
  • The voice assistant understands different interpretations of actions and states, so there is no need to memorize voice commands

The user can quickly and flexibly control the window configuration for each window individually:

  • The user can force close any window at any time by selecting the desired window (selected state), clicking on the "close window" icon (the app remains in the background, the app window is in a hidden mode, to reverse the action, click on the icon in the quick selection menu)
  • The user can maximize the left or right central window (CLW, CRW) to the entire area of ​​the central window (CW) (merged window will be 2.40:1) by selecting the desired window (selected state), clicking on the "maximize window" icon
  • The user can maximize the left or right window (LTAW, LTBW, LBAW, LBBW, RTAW, RTBW, RBAW, RBBW) to the entire left or right area of ​​the window (LW or RW) (merged window will be 1.78:1 x2) by selecting the desired window (selected state), clicking on the "maximize window" icon
  • When you click on the "minimize window" icon, the window returns to its original state
  • The "move window left" icon moves a window to the left, then up to the right, then to the left, changing the position of other windows
  • The "move window right" icon moves a window to the right, then down to the left, then to the right, rearranging other windows
  • The "share screen" icon allows you to send a screenshot of a window or enable broadcasting of this window for another user

Apps priority for windows



LUW CLW CRW RUW
  1. CMPS
  1. CAM (sight view, 2:1 maxi)
  2. RPAC (drone view) / RBRC / FTRC / UVRC
  3. EODD
  1. CAM + FA + CVC widgets
  1. NOTS (notifications, system messages)
  2. TIPS (user tips) + CHAT (new messages)
LTAW LTBW CWJP (left) CWJP (right) RTAW RTBW
  1. none
  2. TSK (2:1)
  3. DVC
  4. SYS
  1. none
  2. P2P
  3. WGR
  4. CAL
  5. WIKI (1:1, 2x)
  6. FILE (1:1, 2x)
  1. RPAC / RBRC / FTRC / UVRC
  1. RPAC / RBRC / FTRC / UVRC
  2. SDRS
  1. none
  2. CVC (setup)
  3. CAC (setup)
  4. FA (setup)
  5. NET (setup)
  6. SDR2 (setup)
  7. ANTC (setup)
  8. STM (setup)
  9. CRPT (setup)
  1. IMSG
  2. SDRS
  3. CHAT (1:2 maxi)
  4. MAIL (1:2 maxi)
LBAW LBBW CWBP (left) CWBP (right) RBAW RBBW
  1. CAM (RHD, (another) user view, drone view)
  2. MAP (NAV, PATH, PLAN, ANLS)
  1. IFF
  2. CA widget
  3. RDF
  4. RFDD
  5. RWRC
  6. PCSR
  1. RPAC / RBRC / FTRC / UVRC
  1. RPAC / RBRC / FTRC / UVRC
  2. SDRS
  1. RPAC (drone view) / RBRC / FTRC / UVRC
  2. CAM (RHD, (another) user view, drone view)
  3. MAP (NAV, PATH, PLAN, ANLS)
  4. MMD
  5. VBS
  1. CHAT (1:1 mini)
  2. MAIL (1:1 mini)
Window Mode Notice
Left Upper window (LUW) Usually used for a long time
Left top A window (LTAW) Used when necessary, not recommended during intense combat so as not to limit the visibility of the environment
Left top B window (LTBW) Usually used for a short time so as not to restrict the visibility of the environment
Left bottom A window (LBAW) Usually used for a long time
Left bottom B window (LBBW) Usually used for a long time
Central left window (CLW) Usually used for a short time so as not to restrict the visibility of the environment
Central right window (CRW) Usually used for a long time
Right Upper window (RUW) Usually used for a long time
Right top A window (RTAW) Usually used for a short time so as not to restrict the visibility of the environment
Right top B window (RTBW) Used when necessary, not recommended during intense combat so as not to limit the visibility of the environment
Right bottom A window (RBAW) Usually used for a long time
Right bottom B window (RBBW) Usually used for a long time

Movement of windows to the left and right

  • General rule "left to right, top to bottom" (list moving of 10 windows):
    • LTAW - left top window A
    • LTBW - left top window B
    • LBAW - left bottom window A
    • LBBW - left bottom window B
    • CLW - central left window
    • CRW - central right window
    • RTAW - left top window A
    • RTBW - left top window B
    • RBAW - left bottom window A
    • RBBW - left bottom window B
  • If the active window is on the left side and it moves to the left, it moves up the list
  • If the active window is on the left side and it moves to the right, it moves down the list in the left side, then from the left to the right in the center of the screen, then moves down the list in the right side
  • If the place where the active window is moved is not occupied by another window, other windows do not change their position
  • The other displayed window that the active window is moved down the list to is automatically moved up the list and takes the previous position of the moved active window
  • The other displayed window that the active window is moved up in the list to is automatically moved down the list and takes the previous position of the moved active window
  • Typically less than 10 large windows are displayed at the same time, so moving the active window does not move most other windows
  • If all 10 large windows are displayed, move the active window by several positions in the list causes the neighboring window to be sequentially automatically temporarily moved, which then returns to the previous position again
  • If the window is maximized and need to move, it is automatically minimized and moved according to the general rule, if necessary, it must be maximized manually again
  • The user can select two or more windows at the same time by holding down the D key (near the joystick) right headphones ear cup (similar to the right mouse button)
  • By selecting a window and holding down the C key (similar to the left mouse button) right headphones ear cup, the user can move the window or several windows at the same time with one action using the joystick - in this case, if a group of windows is moved, they will be located sequentially next to the list (even if in their previous location, they were not sequentially next to each other in the list)
  • Small windows (LUW and RUW) move only to the left or right in place of a similar small window
  • If the small window is moved to the right, the other window is moved to the left (swap)
  • If an app launches a small window while both small windows are displaying information from other applications, then the window with the app with the lowest priority is hidden in the shortcut menu (LQM or RQM)
  • If one small window closes, nothing is displayed in its place, to display the window of another app in this place, you need to click on it in the quick menu (LQM or RQM) or in the main menu (LMM and then RSB)
  • If an app launches a window that has a higher priority than an already occupied window, then the lower priority window is moved to another location, as dictated by the window and app priority rules. When closing a window with a higher priority, the window configuration returns to the previous state
  • If an application launches a window when there is no free more space to display windows, the window with the lowest priority is hidden automatically in the quick menu (LQM or RQM). When you close one of the windows with a higher priority, the previous window display state is restored. If you need to display a window with a lower priority, you need to make free space for it by minimizing one of the windows in the quick menu (LQM or RQM)

Scrolling the window by (4/5 of the window height or in rows - adjustable individually) is done using the up and down icons (scrollbar) or the joystick buttons "< +" and "->" from the bottom of the left or right headphones ear cup (similar to the PageUp, PageDown mouse buttons). Also smooth scrolling of the window with a joystick, voice command or gesture (including a virtual joystick or a separate gesture).

Window control icons, app control icons, app interface buttons located on the Left Features Sidebar (LFS) or Right Features Sidebar (RFS), virtual joysticks - refer only to the active (selected) window. For a passive window (not selected), they are not displayed. The exceptions can be customized, for example CHAT icons in RFS.

Separately for the window control icons - after the window has become active (selected), the window control icons are displayed for 15 seconds, after which they disappear (it can be in the settings disabled). To re-display them, select this window again. The active window's dotted highlight box is displayed for the duration of the window's active state.

For the not selected state you can click on another place on the screen or press the D button on the right headphones ear cup without click.

Also, forced use of some windows can act as a sunshield when the sun is low on the horizon in the line of sight. The user can move the window to the desired area and block the dazzling view of the sun without fading the entire viewing area.

Interface Animation

Interface items motion

  • common view
  • active view when working with the menu
  • the process of selecting a menu item (scrolling elements, changing the view when entering the submenu)
  • permanent (continuous) view (statuses, notifications, active menu items)
  • turning on and changing fading windows (rules, combination options in a simultaneous form)

Modern tools

  1. Qt + C++ (doc, wiki, quick start, framework modules, 6.2.4, QML, Qt Habr, Qt for Startups, Open Source Licences, download for GNL3 Open Source)
    • SaaS (sale) delivery under GPLv3 or LGPL3+ tech support + updates
  2. Electron.JS + React.JS + TypeScript
  3. MRTK-Unity is a Microsoft-driven project that provides a set of components and features, used to accelerate cross-platform MR app development in Unity. Here are some of its functions:
    • Provides the cross-platform input system and building blocks for spatial interactions and UI.
    • Enables rapid prototyping via in-editor simulation that allows you to see changes immediately.
    • Operates as an extensible framework that provides developers the ability to swap out core components.
    • Supports a wide range of platforms.
  4. Unity Mars helps solve the hard problems of AR app creation:
    • Overcome the challenges of authoring for dynamic physical environments
    • Reduce development time
    • Release cross-platform experiences with support for iOS, Android, and HoloLens

Manuals

Troubleshooting

Areas of research

Related fields

Future ideas

Further reading

See also

Public External Sections: Public Wiki Sections: Public Wiki Sections: Not-Public Wiki Sections:

Note: Unless otherwise stated, whenever the masculine gender is used, both men and women are included.


See also product details

Hardware Details: Functional Apps Details: Executive Apps Details: Service Apps Details:


References

  1. ↑ Wikipedia, "Field of view"
  2. ↑ GitHub, Pimax Forum, "HMD Geometry Database"
  3. ↑ Wikipedia, "Outline Computer Vision"
  4. ↑ Wikipedia, "Computer Vision"
  5. ↑ Wikipedia, Artillery sound ranging
  6. ↑ Wikipedia, "Radio direction finding"
  7. ↑ AARTOS, "RF Drone Detection", "Antennas"
  8. ↑ Wikipedia, "List of radar types", "Counter-battery radar"
  9. ↑ Wikipedia, "Counter-battery radar"
  10. ↑ Wikipedia, "Monitoring (medicine)"

External links

Retrieved from "https://wiki.furtherium.com/w/index.php?title=Public:Graphical_User_Interface&oldid=7260"