Introduction
We may have heard of the technology of Brain-Computer Interface (BCI), which enables people to communicate with their computer devices directly from their own thoughts. But as it were, some people have had reservations about this technology - particularly the type that requires a chip to be surgically implanted into the human's brain. But hey! Another option has arisen - you can communicate with these devices using just gestures and body languages. In this survey, we will be considering the technology of Gesture Recognition.
[Gesture Recognition. Source: Wikimedia Commons. Author: Comixboy. CC BY-SA 3.0 licensed]
Concept of Gestures and Gesture Recognition
What are gestures? One may ask. To make this concept clearer, we need to start from the definition of what gesture is. Gestures; in a layman's term; are movements of some parts of the body (particularly the limbs) to communicate something meaningful without using the natural audible language. For example; when babies move their bodies in some manner, it could be interpreted by observant mothers the exact need of the baby, even without saying a word. Also, taking a look at the sign languages made by the deaf and dumb people, these can also be called gestures. And you would agree with me that they do communicate without the audible sounds. So by inference, gestures are quite different from just normal body movements; going by the examples I gave earlier. For instance; when you see someone writing on a paper with a pen; yes; his hands are moving, but this does not qualify to be called "gesture", because no meaningful message is sent to the person observing.
To this end, gestures may as well be considered as a method of human communication in a very natural form. No wonder it has been suggested by some evolutionary biologists that manual gestures preceded the audible language communication [ref]. It is not surprising that some emerging technologies have been seen to leverage on the viability of the human gestures to design systems that would enable us communicate with devices using our body movements (no physical contacts required).
Since we’ve seen what gestures are, let’s look at what “gesture recognition” is: It is the representation and interpretation of the movements generated by the human body in order to pass some certain commands to systems or any other electronic device. In other words; it is the interaction of humans with computers using body motion. People have been trying over the years to interface the human body with machines; now this is one of the ways this can be achieved. Even though majority of these gesture recognition focus on the gestures generated by the hand (tracking-hand movement), but it is worthy to note that gestures are not exclusively preserved for the hands only - they can be generated from other body parts. Maybe you can watch this YouTube video to see how a conventional laptop can be controlled using gestures only.
How do computers recognize gestures?
It is worthy to note that this is an emerging technology, and as such, advances are still made in it. But basically, there are some tools that enable the computer recognize gestures in their basic form; and these tools are also hinged on some mathematical algorithms. These tools form what is called "Perceptual User Interface", which creates a touchless User interface. But before now the only technology available for use as the touchless device is the "Cyber Glove".
The cyber glove is one of man's earliest attempts to create a touchless user interface to communicate human gestures to computers. Here's how it works. The glove is incorporated with micro sensors and some tiny fiber optics strings which are placed under each of the fingers. So, a movement of any of the fingers would create a significant amount of command, and would be interpreted as an instruction for the computers. But unlike the YouTube video I shared with you earlier; in the case of the cyber gloves; the computer only recognizes the gestures when you're wearing the gloves, and because of this, some limitations have been created and some other alternatives have been sought.
[Image Source: Wikimedia Commons. Author: Evan-Amos. Public domain licensed]
But even with these limitations, the cyber glove (otherwise called "data glove") has been majorly integrated in the technology of Mixed Reality (hybrid of Virtual Reality and Augmented Reality), with some newer models being incorporated with other technologies; like impact-detection technology (used by Virtual Reality gamers to simulate the effects of; a punch maybe). But even with all these; it is still bulky and uncomfortable to wear these gloves for an extended period of time. So what other technologies are there in lieu of cyber gloves? Let's look at this:
Two-Dimensional Gesture Recognition Tech
The 2D recognition is also known as the vision-based recognition. Which; like the video I provided earlier; uses the normal 2D camera (and some specialized software) to interpret simple hand gestures. This is done in some ways: Firstly, the 2D camera would mathematically (through the help of some software) segment the pixels surrounding the hand from the background pixels, which also measures the contours of the hand as the movement is made. The differentiation of these contours would represent a particular command. For example; a raise of the middle finger could be interpreted by the computer to mean "launch Microsoft word on the system"
[Image Source: Screenshot from YouTube Video. Author: cvsisterteam. 3 seconds From starting]
This is also the same principle adopted by the facial recognition systems (just like the app Banana Screen that enables you to lock/open your system with your face, or using a smile). But unlike the facial recognition, the 2D gesture tech also measures the direction of the hand movement (like; right, left, down, up, etc), and also some actions performed by the hands, like; grabbing, swinging, pointing, etc. But as it were, this type of gesture recognition is limited by the fact that it only supports simple hand movements. But there is another type that comes with some enhanced features.
3D gesture recognition tech
This type of gesture recognition does not only use software and mathematical algorithms, but employs the use of some 3D devices like; the depth-resolution cameras, motion sensors, projectors, etc. This majorly depends on light distribution and light waves - That is, the depth of an object (“hand”, for example) is measured by the differences in the light waves reflected off from the user's hands; and this can also be incorporated with a triangulation technology, to detect the movement of the hands to a very high precision index.
If you are a gamer, you would have probably heard of a specialized kit called Kinect Motion Controller, which works on the technology of advanced gesture recognition, and enables gamers to communicate with the gaming environment without using hardwares like Joystick, Mouse, or Keyboard, and without any physical contacts being made. But apart from the gaming applications of this, it has also been used in other consumer electronics, like in Smart CCTV cameras to pick up and distinguish between faces of people (as applied to the technology of Internet of Things; IoT).
Asides the depth-resolution camera, the 3D gesture recognition technology can also make use of a specialized infrared stereo camera (like the Leap Motion Camera) which combines the features of depth-resolution camera and infrared motion sensors for optimization.
[Multitouch Gesture Recognition. Source:Wikimedia Commons. Author: touch. CC BY-SA 4.0 licensed]
In this era we are living in; which has been majorly greeted by technological advancements, and with advances in the technology of the “Internet of Things” - In which every of our devices would be connected to a single hub, and also with the establishment of smart cities; the technology of gesture recognition would come in handy. Imagine a scenario like this: You come home after work, when you approach you doorpost, you would wave at the door - the depth resolution door camera would pick up the signals and the door opens automatically. When you're in, you swing your hand in the direction of the bulb, and the light comes on. To put on the television, no need of pressing any button, just point your fingers at the TV and it would on. And to change the channels, you only need to swing your hand. This is just a simulated scenario of what to expect when gesture recognition gets fully integrated into our everyday lives.
Conclusion
Before this era; when considering the input devices we use to give instructions to computer systems, we would be quick to mention the keyboard and the mouse, but now, all these are about to go into oblivion; as another input method has arisen - which is the gesture. And with further advances, we could be having even mobile phones working on this principle... what a time to exist in.
Thanks for reading
References for further reading:
- Techtarget/gesture recognition
- Overview of gesture recognition technology
- 2D/3D gesture recognition technology
- Gesture recognition using kinect
gif by @foundation
This post has been voted on by the steemstem curation team and voting trail.
There is more to SteemSTEM than just writing posts, check here for some more tips on being a community member. You can also join our discord here to get to know the rest of the community!
Awesome post you have here. this is a plausible development and a major breakthrough in alleviating man's stress. thumbs up to our today's scientists. "What a time to exist in"
Thanks a lot for the nice comment. You're appreciated
Very nice!
One technology worth mentioning is the project SOLI from Google. It use radar to detect movements and gestures. Seems pretty promising!
Here's the link :
Keep Steeming!
Thanks a lot for your contribution. it's very appreciated.
I'll check out the link you provided
Thanks a lot for your
Contribution. it's very
Appreciated
- samminator
I'm a bot. I detect haiku.
Are you Japanese? What else can you detect? - let me guess; Ninjutsu (nah, that's way too human for a bot. I hope you evolve soon)
Hi @samminator!
Your post was upvoted by utopian.io in cooperation with steemstem - supporting knowledge, innovation and technological advancement on the Steem Blockchain.
Contribute to Open Source with utopian.io
Learn how to contribute on our website and join the new open source economy.
Want to chat? Join the Utopian Community on Discord https://discord.gg/h52nFrV
Congratulations @samminator! You have completed the following achievement on Steemit and have been rewarded with new badge(s) :
Click on the badge to view your Board of Honor.
If you no longer want to receive notifications, reply to this comment with the word
STOPTo support your work, I also upvoted your post!