WiFi and Keystroke Recognition

Updated July 2018

With the 2017 WikiLeaks revelations about the CIA and its involvement in our computers and online activities, research by Kamran Ali, Alex Liu, Wei Wang and Muhammad Shahzad from Michigan State University in the United States and Nanjing University in China gives us cause to question what little remains of our privacy even further.  

As we are all aware, there is one factor in our lives that has become pervasive, the presence of WiFi signals.  These signals are present in our homes, our workplaces and in many public areas, offering us unprecedented and often "free" access to the internet.  Many of us who access WiFi, particularly in public areas, are at least somewhat concerned about the privacy of our communications, however, we rarely give thought to the idea that someone could actually be detecting every keystroke that we make.  The research by the aforementioned authors of the paper entitled "Keystroke Recognition Using WiFi Signals" have shown that this very issue should be of concern to all of us.

It's obvious that keystroke privacy is important.  Without keystroke privacy, outsiders will be able to access our most private information including passwords and other sensitive data.  In the past, research has studied the effectiveness of keystroke recognition using three main methods:

1.) acoustic-based emission approaches where each key on a keyboard produces a different sound. or the fact that acoustic emanations from different keys arrive at different surrounding smartphones at different times.

2.) electromagnetic emission-based approaches where the electromagnetic emanations from the electrical circuit underneath different keys in a keyboard are different.

3.) vision-based approaches where keystrokes are recognized using video technologies. 

For the first time, the authors of the study noted that WiFi signals can be used to recognize keystrokes, a single -based keystroke recognition system that they call "WiKey".

WiKey consists of two commercial, off-the-shelf WIFi devices, a sender (router) and a receiver (laptop).  The sender continuously emits signals and the receive continuously receives signals.  When a user types on the keyboard on the WiFi signal receiver end, his or her hands and fingers move in a unique formation and direction, generating a unique pattern in the time-series of Channel State Information (CSI) values for each key.  The keystrokes of each keyboard key introduce unique and subtle distortions in the WiFi signals which can be used to recognize exactly which key was pressed.  The authors developed a keystroke extraction algorithm that utilized CSI streams to identify which key was depressed by the user.  As background information, CSI values are already being used to detect larger scale human movements including falling, general household activities, the presence or absence of humans and estimating the number of people in a crowd.   In this research, the authors are looking at the micro-movements of human hands and fingers when they type on a keyboard, taking CSI research to a whole new level.   

  

Here is a quote from the paper showing how data was collected:

“To evaluate the accuracy of WiKey, we collected training and testing dataset from 10 users. These 10 users were general university students who volunteered for the experiments and only 2 out of them had some know how of wire- less communication. Users 1–9 first provided 30 samples for each of the 37 keys (26 alphabets, 10 digits and 1 space and 1 space bar) by pressing that key multiple times. After this, these users typed the sentence; the quick brown fox jumped over the lazy dog two times, without spaces.

To evaluate how the number of training samples impact the accuracy, we collected 80 samples for each of the 37 keys from User 10. Afterwards, this user typed each of the following sentences 5 times, without spaces; the quick brown fox jumps over the lazy dog, nobody knew why the candles blew out, the autumn leaves look like golden snow, nothing is as profound as the imagination, my small pet mouse escaped from his cage.   We asked users to type naturally with multiple fingers but only press one key at a time while keeping the average keystroke inter-arrival time at 1 second. After recording the CSI time series for each of the above experiments, we first applied our keystroke extraction algorithm on those recorded CSI time series to extract the CSI waveforms for individual keys and then extracted the DWT based shape features from each of the extracted keystroke waveforms.”

The authors evaluated the accuracy of their keystroke extraction algorithm in terms of the detection ratio, the ratio of the total number of correctly detected keystrokes in a CSI time series divided by the total number of actually keystrokes.  The authors discovered that WiKey was able to detect that a keystroke had taken place 97.5 percentile of the time with at range of between 91.8 percent and 100 percent for the ten users and that there was a 96.4 percent accuracy rate in classifying single keystrokes.  The keystrokes that were most often missed were those where there is little movement of the user's fingers, for example, when typing the keys "a", "d", "f", "i", "j" and "x", the user's hands and fingers move very little, resulting in undetected variations in CSI values.

Here are two graphs showing the accuracy of WiKey for the 26 alphabetical keys and for 37 keys (26 alphabetical, one space bar and 10 digits) on a standard keyboard:

WiKey is capable of recognizing an overall keystroke accuracy of 83.46 percent in the case of 26 alphabetical keys and 82.87 percent in the case of 37 keys including numerical keys and the space bar.  In real-world experiments, WiKey can recognize keystrokes in a continuously typed sentence with an accuracy of 93.5 percent.  

There are some limitations to WiKey.  It is currently designed for and tested with only two persons in a room (the tester and the user) and would have to be altered (trained) to deal with addition motion (i.e. having multiple people walking around in a library setting).  The authors believe that WIKey can be developed to subtract the waveforms of non-keyboard related motion.   As well, the authors tested WiKey using the same keyboard for all users with transceivers being located at the same distance and in the same direction with respect to the keyboard.  The authors also instructed the users not to move their heads or other body parts significantly when typing, however, they were allowed to make small neutral motions including arm and shoulder movements.

While there are obviously limits to the ability to detect keystrokes using WiFi at this point in time because the technology is still in its infancy, the technique developed by the authors of this paper shows that micro-gestures can be used to detect a user's keystrokes using commercially available equipment and WiKey.  In the future, the authors plan to adapt their system to deal with the real world environment where users' behaviours are less predictable and there is more background "noise".  From what these four gentlemen have proven thus far and with the ubiquitous nature of WiFi in today's world, it is only a matter of time before this technology is widely used by the world's intelligence networks, prying even further into what little remains of our privacy.