RF-based device-free recognition

We consider the detection of activities of device-free entities from the analysis of RF-channel fluctuations induced by these very activities. In analogy to the definition of device-free radio-based localisation systems (DFL) we define device-free radio-based activity recognition systems (DFAR) as systems which recognise the activity of a person using analysis of radio signals while the person itself is not required to carry a wireless device.

Device-free radio-based recognition can extend the awareness of wireless-enabled devices over the device boundaries. It might be applied e.g. for devices in the upcoming IoT as well as for mobile phones, laptop computers or consumer products incorporating an interface to the wireless channel.

Parameter Values
Sensor type Device-free; device-bound
Sensing modality Passive; active
Setup Ad-hoc, non ad-hoc (requires training)
Sensed value Continuous signal (e.g. SDR); Channel-Quality-Indicator (e.g. RSSI)

We categorise radio-based activity recognition systems according to the table above. In particular, we distinguish between passive and active systems depending on whether a transmitter is part of and under control of the radio-based recognition system. Also, an ad-hoc system can be installed in a new environment without re-training the classifier, while a non-ad-hoc system requires initial training or configuration.

Challenges

DFAR classes

Most work conducted in the area of RF-based classification with passive participants is related to the localisation of individuals. The feasibility of this approach was verified in various environmental settings and at various frequencies. The features utilised are mostly the RSSI, its moving average, mean or RSSI fingerprint. Also, 2-way RSSI variance was employed. With these features a localisation accuracy of about 0.5 meters was possible or the simultaneous localisation of up to 5 persons in a changing environment with an accuracy of 1 meter. While the localisation of individuals based on features from the radio channel can therefore generally be considered as solved, recently, some authors considered active DFAR approaches to also detect activities. DFAR is still a mostly unexplored research field. Open research questions regard the optimum frequencies and the impact of the frequency on the classification accuracy, the optimum sampling rate of the signal, the detection range and the impact of this distance on the classification accuracy as well as the minimum Signal-to-Noise Ratio (SNR). Furthermore, a set of activities that can be recognised by RF-based classification is yet to be identified as well as a suitable design of the detection system. In particular, the impact of the count and height of transmitting and receiving nodes has not yet been considered comprehensively as well as even the actual necessity of a transmit node as part of the recognition system since potentially the system might utilise ambient radio. Also, it is not clear whether and how activities of multiple persons can be identified simultaneously and if features exist that enable ad-hoc DFAR systems. A more detailed discussion of most of these aspects is given by Scholz et al.

Contributions


Results

Deployment

Active; continuous signal

We have proposed a non-ad-hoc, active device-free activity recognition system. For this system, we derived feasible features to detect simple activities such as walking, crawling, standing, lying. For these activities we could reach a classification accuracy that is comparable to accelerometer (device-bound) based recognition systems (compared to intertion recognition system in the same scenario). Classification was achieved by k-NN, Naive Bayes and decision tree classifiers with similar classification accuracy.

The presented work is the first to detect the considered activities from RF-channel measurements and also the first to do this with active and passive DFAR systems. Despite some recent advances on device-free radio-based localisation systems, this is also the first study to combine an activity recognition and localisation in one classification algorithm on a common set of features. For the activities lying, crawling, standing and walking we were able to localise them within less than 1 meter in USRP-SDR-based case-studies with the active DFAR system.

The results of this study effectively enable the use of arbitrary wireless devices as sensing equipment.

Simultaneous recognition of multiple activities Furthermore, we have considered the use of environmental RF-sensors to distinguish these basic activities in an indoor environment conducted simultaneously by two subjects not equipped with an antenna.

Generally, and not surprisingly, the recognition accuracy increases when more sensors are utilised. We achieved good results with three or more receive nodes (see the table on the right where four receive nodes have been utilised). Then, the 25 distinct classes of combinations of these simultaneously conducted activities could be well distinguished.



Active; RSSI-based

Recognition hardware We compared the classification accuracy of active and passive SDR-based systems and also to recognition based on accelerometer devices (device-bound) to the accuracy achieved with an active RSSI-based system implemented through sensor nodes deployed in an indoor environment.

With only a single transmitter and receiver (exchanging packets at 100 Hz), the above basic activity cases could be well distinguished. However, the accuracy acheived is lower than for SDR-based and accelerometer-based system.

The main challenge in an RSSI-based system is that the granularity and sample rate of RSSI is lower than for the SDR-based system. As a consequence, only time-domain features are feasible.

Passive; continuous signal

Context prediction algorithm The above mentioned active approaches reqire a transmitter as part of the recognition system. These systems can reach best performances since the signal used for recognition is also under the control of the system. However, the deployment effort and cost are higher for such kind of approaches.

As an alternative, we investigated passive SDR-based DFAR systems in which signals from environmental FM radio stations are leveraged for the recognition of activities.

With this deployment and a single receiver, it was possible to reach similar accuracies as for the active SDR-based system. Also, we investigated the potential to measure attention of persons towards electronic poster frames that have the antenna integrated in their frame. We combined the detection of walking speed and location in order to distinguish attention levels.

Passive; RSSI

Context prediction algorithm We investigated the use of WiFi Received Signal Strength Information (RSSI) at a mobile phone for the recognition of situations, activities and gestures. In particular, we propose a passive activity recognition system that does not require any device carried by the user.
We discussed challenges and lessons learned for the design of such a system on a mobile phone and propose appropriate features for extracting activity characteristics from RSSI signals. We demonstrate the feasibility of recognising activities, gestures and environmental situations from RSSI information obtained by the mobile phone.

The case studies were conducted during a period of about two months in which about 12 hours of continuous RSSI data has been sampled, in two countries and with 11 participants in total. Results demonstrate the potential to utilise RSSI information for the extension of the environmental perception of a mobile device as well as for the interaction with touch-free gestures.

The system achieves an accuracy of 0.51 while distinguishing as many as 11 gestures and can reach 0.72 on average for four more disparate ones.

Links

Publications