Spontaneous secure device interaction
An important factor in the set of security risks is typically the human impact.
People are occasionally careless or incompletely understanding the underlying technology.
This is especially true for wireless communication.
For instance, the communication range or the number of potential communication partners might be underestimated.
This is natural since humans typically base trust on the situation or context they perceive.
Nevertheless, the range of a communication network most likely bridges devices in various contexts.
As context, proximity and trust are related, a security scheme that utilises common contextual features among communicating devices might provide a sense of security which is perceived as natural by individuals and reduce the number of human errors related to security.
Consider, for instance, a meeting with co-workers of a specific project.
Naturally, workers trust the others based on working agreements.
Every group member needs the permission to access common information like mobile phone numbers or shared files.
Communication between group members, however, should be guarded against access from external devices or individuals.
The meeting room defines the borders which shall not be crossed by any confidential data.
Context information that is unique inside these borders, such as ambient audio, can be exploited as the seed to generate a common secret for the secure information exchange and authentication.
Mobile phones can then synchronise their ID-cards ad-hoc without user interaction and secured by their physical proximity.
Similarly, access to shared files on computers of co-workers and communication links among co-workers can be secured.
Another reason why security cautions might be discarded occasionally is the effort required and inconvenience to establish a secure connection.
This is especially true between devices that communicate seldom or for the first time.
We propose a mechanism to unobtrusively (zero interaction) establish an ad-hoc secure communication channel between unacquainted devices which is conditioned on the surrounding context.
In particular, we consider audio as a source of spatially centred context.
We exploit the similarity of features from ambient audio by devices in proximity to create a secure communication channel exclusively based on these features.
At no point in the protocol the secret itself or information that could be used to derive audio feature values is made public.
In order to do so, we generate synchronised audio-fingerprints from ambient sounds and utilise error correcting codes to account for noise in the feature vector.
On each communicating device the feature vector is then used to create an identical key.
The proposed protocol is non-interactive, unobtrusive and does not require specific or identical hardware at communication partners.
Challenges

For device pairing based on context, devices have to be in the same context at a given time. For audio, this means that the audio observed at two devices has to be related. From the audio that is then recorded simultaneously at devices, a representative is created such that the similarity is proportional to the similarity in the recorded audio. We utilised audio fingerprinting techniques in order to create binary fingerprints as representatives for recorded audio. Since these fingerprints are, due to hardware-imperfections, noise and the distance among devices, with high probability not equal for two recordings, we exploitd fuzzy cryptography techniques (basically reverse application of error correcting codes) in order map similar binary fingerprints onto identical keys. In this process, no information on the recorded audio is exchanged so that an eavesdropper is not capable of learning sufficient information to guess the fingerprint of the legitimate devices. Another challenge is the sufficiently accurate synchronisation of devices. Since the similarity of fingerprints is dependent on a strongly synchronised start of the recording, a good synchronisation of devices is obligatory. While for laptop-class devices an NTP-based synchronisation was sufficient, our experiments with mobile phones revealed further timing-related problems: Even on perfectly synchronised devices, the implicit delay to start a recording might differ for various audio hardware. Therefore, we proposed an alignment-based synchronisation method that could achieve high synchronisation without exchanging any information on the recorded audio sequence.
Contributions

- Fuzzy cryptography-based audio-fingerprinting scheme for spontaneous secure device interaction
- Study of the entropy of derived fingerprints
- Extensive case studies with various devices and in multiple environments
- Attack scenarios and cases
- Integration of the security routine into the OpenUAT library
- Implementation of the pairing scheme for android mobile phones
- Alignment-based zero-disclosure device synchronisation based on ambient audio
- Hands-free ZRTP (audio-fingerprinting based ZRTP)
Results

CAse study with laptop-class devices
In four case-studies, we verified the feasibility of the protocol under realistic conditions.
The greatest separation between fingerprints from identical and non-identical audio-contexts was observed indoor with low background noise and a single dominant audio source.
In such an environment we could distinguish devices in the same and in different audio contexts.
It was even possible to clearly identify a device that replicated dominant audio from another room with an equally tuned FM-radio at similar loudness level.
In a case-study conducted in a crowded canteen environment, we observed that the synchronisation quality was generally impaired due to the absence of a dominant audio source.
However, it was still possible to establish a privacy area of about 2m inside which the Hamming distance of fingerprints was distinguishably smaller than for greater distances.
The worst results have been obtained in a setting conducted beside a heavily trafficked road.
In this case, when the noise component becomes dominant and considerably louder, the synchronisation quality was further reduced.
Additionally, due to the increased loudness level, a similar synchronisation quality was possible also at distances of about 9m.
We conclude that in this scenario, a secure communication channel based purely on ambient audio is hard to establish.
We claim that the synchronisation quality in scenarios with more dominant noise components can be further improved with improved features and fingerprint algorithms.
Currently, most ideas are lent from fingerprinting algorithms and features designed to distinguish between music sequences.
Although algorithms have been adapted to better capture characteristics of ambient audio, we believe that features and fingerprint generation to classify ambient audio might be further improved.
Additionally, the consideration of additional contextual features such as light or RF-channel-based should improve the robustness of the presented approach.
Case study with mobile phones
When implementing an audio-based ad-hoc secure device pairing mechanism for previously unacquainted mobile
devices, the diversity of hardware and software can affect the offset in audio recordings of even clock-synchronised
mobile devices. We propose an approximative pattern matching method to align the corresponding audio without
communication between the devices. The devices synchronise their audio sequences without any knowledge about
the recorded audio on the remote device other than their own recorded contextual information. Hence, no information about the audio utilized as a seed for the secure key generation, can leak. To improve the alignment quality, we can choose more than one matching position on each device at the cost of increasing the communication load. We can obtain a synchronization among devices of less than 2 milliseconds when both devices utilize up to 10 matching positions. With 3 trials, a synchronisation in the order of 10 milliseconds is reasonable.
Ad-hoc pairing App
We introduced AdhocPairing, an android application to generate spontaneous secure keys from ambient audio of devices in proximity.
For synchronisation of recorded audio sequences the application utilises an approximative pattern matching that does not require inter-device communication to synchronise audio recordings from remote devices.
The application was instrumented in case studies with different android mobile devices and the entropy of recorded audio was studied.
Furthermore, we integrated the audio-based pairing mechanism into the OpenUAT library.
As a result, audio-based pairing mechanisms can be easily interraged with mobile app development.
The accuracy was tested in case studies and on various mobile devces.
Publications
- Dominik Schuermann and Stephan Sigg: Secure communication based on ambient audio, in IEEE Transactions on Mobile Computing (TMC), Feb. 2013, vol. 12 no. 2 (DOI)
- Dominik Schuermann and Stephan Sigg: Handsfree ZRTP - A Novel Key Agreement for RTP, Protected by Voice Commitments, in 2013 Symposium On Usable Privacy and Security, 2013 (PDF)
- Stephan Sigg, Ngu Nguyen, An Huynh and Yusheng Ji: AdhocPairing: Spontaneous audio based secure device pairing for Android mobile devices, in Proceedings of the 4th International Workshop on Security and Privacy in Spontaneous Interaction and Mobile Phone Use, in conjunction with Pervasive 2012, 2012 (PDF, slides)
- Ngu Nguyen, Stephan Sigg, An Huynh and Yusheng Ji: Pattern-based Alignment of Audio Data for Ad-hoc Secure Device Pairing, in 2012 16th International Symposium on Wearable Computers (ISWC), pp.88-91, 18-22 June 2012 (DOI, slides)
- Ngu Nguyen, Stephan Sigg, An Huynh and Yusheng Ji: Using ambient audio in secure mobile phone communication, 2012 IEEE International Conference on Pervasive Computing and Communications Workshops (PERCOM Workshops), pp.431-434, 19-23 March 2012 (DOI, poster)
- Stephan Sigg, Dominik Schuermann and Jusheng Ji: PINtext: A framework for secure communication based on context, in Proceedings of the 8th International ICST Conference on Mobile and Ubiquitous Systems (MobiQuitous 2011), 2011 (DOI)
- Stephan Sigg: Context-based security: State of the art, open research topics and a case study, in Proceedings of the 5th ACM International Workshop on Context-Awareness for Self-Managing Systems (CASEMANS 2011), 2011 (DOI, slides)
- Stephan Sigg, Matthias Budde, Yusheng Ji and Michael Beigl: Entropy of audio fingerprints for unobtrusive device authentication, in Poster Proceedings of the 7th International and Interdisciplinary Conference on Modeling and Using Context (Context2011), 2011 (Online, poster)