Sensing with smartphones

During the last decade, the fast rise of smartphone distribution amongst the world’s population has been evident. Initially expensive devices that only business individuals could afford, smartphones are now found among all classes and age ranges of the population. Early smartphones featured only minimal internal sensors with targeted usage scenarios in mind. For example, the accelerometer was mainly built into the first iPhone to adapt the display content to the screen orientation. With the introduction of the app and app-store concept, developers started using the sensors in their own unique ways. This thesis focuses on the further extension of the sensing capabilities of smartphones and demonstrates how they can be used to sense and understand its user. First, we show that the touch screen of an Android device can be used to detect who is using it, due to the touch pattern of a person being unique. Second, we extend the capabilities by sensing the characteristics of the gait when the phone is in the pocket. The novelty of our method lies in the fact that the phone does not need to be placed in a certain orientation to achieve high recognition rates. We then widen our focus to external wireless sensors for context sensing. An important part of the context of a person may be medical parameters. We therefore present two systems that rely on custom external sensors to measure health-relevant parameters: UV radiation and foot pressure distribution. In both cases the phone acts as a general purpose computation and interaction platform. In the last part, we show that analyzing the behavior of the user can be applied to mobile music recommendations in two scenarios. To do so, we define a music similarity measure that is based on the listening behavior of many persons, as well as social tags, applicable to millions of artists and tracks. For the first application, we apply this knowledge to define the music taste of a user, based on the songs stored on the smartphone in order to extend the music collection through opportunistic ad hoc file sharing. Second, we describe the implementation of a smart meta-radio that uses thousands of publicly available Internet radio stations and seamlessly switches between them to offer a personalized listening experience.