Another VR paper accepted

Our paper, “Virtual Reality: The Effect of Body Consciousness on the Experience of Exercise Sensations” has been accepted for publication in Psychology of Sport & Exercise. This is a study in collaboration with our Sport Science and Psychology colleague.


Objectives: Past research has shown that Virtual Reality (VR) is an effective method for reducing the perception of pain and effort associated with exercise. As pain and effort are subjective feelings, they are influenced by a variety of psychological factors, including one’s awareness of internal body sensations, known as Private Body Consciousness (PBC). The goal of the present study was to investigate whether the effectiveness of VR in reducing the feeling of exercise pain and effort is moderated by PBC.

Design and Methods: Eighty participants were recruited to this study and were randomly assigned to a VR or a non-VR control group. All participants were required to maintain a 20% 1RM isometric bicep curl, whilst reporting ratings of pain intensity and perception of effort. Participants in the VR group completed the isometric bicep curl task whilst wearing a VR device which simulated an exercising environment. Participants in the non-VR group completed a conventional isometric bicep curl exercise without VR. Participants’ heart rate was continuously monitored along with time to exhaustion. A questionnaire was used to assess PBC.

Results: Participants in the VR group reported significantly lower pain and effort and exhibited longer time to exhaustion compared to the non-VR group. Notably, PBC had no effect on these measures and did not interact with the VR manipulation.

Conclusions: Results verified that VR during exercise could reduce negative sensations associated with exercise regardless of the levels of PBC.



Paper on “Can the crowd tell how I feel?”

Our paper:

Can the crowd tell how I feel? Trait empathy and ethnic background in a visual pain judgment task 

has been published in “Universal Access in the Information Society” journal.

Abstract: Many advocate for artificial agents to be empathic. Crowdsourcing could help, by facilitating human-in-the-loop approaches and data set creation for visual emotion recognition algorithms. Although crowdsourcing has been employed successfully for a range of tasks, it is not clear how effective crowdsourcing is when the task involves subjective rating of emotions. We examined relationships between demographics, empathy, and ethnic identity in pain emotion recognition tasks. Amazon MTurkers viewed images of strangers in painful settings, and tagged subjects’ emotions. They rated their level of pain arousal and confidence in their responses, and completed tests to gauge trait empathy and ethnic identity. We found that Caucasian participants were less confident than others, even when viewing other Caucasians in pain. Gender correlated to word choices for describing images, though not to pain arousal or confidence. The results underscore the need for verified information on crowdworkers, to harness diversity effectively for metadata generation tasks.


Paper accepted in IEEE Access

Right before the start of the new academic year, I am pleased to announce that our paper,  “NotiMind: Utilizing Responses to Smart Phone Notifications as Affective sensors” has been accepted in IEEE Access journal.  The funny thing about this journal is that they ask us to submit a “Multimedia Abstract” along with the paper!

There you go, my very first “Multimedia Abstract”. It does the job I suppose:

And the “traditional” abstract:

Today’s mobile phone users are faced with large numbers of notifications on social media, ranging from new followers on Twitter and emails to messages received from WhatsApp and Facebook. These digital alerts continuously disrupt activities through instant calls for attention. This paper examines closely the way everyday users interact with notifications and their impact on users’ emotion. Fifty users were recruited to download our application NotiMind and use it over a five-week period. Users’ phones collected thousands of social and system notifications along with affect data collected via self-reported PANAS tests three times a day. Results showed a noticeable correlation between positive affective measures and keyboard activities. When large numbers of Post and Remove notifications occur, a corresponding increase in negative affective measures is detected. Our predictive model has achieved a good accuracy level using three different “in the wild” classifiers (F-measure 74-78% within-subject model, 72-76% global model). Our findings show that it is possible to automatically predict when people are experiencing positive, neutral or negative affective states based on interactions with notifications.  We also show how our findings open the door to a wide range of applications in relation to emotion awareness on social and mobile communication.

publication, research

Paper accepted in MobileHCI 2017

Our paper, entitled “Designing a ubiquitous sensor-based platform to facilitate learning for young children in Thailand” has been accepted in MobileHCI conference 2017.


Education plays an important role in helping developing nations reduce poverty and improving quality of life. Ubiquitous and mobile technologies could greatly enhance education in such regions by providing augmented access to learning. This paper presents a three-year iterative study where a ubiquitous sensor based learning platform was designed, developed and tested to support science learning among primary school students in underprivileged Northern Thailand. The platform is built upon the school’s existing mobile devices and was expanded to include sensor-based technology. Throughout the iterative design process, observations, interviews and group discussions were carried out with stakeholders. This lead to key reflections and design concepts such as the value of injecting anthropomorphic qualities into the learning device and providing personally and culturally relevant learning experiences through technology. Overall, the results outlined in this paper help contribute to knowledge regarding the design, development and implementation of ubiquitous sensor-based technology to support learning.

publication, Uncategorized

VR paper accepted in INTERACT conference

Some more good news to share regarding publications. In collaboration with Dr Lex Mauger from Kent Sport and Exercise Sciences, we have been working on how virtual reality (VR) can be used to reduce the pain people feel when exercising. More information about the project can be found here.

Title: How Real is Unreal? Virtual Reality and the Impact of Visual Imagery on the Experience of Exercise-Induced Pain

Abstract. As a consequence of prolonged muscle contraction, acute pain arises during exercise as a result of a build-up of noxious biochemical in and around the muscle. Specific visual cues, e.g., the size of the object in weight lifting exercises, may reduce acute pain experienced during exercise. In this study, we examined how Virtual Reality(VR) can facilitate this “material-weight illusion”, influencing perception of task difficulty, thus reducing perceived pain. We found that when vision understated the real weight, the time to exhaustion was 2 minutes longer. Furthermore, participants’ heart rate was significantly lower by 5-7 bpm in the understated session. We concluded that visual-proprioceptive information modulated the individual’s willingness to continue to exercise for longer, primarily by reducing the intensity of negative perceptions of pain and effort associated with exercise. This result could inform the design of VR aimed at increasing the level of physical activity and thus a healthier lifestyle.

journal, Uncategorized

Paper accepted in Nature Scientific Reports

After two years of collaboration with Dr Yeo from VCU , our work in swallowing sensing with skin-like electronics is finally accepted for publication in Scientific Reports, an open access Nature journal.  More information about the project can be found here.

We are currently working on a second project on wireless chewing monitoring through a mobile app! Watch this space for more updates.

Title: Soft Electronics Enabled Ergonomic Human-Computer Interaction for Swallowing Training

Abstract: We introduce a skin-friendly electronic system that enables human-computer interaction (HCI) for swallowing training in dysphagia rehabilitation. For an ergonomic HCI, we utilize a soft, highly compliant (“skin-like”) electrode, which addresses critical issues of an existing rigid and planar electrode combined with a problematic conductive electrolyte and adhesive pad. The skin-like electrode offers a highly conformal, user-comfortable interaction with the skin for long- term wearable, high-fidelity recording of swallowing electromyograms on the chin. Mechanics modeling and experimental quantification captures the ultra-elastic mechanical characteristics of an open mesh microstructured sensor, conjugated with an elastomeric membrane. Systematic in vivo studies investigate the functionality of the soft electronics for HCI-enabled swallowing training, which includes the application of a biofeedback system to detect swallowing behavior. The collection of results demonstrates clinical feasibility of the ergonomic electronics in HCI- driven rehabilitation for patients with swallowing disorders.


Industry-Sponsored PhD Studentship

Title: Water leakage detection and wastage monitoring through advanced sensing and data modelling

University of Kent – School of Engineering and Digital Arts

Supervisors:  Professor Yong Yan, Dr Jim Ang and Dr Anthony Emeakaroha

Leakage and inadequate use of water costs some organisations a significant amount of fund that could be used for good cause. For instance, a public service organisation in England consumes an estimated 38.8 million cubic metres of water and generates approximately 26.3 million cubic metres of sewage per year. The use of water is also linked to the overall carbon footprint of an organisation through heating for hot water and energy consumption to pump water to the taps. This industry-sponsored PhD research programme aims to apply the state-of-the-art leak detection and data modelling techniques to minimise water leaks and wastage. Two key challenges in water management are the unnoticed leakages and human behaviour towards usage. There are various methods for detecting leaks in a water distribution system. Despite the availability of such methods, many leaks including slow leaks via dripping taps and small holes on underground pipelines are still undetected. New techniques based on advanced signal processing algorithms, precision mass flow balancing and communication techniques have the potential to overcome the limitations of the current techniques. In terms of user behaviour, there have been applications of machine learning techniques to model human behaviour to predict the efficiency of resource usage in recent years. The applicability of such techniques in predicting water usage and minimising water wastage in the service industry will be investigated. It is proposed to combine the improved leak detection techniques, data modelling and human behaviour analytics to minimise leaks and wastage of water.

We are looking for an excellent candidate with a relevant first degree and strong Masters degree in electrical/electronic engineering, physical sciences or related areas relevant to the PhD topic. Experience in sensors, instrumentation and machine learning is advantageous.

The successful applicant will be expected to undertake some teaching commensurate with his/her experience. Principally based on Kent Campus, it is also expected that the student will develop a close working relationship with industrial partners and conduct experimental work on their sites from time to time.

Funding Details:  Funding is available at the home/EU fee rate of £4,121.  There will also be combined maintenance funding of £14,296.

Length of Award: 3 years (PhD)

Eligibility: Open to all applicants (UK students, EU students and international students)

Enquiries:  Any enquiries relating to the project should be directed to Professor Yong Yan y.yan@kent.ac.uk.

Application:  Apply online at https://www.kent.ac.uk/courses/postgraduate/apply-online/262 and select the following:

Programme – PhD in Electronic Engineering

School – School of Engineering and Digital Arts

Research topic – Water leakage detection and wastage monitoring through advanced sensing and data modelling

Deadline:  26 February 2017


Hololens and its future

mhololensWhilst in Parma giving a talk in the Spatial Audio research meeting, I had a chance to play with Microsoft Hololens! In case you have not been following the tech news in the past few years, Hololens is a Augmented Reality headset (not to be confused with Virtual Reality) which blends digital information (e.g. virtual objects, graphical user interfaces, etc.) in the physical environment, allowing the users to experience a “mixed reality” world.

The idea of AR is not new, but two things really got me excited about Hololens: 1) handsfree AR experience, allowing users to use both hands to interact with the mixed reality world, 2) its ability to scan and understand the physical space.

Having tried it, I can say something about these two points. Handsfree experience is well executed to a certain extent. I can point, pinch, swipe digital information quite smoothly. However the tradeoff is that you now have to wear a whole computer on your face, and it is not pleasant at all! In fact, even within a few minutes, my face/head hurt and I cannot imagine myself using it for more than 30 minutes.

The second point on Hololens having “spatial understanding” of the physical environment is what sets Hololens and other AR applications (e.g. Pokemon Go) apart, and this is what makes AR more technologically challenging than VR. Hololens is able to scan a physical room and and map the information digitally in 3D. This is a very important point, as having this digital spatial information allows Hololens to superimpose digital stuff more precisely onto the physical world. For instance, if we would play Pokemon Go on Hololens, the Pokemon would appear on a surface of a room, e.g. on top of a table, instead of just at the centre of the screen regardless where in the room the user is looking at!

At its current price (3000USD), I don’t imagine any casual consumer jumping on it just yet, especially also given other well known problems, such as the small field of vision and low battery life.

I am however very excited by the future of this kind of wearable spatially-aware AR! In fact, we are currently working with a company called Daqri which manufactures a similar AR wearable device used in industrial environments.

research, student work

Happy New Year 2017

Welcome 2017, it is going to be great! To start with, I would like to share a video of a student project on game design (Unity3D). It is work of four students (two designers and two programmers). For a project of 15 credit module (12 weeks), it is pretty impressive! The same students are currently working on their final year project with me, and I can’t wait to see their final output!

I will also be having some research trips in the new year. In two weeks, I will be off to Parma, Italy for a research sandpit meeting, organised by Future Spatial Audio funded by EPSRC. There I will be talking about some of my work in virtual reality and health.

In Feb, I will be going to Northampton to discuss research collaboration with a psychiatric hospital, in the area of sensing technologies and virtual reality. Hopefully there will be more information to share in the new few months.

journal, Uncategorized

End of 2016

Let’s say 2016 has been an eventful year, in my personal professional as well as global political sphere, and I didn’t mean it in a good way! In any case, I am glad that we are now bidding 2016 goodbye. Never have I felt so eager to get rid of a year before, but 2017, here I come.

Well, at least the NIHR (National Institution of Health Research) project I was involved in some years ago has finally produced some results. This is the abstract of the newly published open access paper in Health Technology Assessment:


This report details the development of the Men’s Safer Sex website and the results of a feasibility randomised controlled trial (RCT), health economic assessment and qualitative evaluation.


(1) Develop the Men’s Safer Sex website to address barriers to condom use; (2) determine the best design for an online RCT; (3) inform the methods for collecting and analysing health economic data; (4) assess the Sexual Quality of Life (SQoL) questionnaire and European Quality of Life-5 Dimensions, three-level version (EQ-5D-3L) to calculate quality-adjusted life-years (QALYs); and (5) explore clinic staff and men’s views of online research methodology.


(1) Website development: we combined evidence from research literature and the views of experts (n = 18) and male clinic users (n = 43); (2) feasibility RCT: 159 heterosexually active men were recruited from three sexual health clinics and were randomised by computer to the Men’s Safer Sex website plus usual care (n = 84) or usual clinic care only (n = 75). Men were invited to complete online questionnaires at 3, 6, 9 and 12 months, and sexually transmitted infection (STI) diagnoses were recorded from clinic notes at 12 months; (3) health economic evaluation: we investigated the impact of using different questionnaires to calculate utilities and QALYs (the EQ-5D-3L and SQoL questionnaire), and compared different methods to collect resource use; and (4) qualitative evaluation: thematic analysis of interviews with 11 male trial participants and nine clinic staff, as well as free-text comments from online outcome questionnaires.


(1) Software errors and clinic Wi-Fi access presented significant challenges. Response rates for online questionnaires were poor but improved with larger vouchers (from 36% with £10 to 50% with £30). Clinical records were located for 94% of participants for STI diagnoses. There were no group differences in condomless sex with female partners [incidence rate ratio (IRR) 1.01, 95% confidence interval (CI) 0.52 to 1.96]. New STI diagnoses were recorded for 8.8% (7/80) of the intervention group and 13.0% (9/69) of the control group (IRR 0.75, 95% CI 0.29 to 1.89). (2) Health-care resource data were more complete using patient files than questionnaires. The probability that the intervention is cost-effective is sensitive to the source of data used and whether or not data on intended pregnancies are included. (3) The pilot RCT fitted well around clinical activities but 37% of the intervention group did not see the Men’s Safer Sex website and technical problems were frustrating. Men’s views of the Men’s Safer Sex website and research procedures were largely positive.


It would be feasible to conduct a large-scale RCT using clinic STI diagnoses as a primary outcome; however, technical errors and a poor response rate limited the collection of online self-reported outcomes. The next steps are (1) to optimise software for online trials, (2) to find the best ways to integrate digital health promotion with clinical services, (3) to develop more precise methods for collecting resource use data and (4) to work out how to overcome barriers to digital intervention testing and implementation in the NHS.