How the 'Quantified Self' movement can offer new directions for behavioural science trials
In many fields, but particularly medicine, RCTs (also called trials and experiments) have been a key mechanism for generating knowledge and depends on measures of ‘central tendency’ for evaluating a treatment’s benefits and side effects/risks relative versus a control condition. This has been valuable in offering reliable and valid findings based on group averages to which end it is typically seen as a the ‘gold standard’.
Of course, this is also the case in behavioural science where RCTs have had real success in finding solutions with effective, evidence-based treatments (or interventions) that benefit many people. In a recent paper Steven M. Schwartz, Kevin Wildenhaus, Amy Bucher and Brigid Byrd suggest that this group lens provides a ‘forest view’ of outcomes relative to determining the greatest overall good.
The point they make, however, is that no matter how rigorously the group data is derived, using group statistics alone can never fully address the need to treat individuals uniquely – the ‘tree lens’. They point out:
It is becoming increasingly clear that RCT and group methods, although still quite valuable, are insufficient as treatments and testing move toward the precision of inherent individual differences, which are reflected in every person and every treatment response (positive or negative)
Their focus in the paper is on health and they point out that many common chronic conditions (weight, diet, adherence to prescribed medicine, sleep, etc.) are often only superficially addressed. The authors argue that time and cost constraints mean that this type of care is not catered for in our current healthcare delivery model.
By making use of scalable digital technology, they suggest that the measurement of relevant metrics can be increased and algorithms, developed from ‘aggregate science’, can be used to offer evidence-based feedback for the individual. This reflects the Quantified Self movement, coined by Gary Wolf and Kevin Kelly, then editors at Wired magazine, in 2007.
Whilst the quantified-self phenomenon has arguably had minimal impact on the collective scientific knowledge to date, much has changed. The technology now available for quantifying various biometric, behavioral, emotional, cognitive, and psychosocial factors of daily life has become increasingly diverse, accurate, and accessible. Many believe these burgeoning technologies can and profoundly alter the way lifestyle, health, wellness, and chronic disease are managed in the future.
Schwartz and colleagues suggest that a huge amount of data in healthcare at least which is now much more readily available to us which includes:
Clinically generated data: The data generated by a person’s interactions with the formal healthcare system, including electronic medical records, lab test results, pharmacy data, and health insurance claims, etc.
Commercial real-world health data: This is data generated by programs focused on health management that complement traditional healthcare, including wellness and disease management programmes
Consumer digital health device–generated data: The growing array of commercially available connected digital technologies for care and well-being
Health-suggestive data: Digital data is generated by people from a variety of non-health data that are not explicitly tied to health but reflect other aspects of lifestyle and secondarily can provide additional insights into health (social determinants such as location, local weather, buying habits, etc.)
The authors suggest that this type of data can be used to offer people access to a ‘digital twin’ dashboard, which would mean that treatment development and testing are able to move toward the precision of individual differences inherent in every person and every treatment response (or non-response).
This can be seen as the renaissance of N-of-1 or individual science, creating the opportunity to evaluate each individual uniquely. This is not new, N-of-1 designs have been explored now for over half a century and indeed are part of a tradition of the scientists experimenting on themselves.
For the clinician, this form of evaluation can help test the impact of a given treatment on a given patient with increased efficiency and accuracy. But in addition, users themselves can begin to evaluate the patterns between their own responses with the variety of contingencies that impact them. In other words, we can see the emergence of the self-scientist. By moving from a ‘forest’ to a ‘tree’ level of observation and evaluation, the N-of-1 perspective has potential to complement the RCT.
There are of course a number of challenges to the notion of individual data collection and self-science:
The notion that our health, wellbeing and their determinants is largely quantifiable is quite a leap – there may be a wide range of reasons why we may not be able to sleep, which may be hard to measure.
It assumes that the causes of a health / wellbeing issue are individual rather than structural – if we are holding down multiple jobs with uncertain hours and low wages then the data well collect about our own individual characteristics is unlike to offer much.
The extent to which people will actually engage with technology in this way is debatable – adoption of any specific form of digital technology is far from a given
There is an assumption that people are able to identify that there is a problem and are motivated to resolve it – when in fact may of us struggle to recognise that something such as insomnia is even an issue.
From work we have done, we can see the way that people can get very caught up in the measurement itself (e.g. walking 10,00 steps a day) and optimising the metrics rather than focus on what was the original goal (e.g. getting fit)
A case could be made that the lack of focus on some area of health and wellbeing is less about time and cost constraints but rather structural bias, as many women or ethnic minorities, for example, will surely testify. It is too easy for technologists to jump into solutionism – assume every problem has a technology fix when in fact we should be looking at political or social-cultural fixes.
There are a wide range of issues concerning data ownership and privacy as well as ethical considerations (e.g. insurers raising premiums on the basis of the information collected).
We cannot ignore the wider social-technical environment of these sorts of developments which has implications way beyond the immediate scientific and individual context.
With all this in mind, we nevertheless can see the way in which N-of-1 studies with individuals or small groups are an effective complement to RCTs. The same principles can be used for what we might call ‘behavioural sandpits’, where groups of people agree to participate in a research programme on a longitudinal basis alongside ‘digital-twins’.
This would certainly help support the development and testing of interventions in a more ecologically valid setting but also cater for a more agile form of intervention development. As we move to tackling significant issues such as achieving net-zero, we will need increasing engagement from people to overcome the wide variety of complex challenges they face when changing their behaviour.
As such, N-of-1 studies can provide us with insights from people that are working to make this change happen, so we can help identify ‘what works’ which can be shared with others that are on a similar journey. We can also use this to move from individual ‘tree’ level insights to developing broader ‘forest’ level conclusions concerning what can work for wider populations.