Starting a Company Sparked My Interest in Data Science
Updated: Nov 19, 2019
My curiosity in data science was first piqued in college, when I took a liking to a required statistics class and followed it up with a course in econometrics and data analysis. I didn't delve further into the subjects then, but when I founded Wellinks, a health tech company, in 2013, I began to think more seriously about the role that data has in decision making generally as well as in business and healthcare in particular.
Data affected everything about Wellinks, down to its very raison d'être. From the beginning, we faced four important challenges that required answers with data:
1. Showing that bracing was an effective treatment for adolescent idiopathic scoliosis.
2. Showing that Wellinks' device was successful in improving treatment and outcomes.
3. Establishing market size, i.e. the current population of those with AIS treated with braces and those who might get braces were treatment and reach improved.
4. Demonstrating that the willingness to pay and pricing of our product given the market size was enough to make investors certain expected returns on investment.
I describe the first two below.
Showing that bracing was an effective treatment for adolescent idiopathic scoliosis (AIS).
The entire medical community around AIS sought this answer, yet the literature was inconclusive and sometimes even contradictive. It was only in October 2013, several months after Wellinks began, that the seminal Bracing in Adolescent Idiopathic Scoliosis Trial (BrAIST) ran and its effects published. BrAIST included patients randomly assigned to bracing vs. observation. The researchers found enough evidence demonstrating the efficacy of bracing that they stopped the trial early.
However, the landscape prior to the BrAIST findings was quite discordant. We can look at recommendations by the United States Preventative Services Task Force (USPSTF) regarding screening for AIS as a reflection of how these findings rippled through the medical community. In 2018, they updated their 2004 recommendation (I added the underlining ):
"This recommendation updates the 2004 USPSTF recommendation, in which the USPSTF recommended against routine screening for idiopathic scoliosis in asymptomatic adolescents (D recommendation). In 2004, the USPSTF found fair evidence that treatment of adolescent idiopathic scoliosis leads to health benefits (ie, decreased pain and disability) in a small proportion of persons. The USPSTF bounded the harms of treatment as moderate (eg, unnecessary brace wear or unnecessary referral to specialty care). Therefore, at that time, the USPSTF concluded that the harms of screening exceeded the potential benefits.
To update its recommendation, the USPSTF commissioned a systematic review of the evidence. Because of new research, the USPSTF determined that it no longer has moderate certainty that the harms of treatment outweigh the benefits. The USPSTF found no direct evidence of a benefit of screening for adolescent idiopathic scoliosis on health outcomes. A growing body of evidence suggests that brace treatment can interrupt or slow scoliosis progression; however, evidence on whether reducing spinal curvature in adolescence has a long-term effect on health in adulthood is inadequate.... [T]he USPSTF has determined that the current evidence is insufficient to assess the balance of benefits and harms of screening for adolescent idiopathic scoliosis, leading the USPSTF to issue an I statement."
With this trajectory in mind, Wellinks had an uphill battle at the very start before BrAIST results were published. Based on anecdote, personal experience, and expertise, the three of us co-founders were convinced that braces were effective treatment devices. Poring through the literature, it was clear that there abounded reasons for the varied and contradictory findings. Often it came down to sample sizes – they were almost all too small to capture the large variation inherent in the data. And parents, many of whom were highly involved in treatment management, were naturally not often amenable to randomization for the purposes of studies. Cochrane, a respected organization that conducts systematic reviews of health-care interventions, summarized these challenges:
The evidence was moderate to very low quality. Reasons for downgrading [quality of evidence] were evidence coming from few randomized trials with few participants and many lost at follow-up or from observational prospective controlled studies. An issue in the field of AIS is the high rate of failure of RCTs, since parents want to choose with physicians the preferred treatment for their children. Thus, it is challenging to obtain high quality evidence in this field.
Before the BrAIST study came out, we had to explain our reasoning behind our confidence that brace treatment was effective to mentors and potential investors who rightly performed due diligence on us and the literature we cited. The results and data from the BrAIST study came just after we 3D printed our first proofs of concept and incorporated – good timing for us as we began our path toward commercialization.
The success of the study, which had its findings published in the New York Times, highlights some the challenges of acquiring decisive findings in healthcare fields with few patients, many variables, and practical and ethical considerations of random treatment assignment. Once these data were published, Wellinks had an easier time convincing others of the problem and solution.
Show that Wellinks' device specifically was successful in improving treatment outcomes.
The second hurdle was more central to Wellinks itself. We created a device that essentially recorded the patient's wear-time and tightness level and transmitted it to smartphone and web apps for the patients, their parents, and doctors to see and use to make treatment decisions (see website for more info).
There were two variables we were measuring (that were not independent of each other, mind, on outcomes), one of which (hours worn) was also included in the BrAIST study.
Below is a figure from the BrAIST study. It is so important that it is glued to the walls of scoliosis doctors' offices.
These findings, while extraordinarily helpful in convincing patients of the need to wear their braces properly and investors of the problem that needed solving, also highlight the difficulty of obtaining very clear data. It is easy to make a generalized conclusion that more hours in a brace generally leads to better outcomes ('success' is categorized by not surpassing a threshold curve progression, which is itself up for debate), and there may be diminishing returns after almost 13 hours per day of wear. However, the confidence intervals for success besides 0-6 hours all overlap significantly and the hourly range is quite large.
Deciding dosing is therefore challenging. If these were not human teenagers, you might as well prescribe 24 hrs of daily wear to ensure the optimum treatment results. But of course, brace wear has many costs to children, especially social and psychological ones. It's hard to believe how many kids would rather have the spinal fusion surgery than to wear the brace, for it makes them stick out during the socially intensive years of middle school/early high school. Not only that, but compliance to the treatment is itself affected by the recommendation of the number of hours one should wear the brace. For example, some might find too many prescribed hours so daunting that they might completely fail to wear it at all. Setting realistic goals is incredibly important in this type of treatment program.
Wellinks was able to rely on this data for the general conclusions that more time is better and therefore our device/app will help wearers meet the time dosage as prescribed by their doctor. But when it came to quantifying our success, the data suddenly became more muddled. As mentioned before, there are large confidence intervals for treatment success in the BrAIST data. A clear reason this might be has to do with the tightness with which the brace is worn. It stands to reason that loosely worn braces will be less effective. So the data above captures the number of hours worn, but not the quality of wear time. 12 hours in a completely untightened brace might as well be 0.
Our devices were designed to be able to also give you tightness information.
Part of the challenge for us, then, had to do with obtaining data that precisely defined which thresholds of tightness led to what outcomes for different hours of wear. As this number would likely vary by the physiological differences between patients, this challenge presented a messy combination of variables. Combined with potential data collection issues, it made for a tough project for us.
We decided that a clinical trial with our devices would be a great starting point to eventually define not only a logical connection between buying our device and increasing positive outcomes, but a quantifiable one too that would lead to proper information for pricing vis-a-vis insurance. I personally did not focus on the difficult data-collection and interpretation on the clinical trial end of operations--that was the work for the biostatistician--but it did spark my interest in delving deep into the weeds of data in healthcare in future work.
I might come back later to address challenges 3 and 4 in the future as well as address data collection issues during the clinical trials, but for for now, I will stay away from details as the company is still operational.