Intersections of Machine Learning and Domain Knowledge in Healthcare
Updated: Nov 21, 2019
With so much hope riding on machine learning to perfect diagnostics and realizing better value-based healthcare, I began to wonder how machine learning expertise fits into the field today.
At first glance, the answer is quite obvious. Machine learning, like many tools, should aid experts in making informed decisions. If every doctor were empowered with the knowledge and skillset to develop, run, and interpret machine learning algorithms to whatever particular disease, condition, or problem they focus on, our healthcare system would be much improved in outcomes and efficiency.
However, for many reasons—not the least of which is the time it takes to develop deep knowledge in any healthcare or machine learning field—this ideal scenario is not in line with reality. Expertise is divided among different people at different stages across different pipelines. So the question I have is how experts can optimally work together in the healthcare field.
Take one example in favor of the power of machine learning over healthcare expertise: retinal imaging. Until last year, ophthalmologists didn't think there was a difference in eyes based on sex. However, a program developed by Google could classify the sex of a patient given their retinal scan. This classifier had an AUC of 0.97 (super high)!
When looking at 100 heat maps of features that were most salient in determining the prediction of the algorithm, ophthalmologists could identify some of the different structures--vessels, optic discs, etc.--but they identified 50% of the heat map features as 'non-specific'. In other words, even given the areas of the retinal images that were most influential for the prediction, ophthalmologists, weren't able to figure out exactly how the neural network was making its accurate predictions despite deep domain knowledge.
While predicting sex itself may not be of high utility per se, this example highlights the power of machine learning classifiers to go above and beyond domain knowledge. The tradeoff here is that while the predictions were accurate, the neural network acts as a blackbox, so we gain practical information but don't necessarily advance medical theory.
When it comes to classifying diagnoses from medical images, precision is critical. Medical practitioners and patients often run into tough decision-making and cost-benefit problems when type I and/or type II errors are high.
One of the most interesting examples of this comes from mammogram recommendations. Below is a chart of three organizations' recommendations based on age buckets. The USPSTF embroiled itself in controversy when it changed its recommendations to fewer screenings. This outraged many clinicians and the general public, but was based in the large number of false positives compared to false negatives in screening.
One hypothetical question to consider to appreciate the difference in recommendations is what is the acceptable threshold number of women misclassified as a false positive (i.e. overdiagnosis and potentially overtreatment) to change one false negative to a true positive?
In other words, what is the cost of a false positive? Many people believe there is none, or at least none appreciable to the cost of a false negative. Below, the USPSTF elaborates on its reasoning and places emphasis on harms of false positives whereas few other organizations do.
In this case, machine learning algorithms can be incredibly useful and make a huge change in the healthcare industry. Reducing overtreatment and overdiagnosis and false negatives would save people's lives and lots of time and money.
In most cases however, domain knowledge will still be vital to any machine learning, whether it is in gathering the data, feature engineering, or interpretation. Whenever a physician is well-versed in machine learning, I'm impressed with the results of their algorithms. As machine learning as a field expands, I wonder how professionals in the medical field will interact with ML models and with ML professionals to improve treatment outcomes and improve efficiency in an increasingly expensive healthcare industry.