Course Schedule
Java Performance Tuning
Chania Crete, May 18-21
Java Performance Tuning
Quick Start
Partnering with Informator
Stockholm Sweden,
Jan 29-30
My RSS Feeds
Review of Super Crunchers
posted Monday, 28 July 2008
On a recent trip through Heathrow I ran into Super Crunchers, a book about how business are using regression testing and other statistical methods to change their understanding of how things really work. The book is following in the wake of the very successful Freakonomics. Thought the book isn't quite as easy to read or appealing as Freakonomics, it does contain some very interesting information. The book screams; "Measure, don't guess", my performance tuning mantra.
If you can get past the hand-waving, regression analysis is the next silver bullet propaganda, you will find some very interesting bits of information. More over, this information could help you to understand how companies are using human psychology to target you. The book presents case after case where conventional wisdom just doesn't hold up to proper statistical testing. The bottom line is, regression analysis is becoming cheaper to perform and companies that use it have found it is making an impact on thier bottom lines.
But it isn't only companies that are benefiting from controlled statistical data mining of large sources of data. One chapter looks at how these types of studies have helped the medical proffesion reduce mortality rates by implementing simple changes that have no impact on costs. For example, a significant reduction in respiratory infections for patents on ventilator therapy by simply raising the head. Hospital beds already are design to elevate the head so it's just a matter of turning a crank!
Another significant change in medical care is diagnosis. People are using Google to help their Dr. diagnose their own problems. I’ve personally done this on two occasions the last case being the puzzle of my Mother-in-law’s eyes. In both cases, he “experts” got it wrong but careful use of Google gave us an answer that changed everything. But the use of Google isn’t limited to patients. Super Crunchers points out cases where Doctors themselves have used Google to diagnose odd ball cases.
Do I blame Doctors? Absolutely not! They are suffering from the same problems that we in IT are, the ability to keep up. There is just so much research, some of it good, a lot of it bad, that it is impossible to keep up even in narrow specialties. In this environment it is not so surprising that search technology is getting more attention. Fortunately the medical community is looking at more specialized technology than Google. The book gives hints of the semantic web being applied to the worlds collection of medical studies. Being able to search these documents based on symptoms can often lead to diagnosis's that often surprising.
My Mother-in-law’s eye problems fall into that category. The symptoms spelled Glaucoma with a few other things that no one quite understood. The other things in Google gave us a completely different diagnosis with the warning that the problem looks like Glaucoma. This book only reinforces my opinion that it would be harsh to blame the Doctor’s for this mistake.
Supercrunchers tries to sell regression testing as the next silver bullet. Sorry but I just don’t believe in Silver bullets. The book also doesn't the misunderstandings of statistics that often occurs. People often confuse population statistics and individuals. Another example, testing positive for something using a test that is 99% accurate doesn’t actually mean you have what was tested for. To calculate that we need to do a deeper analysis that involves understanding rates in the population. Yet this bad thinking has been used to convict people in criminal trials. What the book does show is that statistics are like any other sharp tool, you can get wonderful results but used incorrectly and you’ll get hurt. If anything, this book demonstrates how the market is getting more sophisticated in it’s use of statistics and that information by it’s self can help you better navigate everyday choices.
If you can get past the hand-waving, regression analysis is the next silver bullet propaganda, you will find some very interesting bits of information. More over, this information could help you to understand how companies are using human psychology to target you. The book presents case after case where conventional wisdom just doesn't hold up to proper statistical testing. The bottom line is, regression analysis is becoming cheaper to perform and companies that use it have found it is making an impact on thier bottom lines.
But it isn't only companies that are benefiting from controlled statistical data mining of large sources of data. One chapter looks at how these types of studies have helped the medical proffesion reduce mortality rates by implementing simple changes that have no impact on costs. For example, a significant reduction in respiratory infections for patents on ventilator therapy by simply raising the head. Hospital beds already are design to elevate the head so it's just a matter of turning a crank!
Another significant change in medical care is diagnosis. People are using Google to help their Dr. diagnose their own problems. I’ve personally done this on two occasions the last case being the puzzle of my Mother-in-law’s eyes. In both cases, he “experts” got it wrong but careful use of Google gave us an answer that changed everything. But the use of Google isn’t limited to patients. Super Crunchers points out cases where Doctors themselves have used Google to diagnose odd ball cases.
Do I blame Doctors? Absolutely not! They are suffering from the same problems that we in IT are, the ability to keep up. There is just so much research, some of it good, a lot of it bad, that it is impossible to keep up even in narrow specialties. In this environment it is not so surprising that search technology is getting more attention. Fortunately the medical community is looking at more specialized technology than Google. The book gives hints of the semantic web being applied to the worlds collection of medical studies. Being able to search these documents based on symptoms can often lead to diagnosis's that often surprising.
My Mother-in-law’s eye problems fall into that category. The symptoms spelled Glaucoma with a few other things that no one quite understood. The other things in Google gave us a completely different diagnosis with the warning that the problem looks like Glaucoma. This book only reinforces my opinion that it would be harsh to blame the Doctor’s for this mistake.
Supercrunchers tries to sell regression testing as the next silver bullet. Sorry but I just don’t believe in Silver bullets. The book also doesn't the misunderstandings of statistics that often occurs. People often confuse population statistics and individuals. Another example, testing positive for something using a test that is 99% accurate doesn’t actually mean you have what was tested for. To calculate that we need to do a deeper analysis that involves understanding rates in the population. Yet this bad thinking has been used to convict people in criminal trials. What the book does show is that statistics are like any other sharp tool, you can get wonderful results but used incorrectly and you’ll get hurt. If anything, this book demonstrates how the market is getting more sophisticated in it’s use of statistics and that information by it’s self can help you better navigate everyday choices.
comments