I was talking to some medical students recently about the use of outcome measures in rheumatology. I happened to mention that ‘big data’ was the next ‘big thing’ in medicine. Cue blank faces. Followed by quizzical looks and a sly smile. I suddenly realised to my chagrin that not everyone has read Topol’s ‘Creative Destruction of Medicine’ and that my students are beginning to doubt my sanity. I rapidly tried to regain some semblance of dignity by explaining that outcome measures are all the rage in rheumatology right now.
Readers of my blog may have not be surprised to hear that at times I can get a little obsessive about the science of measurement in medicine. So as a form of necessary self-therapy and reflection I am going to explore the darker side of measurement science using a couple of well-known quotations.
“Not everything that can be counted counts, and not everything that counts can be counted.”
So not everything we measure or count actually matters. We need to be very selective before we decide that some measurement is essential or routine. There seems to be an assumption that the more we measure the more scientific our care will be and therefore the outcomes will improve. In the case of rheumatology there is now an assumption that if we have a patient with rheumatoid arthritis and we do not complete a ‘disease activity score’ and a disability assessment in every patient we are in some way negligent as doctors. Now I would be the first to argue that early arthritis trials have demonstrated the need to adopt a ‘treat to target’ approach using objective and quantifiable measures to gauge response – but to extrapolate this to patients with 30 years of arthritis may not be either scientific or kind to the person in front of us, particularly when they mention that they have had a recent bereavement. In times like these, ‘feeling the pain’ will often trump feeling the joints. Some measurements may be eminently ‘scientific’ but completely worthless, particularly in managing the individual.
The second part of this elegant chiasmus claims that not everything that counts can be counted. I hope that every health care worker is acutely aware of this fact, but it is at the core of an empathic approach to medicine. How the patient feels cannot easily be reduced to a score on a questionnaire. How well does the doctor understand the narrative from the patient’s perspective? Can we really quantify how well or badly the doctor or nurse empathises with their patient? There is a feeling that recent changes in the organisation of medical work have disrupted the continuity of care and the integrity of the ‘team’ looking after the patient. Medical ‘clerk-ins’ are reduced to ticking boxes in lengthy proformas and medical handovers are reduced to a rapid fire exchange of facts and figures. Increasingly, doctors complain of being tied to the computer screen completing page after page of data, and having no time to have a proper talk with the patient. In the midst of this swirling fog of data, have we lost sight of the patient? When we teach our students the importance of rigorous data collection and outcome measurement, will we remember to refocus their attention back on the importance of empathy and the art of medicine?
The second quotation has been attributed to AE Houseman or Andrew Lang:
People Use Statistics as a Drunk Uses a Lamppost — For Support Rather Than Illumination
This is justifiably a popular quotation and any medical researcher worth his or her salt will doubtless be familiar with the dark arts of manipulating statistics to make the data seem to support your hypothesis. Reviewers of publications have become wise to this, and statistical scrutiny is a little more rigorous than it used to be. Even better, clinical trials registration requirements and ethics reviewers will insist on you defining the primary and secondary outcome measures before the study begins. This only goes to prove that nowadays you need a higher level of statistical jiggery-pokery’ to baffle your readers and ‘stay ahead of the game’.
Managers Use Statistics as a Dog Uses a Lamppost — For ‘Marking their territory’ Rather Than Illumination
The third quotation could be attributed to a certain PV Gardiner, the writer of this blog. He has clearly borrowed heavily from the afore-named Andrew Lang. He has also used polite and politically correct language within the quotes which readers may wish to replace with more descriptive and direct terminology.
I’ll tell you a story to illustrate the point. A few years ago we were told that we were obliged to carry out an ‘audit’ on our use of biologic drugs in the region. Nothing unusual with that, you may say – they are very expensive drugs, after all. However, we did detect quite a hostile and accusatory sub-text to the discussions. After a little ‘digging’ we discovered that somebody in management ‘on-high’ had ‘discovered’ that we were using biologic drugs twice as much as the average in the UK. We thought we were using the same NICE guidelines to treat similar patients, so this seemed odd. Cue some anxious head-scratching and navel-gazing by the chastened rheumatologists. Strangely, our own figures hadn’t seemed to suggest any problem at all. Eventually, we discovered that someone in management had been using a single figure (the prevalence of RA in our region) that had changed everything. This figure suggested that RA was only half as common as in the rest of the UK. We were most intrigued. Could it be something in the pure Irish spring water? Could it be our fondness for Guinness or Bushmills whiskey? How had they arrived at such a figure, when there had never been a study of the epidemiology of RA in our region? We have never had a diagnostic database to record the prevalence of RA. In fact, they had obtained the figure from a reference found in none other than the official National Audit Office report into RA in England. That article had briefly referred to comparative data in other nations, and the quoted statistic actually came from a paragraph in a local newspaper where a journalist had mentioned in passing the estimated number of people with RA in our region. Well, it is just as well that we discovered the source of these scurrilous rumours. Otherwise the aforementioned honourable rheumatologists might well have wound up propping up the local lamp-posts (or was that the other way round?). We live to fight another day…