Economic Value of Physical Therapy Report from the APTA

A good start - unfortunately need to go further to avoid risk of bias

It’s been a while since I’ve posted here at peripatetic PT. I had some health issues starting in January that took a while to work through. The positive is that it allowed me to reassess and clear my obligations and responsibilities a bit. As I continue to recover, I’ve been enjoying in reading more, and wandering into the world of economics.

Therefore, it was with great interest that I read the report from the APTA on the Economic Value of Physical Therapy, and plan to dig into it further in the coming months. Upon seeing the press release (here), I was pretty excited that this work was being done. The value of clinical research isn’t just to guide clinical decision making. The value of clinical research is also to demonstrate efficacy to those that decide how to allocate scarce resources. That is what economics is:

Economics is the study of the use of scarce resources which have alternative uses. (Basic Economics, Thomas Sowell)

Photo by Towfiqu barbhuiya on Unsplash

Reading through the Executive Summary we are immediately struck by a statement that raises questions that require answers - but answers are not provided.

First we are told that the APTA engaged Nous (“an international management consultant, to examine the costs and benefits of eight condition-based physical therapist services, each of which was chosen based on the prevalence of the condition and its associated level of healthcare spending across the United States”). I have to assume that by “engaged” they mean to say “paid.” As in, the APTA funded this work. But that is not made clear. They should know better. The journal of the APTA (PTJ) requires clear statements of financial arrangements for anyone contributing to the work done for a publication. And clear statements of the conflict of interests inherent in the authors of a publication. Unfortunately, none of these statements are provided in the report. What is also missing - but really necessary - is a statement that Nous has independence in doing the work they were paid to do and were free (in fact compelled) to report what their independent findings were, independent of the the authors of the report we end up with - which are representatives of the APTA and physical therapists (who clearly have a risk of bias towards particular findings of such a report). I’m not saying how bias may have influenced the work. Just that we are at a point in our history were we understand that we are all biased and that science is a set of methods to systematically minimize the impact of bias (Lillenfeld, Ten Lessons Learned from the Study of Pseudoscience). Every member of the working group acknowledged in the report is a physical therapist. Physical therapists are biased towards the demonstration of the effectiveness of physical therapy practice, particularly the economic value of the profession from which they (and I) have devoted our life. It is particularly concerning that there is - as far as I can see - no one from Nous listed in the report. Nor a statement from Nous of what they provided for the report and whether they agree with how the APTA representatives have summarized the results. Better yet, I would prefer to be reading a report from Nous. A completely independent report, funded by the APTA but with Nous having the authority to publish what they find completely. Perhaps that’s what we have - but it’s not clear.

A particularly concerning statement concludes the Executive Summary:

A wider range of conditions initially was considered and then narrowed down to the eight in the report, based on specific economic evaluation criteria applied to the currently available research evidence base. As stronger evidence becomes available, additional conditions may be considered for analysis and inclusion in the future versions of this report. We have identified these potential conditions in the Future Additions to the Report chapter. (emphasis added)

This report only vaguely explains the process by which conditions were narrowed down.

We first identified conditions and applicable interventions that were the most quantifiable, as they would be best suited to being included in our economic model. This does not mean that excluded interventions are not effective physical therapist services, only that they may have lacked sufficient literature that could be accessed within this project’s timeframe.

We are not told what specific economic evaluation criteria were applied to the currently available research evidence based. We are not told whether Nous did an economic evaluation that was then put in a drawer, or whether Nous was told to focus on the eight conditions in the report. When unsure, be explicit. In another post I may dig into the subtle shift from “conditions” to “physical therapy interventions” - but given this is my first post in about 9 months, I’ve got to ease back into such meanderings.

In a randomized controlled trial you cannot just tell a reader that some patients did not meet criteria and therefore were excluded. Especially if data had been gathered and a decision is being made to exclude. In a systematic review you cannot just eliminate evidence from studies determined to be - a posteriori - not helpful to the conclusion that is desired. The criteria not met, that kept these other conditions from having the data for the economic model would be valuable. Especially to those of us in the profession that may be able to help build an economic value assessment for these other conditions. For example, I was co-leader and senior author on the heart failure guideline, and heart failure is one of the “future conditions” in the appendix. But I have no idea why heart failure could not be included.

The methods utilized to reduce the “risk of bias” in all published research should be applied in a report by an organization that adheres to evidence based practice and would like to build a scientific case for the economic value of the profession. Those methods should be applied even if not required by a journal. Perhaps even more so.

Please improve this report with enough information to allow readers to evaluate the risk of bias. Otherwise, it’s difficult for this to be anything other than a possibly pseudoscientific justification to prove to ourselves what we already hope.

As I dig into this report more and attempt to replicate the results based the report itself, I’ll continue to post about it.