MyOutcomes, feedback informed, improve outcomes, reduce costs nrepp certified

The history of the science behind MyOutcomes UK

In 2008, after seven years of research and development, MyOutcomes was launched as a Web-based version of the Partners for Change Outcome Management System (PCOMS), developed by Scott D. Miller, Ph.D. and Barry L. Duncan, Psy.D.In 2010, because of three randomized clinical trials demonstrating the power of feedback conducted by researchers at the Heart and Soul of Change Project and published in top tier, peer-reviewed journals, PCOMS was submitted for inclusion in the US Substance Abuse and Mental Health Services Administration’s (SAMHSA) National Registry of Evidence-based Programs and Practices (NREPP). After an arduous review that included both a research and the ability to disseminate evaluation, both dissemination versions of PCOMS (Heart and Soul of Change Project and International Center for Clinical Excellence) were accepted in early 2013.In the spring of 2014, MyOutcomes released Version 12. Notable updates include: Tagging, Remote Client Access and Group Session Rating Scale.Tagging increases your ability to segregate and aggregate client data to match your reporting needs. Remote Client Access gives therapy clients the ability to log in and register their feedback on their own device. Any computer, laptop or tablet using one of the following browsers will work: Internet Explorer 9 & 10, FireFox (latest version), Google Chrome (latest), Safari (latest), iPad and Android. MyOutcomes Mobile for smart phones provides a simplified version of MyOutcomes that can be downloaded for free on iTunes and Google Play. With all of the client administration options now available, online administration of the ORS/CORS and SRS/CSRS for multiple clients in therapy has become feasible. The Group Session Rating Scale can now be entered by all group members simultaneously for instant feedback and more responsive adjustments to group issues.

In the Spring of 2015, Version 13 of MyOutcomes was released. Some exciting new features introduced in this version are as below:

  • The first is Email PCOMS, a way to send the measures directly to the client so they can enter their responses on their own devices.
  • The second is a Customizable Aggregate Stats Report. Our latest advancement in outcomes, this customizable aggregate stats report offers a number of new metrics as well, including new sections on Recovery, Treatment Duration and Discharge.
  • Our third major new addition is the ORS Audit Dashboard. A new dashboard application that retains the full capacity of our current Dashboard Report Parameters to easily identify cases where data integrity may be at risk.

In the Fall of 2015, MyOutcomes UK in alliance with Trust (LPFT) was released. MyOutcomes®UK is a secure, web-based automatic feedback and data management system that supports the practical application of an Outcomes Orientated Approach to the delivery of Mental Health Services (OO-AMHS). MyOutcomes®UK displays a graphical interpretation of ORS and SRS feedback that enables clinicians to assess a patient’s progress in real-time with ‘flags’ to alert clinicians where there is little or no improvement. Reports can be created from aggregated data to enable clinicians and managers to monitor therapeutic outcomes by case-load, team, or service and will be useful for generating recovery and service activity metrics.

How MyOutcomes UK models the true clinical population

When trying to identify and measure any latent variable, two critical factors play a major role in any success. The first is the sample size. The larger the sample size, the greater the variance and, therefore, the greater the likelihood of extracting the variable. An additional benefit of having a large sample size is that the sample has a higher probability of approximating the true population. The second key factor is the statistical model to be used. The ideal statistical model should be able to detect the variable, as well as be able to model and make predictions about the variable as it truly exists in the population of interest. MyOutcomes UK can easily meet these two critical factors. As a result, MyOutcomes UK® has the power to predict change.

MyOutcomes UK’ database consists of a sample with well over half a million measurements. This sample can be considered to be fairly representative as the measurements come from a broad range of countries and clinical settings. We are confident that MyOutcomes UK easily meets the following four sampling conditions necessary for predictive statistical modeling:

  • Randomized sampling from a defined population
  • Independence of sampling
  • Normal distribution
  • Population variances being equal

MyOutcomes UK’ statistical modeling uses algorithms that have passed extensive cross-validation analyses. The PCOMS (Partners for Change Outcome Management System) development team was led by Dr. Barry Duncan and included Professor Michael Toland, a statistician at the University of Kentucky. A MyOutcomes UK’ independent analysis was also completed and was led by statistician Douglas L. Steinley at the University of Missouri. The teams used their considerable years of clinical experience to develop and validate the model.

Initially, the development team eliminated extreme outliers from the database. These outliers are viewed as errors resulting from the use of MyOutcomes UK by inexperienced clinicians. These errors, which disappear as a function of increased use and experience with PCOMS, could impact the algorithms’ ability to make accurate and practical predictions.

Based upon clinical experience, theory and research in the clinical field, the development team’s a priori assumption was that therapeutic progress and outcomes should be described by a curve following a non-linear growth function. Analyses of models conditional on intake score demonstrated that a cubic model, rather than a quadratic model, provided the best fit for the data.

Considerable in-depth testing was conducted on the statistical model. Using descriptive statistics analyses, the development team evaluated the model’s ability to predict trajectories for each intake score, as well as the means across all sessions in the database. Individual scores were plotted and compared to the expected treatment response predicted by the algorithms. The algorithms passed this very extensive testing process.

To evaluate whether the model predicted too much change, the development team used the data sets from published randomized clinical trials (RCT) of PCOMS (Anker, Duncan, & Sparks, 2009; Reese, Norsworthy, & Rowland, 2009; Reese, Toland, Slone, and Norsworthy, 2010) to examine how much change occurred in the feedback conditions. The algorithms used by MyOutcomes UK were found to be effective in accurately predicting change. The average amount of change across the feedback conditions in all three RCTs was 10.1 points. This value includes all the clients, those who changed and those who did not. Algorithms predicting far less change, not only wouldn’t match the feedback RCTs, but would also inflate outcomes. This would ultimately affect the clinician’s sense of how effective they really are, which is exactly what PCOMS is designed to prevent.

In practical terms, the increased sensitivity of the PCOMS algorithms to detect change and do a better job of predicting what change to expect translates into providers feeling even more confident that MyOutcomes UK helps them to provide their clients with the best quality service.

Based on this information, we are confident that the statistical model currently used by MyOutcomes UK v.13, represents a true clinical population. MyOutcomes UK is an effective clinical tool that provides a vital source of the outcome status of clients. Together with their own theory and knowledge, MyOutcomes UK helps the therapist identify those clients who are not responding to clinical treatment. MyOutcomes UK enables the clinician to address a lack of progress in a positive, proactive way that keeps clients engaged while therapists collaboratively seek new directions.