Now available, the EMR RANKINGS take into account the number of verified raters as well as the overall rating, and calculates a 1st to 10th ranking based on both of these scores according to a ranking algorithm. The purpose is to provide a level of confidence through the application of a weighting system that takes both of these measures into account.
Example:
- Product A has been rated by 50 verified physician raters and has an average overall rating of 4.0
- Product B has been rated by 10 verified physician raters and has an average overall rating of 4.5
- In an un-weighted ranking system, product B would have a higher score than product A and would be ranked above A. However in the weighted ranking system, because A has 50 raters vs. the 10 raters for B, the opposite is now true. By applying a weighting algorithm that takes into account the number of verified raters, a level of confidence is now applied to the rankings. The overall rating scores will remain unchanged.
Ranking Algorithm:
- Each product requires at least 5 verified physician raters before a verified score can be published
- From the start, 5 ghost ratings with a value of 2.0 are added to the overall score for ranking purposes
- The ghost raters fall away at set intervals as the product builds up numbers of verified raters
- At 40 verified raters, 2 ghosts fall away
- At 75, 100 and 125 verified raters, 1 additional ghost falls away
- Once a product has 125+ raters, no ghosts apply
- Taking the example above, product A would have a ranking of (50x4.0)+(3x2.0)/53 = 3.886, product B would have a ranking of (10x4.5)+(5x2.0)/15 = 3.667
From a Weighted Ranking perspective, a level of confidence has been applied to the overall ranking score.
Please Note: Users should also look at all other criteria regarding the company and product including the percentage of raters vs. the overall number of stated users of that product. Due diligence is essential and purchasers are advised to do site visits, get hands-on experience and obtain testimonials from colleagues who are using that EMR.
Potential purchasers of EMRs should not use the CanadianEMR ratings and rankings as their only mechanism for selecting an EMR.
View EMR Rankings by clicking here
What are your thoughts about the weighted ranking system? Share your thoughts or comments, by clicking on the 'Comments' link
Where can I find detailed comments from EMR users about the specific EMR products? What they like, what they don't like etc.
This is much more useful then a simple rating scale. Is this information hidden somewhere or are you afraid to post doctors comments?
Posted by: Ira Bernstein | August 23, 2009 at 06:02 PM
Ira, thanks for the question.
The ideal solution would be to have a completely open rating and feedback capability that allows for open discussion including the ability to post comments as to what is good or bad about any particular vendor. We can do this when we search for television sets, vacations, or restaurants. Why not EMRs?
With respect to CanadianEMR, this is something that we have thought about a great deal. At this time, our decision has been to allow ratings using a simple rating scale that can be interpreted based on the number of individuals who have rated relative to the size of the company and a weighting scale that ranks vendors with more ratings higher than vendors with lower numbers of ratings.
We are still very early in the adoption of EMRs across Canada. Some provinces have more advanced programs providing funding and change support for physicians and their practices. Other provinces do not have funding or any formal program. It is a challenging environment for physicians and for vendors of EMRs.
While we do not plan to allow comments in the immediate future, we will be enhancing the rating system so that a potential purchaser will be able to view current ratings (for example: 'Last 12 months') or view the full history of ratings for that EMR product in a table and a graph. We believe this will be a meaningful enhancement. In addition, we will begin looking at specific capabilities of EMRs e.g. prescribing to allow ratings of specific functionality.
Will we allow comments in the future? This is definitely something that we have considered, however we depend on the input of our physician advisory and vendor advisory groups in terms of ensuring that it is fair and that the correct policies are applied so that the feedback is meaningful.
Posted by: Alan Brookstone | August 29, 2009 at 08:59 AM