Difference between revisions of "EBM"

From Clinfowiki
Jump to: navigation, search
(Rate the quality of the available studies)
 
(18 intermediate revisions by 2 users not shown)
Line 1: Line 1:
'''Evidence-based medicine (EBM)''' is a branch of medicine that emphasizes using clinical and research evidence in making decisions about patients.[http://books.google.com/books/about/Users_guides_to_the_medical_literature.html?id=Kz3_36ejIv4C] The origin of evidence-based medicine was largely due to the work of a group of epidemiologists, such as Sir Austin Bradford Hill, [[Archie Cochrane]], and Richard Doll. The premise of EBM is that medical knowledge grows at a faster rate than health care providers can absorb. To resolve this, EBM concentrates knowledge from published medical literature, via systematic review and meta-analyses, into a more easily accessible format for health care providers with a scarcity of time.
+
'''Evidence-based medicine (EBM)''' is the integration of evidence, experience, and values into clinical decision making. It is this intersection of all 3 components where decisions should be made in medicine. Making decisions based on any of these circles in isolation, would simply be bad medicine. Practicing by experience alone ignores advances in science and medicine and affords little opportunity for growth and advancement and incorporation of new knowledge. Patient values and physician values must be taken into account, but clearly, values without knowledge and experience can lead to bad choices in medicine. Evidence alone does not give us the entire picture. There are too many questions that have been unanswered by evidence and too much overlap. The integration of these 3 circles allows us to incorporate new science, evaluate the evidence, mesh that with our experience, and address the values of our own practice and our patient's preferences providing a means to best care for our patients. This is EBM.<ref name="chisholm">Will the Real Evidence‐Based Medicine Please Stand Up? : Emergency Medicine News [Internet]. LWW. [cited 2015 Oct 27]. Available from: http://journals.lww.com/em-news/Fulltext/2001/06000/Will_the_Real_Evidence_Based_Medicine_Please_Stand.4.aspx</ref>
 +
 
 +
 
 +
The volume of new literature and new evidence is overwhelming. EBM gives us a framework to evaluate new evidence in a rapid fashion and integrate that knowledge as appropriate based on its validity and importance. That being said, no individual can evaluate the constant flow of information. This highlights the importance of these principals and their incorporation into databases of reviews including: the Cochrane Database[http://www.cochranelibrary.com/], ACP journal club[https://www.acponline.org/clinical_information/journals_publications/acp_journal_club/],the TRIP database[https://www.tripdatabase.com/], the NNT[http://www.thennt.com/], and others which allow us to keep current on the best evidence for a clinical question. Understanding these principals guide us in the assimilation of this knowledge and the ability to incorporate new evidence between updates of these reviews. [[Clinical guidelines]] are evidence-based statements that help guide providers towards best practices.
 +
 
  
 
= Introduction =
 
= Introduction =
Line 13: Line 17:
 
The first and sometimes hardest step is developing the focused clinical question that will allow us to most efficiently search the literarture and most efficiently and reliably find the answer. This is done by using the mnemonic "PICO": Patient (or population), Intervention (or Exposure), Comparison, and Outcome. The more specific we can be with each of these elements, the better the question and more likely the desired outcome.
 
The first and sometimes hardest step is developing the focused clinical question that will allow us to most efficiently search the literarture and most efficiently and reliably find the answer. This is done by using the mnemonic "PICO": Patient (or population), Intervention (or Exposure), Comparison, and Outcome. The more specific we can be with each of these elements, the better the question and more likely the desired outcome.
  
It is also helpful to identify the type of question that is being sought:<ref>JAMAevidence | Book [Internet]. [cited 2015 Oct 26]. Available from: http://jamaevidence.mhmedical.com/book.aspx?bookID=847</ref>
+
It is also helpful to identify the type of question that is being sought:<ref name="jama">Guyatt, Gordon H.; Rennie, Drummond (2015). Users' Guides to the Medical Literature: A Manual of Evidence-Based Clinical Practice (third ed.). New York: McGraw-Hill. ISBN 978-0071790710.</ref>
  
 
# Therapy: Determining the effect of interventions on outcomes
 
# Therapy: Determining the effect of interventions on outcomes
Line 34: Line 38:
  
 
There are 3 basic questions that must be answered when evaluating any study:
 
There are 3 basic questions that must be answered when evaluating any study:
# Validity: Are the results valid
+
# Validity: Are the results valid?
# Results: What are the results and are they significant
+
# Results: What are the results and are they significant?
# Applicability: Can I apply these results to the patient (or population) in question
+
# Applicability: Can I apply these results to the patient (or population) in question?
 +
 
 +
The detailed questions that allow us to answer these 3 basic are different for each of the types of clinical questions, but the process is the same. The validity of the study is always asked first because if the methods are not valid then the results are not reliable and are immaterial.
 +
 
 +
The detailed questions are listed for each type below:<ref name="jama"></ref>
  
Answering these questions is a different process for each of the types of clinical questions, but the process is the same. The validity of the methods is always asked first because if the methods are not valid then the results are not reliable and are immaterial.
 
  
 
'''Therapy (RCT)'''
 
'''Therapy (RCT)'''
 
::Validity
 
::Validity
::*Were the patients randomized
+
::*Were the patients appropriately randomized?
 
::*Were patients, treatment providers, and outcome assessors blinded to group assignment?
 
::*Were patients, treatment providers, and outcome assessors blinded to group assignment?
 
::*Were the patients in each group similar at the start of the trial?
 
::*Were the patients in each group similar at the start of the trial?
Line 122: Line 129:
 
::*How precise were the results?
 
::*How precise were the results?
 
::Applicability
 
::Applicability
:::*Were all patient -important outcomes considered?
+
::*Were all patient -important outcomes considered?
 
::*Are any postulated subgroup effects credible?
 
::*Are any postulated subgroup effects credible?
 
::*What is the overall quality of the evidence?
 
::*What is the overall quality of the evidence?
Line 129: Line 136:
 
== Apply the evidence to a particular patient or clinical situation ==
 
== Apply the evidence to a particular patient or clinical situation ==
  
The last step of the EBM process is the ability to translate the ideal findings of carefully controlled studies to the less-ideal and less-carefully-controlled situation of a particular patient.  What are the particular risks and benefits for this patient?  What are his or her preferences?  What are the costs, alternatives, and availability of particular treatments? Even for a statistically significant finding, is the effect size practically significant?  Does the practitioner have the necessary skill or resources to deliver a treatment or to monitor the outcomes?
+
The last step of the EBM process is the ability to translate the ideal findings of carefully controlled studies to the less-ideal and less-carefully-controlled situation of a particular patient.  EBM starts with the patient (forming a clearly defined question) and ends with the patient (translating knowledge back to the patient). As described in the questions above, it is bringing the evidence back to the patient that it is at the heart of EBM. 
  
As medicine has continued to grow and become more complicated, the number of medical specialties has increased and their depth matures.  At the same time, the amount and complexity of the medical literature has similarly grown.  In this sense, Evidenced Based Medicine can be considered a burgeoning medical specialty, with the medical literature itself as the object of study.
+
*What are the particular risks and benefits for this patient? 
 +
*What are his or her preferences? 
 +
*What are the costs, alternatives, and availability of particular treatments?
 +
*Even for a statistically significant finding, is the effect size practically significant? 
 +
*Does the practitioner have the necessary skill or resources to deliver a treatment or to monitor the outcomes?
 +
 
 +
As medicine has continued to grow and become more complicated, the number of medical specialties has increased and their depth matures.  At the same time, the amount and complexity of the medical literature has similarly grown.  In this sense, EBM can be considered a burgeoning medical tool, with the medical literature itself as the object of study.
  
 
= Evaluation methods of EBM =
 
= Evaluation methods of EBM =
Line 141: Line 154:
 
There exists a recognized delay between the time that medical research discovers a profound truth to the time this truth is actually applied within the clinical environment. The lag has been reported as up to 7 years. Even with summaries of relevant research, many providers are still unable to allocate the necessary time to acquire and implement this knowledge. Some providers discount the entire premise of EBM. It is likely that some combination of time-constraints and knowledge management primarily contribute to this lag time.
 
There exists a recognized delay between the time that medical research discovers a profound truth to the time this truth is actually applied within the clinical environment. The lag has been reported as up to 7 years. Even with summaries of relevant research, many providers are still unable to allocate the necessary time to acquire and implement this knowledge. Some providers discount the entire premise of EBM. It is likely that some combination of time-constraints and knowledge management primarily contribute to this lag time.
  
EBM is awash in opportunities for Healthcare Information Technology solutions. Pattern-recognition and algorithm management are IT tools that may dramatically improve the collection of evidence and the application of consistent, high-quality health care services. A serious evaluation of where and how HIT can be inserted into the process of medical knowledge acquisition and application is urgently needed, as of 04/19/2006.
+
EBM is awash in opportunities for Healthcare Information Technology solutions. Pattern-recognition and algorithm management are IT tools that may dramatically improve the collection of evidence and the application of consistent, high-quality health care services. A serious evaluation of where and how HIT can be inserted into the process of medical knowledge acquisition and application is urgently needed, as of 10/27/2015.
  
 
== Advantages ==
 
== Advantages ==
 +
EBM gives the clinician the tools to evaluate new evidence and to incorporate that evidence into experience.<ref>http://ktclearinghouse.ca/cebm/intro/interest</ref>
 +
*It gives us a framework to incorporate new evidence into our knowledge base and give it a weight among what is known.
 +
*It has been the impetus in the creation of systematic reviews and concise summaries of the effects of health care. 
 +
*It has given us the identification and application of effective strategies for life-long learning and for improving our clinical performance.
 +
*It has assisted in the creation of evidence-based journals of secondary publication (that publish the 2% of clinical articles that are both valid and of immediate clinical use)
  
 
== Limitations ==
 
== Limitations ==
 +
 +
There are some potential limitations to EBM.<ref name="Greenhalgh">Greenhalgh T, et al. Evidence based medicine: a movement in crisis? BMJ 2014;348:g3725 doi: http://dx.doi.org/10.1136/bmj.g3725</ref>
 +
*There is concern that the research agenda is often set by industry who can define the disease, the treatment, and the comparison.
 +
*Branding something as "evidence based" can legitimize a treatment or procedure with marginal benefits. 
 +
*The volume of evidence is just too much to evaluate and get a clear picture
 +
*The low hanging fruit has already been well defined and further application of EBM may yield marginal gains
 +
*Overemphasis on algorithmic rules
 +
*May not perform well across patients with multiple comorbidities
  
 
== HIT’s Role in EBM ==
 
== HIT’s Role in EBM ==
 +
 +
 +
HIT is well positioned to become the link between evidence and knowledge translation to the patient. This potential is yet to be fully realized. However, the information systems and the EHR are likely to be a powerful data generator for future evidence. With well set care pathways that based upon best evidence and well respected reviews, HIT can be utilized to translate knowledge into order sets, reminders, and clinical prompts that significantly improve the time from publication and peer review to bedside care.<ref name="Bloomrosen">Bloomrosen M1, Detmer DE.J Am Med Inform Assoc. 2010 Mar-Apr; 17(2): 115–123. doi:  10.1136/jamia.2009.001370</ref>
  
 
The link below describes a process for building health science knowledge-bases used by evidence-based decision support tools. This gives an expanded view of the processes corresponding to the quality metrics, practice guidelines, knowledge services and tools, and CQI feedback loops.
 
The link below describes a process for building health science knowledge-bases used by evidence-based decision support tools. This gives an expanded view of the processes corresponding to the quality metrics, practice guidelines, knowledge services and tools, and CQI feedback loops.
Line 177: Line 206:
 
= Related articles =
 
= Related articles =
  
[[METEOR: An Enterprise Health Informatics Environment to Support Evidence-based Medicine]]
+
*[[METEOR: An Enterprise Health Informatics Environment to Support Evidence-based Medicine]]
  
[[Evidence-based management of ambulatory electronic health record system implementation: an assessment of conceptual support and qualitative evidence ]]
+
*[[Evidence-based management of ambulatory electronic health record system implementation: an assessment of conceptual support and qualitative evidence ]]
  
[[Effectiveness of Evidence-Based Congestive Heart Failure (CHF) CPOE Order Sets Measured by Health Outcomes]]
+
*[[Effectiveness of Evidence-Based Congestive Heart Failure (CHF) CPOE Order Sets Measured by Health Outcomes]]
 +
*[[Clinical Decision Support Systems for the Practice of Evidence-based Medicine]]
  
 
= References =
 
= References =
Line 190: Line 220:
 
[[Category:OHSU-SP-06]]
 
[[Category:OHSU-SP-06]]
 
[[Category:BMI-512-F-07]]
 
[[Category:BMI-512-F-07]]
 +
 +
Edits Submitted by Jason Schaffer
 +
 +
[[Category:BMI512-FALL-15]]

Latest revision as of 01:29, 19 October 2018

Evidence-based medicine (EBM) is the integration of evidence, experience, and values into clinical decision making. It is this intersection of all 3 components where decisions should be made in medicine. Making decisions based on any of these circles in isolation, would simply be bad medicine. Practicing by experience alone ignores advances in science and medicine and affords little opportunity for growth and advancement and incorporation of new knowledge. Patient values and physician values must be taken into account, but clearly, values without knowledge and experience can lead to bad choices in medicine. Evidence alone does not give us the entire picture. There are too many questions that have been unanswered by evidence and too much overlap. The integration of these 3 circles allows us to incorporate new science, evaluate the evidence, mesh that with our experience, and address the values of our own practice and our patient's preferences providing a means to best care for our patients. This is EBM.[1]


The volume of new literature and new evidence is overwhelming. EBM gives us a framework to evaluate new evidence in a rapid fashion and integrate that knowledge as appropriate based on its validity and importance. That being said, no individual can evaluate the constant flow of information. This highlights the importance of these principals and their incorporation into databases of reviews including: the Cochrane Database[1], ACP journal club[2],the TRIP database[3], the NNT[4], and others which allow us to keep current on the best evidence for a clinical question. Understanding these principals guide us in the assimilation of this knowledge and the ability to incorporate new evidence between updates of these reviews. Clinical guidelines are evidence-based statements that help guide providers towards best practices.


Introduction

The EBM Process can be summarized by four steps:

  1. Formulate a sensible, focused clinical question.
  2. Search the medical literature for evidence related to the clinical question.
  3. Rate the quality of the available studies.
  4. Apply the evidence to a particular patient or clinical situation.

Formulate a sensible, focused clinical question

The first and sometimes hardest step is developing the focused clinical question that will allow us to most efficiently search the literarture and most efficiently and reliably find the answer. This is done by using the mnemonic "PICO": Patient (or population), Intervention (or Exposure), Comparison, and Outcome. The more specific we can be with each of these elements, the better the question and more likely the desired outcome.

It is also helpful to identify the type of question that is being sought:[2]

  1. Therapy: Determining the effect of interventions on outcomes
  2. Harm: Ascertaining the effects of potentially harmful agents (including the therapies above) on outcomes
  3. Differential Diagnosis: In patients with a particular clinical presentation, establishing the frequency of underlying disorders
  4. Diagnosis: Establishing the power of a test to differentiate between those with and without the target condition or disease
  5. Prognosis: Estimating a patient's future course

Search the medical literature for evidence related to the clinical question

The second step of the EBM process is to search the medical literature. Aside from understanding the methods, strengths, and weaknesses of various search strategies and search engines, this step also involves looking for the highest level of evidence. In general, Systematic Reviews (not to be confused with general Review Articles) are considered the highest level of evidence, followed by Randomized Controlled Trials (RCT), then Case-Control Studies, followed by Expert Opinion and then, lastly, anecdotal evidence.

see Hierarchy of Evidence

More information on searching can be found here.

Rate the quality of the available studies

The third step--rating the quality of the available studies--involves a knowledge of research methodology that is important to make valid conclusions.

There are 3 basic questions that must be answered when evaluating any study:

  1. Validity: Are the results valid?
  2. Results: What are the results and are they significant?
  3. Applicability: Can I apply these results to the patient (or population) in question?

The detailed questions that allow us to answer these 3 basic are different for each of the types of clinical questions, but the process is the same. The validity of the study is always asked first because if the methods are not valid then the results are not reliable and are immaterial.

The detailed questions are listed for each type below:[2]


Therapy (RCT)

Validity
  • Were the patients appropriately randomized?
  • Were patients, treatment providers, and outcome assessors blinded to group assignment?
  • Were the patients in each group similar at the start of the trial?
  • Was follow-up complete?
  • Were the patients analyzed in the group to which they were assigned?
Results
  • How large was the treatment effect?
  • How precise was the estimate of the treatment effect? (Confidence Intervals)
Applicability
  • Were the patients in the study similar to my patient?
  • Were all patient important outcomes considered?
  • Are the treatment benefits worth the potential harm?

Harm

Validity
  • Were there clearly identified comparison groups that were similar with respect to determinants of outcome other than the one of interest?
  • Were the exposures and outcomes measured in the same was in the groups being compared?
  • Was follow-up sufficiently long and complete?
  • Is the temporal relationship correct?
  • Is there a dose-response gradient?
Results
  • How strong is the association between exposure and outcome
Applicability
  • Are the results applicable to my practice?
  • What is the magnitude of the risk?
  • Should I attempt to stop the exposure?

Diagnosis

Validity
  • Did the participating patients present a diagnostic dilemma?
  • Was the test compared to an appropriate gold standard?
  • Were those interpreting the test and gold standard test blind to the other results?
  • Did the test results interfere with the decision to apply the gold standard test?
Results
  • What likelihood ratios were associated with the range of possible test results?
Applicability
  • Will the reproducibility of the test result and its interpretation be satisfactory in my clinical setting?
  • Are the study results applicable to my patient?
  • Will the test results change the management of my patient?
  • Will patients be better off as a result of the test?


Differential Diagnosis

Validity
  • Did the study patients represent a full spectrum of those with the clinical problem?
  • Was the diagnostic evaluation definitive?
Results
  • What were the diagnoses and their probabilities?
  • How precise were the estimates of disease probability?
Applicability
  • Are the study patients and clinical setting similar to mine?
  • Is it unlikely that the disease possibilities or probabilities have changed since this evidence was gathered?

Prognosis

Validity
  • Was the sample of patients representative?
  • Were the patients sufficiently homogeneous with the respect to their prognostic risk?
  • Was the followup sufficiently complete?
  • Were the outcome criteria objective and unbiased?
Results
  • How likely are the outcomes over time?
  • How precise are the estimates of likelihood?
Applicability
  • Were the study patients and their management similar to those in my practice?
  • Was the follow-up sufficiently long?
  • Can I use the results in the management of my patient(s)?

Systematic Reviews Systematic reviews have their own set of questions that are specifically applicable to them

Validity
  • Did the review include explicit and appropriate eligibility criteria?
  • Was biased selection and reporting of the studies unlikely?
  • Were the primary studies of high methodological quality? (garbage in - garbage out)
  • Were the assessments of the studies reproducible?
Results
  • Were the results similar from study to study?
  • Did the results from one or few studies overwhelmingly drive the overall results?
  • What were the overall results of the study?
  • How precise were the results?
Applicability
  • Were all patient -important outcomes considered?
  • Are any postulated subgroup effects credible?
  • What is the overall quality of the evidence?
  • Are the benefits worth the costs and potential risks?

Apply the evidence to a particular patient or clinical situation

The last step of the EBM process is the ability to translate the ideal findings of carefully controlled studies to the less-ideal and less-carefully-controlled situation of a particular patient. EBM starts with the patient (forming a clearly defined question) and ends with the patient (translating knowledge back to the patient). As described in the questions above, it is bringing the evidence back to the patient that it is at the heart of EBM.

  • What are the particular risks and benefits for this patient?
  • What are his or her preferences?
  • What are the costs, alternatives, and availability of particular treatments?
  • Even for a statistically significant finding, is the effect size practically significant?
  • Does the practitioner have the necessary skill or resources to deliver a treatment or to monitor the outcomes?

As medicine has continued to grow and become more complicated, the number of medical specialties has increased and their depth matures. At the same time, the amount and complexity of the medical literature has similarly grown. In this sense, EBM can be considered a burgeoning medical tool, with the medical literature itself as the object of study.

Evaluation methods of EBM

The evaluation of medical evidence for EBM is primarily based upon research study design parameters (population size, types of controls, analytic methodologies, etc.). The randomized-controlled trial is considered to yield the strongest evidence within medical research. Inherent in this, is the ability to judge the quality of the medical literature, to understand what statements can rationally be made from the medical literature, to appreciate the strength of those inferences, and to realistically apply them to a particular patient or clinical situation.

Controversies

There exists a recognized delay between the time that medical research discovers a profound truth to the time this truth is actually applied within the clinical environment. The lag has been reported as up to 7 years. Even with summaries of relevant research, many providers are still unable to allocate the necessary time to acquire and implement this knowledge. Some providers discount the entire premise of EBM. It is likely that some combination of time-constraints and knowledge management primarily contribute to this lag time.

EBM is awash in opportunities for Healthcare Information Technology solutions. Pattern-recognition and algorithm management are IT tools that may dramatically improve the collection of evidence and the application of consistent, high-quality health care services. A serious evaluation of where and how HIT can be inserted into the process of medical knowledge acquisition and application is urgently needed, as of 10/27/2015.

Advantages

EBM gives the clinician the tools to evaluate new evidence and to incorporate that evidence into experience.[3]

  • It gives us a framework to incorporate new evidence into our knowledge base and give it a weight among what is known.
  • It has been the impetus in the creation of systematic reviews and concise summaries of the effects of health care.
  • It has given us the identification and application of effective strategies for life-long learning and for improving our clinical performance.
  • It has assisted in the creation of evidence-based journals of secondary publication (that publish the 2% of clinical articles that are both valid and of immediate clinical use)

Limitations

There are some potential limitations to EBM.[4]

  • There is concern that the research agenda is often set by industry who can define the disease, the treatment, and the comparison.
  • Branding something as "evidence based" can legitimize a treatment or procedure with marginal benefits.
  • The volume of evidence is just too much to evaluate and get a clear picture
  • The low hanging fruit has already been well defined and further application of EBM may yield marginal gains
  • Overemphasis on algorithmic rules
  • May not perform well across patients with multiple comorbidities

HIT’s Role in EBM

HIT is well positioned to become the link between evidence and knowledge translation to the patient. This potential is yet to be fully realized. However, the information systems and the EHR are likely to be a powerful data generator for future evidence. With well set care pathways that based upon best evidence and well respected reviews, HIT can be utilized to translate knowledge into order sets, reminders, and clinical prompts that significantly improve the time from publication and peer review to bedside care.[5]

The link below describes a process for building health science knowledge-bases used by evidence-based decision support tools. This gives an expanded view of the processes corresponding to the quality metrics, practice guidelines, knowledge services and tools, and CQI feedback loops.

Evidence-based HealthCare Decision Support System

Clinical Trial Registries

See Clinical trial registry

Searching for Evidence

See Searching for Evidence.

Pubmed’s new evidence based searching utility

On pop up windows during drug order entry displaying the latest evidence-based medicine articles from pubmed.

Statement of the problem: there is a lack clinician’s access to contextual information at the point of drug order entry in most CPOEs.

Background information: In the mid-1990s, PubMed introduced it’s search engine. This search engine is a web-based information portal for all types of medically-related journals, which now includes a category for evidence based medicine (EBM). PubMed later introduced limits and automatic web service interfacing to view returns via a web page from a program written in java. Thus, it is possible to build a program that would return the latest evidence based medicine articles limited to those published in the last year to the clinician automatically during drug-order entry on Windows and Unix based CPOE clients. This may be useful in cases where a clinician has incomplete knowledge of a drug’s important recent studies and access to the articles is only provided through a button on the CPOE entry screen.

A description of any alternatives: Alternatives are limiting drug decision support to guidelines.

Major conclusions: With Pubmed’s new evidence based searching utility, one may ask, is it worthwhile to put up a link to PubMed articles for the most recent year during drug order entry? This would be a teaching aid as well, given that only the most recent EBM article would be returned. This would be useful in case of perhaps, when the entering clinical is a pharmacist.--Crawford 18:59, 5 November 2007 (CST)

Related Topics

Basic statistical concepts for interpreting study data.

Related articles

References

  1. Will the Real Evidence‐Based Medicine Please Stand Up? : Emergency Medicine News [Internet]. LWW. [cited 2015 Oct 27]. Available from: http://journals.lww.com/em-news/Fulltext/2001/06000/Will_the_Real_Evidence_Based_Medicine_Please_Stand.4.aspx
  2. 2.0 2.1 Guyatt, Gordon H.; Rennie, Drummond (2015). Users' Guides to the Medical Literature: A Manual of Evidence-Based Clinical Practice (third ed.). New York: McGraw-Hill. ISBN 978-0071790710.
  3. http://ktclearinghouse.ca/cebm/intro/interest
  4. Greenhalgh T, et al. Evidence based medicine: a movement in crisis? BMJ 2014;348:g3725 doi: http://dx.doi.org/10.1136/bmj.g3725
  5. Bloomrosen M1, Detmer DE.J Am Med Inform Assoc. 2010 Mar-Apr; 17(2): 115–123. doi: 10.1136/jamia.2009.001370

Edits Submitted by Jason Schaffer