Vendor Selection Criteria: Vendor assessment

From Clinfowiki
Revision as of 02:28, 24 September 2015 by HannahJoy (Talk | contribs)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

Most vendors typically fall into one of the three categories:

  1. Vendors that develop their own software organically on a single source code, one database, single instance.
  2. Vendors that may operate under one name, but offer several acquired products, including some custom programs.
  3. Vendors that have been acquired/sold/merged as a means to stay more competitive.

The first category of vendor is less likely to run into major market conflicts to keep its software modern and compliant with regulations. Vendors that operate on a single source code have far less difficultly staying current and/or responding to rapid changes in the market and mandated IT standards.[1] However, selection of a vendor also depends on the size of the organization, and the list of services they need. A larger health care organization might need more services, hence the vendor they select must offer multiple products.

Analyzing EHR Business Requirements

The evaluation team or decision team should consist of clinical healthcare professionals including house staff, technical support professionals, administrators, and finance department team from all areas of the institution. Each person brings to the table a different perspective of usage from their daily job responsibilities. Institution-wide involvement creates awareness and knowledge of EMRs and their benefits, as well as an understanding of the upcoming modifications in the workflows. In summary, the following tasks must be followed before starting EHR business requirements analysis:

  • Assemble an Evaluation Team
  • Define the Product, Material or Service
  • Define the Technical and Business Requirements
    • In A Guide to the Project Management Body of Knowledge (PMBOK Guide), the following tools and techniques are defined for collecting requirements: [2]
      • Interviews
      • Focus groups
      • Facilitated workshops
      • Group creativity techniques
      • Group decision techniques
      • Questionnaires and surveys
      • Observation
      • Prototypes
      • Benchmarking
      • Context Diagrams
      • Document Analysis. [2]
  • Define the Vendor Requirements
  • Publish a Requirements Document for Approval

ICD-10 Questions for Vendors

The conversion from ICD-9 codes to ICD-10 codes has a deadline of October 1, 2015. PT in Motion asked vendors for suggestions as to what physical therapists should be concerned about in regards to ICD-10 coding and choosing their electronic medical record (EMR) software. The vendors recommended that four questions be asked regarding a software's compatibility with the new ICD-10 coding.

  1. How will the EMR or practice management system handle ICD-9 to ICD-10 conversions with current patients?
    • There are about 68,000 ICD-10 codes as opposed to 14,000 ICD-9 codes. The EMR should allow for a smooth transition.
  2. Will all ICD-9 codes convert to ICD-10 codes?
    • The EMR may still need some input from clinicians and billers to help determine final codes.
  3. Is the EMR application/system prepared to transition from ICD-9 to ICD-10?
    • Answer should always be Yes.
  4. Can the application or system support both ICD-9 and ICD-10 codes?
    • It is important to make sure the system selected can support both ICD-9 and ICD-10 code sets.[3]

Evaluating Possible Software Vendors

In the article "How to Choose the Right Software Vendor" published in CIO, the following suggestions were offered by IT Executives: [4]

  • Figure out what it is you really need.
  • Check the software provider's credentials and certificates.
  • What do other customers have to say?
  • Can it scale?
  • Check for hidden or additional fees.
  • Who gets custody of your data in the case of divorce?
  • Take a test drive.
  • Agree to key performance indicators (KPIs) before signing a contract.

In addition, other important questions to take into account for software vendor selection included:

  • How easy is it to set up and train users?
  • How will you support me after the sale?
  • How are updates and upgrades managed?

Vendor Product Demonstration

Vendors must be allowed to make demonstrations of their product to the entire work team of the organization. Make sure to prepare a demonstration worksheet beforehand, so that it will help with the team discussions back in the office. This demonstration worksheet should contain the core criteria that the team agreed was important, as indicated on the evaluation tools. In addition, also include a section for comments and further questions you have to ask the vendor.Make sure to see the demonstrations of the product in a live environment as well as stimulated demonstration set up in the office. Following the presentations, the team should rank the vendor presentations by comparing the selection criteria to what it saw. [5]

Reach Out To Peers

Talk to local providers whose offices or organizations are of similar size and patient mix. Ask if they contracted directly with a vendor or worked through a network. Talk with both providers and ancillary staff about their experience implementing and using the system so far.

The U.S. Department of Health and Human Services Health Resources and Services Administration (HRSA) recommends asking peers the following questions[6] regarding their experience with their EHR:

  • How long did it take to get up and running?
  • How much disruption and delay did you experience?
  • What worked to minimize loss in productivity?
  • Did the vendor charge additional or unexpected costs not in the original contract?
  • Have you performed an evaluation of the system? Is it helping you reach your goals?

Regional Selection Center EHR Selection Criteria

The Regional Selection Center suggests that medical practices compare EHR vendors using the following criteria before selecting a specific product. [7]

  • Will the vendor’s product accomplish key practice goals?
  • Clarify EHR system start-up prices before selecting a vendor including costs associated with hardware, software, maintenance and upgrades, phase payment options, lab and pharmacy interfaces, connection to health information exchange, and customized quality reports.
  • What implementation support does the vendor offer?
  • What are the costs, roles, and responsibilities associate with the data migration strategy?
  • What are your sever options?
  • What is the products ability to integrate with other products?
  • What are the privacy and security capabilities of the product and what is the back-up plan?
  • How does the product link payments with EHR incentive rewards, which are necessary to meet the practices EHR implementation goals and milestones?
  • What is the vendor’s stability and market presence?
  • What is the cost to link the product to HIE?
  • What are the costs associated with legal counsel for contract review versus open sources through medical associations?

Cost and Budget

These five cost components of EHR implementation should be considered: [8]

  • Hardware Cost: Database servers, desktops, printers, laptops, scanners, telecom [8]
  • Software Cost: EHR application and upgrades, modular interfaces depending on on-site deployment or SaaS EHR deployment [8]
  • Implementation Assistance: The Regional Extension Center (REC) can be contacted to determine if the organization is eligible for assistance with IT contractor, consultants, hardware/network installation, workflow redesign support, etc. RECs are located throughout the country to help health care providers select, implement and use EHRs [8]
  • Training Cost: Training for all necessary staff and technical support during and after implementation [8]
  • Ongoing Network Fees and Maintenance: License agreements, maintenance agreements, ongoing staff education, IT support fees and system upgrades [8]

Certain systems can require specific hardware, interfaces, networking, training and support resources that all add on to the price. Practices should take these additional services into account to get a comprehensive picture of what the technology will cost. [9]

Associated Costs – in addition to EHR licensing and maintenance, studies report expenses can be expected for; over 130 hrs./physician for system training, reduction in patient volume by as much as 50% during initial implementation, and significant expenses for additional IT infrastructure (e.g. approx. $10,000 for primary care practice). [10]

Vendor Financing

A 2010 Health management Technology article states, a number of healthcare technology vendors are now offering free financing for electronic health record (EHR) purchases, in an effort to increase adoption of electronic records. [11]

Cost is physicians' top concern when considering EHRs, according to a recent Ingenix survey of 1 ,000 physicians and physician-practice administrators. More than 80 percent ranked cost as one of the greatest risks to deploying technology in their practice. [12]

Vendor Proposal (Request)

A request for information (RFI) is often used to solicit information from vendors about their products and services. It is a fact-finding process, often used as the first step in narrowing down the field of vendors when considering future purchases. Most vendors have marketing materials that can provide in-depth information regarding their products. For some organizations, this packet of materials, along with a profile of the company, its history and services, standard agreements, and a cover letter, may be a sufficient response to an RFI.In other instances, an RFI may include more information focused on the specific areas of organizational need as it relates to product functionality. Sending an RFI to vendors is an effective way to stay current and an excellent starting point for the more formal selection process.[13]

Unlike an RFI, a request for proposal (RFP) is a formal request sent to a vendor or group of vendors for specific responses as to how their company, products, and services can meet the organization’s unique requirements. It generally includes a complete summary of related costs (hardware, software) and services (support, training, implementation, and consulting). An RFP can become the basis for a contract, which forms a legal obligation between two parties, the vendor and the solicitor. For this reason, both the vendor and the solicitor should carefully consider the phrasing of the questions and corresponding responses. For instance, in seeking an EHR, the solicitor should request specific information about how the product will support the functionality needed by the healthcare organization.Organizations should consider forming a selection committee consisting of key stakeholders and end users as a first step in preparing the RFP.[13]

  • Requesting a proposal is just as important as selecting the type of medication that will be used in your practice. When you ask for a proposal it is important that as a practice you are clear and concise as to what you and your practice will need. This process, though very monotonous is absolutely crucial to the flow and efficiency of your practice. It also gives you an inside knowledge to the various EHRs that are available and will give you the ability for good comparison to what is on the market [14]
  • Make request to selected vendors
  • A clear list of specific needs (EHR system requirement) should be written
  • Prepare a clear criterion of the system that will be used to make the selection

Proposal rates

The vendor that offers the best fit with your selection criteria may not necessarily submit the best written or most attractive proposal. The most important thing is how each vendor’s proposal rates on the selection criteria that you included in the Request For Proposal. As vendors submit their proposal, use your evaluation tool based on the criteria stated in the proposal to rate their products and make negotiations.[15]

Separate vendors for each identified core IT implementation areas

Identification of core areas which are in dire need of HIT have to be recognized before the HIT implementation. And each area identified should be supervised by a different vendor.This can best be adopted in time bound projects and could prove cost effective.[16]

Personalization of HIT

For the smooth work flow which is primarily patient centered, the HIT should focus on the user friendly for physicians and nurses who can handle them with ease, so they contribute to the successful patient’s outcomes.[16]

Transparency in communication with stakeholders for collaboration

This is essential as there are various stakeholders involved, always giving scope for power struggles and may fail the implementation as well as integration and may eventually face resistance from end users.Enterprise governance with good leader even from non IT can work successfully if utilized from the start. Also, this will enable the focus on patient care rather than technology.Starting from vendor selection to discuss with them, the criteria upon which they were selected and capability to contribute as per needs, also prior feedback from end users, all stakeholders perspectives and opinions before, during and after implementation can build a strong interpersonal relationships with primary target of patient centered care environment.[16]

The ability to have an openness for the reporting of problems in a particular level to the advisory committee/ any which can resolve them within hours rather than dragging them for days and weeks. These problems with solutions can be circulated among all the stakeholders.[16]

Misc considerations

  • It is not uncommon for developers or implementers to impose lock-in features in order to protect their IP rights. How mature is the support community for the particular vendor and is there a marketplace for support and services to their product or is support limited to their own team?
  • A multidisciplinary team, usually a subset of the decision team consisting of a physician, IT personnel and a senior management must conduct a site visit to one of the vendor customer sites and vendor headquarters. Informal discussions at this level were quite informative as they bridged the gap between theory and practice for actual software implementation and usage.
  • An unstructured “drive-by” demonstration of the system must be arranged for, wherein the vendors conduct product demos, demonstrate product futures and distribute promotional materials for a majority of the employees to get the feel of the new systems. It facilitates building a comfort level and relationship with the vendor and the product offered.
  • Can the company demonstrate tangible ways that use of the product can increase hospital revenue? If yes, can you validate this/these claims?
  • Is the vendor willing to contract to go "at risk" for any part(s) of the contract?
  • What other services does the company offer (especially useful for rural or small hospitals or practices)? e.g. consultants to do special projects, data transmission/claims, billing
  • Can the record be accessed at home by clinicians and patients, or do they have to be on-site at the facility?
  • Is the vendor capable of adapting to emerging technology such as open source programming, cloud computing, "tablet" PCs/Macs, demand for smartphones (e.g. iPhone), and Web 2.0 technology?
  • Is the vendor capable of integrating its system with a personal health record and allowing more patient control?
  • Can the company provide return on investment analysis?
  • Vendors may not be chosen if the current managers have been convicted of or have had a civil judgment for the following: fraud, antitrust violations, embezzlement, theft, etc.
  • Reduced pharmaceutical costs derived from having information available at the time it is needed
  • How can this EHR be an asset for the solo practitioner who sees this technology primarily as a financial drain?
  • How much extra financial cost will it be to train outpatient clinics that are a subset of the system?
  • Does the vendor return loss money if their system can not complete established goal in scheduled time?
  • Can the company provide a multi-background consult team from doctors, nurses, IT engineer, to attorney?
  • Does the vendor qualify the organization's acquisition policies? Is the vendor CCHIT certified?
  • Certification can be verified at the Certified Health IT Product List (CHPL) [17]
  • Is the vendor active in national EMR leadership task forces, such as the Electronic Health Records Vendors Association [18]
  • Will the vendor be supporting the organization's desired implementation strategy?
  • Check the track record of vendor for operations and maintenance support?
  • Will the vendor sell or monetize of our clinical data for research or any other purposes?
  • Is Application Support provided by on-shore resources?
  • Clarify the roles, responsibilities and costs for data migration if desired. [19]
  • Does the Vendor demonstrate financial and management stability?
  • Does the vendor have experience with implementing the product in a similar type of organizations? How many?
  • Conduct a site visit [19]
  • Ensure that the site is similar to you - similar number of beds, acuity, rural/urban setting
  • Flexibility that allows for significant changes in product or order lines. Thus, this enables for businesses to modify orders, if necessary, without having to face a penalty from the vendor. Not always do things go as planned and if any modifications are needed, there should not be any consequences that inhibit this from occurring.
  • The ability to provide all the products/services required and/or the complete solution. This in turn can provide the whole picture in allowing for the consumer more transparency in terms of being able to distinguish between the products/services that a particular vendor is offering. As a result, this can save time and money in terms of a consumer being able to choose a certain business that can provide everything that is needed instead of wasting time and effort in purchasing products/services from individual vendors that have a small selection available.
  • The ability to have a consistent supply of products or services readily available for the business to purchase at all times. There is always a need for products and services to be provided on a regular basis; thus a business does not want to have a vendor that has supply issues. If a vendor is not reliable, then it can affect the business’s ability to in turn supply and provide for its customers, which can have a dramatic effect on the reputation and finances of the business.
  • Does vendor volunteering disclose adverse events or near misses data involving vendor’s EMR system to potential buyers? Are there protocol designed to notify federally designated patient safety organizations of IT related adverse events or near misses? [20]
  • Does the EHR have the ability to link payments and incentive rewards to the implementation of milestones and performance goals, as well as conduct data queries for support of quality improvement? [6]
  • Attend Demonstrations with a rating form this will help you write down the important parts of what you DID and DID NOT like and help you make the best EHR decision for your practice. Sales people are awesome and so are the demonstrators but it's important to stay focused on what will work for you.[21]
  • EHR Goals must be set. Goals should be specific, measurable, attainable, relevant, and time bound.[22]
  • Key Decisions should be made by making a list of potential deal-breakers such as deciding whether EHR data should reside in-office, a vendor server, or in web-based storage (“cloud storage”). To help form a list of potential deal-breakers, vendor websites should be researched. Making key decisions up-front will enable a practice to effectively narrow the field..[22]
  • Asking about product experiences and user experiences are crucial before selecting a vendor. [23]
  • An EHR evaluation tool should be utilized before selecting a vendor. EHR evaluation tools can be obtained from medical societies. [23]
  • Online form from HRSA available for establishing EHR specifications to help in vendor section. The form's name is 2011 EHR Selection Guidelines for Health Centers and can be downloaded from [6]
  • Requesting a demo from the vendor keeps a potential buyer informed on prospective features an EHR possesses. [23]
  • Asking if the vendor is committed to training the institution's in house staff. [24]
  • Is there capability to tailor current features of EMRs to enable a smoother changeover from paper to electronic records and to facilitate meeting the criteria for meaningful use. [25]
  • Will the EHR meet present and future requirements? How user friendly is the EHR? [26]
  • Which category do you fall under should be one of the first questions you ask. Are you a small practice 1-15providers, a medium practice 15-100providers, or a large practice greater than 100providers ?[14] Once size of practice has been determined including the potential to reach the size should help you narrow down as to which vendors might best suit your needs.
  • Percentage of research and development reinvested into the company.[27]
  • Life cycle state or maturity of EHR system products should be asked(i.e., the occurrence of software obsolescence)..[27]
  • Frequency of software product updates..[27]
  • Customer support availability..[27]
  • Certification status of the EHR..[27]
  • Setting goals before selecting an EHR. The goals should be specific, measureable, attainable, relevant, and time bound. Having these goals will guarantee that your organization benefits from the EHR. [27]
  • Make a rundown of potential issues and choose where you wish to have your EHR data to reside. Whether you need your EHR information to reside in a live in-office, a vendor server, or web-based storage. Making this list before selecting an EHR will help your practice to narrow the field. [27]
  • Plan site visits. Ask vendors for a list of practices that have successfully implemented the EHR. At site visits it is important to have a list of questions for the practice during, before, and after implementation of the EHR. [27]
  • No vendor can offer all possible services and products. Therefore, it is important to assess if an ecosystem of partners exist for the selected vendor. Is there active developer program? Does the vendor use an open strategy or single a source strategy? [28]
  • A good vendor should have a healthy management team that keeps it growing and improving. Does the vendor have stable, visionary and efficient management team?
  • We certainly do not wish to select a vendor that may bankrupt in the near future. Does the vendor have resources to compete in the market and likely to win in the future?

  • Find out how many other practices and types in the nation use their system
  • Find out how many customers have switched to a different system and why.
  • Ask other users and customers of the system why they chose it [29]

Professional Support

  • What kind of support is included in the initial purchase price of the EHR system?
  • How long will support be provided (e.g, on site, by telephone, or email)?
  • Will the vendor work with contractors to install the system or just be providing with a set of network, telecommunications and electrical specifications?
  • Will the vendor be working with the organization to customize software features such as the templates that will be used? [30]
  • Will the travel expenses of the vendor be included in the implementation estimate? Are the travel expenses fixed or a percentage of the implementation fees?[31]

Vendor viability

Vendor viability is an important criteria that must be checked before a large investment is made in a vendor EHR. Will the vendor be around in nine years (the average life span of a significant IT investment)? If not, can the organisation live without them? Evaluations by neutral third-party analysts like Gartner, Chilmark, KLAS, VHA consulting and The Advisory Board must be considered to see what these analysts are saying about the vendor’s prospects in the market. Questions about vendors viability must be considered such as: Is the vendor in solid financial shape? What’s their monthly burn rate vs. income? How many days cash-on-hand do they maintain? What does their sales pipeline look like? Does the vendor’s executive leadership team have a track record for jumping from one company to another or do they have a track record of longevity and success? How much is the vendor spending on sales staff in comparison to engineering and product development staff? The best products are supported by a very lean sales staff. That’s because great products sell themselves.[32]

There are two types of viability: Financial viability and Strategic viability.

A vendor’s financial viability can be assessed through simple analysis of the vendor’s financial statements or the company’s financials. Strategic viability, meanwhile, requires a more qualitative assessment of the vendor. A vendor may be financially viable enough to keep the doors open, but the questions to be considered are whether they can continue to invest in their product at the same rate as other vendors, or if the product itself is strategic to the vendor relative to the vendor’s other products.

The criteria for Assessing Financial Viability are:

  • Revenue Size :Generally, a big vendor (as measured in annual revenue) is more viable than a very small vendor.
  • Profitability: The vendor generate must enough cash flow from operations to support continued investment in development and support.
  • Balance Sheet: The balance sheet of the vendor must be checked to see if they have enough working capital.The combination of cash on hand and accounts receivable should greatly outweigh the vendor’s accounts payable and short-term debt obligations. Also, too much long term debt is most likely not a good thing.

The criteria for Assessing Strategic Viability are:

  • Ongoing Investment: A vendor must continually invest in product enhancements and customer support.Development is especially important in light of new EHR certification requirements such as ARRA and CCHIT.
  • Product Importance: A very large vendor may have greater resources, but “starve” one product to “feed” another.
  • Role in Consolidation:A large, growing vendor is more likely to be an acquirer. A small, undifferentiated vendor is more likely to be acquired.[33]

Vendor Background check

If the vendor is your only source of information in selecting an EMR, then there is definitely a problem. An EMR site visit is a good step in the right direction for getting outside information on the vendor. Take it a step further and do background check on that EMR vendor. You may not do this with every vendor you demo, but should definitely do it for those you are considering purchasing. A search online will turn up a lot of information. You can also do searches on EMR specific forums like They are full of reviews and information on all sorts of EMR vendors. Read the reviews with some skepticism, but use the information as another point of reference and discussion with your EMR vendor. Plus, you can validate other people's comments as you test drive the EMR yourself. A number of EMR vendors also have online user groups. People participating on these online forums can provide you a wealth of information about that EMR vendor as well. You might also consider searching for any past legal action against the company. Other organizations like KLAS or physician organizations have resources and reviews of EMR vendors that can be another source of information on an EMR vendor. [34]

Extensive Testing of EMR Software Prior to Implementation

Vendors should be able to demonstrate the conduction of rigorous testing of their EMR, such as: [35]

  • Unit Testing – testing of software codes and algorithm to detect errors which might lead to inaccuracies in patients’ medical records i.e., transposition of patient information, omission of diagnostic test results [35]
  • Regression Testing – Testing “end to end” functionality after the introduction of an additional ‘unit’ of functionality or a “bug fix” to confirm that the newly introduced change does not “break” related processes. [36]
  • Integration Testing – testing for integration and interface between different vendor modules i.e., faulty integration of CPOE and Pharmacy modules could result in physicians orders not being forwarded to the pharmacy department [35]
  • Systems Testing – testing of the EMR system’s ability to run on a multiple user environment, have optimal runtime availability, have an interface that is highly usable and intuitive, with strict adherence to information security regulations such as HIPAA [35]
  • User Acceptance Testing – testing by end users (physicians, nurses, administrative personnel) that validates whether all specified requirements and functionalities for the EMR have been met [35]
  • Usability Testing - Realization of EMR benefits depends largely upon usability. Usability testing with representative end users validates “the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use” (p. 331). [37]
  • Stress Testing - "running the software under less-than-ideal conditions. Look at your software and determine external resources and dependencies it has." [38]
  • Load Testing - "operate software with the largest possible data files. If software operates on peripherals such as printers, connect as many as you can." [38]

Face the Interfaces

One of an EHR’s most valuable assets is its potential to share information with other IT systems such as lab and diagnostic services. Interfaces allow the EHR system to communicate with other applications. These applications can reside outside the practice, such as lab applications, or can be another system within the practice, such as a billing system.[39]

Interface development is completed during the implementation phase; however, interface functions and costs must be a consideration during system selection. One cannot assume that a given EHR product will automatically communicate with other electronic functions such as appointment scheduling or billing. Vendors should be asked to address the issue of integrating lab and radiology results into the electronic record.[39]

Many vendors offer ancillary services such as automatic statement processing, billing services, consulting services, additional coding enhancements, and claim scrubbing. One may want to take advantage of these services later. These items are considered third-party or add-on applications.[39]

Many practices are considering the implementation of portals that offer patients a secure Internet connection to the practice. The portal allows patients to schedule appointments, complete paperwork, and access test results (or even their medical record), depending on the services the practice offers. These portals have the potential for increased practice production, increased revenues, and increased patient satisfaction. [39]

Interfaces can be a tricky piece of system selection and the implementation that follows. Red flags during the selection process include:

  • One-way or bidirectional interface not specified. The system may only accept demographic or appointment data from the practice management system but not feed data back to it.
  • Data format not specified. Data can be entered into EHRs in one of two formats--as free text (unstructured data) or as defined elements (structured data) such as forms and pick lists. Each practice must weigh the pros and cons for itself.1 However, ask vendors if their products support standard vocabularies such as SNOMED, Medcin, and LOINC.
  • Additional license costs not specified. Additional fees for third-party applications could apply, such as plug-ins for a drug database or CPT and ICD-9 coding, code scrubbing, database user seats, and anti-virus software.
  • Interface promised but not available. A vendor may say it can interface with your billing application or a lab system, but after the contract is signed it may advise you that the interface is not available. That can place a serious strain on the practice and require you to enter data twice.[39]

Interface History

One of the more efficient methods of determining a particular EMR's ability to handle interfaces is by reviewing the history of interfaces which have already been built and established. [40] The following questions should be answered in order to establish if the EMR being considered meets your organization's needs:

  • Which vendors (and which of their applications) have they interfaced with?
  • What type of information (patient demographics, order entry, laboratory results, billing, radiology findings, etc.) was interfaced?
  • How many interfaces were built, and what is the maximum the system can support?
  • What is the capacity of the interface; how many messages where sent and/or received per day; and what is the theoretical maximum amount allowed?
  • What, if any, additional costs were involved in creating, operating, and maintaining the interface?
    • Where there additional fees depending on the amount of data being sent/received?
    • Were there any costs to expand and/or modify the capabilities of the interface in the event the organization grows and requires additional functionality?

User Satisfaction : User-Centric Selection

  • User satisfaction of the clinical staff plays a key role in vendor selection and can predict the success or failure of the EMR system implementation [41]
  • Since users may have varied experiences according to Kannry, Mukani, and Myers (2006, p. 89)[41] it is important to include a multidisciplinary team from clinical, administrative and IT personnel as part of the vetting and selection process.
  • The creation and use of Scripted Scenarios representative of user workflow was helpful even when users had little technical knowledge of EMR systems. [41]

Check Vendor References

  • Each vendor being considered should provide minimal of three (3) references, to include a) physician users, b) IT person, and c) senior manager of the facility or practice.
  • Vendor provided references may be considered 'happy customers' with biased viewpoints.
  • Check several references on your own, outside of the provided references from the proposed vendor.
  • The chosen reference should be of a comparable size and structure
  • Have a prepared list of questions to ask.
  • Compare vendor satisfaction with current customers.
  • Ask supplied list of references how they acquired the EHR system, eg did they purchase the system or was it provided by the vendor?* Ask each reference to rate the vendor on a scale from 1 (very dissatisfied/strongly disagree) to 5 (very satisfied/strongly agree) for each criterion
  • Each vendor reference should provide reference on background information such as version of system currently being used, duration using current system, length of implementation, number of interfaces practice currently installed and estimated time to install, training provided on-site or remote.
  • Each vendor reference should provide reference on current system overview such as system reliability, satisfaction with vendor, and overall satisfaction with system.
  • Research references the vendor did not mention, visit facility to get an unbiased viewpoint
  • Must use listservs, internet searches and networking to identify other users to interview to get more objective viewpoint.
  • All interviewers should utilize the same questionnaire approved by the project manager or steering team.
  • Notes must be recorded by each interviewer for each customer interviewed.
  • These notes should be made part of the overall evaluation process, ideally kept by the project manager. [42]
  • Is the vendor currently involved in or have a history of any litigation with customers?
  • Does the vendor have a track record?
  • Has the vendor been in business for long?
  • Bring a diversity of positions from your committee.
    • At minimum bring a physician, IT specialist, and senior management person.
  • Observe during actual patient encounters.
  • Talk to all personnel including front office, IT, billing, nursing, nursing support staff, etc.
  • Observe other departments such as billing and labs using the vendor.
  • Choose a site to visit based on your own research rather than one provided by the vendor.
  • Perform site visits to similar organizations that are currently implementing the vendor’s product and ask plenty of questions to all end users (physicians, administrative staff, nurses, etc.) [43] [44]

Site Visits

Once you have narrowed down your search to two vendors, you’ll want to visit a practice where the system has already been implemented. Ask the vendors for a client list and contact the references yourself. If there is no local client base, keep in mind that you won’t have the support of a local colleague, and that can be challenging. Only visit practices that are similar in size and specialty to yours; if they use different templates you may not get the full idea of how the system can be applied to your specialty.[39] During site visits, ask how engaged the vendor was in the implementation. For a truly successful implementation, you’ll need a close relationship with your vendor. Be sure that the vendor’s project manager provides face-to-face interaction with your staff. The project manager should also attend staff meetings and produce regular project updates as the implementation progresses.[39] Conducting site visits at other institutions that have already implemented the vendor’s system may provide additional and practical insight. Below is a list of criteria that should be observed on site visits. The following are criteria established by MetaStar and can be found here. [45]


EHR vendors may enter contracts with confidentiality or non-disclosure terms that can prevent transparency. Examples include:

  • Disclosures required by law or regulation, sometimes with an obligation to give the other party advance notice and the opportunity to oppose the disclosure or seek confidential treatment.
  • Disclosure of information that has been independently developed by the disclosing party
  • Disclosure of information that is available to the general public or has been provided separately to the disclosing party without violation of an agreement.
  • Are there any hidden fees associated with training, support, consultant costs?

The definition of confidential information may be broad and could restrict your ability to share access to the EHR technology developer’s software in order to compare different EHR technology developer systems, provide access to researchers, or even address possible patient safety concerns. You should review the confidentiality and non-disclosure language carefully to make certain it does not inhibit your ability to conduct activities you value. [46]

Provisions for Vendor Assessment

A list of resources is available for use and provided by the National Learning Consortium (NLM). It is listed as “examples of tools that are used in the field today” for vendor assessment.[47]

This list is as follows:[47]

  • EHR Contracts: Key Contract Terms for Users to Understand
  • EHR Demonstration Scenario, Evaluation and Vendor Questions
  • NLC Contracting Guidelines and Checklist for EHR Vendor Selection
  • NLC Reference Checking for EHR Vendors
  • Request for Proposal (RFP) Template for Health Information Technology
  • Vendor Evaluation Matrix Tool
  • Vendor Meaningful Use Compare Tool
  • Vendor Pricing Template

All of the above resources are documents that assist in the development of vendor assessment.

return to Vendor Selection Criteria


  1. Types of vendors
  2. 2.0 2.1 A Guide to the Project Management Body of Knowledge (PMBOK Guide). (5th Edition). (p.114-117)
  3. Anonymous. (2013). 4 Questions You Should Ask EMR Vendors: ICD-10 Codes. PT in Motion; 5(8): 12.
  4. How to Choose the Right Software Vendor
  6. 6.0 6.1 6.2 Health Resources and Services Administration
  7. Selecting or Upgrading to a Certified EHR.
  8. 8.0 8.1 8.2 8.3 8.4 8.5 (2014). How much is this going to cost me?
  9. McCarthy, K. 5 important considerations when choosing EHR. 2014.
  10. Understanding the True Costs of an EHR implementation
  11. EHR financing offered. (2010). Health Management Technology, 31(2), 10. Retrieved from
  12. EHR financing offered. (2010). Health Management Technology, 31(2), 10. Retrieved from
  13. 13.0 13.1 Request for information
  14. 14.0 14.1 Kenneth G. Adler, MD, MMM Fam Pract Manag. 2005 Feb;12(2):55-62How to Select an Electronic Health Record System
  16. 16.0 16.1 16.2 16.3 Blake, R.T., Massey, A.P., Bala, H., Cummings,J., Zotos,A. (2010).Driving health IT implementation success: Insights from The Christ Hospital.53(2),131-138
  17. Certified Health IT Product List (CHPL) list of EMRs.
  18. HIMSS Electronic Health Record Association
  19. Cite error: Invalid <ref> tag; no text was provided for refs named factors_selecting_vendor
  20. Better Patient Safety is Goal of Confidential EHR Error Reports .
  21. HEY, WHO DID THIS? Note: there needs to be something here
  22. 22.0 22.1 Vendor Assessment
  23. 23.0 23.1 23.2 - Vendor Assessment
  24. Weber, M. Selecting an EHR, now what???? [PDF document]. Retrieved from Lecture Notes Online Web site:
  25. ‘’Medical Mastermind’’, EMR Vendor Checklist: Ten Important Product Questions to Ask the Vendor, July 31, 2012
  26. ‘’Practice Fusion Blog”, 4 questions to ask when selecting an EHR, July 7, 2014
  27. 27.0 27.1 27.2 27.3 27.4 27.5 27.6 27.7 Vendor Selection
  28. Enterprise EHR Vendor Evaluation
  29. Choosing the Right EHR Vendor for Your Practice
  30. How Much Will That EMR System Really Cost?
  31. vendor viability
  32. Types of vendor viability
  34. 35.0 35.1 35.2 35.3 35.4 Valacich, J. S., George, J. F., & Hoffer, J. A. (2012). Essentials of systems analysis and design (5th ed.)
  35. Proposing Regulatory-Driven Automated Test Suites
  36. Harrington, L., Porch, L., Acosta, K., & Wilkens, K. (2011). Realizing electronic medical record benefits: An easy-to-do usability study. Journal of Nursing Administration, 41(7/8), 331-335.
  37. 38.0 38.1 Patton, R. (2001). Software Testing (1st ed.)
  38. 39.0 39.1 39.2 39.3 39.4 39.5 39.6
  39. How to Select an Electronic Health Record System.
  40. 41.0 41.1 41.2
  41. Murphy E., et. al. An Electronic Medical Records System for Clinical Research and the EMR–EDC Interface
  42. Selecting An EHR, Now What?
  43. MetaStar
  44. EHR Contracts: Key Contract Terms for Users to Understand.
  45. 47.0 47.1 National Learning Consortium Resources