Difference between revisions of "Evaluation and Design Methodologies"

Revision as of 16:36, 26 February 2007

Evaluation Methods in Informatics

This is a collection of short descriptions of evaluation methods used in informatics research.

Ethnography

Cognitive ethnography is one of the tools which could be used to study effective manners for instruction of BMI in professional schools. A study to delineate the present state of informatics curricula at healthcare professional schools, the present status of BMI knowledge held by students, at the beginning and completion of their healthcare professional education, and a systematic analysis of what should be included in a standardized, structured BMI curriculum to meet the Liaison Committee on Medical Education (LCME) standards would benefit from the use of cognitive ethnography.

Critical Incident Technique

Critical Incident Technique (CIT) is a form of criterion sampling. [Patton, 2002] CIT relies upon interviews of subjects who fall into a defined category or categories and who carry out defined work tasks. Recurring interview topics are recorded as incidents, and the incidents are analyzed so to develop a psychological profile of the subjects. The goal of CIT research is to understand the weaknesses involved with a particular task and to provide solutions to resolve those weaknesses. [Wikipedia, 2007]

Focus Groups

Oral History

Cognitive Ethnography

Cognitive ethnography is rooted in traditional ethnography but differs from it in a fundamental way. Whereas traditional ethnography is concerned with the meanings that members of a cultural group create, cognitive ethnography is concerned with how members create those meanings. Cognitive ethnography employs traditional ethnographic methods to build knowledge of a community of practice and then applies this knowledge to the micro-level analysis of specific episodes of activity. The principal aim of cognitive ethnography is to reveal how cognitive activities are accomplished in real-world settings. Cognitive ethnography is a particularly apt method for studying instruction in both formal and informal settings, such as that found in medical instruction - in the classroom or on the wards. Cognitive ethnography looks at process: at the moment-to-moment development of activity and its relation to socio-cultural (often institutional) processes unfolding on different time scales. Traditional ethnography describes knowledge; cognitive ethnography describes how knowledge is constructed and used [Williams, 2006].

Ethnomethodology

Ethnomethodology is a sociological discipline which focuses on the ways in which people make sense of their world, display this understanding to others, and produce the mutually shared social order in which they live. It is distinct from traditional sociology, and does not seek to compete with it, or provide remedies for any of its practices. Furthermore, ethnomethodology is concerned with the"how" (the methods) by which that social order is produced, and shared (1,2). It seeks to describe the practices (the methods) these individuals use in their actual descriptions of those settings. Ethnomethology may ask, how do people perceive understand and explain the world in which they live? Specifically, with patient care, it may ask, are these perceptions and beliefs about the world changed when we become ill? Ethnomethodology is particularly concerned with cultural differences in explanations offered, especially the influence of social norms on the communication process. This cultural context is particularly significant when a patient tries to understand or the doctor tries to explain a condition, for example, a disease or newly diagnosed cancer.

Grounded Theory

Protocol Analysis

Protocol Analysis uses verbal descriptions of though processes and tasks to generate data about a given scenario or cognitive act. The primary technique used in protocol analysis is to ask subjects to "think-aloud" while performing a task. These verbal descriptions provide a set of explicit procedures that more thoroughly describe the solution to a task than other introspective techniques. According to K.A. Ericsson, "the central assumption of protocol analysis is that it possible to instruct subjects to verbalize their thoughts in a manner that doesn’t alter the sequence of thoughts mediating the completion of a task, and can therefore be accepted as valid data on thinking. Based on their theoretical analysis, Ericsson and Simon (1993) argued that the closest connection between thinking and verbal reports is found when subjects verbalize thoughts generated during task completion."