Simulation in healthcare : journal of the Society for Simulation in Healthcare, 2009
Objective outcome measures for use with simulator-based assessments of cardiac physical examinati... more Objective outcome measures for use with simulator-based assessments of cardiac physical examination competence are lacking. The current study describes the development and validation of an approach to scoring performance using a cardiac findings checklist. A cardiac findings checklist was developed and implemented for use with a simulator-based assessment of cardiac physical examination competence at a Canadian national specialty examination in internal medicine. Candidate performance as measured using the checklist was compared with global ratings of clinical performance on the cardiac patient simulator and with overall examination performance. Interrater reliability for scoring the checklist ranged from 0.95 for scoring correct findings to 0.72 for scoring incorrect findings. A summary checklist score had a Pearson correlation of 0.60 with overall candidate performance on the simulator-based station. Use of a cardiac findings checklist provides one objective measure of cardiac physical examination competence that may be used with simulator-based assessments.
Journal of Educational Evaluation for Health Professions, Apr 20, 2020
Purpose: It aimed to know the performance of the Ebel standard-setting method in in spring 2019 R... more Purpose: It aimed to know the performance of the Ebel standard-setting method in in spring 2019 Royal College of Physicians and Surgeons of Canada internal medicine certification examination consisted of multiple-choice questions. Specifically followings were searched: the inter-rater agreement; the correlation between Ebel scores and item facility indices; raters' knowledge of correct answers' impact on the Ebel score; and affection of rater's specialty on theinter-rater agreement and Ebel scores. Methods: Data were drawn from a Royal College of Physicians and Surgeons of Canada certification exam. Ebel's method was applied to 203 MCQs by 49 raters. Facility indices came from 194 candidates. We computed Fleiss' kappa and the Pearson correlation between Ebel scores and item facility indices. We investigated differences in the Ebel score (correct answers provided or not) and differences between internists and other specialists with t-tests. Results: Kappa was below 0.15 for facility and relevance. The correlation between Ebel scores and facility indices was low when correct answers were provided and negligible when they were not. The Ebel score was the same, whether the correct answers were provided or not. Inter-rater agreement and Ebel scores was not differentbetween internists and other specialists. Conclusion: Inter-rater agreement and correlations between item Ebel scores and facility indices wee consistently low; furthermore, raters' knowledge of correct answer and rater specialty had no effect on Ebel scores in the present setting.
physicians on a contractual instead of a fee-for-service basis, have strengthened the ability of ... more physicians on a contractual instead of a fee-for-service basis, have strengthened the ability of the two departments to deliver on their clinical, research and education missions. This commentary describes why and how they were developed. To understand the context in which these AARPs were developed, recognition of the health care reform agenda in Canada and specifically in Alberta is required. Several national and provincial reports, including the Romanow, 1 Kirby 2 and Mazankowski 3 reports, suggested the need to reframe and refocus the health care system to ensure access, quality and sustainability. These reports Bichel et al. all recommended that alternative mechanisms be considered for physician remuneration in an effort to encourage
To examine the relationship between a physician&a... more To examine the relationship between a physician's ability to examine a standardized patient (SP) and their ability to correctly identify related clinical findings created with simulation technology. The authors conducted an observational study of 347 candidates during a Canadian national specialty examination at the end of post-graduate internal medicine training. Stations were created that combined physical examination of an SP with evaluation of a related audio-video simulation of a patient abnormality, in the domains of cardiology and neurology. Examiners evaluated a candidate's competence at performing a physical examination of an SP and their accuracy in diagnosing a related audio-video simulation. For the cardiology stations, the correlation between the physical examination scores and recognition of simulation abnormalities was 0.31 (p < 0.01). For the neurology stations, the correlation was 0.27 (p < 0.01). Addition of the simulations identified 18% of 197 passing candidates on the cardiology stations and 17% of 240 passing candidates on the neurology stations who were competent in their physical examination technique but did not achieve the passing score for diagnostic skills. Assessments incorporating SPs without physical findings may need to include other methodologies to assess bedside diagnostic acumen.
BACKGROUND: In ambulatory care settings, patients with limited English proficiency receive lower ... more BACKGROUND: In ambulatory care settings, patients with limited English proficiency receive lower quality of care. Limited information is available describing outcomes for inpatients. OBJECTIVE: To investigate the effect of English proficiency on length of stay (LOS) and in-hospital mortality.
Background Many standardized patient (SP) encounters employ SPs without physical findings and, th... more Background Many standardized patient (SP) encounters employ SPs without physical findings and, thus, assess physical examination technique. The relationship between technique, accurate bedside diagnosis, and global competence in physical examination remains unclear.
CONTEXT Competency-based medical education frameworks are often founded on a combination of exist... more CONTEXT Competency-based medical education frameworks are often founded on a combination of existing research, educational principles and expert consensus. Our objective was to examine how components of the attending physician role, as determined by observing preceptors during their real-world work, link to the CanMEDS Physician Competency Framework.
Journal of Educational Evaluation for Health Professions
Purpose: It aimed to know the performance of the Ebel standard-setting method in in spring 2019 R... more Purpose: It aimed to know the performance of the Ebel standard-setting method in in spring 2019 Royal College of Physicians and Surgeons of Canada internal medicine certification examination consisted of multiple-choice questions. Specifically followings were searched: the inter-rater agreement; the correlation between Ebel scores and item facility indices; raters’ knowledge of correct answers’ impact on the Ebel score; and affection of rater's specialty on theinter-rater agreement and Ebel scores. Methods: Data were drawn from a Royal College of Physicians and Surgeons of Canada certification exam. Ebel’s method was applied to 203 MCQs by 49 raters. Facility indices came from 194 candidates. We computed Fleiss’ kappa and the Pearson correlation between Ebel scores and item facility indices. We investigated differences in the Ebel score (correct answers provided or not) and differences between internists and other specialists with t-tests.Results: Kappa was below 0.15 for facili...
Competency in the performance of a number of bedside procedures is an objective of training, set ... more Competency in the performance of a number of bedside procedures is an objective of training, set forth by the Royal College of Physicians and Surgeons of Canada. These procedures include venous access including central line placement, lumbar puncture, peripheral arterial catheter insertion, abdominal paracentesis, endotracheal intubation, thoracentesis, and knee arthrocentesis.
CMAJ : Canadian Medical Association journal = journal de l'Association medicale canadienne, May 14, 2018
Impaired vision is a risk factor for functional limitations, fallrelated injuries and quality of ... more Impaired vision is a risk factor for functional limitations, fallrelated injuries and quality of life in older people. • Most provincial governments in Canada cover comprehensive eye examinations by eye care practitioners for adults aged 65 years and older. 6 • A systematic review found low-quality evidence that screening in primary health care settings for impaired vision in adults aged 65 years and older does not appear to improve vision-related functional limitations or quality of life. 5 • The task force recommends against screening in primary health care settings for impaired vision in community-dwelling adults aged 65 years or older.
The Canadian Task Force on Preventive Health Care has not previously developed a recommendation o... more The Canadian Task Force on Preventive Health Care has not previously developed a recommendation on screening for HCV. Reasons for developing this recommendation include the availability of new treatments for chronic HCV infection and the lack of Canadian guidelines for screening. 13 The current recommendations are intended to provide clinicians and policy-makers with guidance on screening asymptomatic Canadian adults for HCV.
Academic medicine : journal of the Association of American Medical Colleges, Dec 20, 2017
To generate an empiric, detailed, and updated view of the attending physician preceptor role and ... more To generate an empiric, detailed, and updated view of the attending physician preceptor role and its interface with the complex work environment. In 2013, the authors conducted a modified collective ethnography with observations of internal medicine medical teaching unit preceptors from two university hospitals in Canada. Eleven observers conducted 32 observations (99.5 hours) of 26 preceptors (30 observations [93.5 hours] of 24 preceptors were included in the analysis). An inductive thematic approach was used to analyze the data with further axial coding to identify connections between themes. Four individuals coded the main data set; differences were addressed through discussion to achieve consensus. Three elements or major themes of the preceptor role were identified: (1) competence or the execution of traditional physician competencies, (2) context or the extended medical teaching unit environment, and (3) conduct or the manner of acting or behaviors and attitudes in the role. M...
Competency-based medical education frameworks are often founded on a combination of existing rese... more Competency-based medical education frameworks are often founded on a combination of existing research, educational principles and expert consensus. Our objective was to examine how components of the attending physician role, as determined by observing preceptors during their real-world work, link to the CanMEDS Physician Competency Framework. This is a sub-study of a broader study exploring the role of the attending physician by observing these doctors during their working day. The parent study revealed three overarching elements of the role that emerged from 14 themes and 123 sub-themes: (i) Competence, defined as the execution of traditional physician competencies; (ii) Context, defined as the environment in which the role is carried out, and (iii) Conduct, defined as the manner of acting, or behaviours and attitudes in the role that helped to negotiate the complex environment. In this sub-study, each sub-theme, or 'role-related component', was mapped to the competencies d...
Simulation in healthcare : journal of the Society for Simulation in Healthcare, 2009
Objective outcome measures for use with simulator-based assessments of cardiac physical examinati... more Objective outcome measures for use with simulator-based assessments of cardiac physical examination competence are lacking. The current study describes the development and validation of an approach to scoring performance using a cardiac findings checklist. A cardiac findings checklist was developed and implemented for use with a simulator-based assessment of cardiac physical examination competence at a Canadian national specialty examination in internal medicine. Candidate performance as measured using the checklist was compared with global ratings of clinical performance on the cardiac patient simulator and with overall examination performance. Interrater reliability for scoring the checklist ranged from 0.95 for scoring correct findings to 0.72 for scoring incorrect findings. A summary checklist score had a Pearson correlation of 0.60 with overall candidate performance on the simulator-based station. Use of a cardiac findings checklist provides one objective measure of cardiac physical examination competence that may be used with simulator-based assessments.
Journal of Educational Evaluation for Health Professions, Apr 20, 2020
Purpose: It aimed to know the performance of the Ebel standard-setting method in in spring 2019 R... more Purpose: It aimed to know the performance of the Ebel standard-setting method in in spring 2019 Royal College of Physicians and Surgeons of Canada internal medicine certification examination consisted of multiple-choice questions. Specifically followings were searched: the inter-rater agreement; the correlation between Ebel scores and item facility indices; raters' knowledge of correct answers' impact on the Ebel score; and affection of rater's specialty on theinter-rater agreement and Ebel scores. Methods: Data were drawn from a Royal College of Physicians and Surgeons of Canada certification exam. Ebel's method was applied to 203 MCQs by 49 raters. Facility indices came from 194 candidates. We computed Fleiss' kappa and the Pearson correlation between Ebel scores and item facility indices. We investigated differences in the Ebel score (correct answers provided or not) and differences between internists and other specialists with t-tests. Results: Kappa was below 0.15 for facility and relevance. The correlation between Ebel scores and facility indices was low when correct answers were provided and negligible when they were not. The Ebel score was the same, whether the correct answers were provided or not. Inter-rater agreement and Ebel scores was not differentbetween internists and other specialists. Conclusion: Inter-rater agreement and correlations between item Ebel scores and facility indices wee consistently low; furthermore, raters' knowledge of correct answer and rater specialty had no effect on Ebel scores in the present setting.
physicians on a contractual instead of a fee-for-service basis, have strengthened the ability of ... more physicians on a contractual instead of a fee-for-service basis, have strengthened the ability of the two departments to deliver on their clinical, research and education missions. This commentary describes why and how they were developed. To understand the context in which these AARPs were developed, recognition of the health care reform agenda in Canada and specifically in Alberta is required. Several national and provincial reports, including the Romanow, 1 Kirby 2 and Mazankowski 3 reports, suggested the need to reframe and refocus the health care system to ensure access, quality and sustainability. These reports Bichel et al. all recommended that alternative mechanisms be considered for physician remuneration in an effort to encourage
To examine the relationship between a physician&a... more To examine the relationship between a physician's ability to examine a standardized patient (SP) and their ability to correctly identify related clinical findings created with simulation technology. The authors conducted an observational study of 347 candidates during a Canadian national specialty examination at the end of post-graduate internal medicine training. Stations were created that combined physical examination of an SP with evaluation of a related audio-video simulation of a patient abnormality, in the domains of cardiology and neurology. Examiners evaluated a candidate's competence at performing a physical examination of an SP and their accuracy in diagnosing a related audio-video simulation. For the cardiology stations, the correlation between the physical examination scores and recognition of simulation abnormalities was 0.31 (p < 0.01). For the neurology stations, the correlation was 0.27 (p < 0.01). Addition of the simulations identified 18% of 197 passing candidates on the cardiology stations and 17% of 240 passing candidates on the neurology stations who were competent in their physical examination technique but did not achieve the passing score for diagnostic skills. Assessments incorporating SPs without physical findings may need to include other methodologies to assess bedside diagnostic acumen.
BACKGROUND: In ambulatory care settings, patients with limited English proficiency receive lower ... more BACKGROUND: In ambulatory care settings, patients with limited English proficiency receive lower quality of care. Limited information is available describing outcomes for inpatients. OBJECTIVE: To investigate the effect of English proficiency on length of stay (LOS) and in-hospital mortality.
Background Many standardized patient (SP) encounters employ SPs without physical findings and, th... more Background Many standardized patient (SP) encounters employ SPs without physical findings and, thus, assess physical examination technique. The relationship between technique, accurate bedside diagnosis, and global competence in physical examination remains unclear.
CONTEXT Competency-based medical education frameworks are often founded on a combination of exist... more CONTEXT Competency-based medical education frameworks are often founded on a combination of existing research, educational principles and expert consensus. Our objective was to examine how components of the attending physician role, as determined by observing preceptors during their real-world work, link to the CanMEDS Physician Competency Framework.
Journal of Educational Evaluation for Health Professions
Purpose: It aimed to know the performance of the Ebel standard-setting method in in spring 2019 R... more Purpose: It aimed to know the performance of the Ebel standard-setting method in in spring 2019 Royal College of Physicians and Surgeons of Canada internal medicine certification examination consisted of multiple-choice questions. Specifically followings were searched: the inter-rater agreement; the correlation between Ebel scores and item facility indices; raters’ knowledge of correct answers’ impact on the Ebel score; and affection of rater's specialty on theinter-rater agreement and Ebel scores. Methods: Data were drawn from a Royal College of Physicians and Surgeons of Canada certification exam. Ebel’s method was applied to 203 MCQs by 49 raters. Facility indices came from 194 candidates. We computed Fleiss’ kappa and the Pearson correlation between Ebel scores and item facility indices. We investigated differences in the Ebel score (correct answers provided or not) and differences between internists and other specialists with t-tests.Results: Kappa was below 0.15 for facili...
Competency in the performance of a number of bedside procedures is an objective of training, set ... more Competency in the performance of a number of bedside procedures is an objective of training, set forth by the Royal College of Physicians and Surgeons of Canada. These procedures include venous access including central line placement, lumbar puncture, peripheral arterial catheter insertion, abdominal paracentesis, endotracheal intubation, thoracentesis, and knee arthrocentesis.
CMAJ : Canadian Medical Association journal = journal de l'Association medicale canadienne, May 14, 2018
Impaired vision is a risk factor for functional limitations, fallrelated injuries and quality of ... more Impaired vision is a risk factor for functional limitations, fallrelated injuries and quality of life in older people. • Most provincial governments in Canada cover comprehensive eye examinations by eye care practitioners for adults aged 65 years and older. 6 • A systematic review found low-quality evidence that screening in primary health care settings for impaired vision in adults aged 65 years and older does not appear to improve vision-related functional limitations or quality of life. 5 • The task force recommends against screening in primary health care settings for impaired vision in community-dwelling adults aged 65 years or older.
The Canadian Task Force on Preventive Health Care has not previously developed a recommendation o... more The Canadian Task Force on Preventive Health Care has not previously developed a recommendation on screening for HCV. Reasons for developing this recommendation include the availability of new treatments for chronic HCV infection and the lack of Canadian guidelines for screening. 13 The current recommendations are intended to provide clinicians and policy-makers with guidance on screening asymptomatic Canadian adults for HCV.
Academic medicine : journal of the Association of American Medical Colleges, Dec 20, 2017
To generate an empiric, detailed, and updated view of the attending physician preceptor role and ... more To generate an empiric, detailed, and updated view of the attending physician preceptor role and its interface with the complex work environment. In 2013, the authors conducted a modified collective ethnography with observations of internal medicine medical teaching unit preceptors from two university hospitals in Canada. Eleven observers conducted 32 observations (99.5 hours) of 26 preceptors (30 observations [93.5 hours] of 24 preceptors were included in the analysis). An inductive thematic approach was used to analyze the data with further axial coding to identify connections between themes. Four individuals coded the main data set; differences were addressed through discussion to achieve consensus. Three elements or major themes of the preceptor role were identified: (1) competence or the execution of traditional physician competencies, (2) context or the extended medical teaching unit environment, and (3) conduct or the manner of acting or behaviors and attitudes in the role. M...
Competency-based medical education frameworks are often founded on a combination of existing rese... more Competency-based medical education frameworks are often founded on a combination of existing research, educational principles and expert consensus. Our objective was to examine how components of the attending physician role, as determined by observing preceptors during their real-world work, link to the CanMEDS Physician Competency Framework. This is a sub-study of a broader study exploring the role of the attending physician by observing these doctors during their working day. The parent study revealed three overarching elements of the role that emerged from 14 themes and 123 sub-themes: (i) Competence, defined as the execution of traditional physician competencies; (ii) Context, defined as the environment in which the role is carried out, and (iii) Conduct, defined as the manner of acting, or behaviours and attitudes in the role that helped to negotiate the complex environment. In this sub-study, each sub-theme, or 'role-related component', was mapped to the competencies d...
Uploads
Papers by Maria Bacchus