Zaid Al-fagih

Aims and Objectives Accurate and efficient transcription of patient consultations is essential for maintaining high quality clinical documentation in healthcare. This study aims to evaluate the effectiveness of an artificial intelligence (AI) transcription tool in comparison to traditional manual transcription by medical students. The primary hypothesis is that AI-assisted transcription improves both the speed and quality of clinical documentation.

Method and Design

Study Design:… Show more

Aims and Objectives Accurate and efficient transcription of patient consultations is essential for maintaining high quality clinical documentation in healthcare. This study aims to evaluate the effectiveness of an artificial intelligence (AI) transcription tool in comparison to traditional manual transcription by medical students. The primary hypothesis is that AI-assisted transcription improves both the speed and quality of clinical documentation.

Method and Design

Study Design: Observational, comparative study

Study Dates: August – September 2024

Commercial Artificial intelligence (AI) Transcription Tool used: Rhazes.AI

Participants: Five medical students transcribed 15 patient consultation videos sourced from YouTube. The transcriptions were compared with those generated by the AI tool.

Intervention: Use of an AI transcription tool for clinical documentation

Outcomes: Transcription time, note completeness (standardization), and the quality of clinical notes

Statistical Methods: Descriptive statistics were used to analyze transcription time and quality. Comparisons were made using paired t–tests to assess statistical significance.

Results and Conclusion Fifteen videos were transcribed using both artificial intelligence (AI) transcription tool and manual transcription. Human transcription took an average of 10 minutes 34 seconds (SD = 2 minutes 28 seconds), with an additional average processing time of 2 minutes 25 seconds (SD =55 seconds). ASR transcription averaged 9 minutes 36 seconds (SD = 1 minute 23 seconds), with an additional average processing time of 52 seconds (SD = 21 seconds). The AI transcription tool demonstrated a significant improvement in transcription speed, with an average of 44.06 seconds per consultation compared to 115.19 seconds for manual transcription, representing a time reduction of over 60% (p < 0.001).
Show less

On average, one in ten patients die because of a diagnostic error and medical errors are the third largest cause of death in the word. While LLMs have been proposed to help doctors with diagnoses, no research results have been published on comparing the diagnostic ability of many popular LLMs on an openly accessible real-patient cohort. In thus study, we compare LLMs from Google, OpenAI, Meta, Mistral, Cohere and Anthropic using our previously published evaluation methodology and explore… Show more

On average, one in ten patients die because of a diagnostic error and medical errors are the third largest cause of death in the word. While LLMs have been proposed to help doctors with diagnoses, no research results have been published on comparing the diagnostic ability of many popular LLMs on an openly accessible real-patient cohort. In thus study, we compare LLMs from Google, OpenAI, Meta, Mistral, Cohere and Anthropic using our previously published evaluation methodology and explore improving their accuracy with RAG. Show less

Background:
Given the strikingly high diagnostic error rate in hospitals, and the recent development of Large Language Models (LLMs), we set out to measure the diagnostic sensitivity of two popular LLMs: GPT-4 and PaLM2. Small-scale studies to evaluate the diagnostic ability of LLMs have shown promising results, with GPT-4 demonstrating high accuracy in diagnosing test cases. However, larger evaluations on real electronic patient data are needed to provide more reliable… Show more

Background:
Given the strikingly high diagnostic error rate in hospitals, and the recent development of Large Language Models (LLMs), we set out to measure the diagnostic sensitivity of two popular LLMs: GPT-4 and PaLM2. Small-scale studies to evaluate the diagnostic ability of LLMs have shown promising results, with GPT-4 demonstrating high accuracy in diagnosing test cases. However, larger evaluations on real electronic patient data are needed to provide more reliable estimates.

Methods:
To fill this gap in the literature, we used a deidentified Electronic Health Record (EHR) data set of about 300,000 patients admitted to the Beth Israel Deaconess Medical Center in Boston. This data set contained blood, imaging, microbiology and vital sign information as well as the patients' medical diagnostic codes. Based on the available EHR data, doctors curated a set of diagnoses for each patient, which we will refer to as ground truth diagnoses. We then designed carefully-written prompts to get patient diagnostic predictions from the LLMs and compared this to the ground truth diagnoses in a random sample of 1000 patients.

Results:
Based on the proportion of correctly predicted ground truth diagnoses, we estimated the diagnostic hit rate of GPT-4 to be 93.9%. PaLM2 achieved 84.7% on the same data set. On these 1000 randomly selected EHRs, GPT-4 correctly identified 1116 unique diagnoses.

Conclusion:
The results suggest that artificial intelligence (AI) has the potential when working alongside clinicians to reduce cognitive errors which lead to hundreds of thousands of misdiagnoses every year. However, human oversight of AI remains essential: LLMs cannot replace clinicians, especially when it comes to human understanding and empathy. Furthermore, a significant number of challenges in incorporating AI into health care exist, including ethical, liability and regulatory barriers. Show less

Vaccines are one of the most effective tools humanity has in the fight against pandemics. One of the major challenges of vaccine distribution is achieving fair and equitable allocation across the countries of the world, regardless of their economic wealth. The self-interested behaviour of high-income countries and the underutilisation of vaccines allocated to underprepared countries are some of the failures reported during COVID-19 vaccine roll-out. These shortcomings have motivated the need… Show more

Vaccines are one of the most effective tools humanity has in the fight against pandemics. One of the major challenges of vaccine distribution is achieving fair and equitable allocation across the countries of the world, regardless of their economic wealth. The self-interested behaviour of high-income countries and the underutilisation of vaccines allocated to underprepared countries are some of the failures reported during COVID-19 vaccine roll-out. These shortcomings have motivated the need for a central market mechanism that takes into account the countries’ vulnerability to COVID-19 and their readiness to distribute and administer their allocated vaccines. In this paper, we leverage game theory to study the problem of equitable global vaccine distribution and propose a fair market mechanism that aligns self-interested behaviour with optimal global objectives. First, we model the interaction between a central vaccine provider (e.g. COVAX) and a country reporting its demand as a two-player game, and discuss the Nash and mixed Nash equilibria of that game. Then, we propose a repeated auction mechanism with an artificial payment system for allocating vaccines among participating countries, where each auction round is based on a Vickrey-Clarke-Groves (VCG) mechanism. The proposed allocation mechanism aims at minimising deaths and incentivises the self-interested countries to report their demand truthfully. Compared with real-world COVAX allocation decisions, our results show that the proposed auction mechanism achieves more efficient outcomes that maximise the number of averted deaths. Pragmatic considerations are investigated and policy recommendations are discussed. Show less

The emergency department (ED) is an indispensable part of most hospitals worldwide that delivers immense value to patients and clinicians alike. The stressful and high-pressure nature of the work carried out by ED medical and administrative staff can make it difficult for them to continuously assess and consider process improvements that may positively impact their day-to-day activities. The ED is visited by patients who vary in the urgency of their need for treatment. As part of the triage… Show more

The emergency department (ED) is an indispensable part of most hospitals worldwide that delivers immense value to patients and clinicians alike. The stressful and high-pressure nature of the work carried out by ED medical and administrative staff can make it difficult for them to continuously assess and consider process improvements that may positively impact their day-to-day activities. The ED is visited by patients who vary in the urgency of their need for treatment. As part of the triage process, a considerable amount of patients are considered low- or medium-urgency cases, which can be labeled as low priority, leading to overcrowded ED facilities and waiting areas that eventually cause increased pressure on ED staff, longer wait times, and higher leave-without-being-seen rates. This paper: (i) introduces a model applying technology to streamline the ED admission process by merging processes for registration and measuring vital signs, (ii) uses expert assessments to engage in the beginning of the process, and (iii) presents methods to better utilize new or existing waiting areas for patient treatment. Implementation of the model targets a better allocation of scarce resources to minimize overcrowding, reduce waiting times, and improve discharge rates, thereby increasing patient satisfaction. Show less

Since the outbreak of the COVID-19 pandemic, many healthcare facilities have suffered from shortages in medical resources, particularly in Personal Protective Equipment (PPE). In this paper, we propose a game-theoretic approach to schedule PPE orders among healthcare facilities. In this PPE game, each independent healthcare facility optimises its own storage utilisation in order to keep its PPE cost at a minimum. Such a model can reduce peak demand considerably when applied to a variable PPE… Show more

Since the outbreak of the COVID-19 pandemic, many healthcare facilities have suffered from shortages in medical resources, particularly in Personal Protective Equipment (PPE). In this paper, we propose a game-theoretic approach to schedule PPE orders among healthcare facilities. In this PPE game, each independent healthcare facility optimises its own storage utilisation in order to keep its PPE cost at a minimum. Such a model can reduce peak demand considerably when applied to a variable PPE consumption profile. Experiments conducted for NHS England regions using actual data confirm that the challenge of securing PPE supply during disasters such as COVID-19 can be eased if proper stock management procedures are adopted. These procedures can include early stockpiling, increasing storage capacities and implementing measures that can prolong the time period between successive infection waves, such as social distancing measures. Simulation results suggest that the provision of PPE dedicated storage space can be a viable solution to avoid straining PPE supply chains in case a second wave of COVID-19 infections occurs. Show less

Introduction:
Incorporating 'gaming' elements (gamification) as behaviour rewards into mHealth apps is becoming popular. We assessed the effect of the gamification of mHealth on health behaviour, specifically as an adjunct to smoking cessation therapies.

Methods:
In a qualitative longitudinal-study,16 active smokers were divided into 2 cohorts: cohort-A used a gamified app, cohort-B used a non-gamified app. All participants underwent 4 semi-structured interviews over a period of 5… Show more

Introduction:
Incorporating 'gaming' elements (gamification) as behaviour rewards into mHealth apps is becoming popular. We assessed the effect of the gamification of mHealth on health behaviour, specifically as an adjunct to smoking cessation therapies.

Methods:
In a qualitative longitudinal-study,16 active smokers were divided into 2 cohorts: cohort-A used a gamified app, cohort-B used a non-gamified app. All participants underwent 4 semi-structured interviews over a period of 5 weeks. We triangulated three further data sources to increase the validity of our primary dataset: a systematic literature review, expert interviews, and a secondary dataset.

Results:
We observed, 'perceived behavioural control' and 'intrinsic motivation' acted as positive drivers to game engagement, and subsequently positive health behaviour. External social influences exerted a negative effect. We identified three critical factors, whose presence was necessary for game engagement; purpose, user alignment, and functional utility. We developed these findings into a framework, which we propose to guide the future development of gamified mHealth interventions.

Conclusion:
Gamification holds potential for highly effective mHealth solutions with widespread reach that may replace or supplement the behavioural support component found in current smoking cessation programmes. Show less

Finding ways to increase and sustain engagement with mHealth interventions has become a challenge during application development. While gamification shows promise and has proven effective in many fields, critical questions remain concerning how to use gamification to modify health behavior.

The objective of this study is to investigate how the gamification of mHealth interventions leads to a change in health behavior, specifically with respect to smoking cessation.

We conducted a… Show more

Finding ways to increase and sustain engagement with mHealth interventions has become a challenge during application development. While gamification shows promise and has proven effective in many fields, critical questions remain concerning how to use gamification to modify health behavior.

The objective of this study is to investigate how the gamification of mHealth interventions leads to a change in health behavior, specifically with respect to smoking cessation.

We conducted a qualitative longitudinal study using a sample of 16 smokers divided into 2 cohorts (one used a gamified intervention and the other used a nongamified intervention). Each participant underwent 4 semistructured interviews over a period of 5 weeks. Semistructured interviews were also conducted with 4 experts in gamification, mHealth, and smoking cessation. Interviews were transcribed verbatim and thematic analysis undertaken.
Results indicated perceived behavioral control and intrinsic motivation acted as positive drivers to game engagement and consequently positive health behavior. Importantly, external social influences exerted a negative effect. We identified 3 critical factors, whose presence was necessary for game engagement: purpose, user alignment (congruency of game and user objectives), and functional utility (a well-designed game). We summarize these findings in a framework to guide the future development of gamified mHealth interventions.
Gamification holds the potential for a low-cost, highly effective mHealth solution that may replace or supplement the behavioral support component found in current smoking cessation programs. The framework reported here has been built on evidence specific to smoking cessation, however it can be adapted to health interventions in other disease categories. Future research is required to evaluate the generalizability and effectiveness of the framework, directly against current behavioral support therapy interventions in smoking cessation and beyond. Show less

Introduction and objectives:
Increasing emphasis has been placed on behavioural therapy in smoking cessation efforts. mHealth aims to join today’s arsenal of smoking cessation techniques. Many apps are utilising ‘gamification’ (the use of game design elements in non-game contexts) as a tool to drive positive behaviour change. However, a significant knowledge gap currently remains regarding how gamification can affect health behaviour. Our study seeked to elucidate the motivational mechanisms… Show more

Introduction and objectives:
Increasing emphasis has been placed on behavioural therapy in smoking cessation efforts. mHealth aims to join today’s arsenal of smoking cessation techniques. Many apps are utilising ‘gamification’ (the use of game design elements in non-game contexts) as a tool to drive positive behaviour change. However, a significant knowledge gap currently remains regarding how gamification can affect health behaviour. Our study seeked to elucidate the motivational mechanisms exploited by gamification in promoting positive health behaviours in the context of smoking cessation, with a view to generating recommendations on how to create effective gamified mHealth interventions.

Methods:
We conducted a qualitative longitudinal study using a sample of 16 smokers divided into two cohorts. The first cohort used a non-gamified mHealth intervention, whilst the second used a gamified mHealth intervention. The added game components allowed us to isolate the effects of gamification. Each participant underwent 4 one-on-one, semi-structured interviews over a period of 5 weeks. Interviews were transcribed verbatim after which thematic analysis was undertaken.

Results:
We observed that perceived behavioural control and intrinsic motivation acted as positive drivers to game engagement and consequently positive health behaviour. Importantly, external social influences exerted a negative effect. We identified three critical factors, whose presence was necessary for game engagement; purpose (explicit purpose known by the user), user alignment (congruency of game and user objectives), functional utility (a well-designed game). We summarise these findings in a framework, which we propose to guide the development of gamified mHealth interventions.

Show less

Smoking is the largest cause of preventable mortality in the United Kingdom, and whilst 68% of smokers want to quit, only 3% of them are successful. mHealth looks to join today’s arsenal of smoking cessation techniques, with an increasing number of apps looking to gamification as a tool to drive positive behaviour change. Briefly, gamification is the use of game design elements in non-game contexts. Here we aim to review extant knowledge and offer an outlook in terms of avenues that may inform… Show more

Smoking is the largest cause of preventable mortality in the United Kingdom, and whilst 68% of smokers want to quit, only 3% of them are successful. mHealth looks to join today’s arsenal of smoking cessation techniques, with an increasing number of apps looking to gamification as a tool to drive positive behaviour change. Briefly, gamification is the use of game design elements in non-game contexts. Here we aim to review extant knowledge and offer an outlook in terms of avenues that may inform future work. Show less