This retrospective before-after study was conducted at the Hospital Group Twente, a teaching hospital in the Eastern region of the Netherlands, from March 2023 to December 2024. To evaluate the effectiveness of prioritizing patients we compared the period after introduction of the hospital admission prediction dashboard (between March 15 and December 31 in 2024) with the same period in 2023, when selecting the patients for MR was performed by the pharmacy technicians without the dashboard.

Approximately 30,000 patients per year visit the ED. At the ED, pharmacy technicians perform MR during working hours (11:00 to 20:00), seven days a week. After working hours, the attending physician is responsible for the MR on the ED. As a fallback strategy, when patients are admitted to the ward without MR performed in the ED, pharmacy technicians perform MR on the ward the next day. MR is performed using the nationwide medication record system that contains pharmacy dispensing data in combination with a patient interview [12].

All patients admitted to the ED were included in the study before and after implementation of the hospital admission prediction dashboard. Some patients were admitted multiple times during the study period, each admission was separately analyzed. Patients were excluded if the visit to the ED occurred outside the pharmacy technicians’ working hours and when patients objected to sharing data for scientific research.

Extreme Gradient Boosting (XGBoost) models were developed to predict the likelihood of admission from the ED. The models were trained on retrospective data collected between 2015 and 2022, which included patient demographics, mode and time of arrival at ED, main presenting complaint, lab results, and vital sign measurements (for details, see Supplementary files). The training dataset comprised a total of 188,754 ED visits. Model validation was performed using data from 2022, which included 28,830 ED visits. To account for the temporal nature of ED data, a separate XGBoost model was trained for each 10 min interval after patient arrival. This approach enabled the dashboard to reflect changes in a patient's condition throughout their ED stay. The models had a mean area under the curve (AUC) of 0.880 (0.851–0.893). The models have been published open-source and are available at: https://github.com/HospitalGroupTwente/SEH_Opname_Voorspelling.

Predictions were generated in real-time and displayed on a clinical dashboard, where each patient was assigned a continuously updated probability score indicating their risk of admission. These scores were recalculated every 10 min to incorporate newly available data such as vital signs and laboratory results. This dashboard was integrated into the workflow of the pharmacy technicians on March 1, 2024, resulting in a two week familiarization period before post-intervention data collection. All technicians working on the ED (in total 51), used the dashboard to select patients for MR, starting with the patient with the highest probability score and proceeding through the list in descending order.

The primary outcome of this study was the proportion of patients with correct MR, i.e. patients admitted to the ward with a completed MR on the ED. The secondary outcome was the proportion of patients with potentially unnecessary MR, i.e. patients with a completed MR who are not admitted to the ward and discharged from the ED.

Data were collected by extracting information from the Hospital Information System (HIS) HiX 6.3 (Chipsoft, Amsterdam, the Netherlands) database using Structured Query Language (SQL). All data were retrospectively collected. The collected data consisted of baseline characteristics with their admission status, vital signs, laboratory data (see appendix A) and whether or not an MR was completed on the ED.

No formal sample size calculation was performed as we could use a large set of retrospective data. Continuous variables were summarized as means and standard deviations in case of normal distribution, or medians and interquartile ranges in case of non-normal distribution. Categorical variables were presented as percentages. Patient characteristics were compared between the two measurement periods, using the chi-square (χ2) test or Fisher’s exact test for categorical variables. The t-test was used for normally distributed continuous variables, and the Mann–Whitney U test for non-normally distributed continuous variables. For the primary and secondary outcome the chi-square (χ2) test was used to compare the proportions before and after the intervention. For patients with multiple admissions, each admission was analysed separately. All data processing and statistical analyses were performed using Python version 3.8.10 (Python Software Foundation, Wilmington, USA) with the Pandas and SciPy libraries.

The study was approved by the local review board of the hospital (Advies commissie Lokale Uitvoerbaarheid wetenschappelijk onderzoek (ALU); registration number 24–52) on November 11, 2024. The ALU waived the need for informed consent due to the large amount of data and the retrospective nature of the data. Patient data were obtained and handled in accordance with privacy regulations.