Print
facebook
twitter
Linkedin
google+
Introduction
Atherosclerotic cardiovascular disease (CVD) is a leading cause of morbidity and mortality [1]. The presence and extent of atherosclerosis, indicated by carotid plaques (CPs), are utilized to estimate and classify or reclassify an individual's cardiovascular risk [2]. Beyond general risk stratification, carotid atherosclerosis (CAS) is also recognized as a predictor of other CVD events, including strokes caused by luminal vessel stenosis and CP rupture [3,4]. CAS and the buildup of CPs represent a significant global health concern. Therefore, the early detection and consistent monitoring of CPs are critically important [5].
Several imaging modalities, including ultrasound (US), computed tomography (CT), and magnetic resonance imaging (MRI), are utilized to detect CPs. US is particularly favored due to its non-invasive, radiation-free nature and its capability for real-time scanning [6]. Additionally, carotid US imaging offers a unique opportunity to identify underlying cardiovascular risks [7]. However, the distribution of medical resources is often uneven, and professional US services are frequently unavailable in low-resource or underserved healthcare settings, particularly in rural and remote areas. Consequently, widespread screening for CPs using conventional US is restricted.
Recently, a new type of mobile US device has been developed and is increasingly being used in clinical practice. This innovation was first proposed in 2011 and has significantly contributed to the expansion of point-of-care ultrasound (POCUS). POCUS involves clinical doctors, rather than radiologists or sonographers, obtaining and interpreting US images directly at the patient's bedside [8]. Most research on POCUS has been conducted in the fields of critical care and emergency medicine [9]. With technological advancements, several smaller, higher-quality, and more convenient handheld US devices have been introduced [10]. These devices have been tested for a variety of clinical applications, including health examinations [11-14]. However, US examination remains a technology dependent on the operator's experience, typically performed by specialists such as sonographers and radiologists. This dependency may pose challenges to widespread implementation of comprehensive screening programs.
Fortunately, the development of artificial intelligence (AI) technology has led to several reports on its application in traditional carotid US examinations. These applications include detecting the vascular lumen [15], measuring and classifying carotid intima-media thickness (CIMT) [16], and detecting and characterizing CPs [17,18]. To our knowledge, there have been no studies specifically focusing on the use of intelligent handheld US devices by non-expert general practitioners (GPs) for detecting CPs in community populations. An intelligent handheld US device equipped with AI could potentially enable non-expert GPs to independently conduct CP detection examinations.
Therefore, this study proposes a novel approach for CP examination and management, wherein radiology specialists train non-expert GPs to independently perform CP detection examinations and risk stratification using an intelligent handheld US device. A prospective, parallel controlled trial was conducted to explore the feasibility of non-expert GPs using an intelligent handheld US device for detecting CPs in community populations.
Materials and Methods
Compliance with Ethical Standards
All procedures were performed in compliance with relevant laws and institutional guidelines and have been approved by the institutional Clinical Research Ethics Committee of Zhongshan Hospital, Fudan University (No. B2022326R). And informed consent was obtained from all community residents involved in the study.
Subjects
A prospective, parallel controlled trial was conducted in China to assess the ability of non-expert GPs to independently detect CPs using an intelligent handheld US device in community populations. The study recruited consecutive community residents at the Lian'an Health Service Station in the Beicai Community, Pudong New Area, Shanghai, from May 2022 to June 2023. The inclusion criteria included: (1) community residents aged 18 years or older; (2) community residents who visited the Health Service Station. The exclusion criteria were as follows: (1) incomplete image and clinical data; and (2) lack of an informed consent form. All participants provided written informed consent before being included in the study.
Baseline characteristics, including age, sex, height, body weight, and body mass index were recorded. Each participant's medical history, including known hypertension, diabetes, smoking habits, and hyperlipidemia, underwent a thorough review. A flowchart depicting the inclusion of community residents in this study is presented in Fig. 1. The study received approval from the Institutional Clinical Research Ethics Committee (No. B2022326R) and was registered with the China Clinical Trial Registration Center (CHiCTR2300073323).
Intelligent Handheld US System
An intelligent handheld US device, the Cloud-35V (Stork Healthcare Technology Co., Ltd., Chengdu, China), was utilized for this trial. This device comprises a tablet computer and an 8.5 MHz linear-array probe (Fig. 2A, B). The tablet computer and US probe connect via WiFi. Additionally, the stored images can be uploaded to a cloud server through the internet.
Real-time AI is the key component of the system, which is run on a local graphical processor. It employs a convolutional neural network (CNN) with a multiclass support vector machine model and uses US signal data, instead of compressed US image data, to make better use of numerous physical characteristics, such as in-phase/quadrature and radio frequency (Fig. 2C). Real-time vascular navigation technology aids users in locating the standard reference section for automatically measuring the CIMT, which aligns closely with clinical diagnoses. In this context, it is unnecessary for the reference section to include potential lesions. Real-time recognition technology enables dynamic identification of the carotid arteries, CIMT, and CPs (Fig. 2D, E). It can also mark CP targets, automatically analyze their features, and perform risk stratification.
The device can be switched between AI and non-AI modes using the "AI C" button located at the lower left corner of the tablet computer (Fig. 2D). The chromatogram, displayed in the lower-right corner of the tablet screen, assists the operator in determining whether the sample is consistent with the standard. Typically, the closer the alignment of the chromatogram peaks with the green reference markers, the better the match of the sample to the standard.
This intelligent handheld US device offers several advantages, including its compact size (20×16×25 cm), light weight (net 1,200 g), and extended operational capability. It supports Type-C DC power, ensuring 24 hours of battery life in outdoor mobile power mode, and features a built-in 6,000 mAh battery that provides up to 2 hours of use without external power.
Non-expert GPs and Specialist Doctors
Four non-expert GPs received training and evaluation on both the theoretical knowledge and practical operation of the US device. This training was conducted in advance by two specialists from a tertiary hospital, who were recruited by the Health Service Station.
Each non-expert GP participated in a 3-hour didactic session to become familiar with the carotid US scanning method, equipment operation, and US characteristics of CPs. Prior to commencing the study, these GPs performed practice scans on volunteers to demonstrate their proficiency with the software. Following a week of repeated practice, the trainees were independently tested.
Design and Setting
The entire trial was structured into three distinct phases: In setting A, two non-expert GPs, J.M.Z. and S.S.C., who had approximately 2 and 4 years of experience respectively, conducted carotid examinations using handheld US devices without the aid of AI; in setting B, two other non-expert GPs, Y.L.S. and X.C.L., with approximately 4 and 3 years of experience respectively, performed similar carotid examinations using handheld US devices but with AI assistance; in setting C, two specialist doctors, P.S. and Y.K.S., who had roughly 9 and 6 years of experience in carotid US examinations respectively, also conducted the examinations using the handheld US device without AI assistance. Each community resident was examined sequentially by the three groups of doctors (settings A, B, and C). The procedures in settings A, B, and C were conducted blind to each other. The results from setting C were considered the gold standard.
Finally, the collection time was recorded for each setting. In setting B, community residents and non-expert GPs were required to complete a questionnaire survey. An independent coordinator collected and organized all the data.
The settings were identical in the following aspects: (1) settings A and B possessed a similar level of knowledge regarding US theory and had limited experience with US examinations; (2) the US courses completed by both settings were identical; and (3) the coordinator oversaw the entire trial process, ensuring that settings A, B, and C did not interfere with one another.
Measurement and Diagnostic Criteria of CIMT and CPs
The doctors were asked to scan all the carotid arteries using the handheld US device through the conventional process using the same protocol in all three settings. All carotid US examinations included three steps (Fig. 3): In step 1, the carotid artery was continuously scanned and recorded in both transverse and longitudinal sections. Step 2 involved measuring the thickness of the CIMT. During step 3, if CPs were identified, they were measured, and details such as their position, size, echogenicity, risk stratification, and classifications according to the Plaque-Reporting and Data System (RADS) were documented. For community residents displaying two or more CPs, the thickest plaque was measured. If no CPs were detected, the examination concluded at step 2. If one or more CPs were present, either the single CP or the thickest CP was selected for a detailed assessment.
CIMT was defined as the vertical distance from the upper edge of the inner membrane to the upper edge of the outer membrane. A normal CIMT was considered to be less than 1.0 mm, while a CIMT ranging from 1.0 mm to less than 1.5 mm indicated thickening. In this study, CIMT measurements were taken at the common carotid artery (CCA), specifically referring to a 10 to 15 mm section from the bifurcation. If plaque was present, the largest CIMT measurement was recorded.
The locations of the CPs were determined comprehensively using both transverse and longitudinal sections. In the longitudinal section, the carotid artery was segmented into the CCA, bifurcation, internal carotid artery (ICA), and external carotid artery. In the transverse section, the carotid artery was classified according to the "eightpart method," which includes the anterior, posterior, medial (tracheal side), lateral, anterior medial, posterior medial, anterior lateral, and posterior lateral walls.
The diagnostic criteria for CPs were as follows [19]: the local CIMT was ≥1.5 mm, or the measurement of the locally thickened wall was 0.5 mm or 50% thicker than the surrounding wall in the longitudinal section.
The cardiovascular risk stratification was conducted using the CP grading system [2]: low risk for CP thickness less than 1.5 mm; medium risk for CP thickness between 1.5 mm and 2.5 mm; and high risk for CP thickness of 2.5 mm or greater. The stroke risk classification followed the Plaque-RADS guidelines [18]: type I, normal vessel wall; type II, maximum wall thickness less than 3 mm; type III, maximum wall thickness of 3 mm or more, or presence of a healed ulcerated CP; type IV, presence of plaque ulceration regardless of CP thickness, without intra-plaque hemorrhage, fibrous cap disruption, or intra-luminal thrombus.
Statistical Analysis
SPSS version 24.0 (IBM Corp., Armonk, NY, USA) was utilized for the statistical analysis. The dependent t-test was employed to compare the quantitative data, presented as mean±standard deviation. The chi-square test was applied to assess the categorical variables, which are reported as numbers and percentages. The receiver operating characteristic (ROC) curve for plaque detection by non-expert GPs was constructed, and the area under the curve (AUC), along with sensitivity, specificity, accuracy, positive predictive value (PPV), and negative predictive value (NPV) for settings A and B, were calculated. The interoperator consistency of the CIMT measurement and the identification of target CPs were assessed using the intragroup correlation coefficient (ICC). The levels of agreement were classified as poor (ICC, 0-0.2), fair (ICC, 0.2-0.4), moderate (ICC, 0.4-0.6), good (ICC, 0.6-0.8), and excellent (ICC, 0.8-1.0). Statistical significance was established at P<0.05.
Results
Population
A total of 114 community residents participated in the trial. However, three were excluded due to incomplete US images and clinical data. Consequently, 111 residents (mean age, 70.29±6.58 years; age range, 49 to 89 years) with complete US images and clinical data formed the study cohort. Of these, 66 (59.5%) were female and 45 (40.5%) were male (Table 1).
Diagnostic Performance of Non-expert GPs in Detecting CPs
In accordance with the protocol described above, 112 plaques from 222 carotid arteries were analyzed. CPs were detected in 50.5% (112/222) of the samples, using setting C as the reference standard. Among these, 68 carotid arteries (60.7%) contained one plaque, while 39.3% had two or more plaques. Fifty-one plaques (45.5%) were located in the right carotid artery, and 61 (54.5%) in the left carotid artery. Eighty-five plaques (75.9%) were found in the posterior wall of the bifurcation, and 18 (16.1%) in the posterior wall of the ICA. The average length of the CPs was 9.23±5.61 mm (range, 1.5 to 28.4 mm), and the average thickness was 2.09±0.75 mm (range, 0.6 to 4.5 mm). The CP gradings were 16.1% (18/112) low risk, 58.9% (66/112) medium risk, and 25.0% (28/112) high risk. The Plaque-RADS categories of the CPs were 87.5% (98/112) for category II, 11.6% (13/112) for category III, and 0.9% (1/112) for category IV.
CPs were detected in 36.0% (80/222) and 43.2% (96/222) of community residents using settings A and B, respectively (Table 2). The participants in setting B detected the same CPs more consistently compared with setting C than setting A (setting A, 85.0% vs. setting B, 91.7%, P<0.001). The ROC curve of non-expert GPs for CP evaluation is shown in Fig. 4. Based on the ROC curve analysis, setting B exhibited better diagnostic performance than setting A (AUC, 0.856; 95% confidence interval [CI], 0.803 to 0.910 vs. AUC, 0.749; 95% CI, 0.683 to 0.815; P<0.05). The diagnostic results (sensitivity, specificity, accuracy, PPV, and NPV) for settings A and B are summarized in Table 2. Setting B achieved a higher overall diagnostic performance than setting A for all these parameters (all P<0.05).
Inter-observer Agreement
The inter-observer agreement for the CIMT and the target CP measurements, features, and categories obtained by settings A, B, and C are summarized in Tables 3 and 4. In terms of the interobserver agreement between settings B and C, there was excellent consistency in determinations of length, thickness, echogenicity, and surface morphology (all ICCs>0.80). The inter-observer agreement for CIMT, CP grading, and Plaque-RADS classification showed good consistency, with ICCs>0.60. Notably, a CP (intermediate risk, Plaque-RADS II) located on the posterior wall of the CCA bifurcation was identified by settings B and C but was overlooked by setting A (Fig. 5).
Collection times
Setting B required less time than settings A and C for detecting CPs (44.59±17.27 seconds vs. 126.13±69.61 seconds and 108.87±54.84 seconds, both P<0.01).
Questionnaire
In total, 96.4% (107/111) of the community residents underwent examinations with the intelligent handheld US device. Only 1.8% (2/111) felt uncomfortable, and 4.5% (5/111) thought the examination time was too long. Furthermore, 90.1% (100/111) believed the results, 89.2% (99/111) agreed to pay for the examination, and 97.3% (108/111) would recommend it to others (Table 5).
The non-expert GPs from setting B believed that 74.8% (83/111) of the AI-based handheld US examinations were easy to perform. Additionally, the participants believed that AI was helpful in 76.6% (85/111), unhelpful in 22.5% (25/111), and uncertain in 0.9% (1/111) of cases (Table 5).
Discussion
This prospective, parallel controlled trial confirmed that an intelligent handheld US device could enhance the ability of non-expert GPs to perform carotid examinations across community populations. The study revealed that non-expert GPs using a handheld US device equipped with AI could detect CPs as consistently as specialists, unlike their counterparts using the device without AI support. Furthermore, the diagnostic performance of non-expert GPs using the AI-enhanced device was not only satisfactory but also significantly better than that of GPs using the non-AI device (AUC, 0.856 vs. 0.749), closely matching the performance of specialists. In terms of sensitivity, specificity, accuracy, PPV, and NPV, non-expert GPs using the AI-equipped handheld US device achieved superior overall diagnostic performance compared to those using the device without AI. Additionally, these GPs maintained high consistency with specialists in measuring CIMT and CP characteristics such as length, thickness, echogenicity, surface morphology, and risk stratification. Consequently, non-expert GPs could preliminarily detect CPs and conduct risk stratification, potentially reducing the frequency of patient visits to tertiary hospitals. Moreover, the use of AI significantly reduced the data collection time for non-expert GPs using the handheld US device, also outperforming specialists who did not use the device. This efficiency could facilitate widespread CP screening in the future.
To our knowledge, this study is the first to explore the feasibility of using an intelligent handheld US device to assist non-expert GPs in independently detecting CPs in community populations. The four non-expert GPs, recruited from the Health Service Station, had no prior experience performing US examinations before participating in this trial. In contrast, the two specialist US doctors from a tertiary hospital possessed extensive experience in carotid US examinations. All data were collected and organized rigorously and systematically by an independent coordinator. To ensure the trustworthiness of the findings, a co-author who is a medical PhD with extensive clinical research experience reviewed the research design, setting, and framework. The analysis benefited from the support of practicing GPs and engineers on the authorship team, who played a crucial role in the smooth completion of this study.
To our knowledge, no studies have explored whether an intelligent handheld US device can enhance the ability of non-expert GPs to perform carotid examinations in community populations. This oversight may stem from the fact that traditional carotid US examinations are typically conducted by specialists such as sonographers and radiologists. However, there is currently a significant shortage of US specialists, particularly in low-resource areas [20], which restricts the ability to conduct large-scale CP screening in the general population. Moreover, financial constraints and equipment costs limit the widespread use of US devices [21]. Consequently, there is a need for an experience-independent, cost-effective, and convenient US device in the future. A portable US device, which is relatively easy to learn to use [8], was first developed in the 1990s [22] and has since become widely used in clinical settings for rapid diagnosis and treatment at the bedside (POCUS). Since the turn of the millennium, there has been an exponential increase in reports on the use and training of portable US devices [23]. The recent commercial availability of "pocket, or handheld, probes" has made POCUS more accessible [21]. However, many practicing physicians still lack formal training in this area. Additionally, challenges such as intermittent power supply for recharging batteries and unsafe storage conditions can pose significant barriers, potentially limiting the use of these devices in mobile clinics or peripheral healthcare centers [23]. Recently, GE developed a portable US device called VScan Air, which resembles a flip phone and has a battery life of 50 minutes [24]. In comparison, the handheld US device used in this trial offers a longer battery life, allowing for 2 hours of use with a built-in 6,000 mAh battery and supports Type-C DC power for 24 hours of battery life in outdoor mobile power mode.
Furthermore, AI can analyze the CIMT, CP area, and internal components of CPs based on images, subsequently evaluating CP stability [17]. Some reports have been published on the application of AI in carotid US examinations [25,26]. In 2020, Vila et al. [27] used a densely connected CNN (DenseNet) for the semantic segmentation of entire images. AI was then employed to perform a comprehensive analysis of the total volume, area, morphology, and composition of CPs, assessing their stability. In this study, an intelligent handheld US device equipped with AI used a CNN + multiclass support vector machine model with US signal data, rather than compressed US image data. This approach better utilizes numerous physical characteristics, such as in-phase/quadrature and radio frequency, compared to the device used by Vila et al. [27] Additionally, AI can enhance the quality of screening examinations conducted by non-expert GPs and may improve the early detection rate of CPs, potentially reducing the incidence of CVD and stroke [28,29]. This trial preliminarily confirmed that non-expert GPs, equipped with intelligent handheld US devices, were able to independently complete CP screening in community populations after receiving professional training. Moreover, AI has the potential to replace a significant amount of mechanical and repetitive work, thereby reducing the labor force required. This could help alleviate the shortage of professional doctors and conserve medical resources [30]. AI can also perform measurements automatically, which may help reduce collection time. This trial demonstrated that non-expert GPs using intelligent handheld US devices spent less time screening for CPs than specialist doctors.
Recruitment for this study was conducted through GP groups and by word-of-mouth among community residents. All participants from the community were required to complete a questionnaire to assess their acceptance of and satisfaction with the intelligent handheld US examination. In this study, non-expert GPs reported that 74.8% of the AI-based handheld US examinations were easy to perform, and they found AI to be helpful in 76.6% of the cases. Additionally, most community residents were receptive to the examination, trusted the results, and expressed willingness to recommend it to others in the future. However, a minority of participants reported discomfort and believed that the examination duration was excessively long. Despite this, considering the necessary infrastructure, it would be essential for community residents to incur certain fees in the future. Currently, the cost of carotid US in China is approximately $27 USD, and this is covered by health insurance. According to our findings, the majority of community residents were willing to pay these fees, and an additional cost of about $27 USD was deemed acceptable. However, the cost of carotid US may differ across countries, necessitating further research to assess its cost-effectiveness. Furthermore, this study involved interviewing non-expert GPs from setting B using questionnaires to collect their recommendations on the use of intelligent portable handheld US devices.
Based on the findings of this study, it is recommended that community populations undergo carotid US examinations at local community hospitals or health service stations, where the quality of care is comparable to that of tertiary hospitals. Additionally, GPs can stratify risk levels among CPs, facilitating comprehensive management of chronic diseases within the community. This approach not only reduces unnecessary travel to tertiary hospitals for low-risk residents, thereby alleviating the burden on these facilities, but also helps establish a rapid referral system between GPs and tertiary hospitals. This ensures that high-risk residents receive timely medical attention. This model could potentially be expanded to remote or medically underserved areas, benefiting a broader population. However, further research is necessary before this model can be widely implemented. The outcomes of this study provide a solid basis for future large-scale multicenter studies.
However, this study has several limitations. First, it was conducted at a single center, which may affect the generalizability of the findings. Further verification through multicenter studies is needed. Second, the degree of stenosis and CPs in the ICA are more clinically relevant in terms of stroke risk and other serious CVDs. Future subgroup analyses will focus on the degree of stenosis and CP locations. Third, our sample size was relatively small. In theory, non-expert GPs will become more skilled as the sample size increases, potentially enhancing diagnostic efficiency and reducing the time required for US examinations. Fourth, there was no verification through pathological or other imaging data (CT or MRI), which would definitively confirm the accuracy of the diagnoses. Future studies will need to include subgroup analyses to minimize result bias due to different operators, and further validation of our findings will be essential. Fifth, the evaluation after each US examination was based on a questionnaire and thus subjective. Future studies that provide objective or quantitative data may help address this issue.
In conclusion, this prospective and parallel controlled trial demonstrated that an AI-based handheld US device is effective for non-expert GPs conducting carotid US examinations. AI can help non-expert GPs reach a diagnostic proficiency comparable to that of specialist doctors in handheld carotid US examinations. The performance of non-expert GPs using the intelligent handheld US device in detecting carotid abnormalities was deemed acceptable for community residents.
Download Citation
PDF Links
PubReader
ePub Link
XML Download
Full text via DOI
