A laboratory feasibility study using a computer algorithm for anastomosis segmentation of epicardial ultrasonography images from distal coronary artery bypass anastomoses

Abstract

Background

The outcome of coronary artery bypass grafting (CABG) depends on several factors, including the quality of the distal anastomoses to the coronary arteries. Early graft failure may be caused by, e.g., technical suture failures, and such failures may be detected using intraoperative quality assessment. High-intensity epicardial ultrasonography (ECUS) allows anatomical visualization of the anastomoses during surgery, but currently, the images must be assessed manually. Here, we aim to describe an automatic quality assessment of distal coronary anastomoses using in-house software for vessel area and diameter extraction.

Methods

A postoperative, laboratory, investigational feasibility study comparing computer readings of longitudinal and transverse ultrasonographic images of distal coronary artery anastomoses with manual readings was performed, including ECUS images from 30 patients undergoing elective, isolated on-pump CABG. Vessel and anastomosis segmentation performance metrics from images obtained intraoperatively were compared to assess agreement between the manual and automatic segmentation methods. Scatter plots, the Dice coefficient and correlation analyses were used as measures of similarity between the two readings. p < 0.05 was considered significant.

Results

The number of dimensions of anastomotic vessel structures that are relevant for stenosis quantification and the Dice coefficient were 0.888 between the automatic and manual segmentations. The correlation coefficient between the manual and automatic stenotic rates was 0.674.

Conclusions

An anastomosis segmentation software for automatic and objective extraction of the anatomical dimensions of relevant distal coronary anastomotic structures from ECUS images obtained during CABG was developed. The framework allows for quantifying stenotic in the anastomotic structures and has the potential to assist surgeons during quality assessment of coronary anastomoses when the described segmentation of vessels and anastomoses is available for real-time epicardial ultrasonography use during surgery.

Trial registration

The study was registered on September 29, 2016, before enrolment of the first participant (ClinicalTrials.gov ID: NCT02919124).

Background

Coronary artery bypass grafting (CABG) is widely used for invasive treatment of coronary arteriosclerosis. Among several other factors, the outcome of CABG depends on the quality of the distal graft anastomosis, which should ensure the reestablishment of sufficient blood flow to the ischemic myocardium [1].

Early graft failures caused by, e.g., technical suture errors with the need for graft revision at the site of a peripheral coronary anastomosis may have been described in approximately 4–5% of the anastomoses [2,3,4], and several methods for intraoperative quality assessment of coronary artery bypass grafts, including distal anastomoses, have been proposed, including coronary angiography, transit-time flow measurement (TTFM), intraoperative fluorescence imaging and high-intensity epicardial ultrasonography (ECUS) [5, 6]. ECUS allows visual assessment of anastomotic structures, including the inner wall of an anastomosis, and may thus be able to detect an anastomotic stenosis caused by, e.g., a purse-string constrictive effect of a continuous suture technique or when a suture stitch has produced stenosis at the toe of an anastomosis. The ability to perform intraoperative quality assessments of coronary artery bypass grafts allows surgeons to correct graft failures before chest closure, and intraoperative detection of graft failures may be further improved if ECUS is combined with TTFM [7].

ECUS enables visualization of longitudinal and transverse anastomotic structures, allowing surgeons to identify anastomotic stenoses and to quantify the exact grade of stenosis. However, current readings of ECUS images require manual delineation of the vessel area or diameters in anastomotic structures, as no objective methods are available for this purpose. Therefore, interpretation of the anastomotic structures in ECUS images is subjective and may be time consuming [8, 9]. Assessment of anastomosis quality may be improved by obtaining a method for automated detection and segmentation of anastomotic structures in ECUS images. Automated detection and segmentation of anastomotic structures have previously been proposed for segmentation of transverse vessel structures in porcine end-to-end anastomoses with promising performance [10, 11]. However, no methods have been proposed to perform automated and generalized anastomosis segmentation of both longitudinal and transverse anastomotic vessel structures in human coronary artery bypass anastomoses.

The aim of this study was to evaluate the performance of postoperative viewer-independent distal coronary artery bypass anastomoses in transverse and longitudinal ECUS images using an anastomosis segmentation algorithm. We hypothesized that it would be possible to obtain the dimensions of anastomotic structures (vessel area and diameters) by identifying the inner wall of an anastomosis to assess potential stenotic grades within the anastomoses using a computer algorithm.

Methods

This study was a postoperative laboratory study comparing computer and manual readings of longitudinal and transverse ECUS images of distal coronary artery bypass anastomoses obtained from patients undergoing elective, isolated on-pump CABG in the Department of Cardiothoracic Surgery, Aalborg University Hospital, Denmark.

Patients and ultrasonographic data

ECUS images from the initial 30 on-pump CABG patients included in a previous study between September 27, 2016 – March 4, 2019, on ECUS were used for postoperative analyses in the present study [12]. No formal power calculation was performed because the study was a feasibility study. Only the primary ECUS images obtained during surgery were sent for postoperative computer and secondary manual interpretation of the images. All manual postoperative readings of the ECUS images were performed by an ECUS imaging expert, with no a priori information with respect to any anastomoses being revised per- or postoperatively. ECUS images sent for analyses were images with the least artefacts.

A 15-MHz ultrasonography probe (Medistim VeriQ System, Medistim A/S, Oslo, Norway) was used to obtain high-resolution ECUS sequences of longitudinal and transverse images of the heel, middle and toe in peripheral coronary bypass anastomoses (see Fig. 1). The ultrasound transducer was connected using a rotatable stabilizing device (Echoclip, Aalborg University Hospital, Denmark) to ensure air-free contact between the transducer and the anastomosis of interest without the risk of any deformation of the anastomosis [12]. As we were not aware of any sterile ultrasound gel approved for application into the pericardial space, we used coagulated blood from the patients as a coupling agent in some cases [13]. Epicardial ultrasonography of anastomoses were performed during cross-clamp while free conduits were infused with pressure controlled cold blood cardioplegia through the proximal end of the graft using an infusion pump. Imaging of in-situ internal mammary artery grafts were performed after the cross-clamp was released while the patients were still on-pump.

Visual assessments of the ECUS sequences were performed to identify ECUS frames that contained longitudinal or transverse images of the heel, middle and toe for each anastomosis.

Fig. 1

Examples of anastomotic structures in different views. G: Graft; CA. Coronary artery. (A) Longitudinal view of an end-to-side anastomosis where the location of the heel, middle, and toe sections of the anastomosis are shown. (B) A transverse view of the heel containing a graft and a coronary artery. (C) Transverse view of the mid-section of the anastomosis, where the red line represents the width of the anastomotic orifice in the frame. The yellow contours represent manual segmentations of the anastomotic vessel structures

Five frames were selected for manual segmentation of the anastomotic vessel structures. In each frame, the graft (G) and coronary artery (CA) were segmented separately and labelled as either a transverse vessel or longitudinal vessel, which were further subcategorized into four different vessel cases: Separate transverse vessels (TV_sep), separate longitudinal vessels (${L}_{sep}$), transverse midanastomosis vessels (${A}_{tv}$), and longitudinal mid-anastomosis vessels (${A}_{l}$). $T{V}_{sep}$ and ${L}_{sep}$ were defined as manual vessel segmentations that were not obtained in the mid-section of the anastomosis. ${A}_{tv}$ and ${A}_{l}$ were defined as manual segmentations obtained in the mid-section of the anastomoses that only consisted of either transverse or longitudinal vessel structures, respectively. The categories were selected for anastomosis quality assessment validation, as different performance metrics were relevant in each case. This resulted in a total of 3640 $T{V}_{sep}$, 617 ${L}_{sep}$, 309 ${A}_{l}$, and 385 ${A}_{tv}$ manual segmentations. All manual segmentations from 10 randomly selected patients were selected for segmentation algorithm training, and the remaining 20 patients were selected as test data for validation.

Automatic segmentation

Automatic segmentation of the ECUS images from which vessel structures were extracted was performed using an in-house anastomosis segmentation algorithm framework initially designed for segmentation of both transverse and longitudinal anastomotic vessel structures [11]. The framework started by identifying potential lumen regions in all the selected ultrasound frames using contrast analysis. An active shape model framework was then used to acquire the initial transverse vessel segmentations in all the ultrasound frames. The same active shape model was used to identify initial longitudinal vessels and anastomotic structures in the ultrasound frames. If any segments were missing from the longitudinal vessels or anastomotic structures, the segmentation was supplemented by using the segmented transverse vessels. Vessel and anastomosis segmentation performance metrics were analysed to assess the agreement between the automatic and manual segmentations (see Fig. 2). A detailed technical description of the manual and computer anastomosis segmentation framework and the anastomosis segmentation process is provided in an additional file [see Additional file 1].

Fig. 2

Different steps of anastomosis segmentation. (A) Intermediate steps of initial transverse vessel (TV) segmentation that is inflated in a large graft due to low tissue contrast in the initial contour. (B) Intermediate steps in segmentation of a longitudinal vessel. Two low tissue contrast segments are detected (red and magenta) in the initial TV segmentation to the right in the 1D-tissue contrast analysis (graph below), causing an additional TV segmentation to be added. (C) Intermediate segmentation steps of an. A missing tissue segment is detected at the bottom of the initial TV segment (green). The added TV segmentation allows the extraction of the size of the anastomotic orifice for stenosis quantification

Statistical analyses

The Dice coefficient and correlation analyses were used as measures of similarity between the automatic and manual segmentations of the ECUS images. The Dice coefficient is a statistical measure of similarity between two measurements [14]. A scatter plot was used to visualize the correspondence between the manual and automatic stenotic rates as a function of the mean manually segmented vessel diameter of each landmark segment of the anastomosis. Additionally, the correlation coefficients between the manual and automatic stenotic rates were calculated. p < 0.05 was considered significant.

Results

Both single and sequential coronary artery bypass grafts (internal mammary arteries, radial arteries and saphenous vein grafts) were used during surgery. ECUS images obtained from all coronary territories were used for segmentation in the present study. Patients received an average of 2.9 peripheral anastomoses. Selected preoperative and operative data are shown in Table 1.

Table 1 Preoperative patient characteristics and operative data (n = 30)

The anastomosis segmentation performances in four different vessel cases are shown in Table 2.

Table 2 Quantitative metrics of segmentation performance for each of four segmentation cases

The scatter plot and contrast plots are shown in Fig. 3, and the qualitative results for each case are shown in Fig. 4.

Fig. 3

Scatter plots between the automatic and manual segmentation methods. (A-C) Segmentation area precision as a function of contrast in TV vessels (C), segmentation diameter precision as a function of vessel contrast from L/-cases (E), and scatter plot between the automatically and manually segmented anastomosis diameters in -cases

Fig. 4

Representative examples of the anastomosis segmentations. (Yellow contours = automatic segmentation, green contours = manual segmentation). (A) Good TV segmentation at a heel site, (B) Open ${A_tv}$ with a single TV segmentation, (C) Good ${A_l}$ segmentation. (D) ${A_l}$ with stenosis in the toe detected, (E) ${A_l}$ with lumen overestimation due to low-contrast plaque, (F) TV heel site with underestimation of CA due to close proximity of another vessel, (G) ${A_tv}$ where the size of the anastomotic orifice was extracted (magenta line), (H) TV heel site where the graft is underestimated due to lumen artefacts, (I) ${A_l}$ with underestimation of the graft due to high lumen intensity, (J) A small longitudinal vessel, which is estimated as a missing tissue segment that was not detected during segmentation

$\varvec{T}{\varvec{V}}_{\varvec{s}\varvec{e}\varvec{p}}$:

In the cases of $T{V}_{sep}$ and ${A}_{tv}$, the overall vessel Dice score was 0.887, with similar performances for G and CA, with Dice scores of 0.888 and 0.885, respectively. Scatter plots showing a linear trend were obtained between the manual and automatic vessel areas, as shown in Fig. 3a, which shows strong agreement. However, a decrease in accuracy was observed for vessels with an area less than 500 pixels. Most of the outliers were obtained in low-contrast vessels or in coronary arteries containing large septal vessels, which causes missing tissue information. This was supported by a larger error in vessels with lower contrast.

${L_{sep}}/{A_l}$:

In ${L_{sep}}$, an overall vessel Dice of 0.848 was obtained, where the highest Dice of 0.872 was obtained for CA vessels. The average diameter difference was pixels with a linear trend. Outliers caused by septal vessels and/or low-contrast segments were observed, showing that a lower diameter precision was obtained in low-contrast segments (see Fig. 3d). The same segmentation accuracy tendencies were obtained for the ${A_l}$-frames, where the overall Dice score was 0.8838. In ${A_tv}$ the frames, an anastomosis Dice score of 0.864 was obtained. The difference between the manual and automatic segmented anastomosis lengths was the number of pixels. A linear trend was observed in the scatter plot in Fig. 3e. Segmentation issues could occur if multiple vessel structures were located proximal to each other or in patients with high lumen intensity or artifacts (see Fig. 4f, i and h). The iterative segmentation setup allowed segmentation of both single transverse vessels, ${A_tv}$, and ${A_l}$, and extracted the size of the anastomotic orifice in ${A_tv}$ for small stenoses but not when no stenosis was present (see Fig. 4b, g and j).

$\varvec{T}{\varvec{V}}_{\varvec{s}\varvec{e}\varvec{p}}$:

An overall vessel Dice coefficient of 0.877 was found for $T{V}_{sep}$, with similar results for both the graft and CA (Table 1). The Dice coefficients for ${L}_{sep}$ were slightly lower. A linear trend was obtained between the manual and automatic vessel areas with a few outliers within the 95% limits of agreement, which shows that high agreement was obtained in most cases (see Fig. 3a).

${A_l}$ Stenotic severity quantification:

The results from extracting stenosis severity in the anastomoses from the heel and toe sites in the ${A_l}$-cases are shown in Fig. 5. The correlation coefficient between the manual and automatic stenotic severity was 0.6741 (p < 0.001). A total of 87% manually segmented stenotic single frames showing a stenotic grade > 50% were correctly detected using the automatic segmentation framework, while the overall accuracy was 77.11%. A total of 46.7% of the insignificant manually segmented stenoses was correctly detected, indicating the overestimation of significant stenoses by the algorithm (see Fig. 5b).

The algorithm uses the same frames as was manually segmented by the operator. There was a total of 385 anastomotic structures (Atv), and 3649 total TVsep segmentations, so there were 10 TV segments per anastomotic structure on average.

Fig. 5

Results from extracting stenotic severity from A_l frames. (A) Scatter plot between the manually and automatically segmented stenosis severity. (B) Confusion matrix of the stenosis classification into two categories: those with a stenotic grade either above or below a threshold of 0.5

Postoperative laboratory analyses of the ECUS images did not lead to any postoperative graft revisions. However, per-operative combined use of TTFM and ECUS resulted in immediate revision of a peripheral anastomosis in five different patients: TTFM was not acceptable in the left internal mammary artery (LIMA) graft in two patients (Flow < 20 ml/min; Pulsatility index (PI): > 5), and in a third patient TTFM indicated a problem with a right internal mammary artery (RIMA) sequential side-to-side anastomosis to an intermediate coronary artery (Flow < 20 ml/min, PI: > 5). ECUS indicated a graft problem in two of these patients, and ECUS was not obtained from the side-to-side anastomosis in the third. During revision of the anastomoses a stenosis due to a suture was observed in two anastomoses, and no explanation was found in a LIMA graft. Flow was increased in all anastomoses after revision. A fourth patient had a saphenous vein graft anastomosed to a marginal coronary artery with no indication of a flow problem while infusing cardioplegia into the graft. However, ECUS showed antegrade obstruction at the toe of the anastomosis. In a fifth patient, ECUS indicated an obstruction in a LIMA to the LAD anastomosis while the TTFM was acceptable (Flow: 26 ml/min; PI: 1.8¸ Diastolic filling: 69%). Revision of these anastomoses showed an obstructing intimal flap at the toe in both anastomoses. One patient experienced recurrent angina and ventricular fibrillation in the ward on postoperative day four. After cardiopulmonary resuscitation, a coronary re-angiography showed low flow from the LIMA to the LAD and spasm with thrombus formation in the sequential RA graft distal to the first and a still-patent side-to-side anastomosis to a diagonal branch. The thrombus was causing occlusion of the anastomoses to the intermediate and the first obtuse marginal artery. The patient was brought to the operating room, and new anastomoses were constructed. The patient was discharged after one week following the re-operation without any physical sequelae. ECUS did not indicate any problems during the primary operation, and TTFM did not indicate any problems with the sequential graft flow either.

The postoperative analyses of ECUS images in the present study did not give rise to any additional laboratory or clinical investigations of patients included in the study.

Discussion

In this postoperative laboratory study, we showed that it is feasible to automatically segment and extract vessel diameters and areas from ECUS images of distal coronary artery anastomotic structures obtained during on-pump CABG. Furthermore, automatic estimates of stenotic severity may be calculated. The anastomosis segmentation obtained Dice coefficients of approximately 0.88 between the automatic and manual segmentations. A correlation of 0.674 was obtained between manual and automatic stenotic grades in the ECUS frames of longitudinal anastomoses.

The long time-period from inclusion of the patients in the study [12] to these segmentation analyses were performed, was primarily caused by the need for developing the computer algorithm to be used for longitudinal segmentation.

The reliability of automatic vessel segmentation was negatively affected by vessel contrast with respect to the background. It was possible to show direct feasibility in extracting the stenotic severity in ${A}_{l}$, as the segmentation framework was applied to single ECUS frames. It was demonstrated that the vessel area could be automatically extracted largely independently of vessel size. Furthermore, it was demonstrated that the vessel area could be automatically extracted largely independently of vessel size, allowing automatic stenotic severity quantification. Automatic anastomosis quality assessment may also be possible due to automatic stenosis severityestimation while providing anatomical information on the location of a stenosis. However, automatic anastomosis quality assessment should always be seen as a supportive technique for the surgeon’s manual readings of the ECUS images, the TTFM measurements and clinical judgement, as there will be a risk of over- and underestimating the stenosis rate in the anastomosis. We do not think that a specific cut-off value can be set at which the surgeon must make a revision of an anastomosis, however, as readings of ECUS imaging requires some expertise, automatic analyses may be most helpful for surgeons with less experience in analyzing such images.

The main reason for low segmentation performance was caused by strong vessel lumen artifacts, low contrast vessels or vessel segments, or vessels containing septal vessels emanating from the coronary artery (see Fig. 4h, j and f). Both lumen artefacts and low contrast vessels may be considered operator dependent, for which the contrast model may be used to indicate if the current vessel segmentation contains reliable contrast. Septal vessels cause loss of tissue information and can be difficult to handle in a single-frame segmentation framework. However, this could be improved by utilizing temporal information during the sequence. As shown, the segmentation performance depends on sufficient tissue contrast, which can be considered operator dependent. The top of the grafts often contained low contrast, which caused overestimation of the vessel structures. However, this could be improved by ensuring that vessel segmentations are only performed when the measured vessel contrast is sufficiently high. This will also provide feedback to the operator that the image quality is not sufficient. Use of UL gel as a coupling agent instead of the use of coagulated blood will reduce the risk of artefacts in top of the anastomosis [13] and may improve vessel segmentation during ECUS.

All ECUS images were obtained using an ultrasonography probe attached to a stabilizing device. We have no reason to believe that this technique for obtaining ultrasonographic images has any impact on the quality of the images obtained compared with images obtained without the use of a stabilizing device. However, it remains to be shown whether segmentation of anastomotic structures in off-pump procedures may require additional tracking of the vessel structures during an ECUS sequence due to the pumping heart.

Automatic quality assessment of distal coronary anastomoses was performed postoperatively in the laboratory in the present study, and the method must therefore be further developed for the technique to be available for surgeons in real time in the operating room. An algorithm for real-time use may be integrated into devices used for ECUS imaging.

One of the detrimental remarks that surgeons often make about surface ultrasound in the graft verification procedure is that the use ECUS will increase the operative time. An ECUS procedure may increase the operative time by about 1½ minutes for each distal anastomosis [8], but as use of ECUS in addition to TTFM have been shown to increase the diagnostic accuracy in relation to intraoperative graft verification [7], this time may be well spent.

We were not able to compare imaging quality obtained from different kinds of peripheral anastomoses due to the low number of patients included in the study. However, as reported in a previous study [12], imaging could not be obtained from all anastomoses performed during the study e.g. due to a curved sequential graft with an anastomosis that could not be contained in the straight cavity of the stabilizing devise which was used during the study. Echo artefacts from a Titanium clip located in the roof of an anastomosis and challenges in interpreting the images in the operating room also affected the imaging quality.

Segmentation analyses in the present study did not result in information on the percent of ECUS evaluations with a true anastomotic stenosis of > 50%. Stenotic grades were determined based only on the content in each single frame of an anastomotic site to simplify the study. An anastomosis may appear more or less stenotic, dependent on the two-dimensional viewing angle in the ECUS image. This alone does not reflect the real three-dimensional appearance of an anastomosis. Single frame stenoses were used to indicate performance when anastomotic sites had different stenotic grades. ECUS is a dynamic investigation technique, and to determine an accurate stenotic grade of an anastomosis would require an in-depth analysis of all frames in the video content from an actual anastomotic site. The main objective in the present study was to try to identify the inner wall of the anastomoses in selected ECUS images with as little artefacts as possible, and these images were not always obtained from true stenotic anastomoses. Determining the stenotic grades based on the content of a single frames was considered sufficient for this study.

Apart from this study being a laboratory study, there are several other limitations to the study that must be acknowledged. The results were based on data from only 30 patients. The current segmentation framework was mainly based on extracting vessel appearance and a few additional parameters that require less tuning than, e.g., convolutional neural networks. Furthermore, only 10 patients were used for training the current segmentation. We would therefore expect the method to be more robust when applied to new and more data. The recruitment of a larger dataset for training and testing may benefit from a more accurate interpretation of the system performance.

Furthermore, it should be noted, that there are inherent difficulties involved in optimal probe location and angle for both longitudinal and transverse vessel images when the operator is attempting to capture all the information of a 3D object, in a simple image. However, we have no reason to believe that this have any impact on the results of the present study: We only tried to be able to identify the inner surface of an anastomotic structure, thus being able to show that potential clinical significant stenotic anastomotic structures may be identified if an in-dept computer analysis of full ECUS sequences are performed.

Finally, TTFM was used as a routine at different time points during surgery in all patients, but as ECUS imaging was not performed at the same time as TTFM, we were not able to investigate whether the Dice coefficients are dependent on graft flow.

Conclusions

An anastomosis segmentation software for automatic and objective extraction of the anatomical dimensions of relevant distal coronary anastomotic structures from ECUS images obtained during CABG was developed. As the framework allows identification of the inner wall of an anastomosis this framework may be able to quantify a potential stenosis in the anastomotic structures. Thus, this framework has the potential to assist surgeons during quality assessment of coronary anastomoses when the described segmentation of vessels and anastomoses is available for real-time ECUS use during surgery.