This article has Open Peer Review reports available.

# Design of a factorial experiment with randomization restrictions to assess medical device performance on vascular tissue

- Wiebke S Diestelkamp
^{1}Email author, - Carissa M Krane
^{2}Email author and - Margaret F Pinnell
^{3}

**11**:75

https://doi.org/10.1186/1471-2288-11-75

© Diestelkamp et al; licensee BioMed Central Ltd. 2011

**Received: **10 May 2010

**Accepted: **20 May 2011

**Published: **20 May 2011

## Abstract

### Background

Energy-based surgical scalpels are designed to efficiently transect and seal blood vessels using thermal energy to promote protein denaturation and coagulation. Assessment and design improvement of ultrasonic scalpel performance relies on both in vivo and ex vivo testing. The objective of this work was to design and implement a robust, experimental test matrix with randomization restrictions and predictive statistical power, which allowed for identification of those experimental variables that may affect the quality of the seal obtained ex vivo.

### Methods

The design of the experiment included three factors: temperature (two levels); the type of solution used to perfuse the artery during transection (three types); and artery type (two types) resulting in a total of twelve possible treatment combinations. Burst pressures of porcine carotid and renal arteries sealed ex vivo were assigned as the response variable.

### Results

The experimental test matrix was designed and carried out as a split-plot experiment in order to assess the contributions of several variables and their interactions while accounting for randomization restrictions present in the experimental setup. The statistical software package SAS was utilized and PROC MIXED was used to account for the randomization restrictions in the split-plot design. The combination of temperature, solution, and vessel type had a statistically significant impact on seal quality.

### Conclusions

The design and implementation of a split-plot experimental test-matrix provided a mechanism for addressing the existing technical randomization restrictions of ex vivo ultrasonic scalpel performance testing, while preserving the ability to examine the potential effects of independent factors or variables. This method for generating the experimental design and the statistical analyses of the resulting data are adaptable to a wide variety of experimental problems involving large-scale tissue-based studies of medical or experimental device efficacy and performance.

## Keywords

## Background

Blood vessel transection and sealing requires that tissue segments be held together with an immediate and temporary bond that prevents leakage and holds long enough to allow the adjacent tissue margins to re-grow, heal, and regenerate structural connections through the natural healing process. Energy-based vessel sealing methods use no exogenous materials to accomplish vessel closure, but rely on acute vessel compression, thermally induced tissue injury, and tissue healing to generate the seal. This process limits inflammation and eliminates concerns with biocompatibility and immunogenicity associated with using an exogenous material such as sutures, staples, or bonding adhesives. In addition, energy-based methods are easier and faster to use than materials-based methods, especially for endoscopic surgeries, resulting in a decrease in operative time and surgical invasiveness, and reduced infection rates. Ultrasonic scalpel technology has been used extensively in fine tissue dissection, hemostasis, vessel closure and transection [1–3]. The optimization of ultrasonic device performance relies on both in vivo animal testing and ex vivo tissue testing at a level and degree that achieves statistically significant evidence of performance standards. Many physiological and anatomical properties of biological tissues exhibit inter-individual variability due to the sex, age, and health of the animal from which the tissue was harvested. Moreover, it is technically difficult to recapitulate physiological conditions ex vivo. However, proof of device efficacy through ex vivo testing is required prior to in vivo testing. As such, it is critical to control as many experimental variables as possible, and to measure those that cannot be controlled during ex vivo testing in order to achieve results that realistically assess device performance. Therefore, the challenge is to identify those experimental conditions that should be tested, and those that should be controlled. The generation of statistically relevant observations is dependent on the careful design of a test matrix that facilitates the assessment of the effects of fixed and variable test conditions while taking into account technical or procedural restrictions to randomization that exist for the proposed experiments. In this report, the computational and biostatistical tools that were employed to generate a robust and randomized test matrix to assess ex vivo ultrasonic scalpel performance are described.

## Methods

### Porcine vessel sealing and burst pressure test

Porcine vessel sealing and burst pressure testing were performed as previously described [4]. Briefly, porcine carotid and renal arteries were harvested from tissue remnants from animals slaughtered for food consumption at a USDA-inspected slaughterhouse facility by technicians from Animal Technologies (Tylersville, TX). Institutional Animal Care and Use Committee (IACUC) approval was not required for the use of these tissue remnants since no live animals were used in experiments, and the tissues were obtained from a food-purposed slaughterhouse source. Luer locks were sutured to each end of the artery. The arteries were mounted onto a custom-designed tension device that stretched the vessel at a constant rate of 0.7 mm/s to a load of ~113 gram-force (0.25 lbf). The tensioned vessel was infused with one of three test solutions pre-equilibrated to ambient temperature (23°C) or 37°C. An HA2001 PHD2000 Harvard Apparatus dual syringe infusion pump supplied a constant infusion rate of 1.5 ml/min. During infusion, each artery was transected and sealed using an ultrasonic scalpel connected to an ultrasonic generator set at power level 3. Subsequent to transection and sealing, each vessel was subjected to pressure burst testing. A 10,340 mmHg (200 psi) Measurements Specialties MSP-300-200-P-N-1- pressure transducer was used to record the pressure that was generated in the sealed vessel during constant infusion with the test solution at a rate of 20 ml/min. Vessel burst pressure, the response variable in this experiment, was defined as the pressure at which the test solution was observed to leak from the seal and/or a rapid drop in pressure was noted in the pressure versus time curve [5, 6]. The load cell and pressure transducers were connected to a Vishay Micromeasurements System 5000 Model 5100 B Scanner and Strain Smart^{®} software was used to record the data.

### Biostatistical experimental matrix design

The design included three factors: The temperature of the solution (Temp - two levels; ambient (23°C) or 37°C), the type of solution that was utilized (Sol - three types; 0.9% saline; 6% Hetastarch (HES); "Plasma protein" enriched buffer (95% Hanks balanced salt solution, 4.7% bovine serum albumin fraction V, 0.36% fibrinogen type IV)) and the type of vascular tissue used (Vessel - two types; carotid or renal artery).

### Sample size calculation

A number of preliminary experiments (n = 4-12) were carried out as described above. The data were used to perform basic sample size computations to aid in determining the sample size for the main experiment. All computations were based on comparisons of two means. To determine the sample size necessary to detect a significant difference between the means of two factors at a desired significance level and with given power, the observed sample means and the pooled standard deviation corresponding to the two factors from the pilot were computed. The Analyst application in SAS was utilized to compute the sample sizes. The input consisted of the two observed sample means, the pooled standard deviation, the desired significance level and the desired power for the test.

### Statistical analyses of the data

The statistical package SAS was used to carry out the data analysis. The factors temperature, solution and vessel type, as well as "Session" (with 48 levels) were declared as class variables for the analysis of variance. To incorporate the information about the randomization restrictions discussed above, PROC MIXED was used, and Session was declared as a random effect as suggested in [7] (the others are treated as fixed effects by default). The data set was imported into SAS from Excel, with columns for session, temperature, solution, vessel type and the response variable.

### SAS code

PROC MIXED data = dataset;

CLASS Session Temp Sol Vessel;

MODEL response = Temp|Sol|Vessel;

RANDOM session;

RUN;

### Interpretation of SAS output

SAS computed the ANOVA table displaying the p-values for the main effects (temperature, solution and vessel type) and their interactions. Using the significance level of 0.05, a p-value of less than 0.05 indicates a (statistically) significant main effect or interaction effect.

## Results

The objective of this work was to design and implement a robust, experimental test matrix with randomization restrictions and predictive statistical power, which allowed for identification of those experimental variables that may affect the quality of the seal obtained ex vivo. A number of pilot studies using small sample sizes were initially performed to determine which variables likely contribute to seal quality and therefore should be included in the test design. The factors initially studied included temperature of the solution, type of infusion solution, vessel type, and infusion flow rate during blood vessel transection/sealing. The response variable in each case was the burst pressure (mmHg) at which the seal failed.

In addition, the pilot data was used to estimate the sample size needed to obtain significant results (at a significance level of 0.05) with a power of 0.9, or the probability that a test with the specified assumptions (sample size, observed difference of means, observed pooled standard deviation, and type of alternative hypothesis) would correctly reject the null hypothesis when the alternative hypothesis was true. Given the minimum desired detectable difference, as well as (an estimation for) the standard deviation and the desired significance level of the test, the sample size that was necessary for a two-sample test to have a given power was computed. The sample size computations were based on pairwise comparisons, in which each factor level was examined as to its potential effect on burst pressure. In some cases, it was clear that these comparisons were not trending towards showing significant differences, and the corresponding factors (or factor levels) were therefore not included in the experimental design. However, those comparisons that were not statistically significant in the pilot study due to small sample size but showed a trend toward predicted statistical significant differences with an increased sample size (based on the power calculation) were included in the experimental design. In addition, the interaction effects of the included factors (comparisons between combinations of two or three factors) were deemed important to assess.

Two copies of the six Temperature/Solution combinations, plus a column of random numbers.

Temperature | Solution | Random number |
---|---|---|

Temp1 | Sol1 | 0.617940611 |

Temp1 | Sol2 | 0.739488341 |

Temp1 | Sol3 | 0.088714721 |

Temp2 | Sol1 | 0.556090887 |

Temp2 | Sol2 | 0.55227804 |

Temp2 | Sol3 | 0.691030612 |

Temp1 | Sol1 | 0.286876071 |

Temp1 | Sol2 | 0.535759581 |

Temp1 | Sol3 | 0.882488882 |

Temp2 | Sol1 | 0.981425728 |

Temp2 | Sol2 | 0.228304389 |

Temp2 | Sol3 | 0.606414609 |

Two copies of the six Temperature/Solution combinations, ordered by the size of the random numbers.

Temperature | Solution | Random number |
---|---|---|

Temp1 | Sol3 | 0.088714721 |

Temp2 | Sol2 | 0.228304389 |

Temp1 | Sol1 | 0.286876071 |

Temp1 | Sol2 | 0.535759581 |

Temp2 | Sol2 | 0.55227804 |

Temp2 | Sol1 | 0.556090887 |

Temp2 | Sol3 | 0.606414609 |

Temp1 | Sol1 | 0.617940611 |

Temp2 | Sol3 | 0.691030612 |

Temp1 | Sol2 | 0.739488341 |

Temp1 | Sol3 | 0.882488882 |

Temp2 | Sol1 | 0.981425728 |

The twelve sessions in random order.

Session | Temperature | Solution |
---|---|---|

1 | Temp1 | Sol3 |

2 | Temp2 | Sol2 |

3 | Temp1 | Sol1 |

4 | Temp1 | Sol2 |

5 | Temp2 | Sol2 |

6 | Temp2 | Sol1 |

7 | Temp2 | Sol3 |

8 | Temp1 | Sol1 |

9 | Temp2 | Sol3 |

10 | Temp1 | Sol2 |

11 | Temp1 | Sol3 |

12 | Temp2 | Sol1 |

ANOVA Table.

Effect | Num DF | Den DF | F Value | Pr > F |
---|---|---|---|---|

Sol | 2 | 426 | 0.21 | 0.812 |

Temp | 1 | 426 | 1.14 | 0.2857 |

Sol*Temp | 2 | 426 | 0.05 | 0.949 |

Vessel | 1 | 426 | 2.65 | 0.1045 |

Sol*Vessel | 2 | 426 | 0.83 | 0.4354 |

Temp*Vessel | 1 | 426 | 1.36 | 0.2449 |

Sol*Temp*Vessel | 2 | 426 | 3.64 | 0.027 |

## Discussion

Assessing the efficacy of surgical device performance ex vivo is dependent upon the quality of the experimental design, and the ability of the test to control or measure those variables that contribute to device performance. Careful design of the experimental test matrix is an important first step in this process. Failure to appropriately consider practical limitations on randomization can substantially compromise the utility and significance of the data generated. The biostatistical methods described in this report provide a procedural model for the development of the design of factorial experiments that have randomization restrictions. Since the impact of more than two factors on a response variable were assessed in this experiment, it was appropriate to carry out an analysis of variance. However, an underlying assumption for the validity of the usual analysis of variance for a complete factorial design is that the runs are observed in a completely randomized order [8]. This usually requires frequent changes in the levels of the factors, which proved to be problematic in this experiment. While it was possible to randomly assign one of the two tissue types to an experimental run, there were several constraints on the other two factors due to the setup of the experiment. Solution and temperature were the hard-to-change factors. Changing the temperature required considerable time and effort, as the solution, the tissue and the apparatus needed to be adjusted to the new temperature. Changing the solution required a complete disassembly of the apparatus since the entire apparatus had to be flushed, cleaned and dried before a different solution could be employed. Due to these constraints, a completely random order of the 480 runs was not feasible. Instead, a split-plot design was used [9]. A split-plot experiment is a blocked experiment, but the blocks themselves are experimental units for a subset of the factors. The blocks are called whole plots, while the experimental units within blocks are split plots. This introduces two levels of randomization: The random order of the whole plots, and the random assignment of the treatments within the whole plots. This approach has been successfully used in industrial experiments involving factors that are hard or costly to change [7, 10–13]. In the experiment described here, the whole plots consisted of the sessions (with solution and temperature fixed in each session), and each type of vessel was observed five times within each session (in random order). Using this experimental design and analysis, it was determined that the combination of temperature, solution, and vessel type affected seal quality in a statistically significant way. Therefore, experimentally, it is apparent from these data that all three of these factors need to be considered when developing an ex vivo test method for the assessment of harmonic scalpel performance and device optimization.

Acute collateral damage, although expected, was not examined in this study. Histopathology is commonly used to estimate the extent and degree of thermally-induced coagulative necrosis and protein denaturation incurred in relation to the distance from the point of energy application [14]. However, measurements of tissue injury margins obtained by histopathology may not reflect the true distance of thermal impact due to the tissue shrinkage that occurs during processing [15]. Several methods exist for determining temperature elevation and thermal spread in tissues at and at fixed distances from the energy source [16]. However, differences exist between in vivo and ex vivo analyses of collateral damage and thermal spread, especially at larger distances from the blade [17]. These differences are likely due to the effects of perfusion on the efficient removal of heat and the rapid activation of the innate acute wound response in vivo [17]. Nevertheless, the assessment of collateral tissue damage is an important aspect of surgical device performance. In future experiments, measurements of collateral damage obtained through histopathology along with temperature elevation and thermal spread can be incorporated as response variables within the optimized test-matrix described here to provide a more comprehensive analysis of device performance ex vivo.

## Conclusions

From this investigation, it was determined that the design and implementation of a split-plot experimental test-matrix provided a mechanism for addressing the existing technical randomization restrictions in a statistically meaningful way, while preserving the ability to examine the potential combinatorial effects of independent factors or variables that can be assessed in ex vivo testing. This type of approach would likely be useful to a broad spectrum of medical device researchers who must attempt to recapitulate in vivo conditions, ex vivo.

## Declarations

### Acknowledgements

The authors would like to thank University of Dayton undergraduate students Courtney Gardner, Emily Kitchin, Anne Klapheke, Mercedes Thompson, Kevin Timms, and Eric Whitney for their technical assistance, and Dr. Peter Hovey, Department of Mathematics, University of Dayton, for critical review of this manuscript. This research was funded by an industrial research contract from Ethicon Endo-Surgery, Blue Ash, OH, to the University of Dayton.

## Authors’ Affiliations

## References

- Foschi D, Cellerino P, Corsi F, Tiadelli T, Morandi E, Rizzi A, Trabucchi E: The mechanisms of blood vessel closure in humans by the application of ultrasonic energy. Surg Endosc. 2002, 16 (5): 814-819. 10.1007/s00464-001-9074-x.View ArticlePubMedGoogle Scholar
- Harold KL, Pollinger H, Matthew BD, Kercher KW, Sing RF, Heniford BT: Comparison of ultrasonic energy, bipolar thermal energy, and vascular clips for the hemostasis of small-, medium- and large-siazed arteries. Surg Endosc. 2003, 17 (8): 1228-1230. 10.1007/s00464-002-8833-7.View ArticlePubMedGoogle Scholar
- Landman J, Kerbl K, Rehman J, Andreoni C, Humphrey PA, Collyer W, Olweny E, Sundaram C, Clayman R: Evaluation of a vessel sealing system, bipolar electrosurgery, harmonic scalpel, titanium clips, endoscopic gastrointestinal anastomosis vascular staples and sutures for arterial and venous ligation in a porcine model. J Urol. 2003, 169 (2): 697-700. 10.1016/S0022-5347(05)63995-X.View ArticlePubMedGoogle Scholar
- Krane C, Pinnell M, Gardner C, Thompson M, Coleman J, Wilkens R: Mechanical test methods for assessing porcine carotid and uterine artery burst pressure following ex vivo ultrasonic ligature seal and transection. J Test Eval. 2011, 39 (4): 1-8.Google Scholar
- Bubenik LJ, Hosgood G, Vasanjee SC: Bursting tension of medium and large canine arteries sealed with ultrasonic energy or suture ligation. Vet Surg. 2005, 34 (3): 289-293. 10.1111/j.1532-950x.2005.00043.x.View ArticlePubMedGoogle Scholar
- Novitsky YW, Rosen MJ, Harrell AG, Sing RF: Evaluation of the efficacy of the electrosurgical bipolar vessel sealer (LigaSure) devices in sealing lymphatic vessels. Surg Innov. 2005, 12 (2): 155-160. 10.1177/155335060501200215.View ArticlePubMedGoogle Scholar
- Webb DF, Lucas JM, Borkowski JJ: Factorial experiments when factor levels are not necessarily reset. J Quality Technology. 2004, 36 (1): 1-11.Google Scholar
- Montgomery DC: Design and Analysis of Experiments. 2005, New Jersey: John Wiley & Sons, Inc, 7Google Scholar
- Federer WT, King F: Variations on Split Plot and Split Block Experiment Designs. 2007, New Jersey: John Wiley & Sons, IncView ArticleGoogle Scholar
- Ganju J, Lucas JM: Detecting randomization restrictions caused by factors. J Stat Plann Inference. 1999, 81: 129-140. 10.1016/S0378-3758(99)00002-6.View ArticleGoogle Scholar
- Ganju J, Lucas JM: Bias in test statistics when restrictions in randomization are caused by factors. Comm Statist Theory Methods. 1997, 26 (1): 47-63. 10.1080/03610929708831901.View ArticleGoogle Scholar
- Kovalski SM, Vining GG, Montgomery DC, Borror CM: Modifying a central composite design to model the process mean and variance when there are hard-to-change factors. J Roy Statist Soc Ser C. 2006, 55 (5): 615-630. 10.1111/j.1467-9876.2006.00556.x.View ArticleGoogle Scholar
- Webb DF, Lucas JM: Blocking strategies for factorial experiments with hard-to-change factors. Proceedings of the Section on Quality and Productivity of the American Statistical Association. 2004, 2181-2188.Google Scholar
- Dodde R, Shih A, Advincula AP: A novel technique for demonstrating the real-time subsurface tissue thermal profile of two energized surgical instruments. Journal of Minimally Invasive Gynecology. 2009, 16: 599-603. 10.1016/j.jmig.2009.05.018.View ArticlePubMedGoogle Scholar
- Lim CBB, Goldin RD, Elson DS, Darzi A, Hanna GB: In vivo thermography during small bowel fusion using radiofrequency energy. Surg Endosc. 2010, 24: 2465-2474. 10.1007/s00464-010-0987-0.View ArticlePubMedGoogle Scholar
- Wallwiener CW, Rajab TK, Krämer B, Isaacson KB, Bruckner S, Wallwiener M: Quantifying electrosurgery-induced thermal effects and damage to human tissue: An exploratory study with the fallopian tube as a novel in-vivo in-situ model. Journal Minimally Invasive Gynecology. 2010, 17: 70-77. 10.1016/j.jmig.2009.09.007.View ArticleGoogle Scholar
- Koch C, Friedrich T, Matternich F, Tannapfel A, Reimann HP, Eichfeld U: Determination of temperature elevation in tissue during the application of the harmonic scalpel. Ultrasound in Med & Biol. 2003, 29 (2): 301-309. 10.1016/S0301-5629(02)00727-5.View ArticleGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2288/11/75/prepub

### Pre-publication history

## Copyright

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.