Skip to main content

Advertisement

Health technology assessment (HTA) of optoelectronic biosensors for oncology by analytic hierarchy process (AHP) and Likert scale

Article metrics

Abstract

Background

The multicriteria decision method (MCDM) aims to find conflicts among alternatives by comparing and evaluating them according to various criteria to reach the best compromise solution. The evaluation of a new health technology is extremely important in the health sciences field. The aim of this work is to evaluate a new health technology to assay thyroglobulin in patients with differentiated thyroid cancer to improve its service from an organizational point of view, by planning new and appropriate training activities, ensuring proper use of resources and satisfying the needs of different users.

Methods

The evaluation was performed using two methodologies: the analytic hierarchy process (AHP) and the Likert scale. The AHP is a multicriteria decision approach that assigns a weight to each evaluation criterion according to the decision maker’s pairwise comparisons of the criteria. The Likert scale is a psychometric scale employed to study the degree of user satisfaction by measuring opinions.

Results

Results show the need of particularly improving clinical efficiency, effectiveness, and return on sales (ROS) related to the technology; technological safety, human resources and other parameters do not need to be improved because of the high satisfaction results of the users.

Conclusions

The application of both methods provided the necessary information to improve the quality of the service, allowing the decision maker to identify the most valuable service features and to improve these to ensure user satisfaction and to identify possible service improvements.

Background

The analysis of the multicriteria decision method (MCDM) aims to identify conflicts among alternatives by comparing and evaluating them according to various criteria to reach the best compromise solution [1,2,3,4]. Today, many MCDM methods are in use [5,6,7]. Among these, the Analytic Hierarchy Process (AHP) is of considerable interest. The AHP, or hierarchical analysis, is a methodology, developed in the 1970s by Thomas L. Saaty [8], which evaluates a set of alternatives and creates a final version of the same problem by splitting the problem into many sub-problems for decision making [1, 9,10,11,12,13,14,15,16,17,18,19,20].

This method can handle a large number of different factors, which are often in conflict with each other, and it can also compare different alternatives in relation to a number of criteria [21,22,23,24,25,26,27,28,29].

Among the several application of AHP, this approach has been applied by Suner et al. [30, 31] to the management of rectal cancer. They constructed a sequential decision tree for the best treatment decision process, using priorities determined by the AHP method. Moreover, they were able to develop a web-based clinical decision support tool for physicians in the selection of potentially beneficial treatment options for patients with rectal cancer by combining the AHP, which determines the priority of criteria, and decision tree that formed using these priorities. More recently, Suner et al. [32] also applied AHP in the evaluation of infectious diseases to determine the best hand hygiene preference of the infectious diseases and clinical microbiology specialists to prevent transmission of microorganisms from one patient to another. They examined opinions of the specialists with two widely used multi-criteria decision analysis methods, the Multi-Attribute Utility Theory (MAUT) and the AHP, showing that both decision models indicate that rubbing the hands with alcohol-based antiseptic solution is the most favorable choice for specialists to prevent nosocomial infection.

Through the AHP methodology, a weight is assigned to each alternative, relying on the reviews provided by the purely qualitative decision maker, and each weight vector is placed in a final matrix that is used to sort the priority of each alternative. Several works discuss the use of AHP in solving health technology assessment problems [33, 34]. These studies demonstrate the ability of AHP to facilitate the understanding of the criteria and the priorities to successfully evaluate hospital technologies. Therefore, the AHP may be considered as a decision support tool for the Health Technology Assessment (HTA) projects [35,36,37,38].

This work focuses on a specific clinical application of the HTA. Indeed, HTA enables the analysis and assessment of health technologies, considering all medical-clinical, organizational, economic, social, legal and ethical implications, both directly and indirectly caused, and both short- and long-term implications [39, 40].

Many scientific papers [41, 42] focus on the innovative contribution of HTA [43,44,45,46] using the project management methodology to improve the quality of health services, in particular aiding the optimal allocation of biomedical systems, by evaluating all safety, ethical, legal and social, economic, technical and technological, and organizational parameters. If these requirements are met, hospitals can proceed with installing the system in the assessed area only; otherwise, the target area will not comply with the installation of the system, and then the assessment can be repeated for another area or to choose a different type of system.

In this work, the AHP approach is applied to a HTA [36, 43, 46,47,48,49] issue to evaluate an assessment that involves the use of optoelectronic systems in oncology (specifically, cancer of the thyroid). In addition, it offers the most suitable solution for its application since the choice of medical technology produces complex interactions among the variables involved [50]. Customer satisfaction is measured and evaluated not only by the AHP [51,52,53,54,55] but also using the Likert scale. The Likert scale is used to evaluate the degree of user satisfaction by measuring their opinions. This enables numerically quantifying the satisfaction of users in relation to the defined items, thereby allowing us to investigate the experts’ preferences about various defined “items”.

Methods

The methodological approach adopted to evaluate the perceived satisfaction is the questionnaire, which was administered to a sample of 80 users. Two different methods were implemented to analyse the data acquired by the questionnaire, the Likert scale and the AHP, which have been widely used in the literature in many application areas and in the health care sector [56,57,58].

The Likert scale is a psychometric scale widely used in many different research areas to evaluate questionnaires. Questionnaires assessed by the Likert scale formula usually a n-point scale (generally, from 4- to 7-point scale). Respondents are forced to choose in even-numbered scales because odd-numbered scales allow for indecision or neutrality. Questions designed using the Likert scale must either be in agreement or disagreement. This method has been found to be reliable at obtaining valid measurements of training effectiveness, reaction (satisfaction with conducted training), and overall training impact at work [59]. The Likert scale also has some drawbacks, including limitations due to the honesty of respondents in answering the questions, reproducibility and validity of the results [60].

AHP is a multicriteria decision making tool for organizing and analysing complex decisions based on mathematics and psychology. AHP incorporates group census from a questionnaire by comparing each element and geometric mean to arrive at a decisive solution.

AHP is used around the world in a wide variety of decision situations, in fields such as government, business, industry, healthcare and education, to plan, select best alternatives, allocate resources, resolve conflicts and optimize decisions [61]. Training programmes are efficiently evaluated by the AHP as this method provides the highest value and ranks the content and trainer based on logistics [62].

The top-level criteria and indicators of each criterion highlighted by HTA are listed in Fig. 1.

Fig. 1
figure1

Top-level criteria and indicators of each criterion

This approach provides a hierarchical decomposition of the problem, first into goals, criteria, sub-criteria and alternatives, and then calculates the global sub-criterion weight.

It is important to consider all aspects related to the definition of the minimum technical quality of an evaluated technology, as these aspects can subsequently result in the minimum level guarantee of the effectiveness and efficiency of the process under study.

To assess the technical aspects, the following indicators/sub-criteria are considered:

  • Technological efficiency is a technological performance level indicator that measures the amount of useful work produced, depending on time and available resources. It represents the ability to act with maximum efficiency and minimal waste of resources. The sum, at each time step of the executed exams, is the indicator of the sub-criterion technological efficiency; the higher this indicator, the greater the efficiency of the technology:

    $$ {\sum}_{timestep} executed\ exams\kern0.5em $$
    (1)
  • Reliability aims to describe and measure the operation of the devices or equipment. For any given system, this measure quantifies the degree of confidence we have in the good functioning of the system, i.e., the fulfilment of the purposes for which the system itself is designed and built. An important indicator of the reliability of the technology, especially for a device, is the number of faults that the technology undergoes per unit of time. The sum at each timestep of queued examinations for faults is the indicator of higher reliability; the higher this indicator sub-criterion, the higher the number of tests for faults, so the technology is less reliable:

    $$ {\sum}_{timestep} queued\ exams\ for\ faults $$
    (2)
  • Safety technology of the equipment depends on strict compliance with the requirements of the equipment, in terms of both design and construction, during installation, use and maintenance over time. To assess this indicator, another system, separate from the one above, is constructed. The indicator includes a number of variables and constants: visual evaluation, compliance with safety standards, the availability of user manuals, the presence of alarm signals and control systems and the status of the use of the rooms and facilities. The sum of the values of these variables (each of which will have a value from 1 to 10) constitutes the indicator of the sub-criterion “Security Technology”. The higher the level of security associated with the technology, the greater the indicator. By normalizing these three vectors at each time step, the local weight of the alternatives relative to the three indicators is obtained.

The following indicators/sub-criteria are considered in the assessment of the organizational aspect:

  • the procedural complexity is expressed as:

    $$ procedural\ Complexity=\frac{total\ exams\ average\ executed\ daily}{total\ exams\ to\ be\ performed\ daily} $$
    (3)

This relationship, with appropriate adjustments (which can vary from days to hours), is the indicator of the procedural complexity, and this sub-criterion indicates the hours spent to run the exams for each alternative. The greater the number of hours, the greater the procedural complexity associated with the technology.

  • the human resources.

The number of staff needed and available to run sampling and analysis is the indicator of human resources, and the greater the sub-criterion of this indicator, the greater the necessary staff for the technology:

$$ human\ resources= medical\ specialists+ biologists $$
(4)

By normalizing these two vectors at each timestep, the local weight of the alternatives for the two indicators is obtained.

The following indicators/sub-criteria are considered in assessing the economic aspects:

  • Investments are considered mainly for the purchase cost of the technology. The relationship between investment and the purchase cost of the technology is the indicator of the sub-criterion investments. The higher this indicator, the smaller the investment associated with the technology:

    $$ investiments=\frac{investiment}{purchase\ cost\ of\ the\ system} $$
    (5)
  • The profit is the profit or loss from business management over a period of time. The sum of the net profits at each timestep is a sub-criterion for the usefulness indicator: the higher this indicator, the greater the gains associated with the technology:

    $$ profit={\sum}_{timestep} net profit $$
    (6)
  • The ROS, or return on sales, measures the sales profitability and lucrative revenue streams typical of the capacity of the enterprise. The ratio of net operating margin (MON) and net revenues, obtained by multiplying the tests processed with drug revenues, constitute the sub-criterion for ROS. The higher this indicator, the more profitable the product (in this case medical examination):

    $$ ROS=\frac{MON}{net\ revenues} $$
    (7)

By normalizing these three vectors at each time step, the local weight of the alternatives are obtained for the three indicators.

We consider any provision that can enclose two primary qualities, such as effectiveness and efficiency, as appropriate, and at the same time, we consider both acceptable by those who receive and dispense them [63].

Therefore, to assess the clinical aspects, the following indicators/sub are considered:

  • Clinical efficiency [64]. The difference between the sum of the examinations that require no repetitions (between sampling and analysis) and the sum of examinations requiring repetitions (between sampling and analysis) at each timestep constitutes the sub-criterion: the greater the clinical efficiency indicator, the greater the efficiency of the technology:

    $$ Clinical\ efficiency=\Big[\left(\sum tests\ that\ require\ no\ additional\ withdrawals\right)+\left(\sum tests\ that\ do\ no t\ require\ further\ investigation\right)-\left(\sum exams\ that\ require\ further\ withdrawals\right) $$
    (8)
  • Effectiveness. The ratio of the sum of effective tests and the sum of incoming requests is the sub-criterion for the effectiveness indicator: the greater this efficiency indicator, the greater the degree of effectiveness of the technology:

    $$ \mathrm{Effectiveness}=\frac{\sum_{\mathrm{timestep}}\mathrm{effective}\ \mathrm{tests}\ }{\sum_{\mathrm{timestep}}\mathrm{incoming}\ \mathrm{requests}} $$
    (9)
  • Side effects. The sum of the examinations that have side effects and effective examinations with error constitutes the side effects sub-criterion: the greater the side effects indicator, the more side effects the technology has:

    $$ Side\ effects=\sum \limits_{timestep} examinations\ that\ have\ side\ effects+\sum \limits_{timestep} effective\ examinations\ with\ error $$
    (10)

The local weight of the alternatives relative to these three indicators is obtained by normalizing these three vectors at each timestep.

The following indicators/sub-criteria are considered for the purposes of social, ethical and legal aspects.

  • the respect for social aspects refers to following the principle of justice, which provides for the equality of treatments based on clinical conditions, i.e., the possibility that the technology at issue has equal access for all patients, especially in terms of the costs it entails. The sum of examinations is the indicator of the sub-criterion of compliance with social aspects; the higher this indicator, the more the testing, and hence the lower economic unavailability due to missed examinations. This means that the technology is more respectful of social aspects.

    $$ \mathrm{R} espect\ for\ social\ aspects={\sum}_{timestep} Examinations $$
    (11)
  • The respect for ethical principles refers to the moral significance of biomedical and biotechnological practices at the borders of right and not right concerning who you can hire, the rights of the person, the decisions entitled to the people, the decision criteria and values at stake. Additionally, this criterion includes a sense of current concepts such as respect for human life and respect for the dignity of the person. The sum of examinations not performed for ethical reasons, related to both the surgeon and the patient, is the indicator of the sub-criterion for compliance with the ethical principles:

    $$ \mathrm{Respect}\ \mathrm{for}\ \mathrm{ethical}\ \mathrm{principles}={\sum}_{timestep}\mathrm{examinations}\ \mathrm{not}\ \mathrm{performed}\ \mathrm{for}\ \mathrm{ethical}\ \mathrm{reasons}\ \mathrm{related}\ \mathrm{to}\ \mathrm{the}\ \mathrm{surgeon}+{\sum}_{timestep}\mathrm{examinations}\ \mathrm{not}\ \mathrm{performed}\ \mathrm{for}\ \mathrm{ethical}\ \mathrm{reasons}\ \mathrm{related}\ \mathrm{to}\ \mathrm{the}\ \mathrm{patient} $$
    (12)
  • The respect for legal issues implies that the field of medical equipment and technology must be regulated through legislation, the so-called European directives, technical standards issued by the relevant standardization bodies. To calculate this indicator, variables were given a score between 1 and 10, where 1 is the best value, and 10 the worst. The indicator was calculated by adding up all the scores. The smaller the indicator, the higher respect for the legal issues. The sum of these variable provides the respect for legal issues indicator:

Thus,

legal = noise + electromagnetic pollution + respect the value of life +.

health risk + compliance with mandatory standards + lawsuits.

By standardizing the three vectors obtained at each timestep, the local weight of the alternatives is obtained and compared for the three indicators.

Data processing using the AHP (analytic hierarchy process) and confirming the consistency of the matrices

The scores from the comparisons in the AHP are used to construct a matrix of pairwise comparisons. To aggregate the individual judgements, the weighted geometric mean method (WGMM) is used according to the following formula:

$$ \prod \limits_{k=1}^N{\left({a}_k\right)}^{\beta_k} $$
(13)

by applying the constant weight 1/N, the equation is rewritten:

$$ \prod \limits_{k=1}^N{\left({a}_k\right)}^{\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$N$}\right.}=\sqrt[N]{\prod \limits_{k=1}^N{a}_k} $$
(14)

and by applying a logarithm operation, the computation is simplified as a sum:

$$ {10}^{\frac{1}{N}\sum \limits_{\mathrm{k}=1}^N\log {a}_k} $$
(15)

Obviously, as the geometric mean returns a non-integer value, we round the obtained value to the nearest integer.

From the average values thus calculated and the properties of the matrix of pairwise comparisons, a matrix inherent to the three main categories can be constructed. A matrix was also constructed for all other pairwise comparisons (matrices are reported in Table 3).

Now we need to analyse the consistency, which is accomplished by calculating the ratio of texture CR = (CI/RI) where:

$$ \mathrm{CI}=\frac{\uplambda_{\mathrm{max}}-\mathrm{n}}{\mathrm{n}-1} $$
(16)

and RI is a random index that assumes a certain value depending on the order of the array. λmax is the largest eigenvalue, n is the matrix dimension, and RI is the Saaty random consistency index.

According to Saaty (2008) [65], if the consistency ratio exceeds 0.1, the set of judgements may be too inconsistent to be reliable. In practice, CRs of more than 0.1 sometimes are accepted. If CR equals 0, then the judgements are perfectly consistent [66].

All data elaboration and analysis fo the AHP has been carried out by using Matlab (Mathwork).

Case study and implementation of the AHP

The present case study assessed two new health technologies used in oncology (opto system and fluidic system), as displayed in Fig. 2. These two systems combine fibre optics, electronics and fluid dynamics to determine the levels of thyroglobulin (Tg) in needle-aspiration samples taken from the thyroid. Several treatment options are available for the management of thyroid cancer. Tg is a large glycoprotein composed of two identical polypeptide chains that form a molecule of approximately 660 kDa and localized in the follicular thyroid colloid. Currently, Tg is assayed in suspicious lymph aspirates, the results of which are then used in a cytological analysis.

Fig. 2
figure2

Representation of optofluidic system. Architecture of the system

To specify the concentration of Tg (with a sensitivity limit above 100 ng/ml), biosensors with fibre-optic transducers should be characterized by:

  • high sensitivity, to reveal even small concentrations of tumour markers in biological liquids;

  • high selectivity, to prevent the influence of other substances in the lymph node that are irrelevant to the analysis of sick nodules;

Two different types of fibre-optic transducers are considered to meet the sensitivity requirement:

  1. (1)

    the first transducer, named a lab on fibre, (LOF), integrates nanoscale metallic patterning on the tip of optical fibres. The resulting structures are plasmonic crystals capable of trapping light at a specific wavelength of resonance (Fig. 3);

  2. (2)

    the second configuration, called LPG (long period gratings), involves the use of long-wheelbase patterns inscribed within the core of the fibre, which is covered with nanoscale layers of functional polymeric materials (Fig. 3 b).

Fig. 3
figure3

Schematic representation of the two different types of sensors: with integration and patterning on the tip of the fibre, LOF (a), and based on the long-wheelbase patterns coated with nanoscale layers of functional polymer materials, LPG (b)

However, the surface must be appropriately engineered to meet the selectivity, so that only the target molecule (and not other molecules present in the biological solution) is immobilized and detected by the sensor.

An optofluidic system integrates fibre-optic sensors with fluidic circuits that can manage the flow of biological solutions while ensuring the correct operation of the sensor. It consists of two major subsystems:

  • the “opto” subsystem is responsible for analysing biological fluids sampled by needle aspiration and indicating the concentration of tumour markers contained therein;

  • the “fluidic” subsystem manages the sampled body fluids and ensures proper performance of the analysis.

Both are managed by the same software with a user-friendly graphical interface that can be used even by unskilled personnel.

Results and discussion

Data processing using AHP (analytic hierarchy process)

The judgements used to construct the matrices of pairwise comparisons for the AHP were provided in collaboration with the National Research Council staff working in the field of biosensing systems.

Three options are considered, including decisions for each criterion. The first option is cytological analysis, which represents AS IS, i.e., the current method of assessing Tg levels. The other two options are the two biosensors that should be used to determine Tg, the one with the sensor on the tip of a fibre (LOF) and the one with the fibre along the surface (LPG), which represent TO BE. The elements of the array represent the relative importance of one criterion over another, using the scale proposed by Saaty and reported in Table 1.

Table 1 Saaty’s pairwise comparison scale

On the basis of the Saaty’s scale, the following Matrix of pairwise comparisons has been built (Table 2). Each value reported in the Table 2 represents the geometric logarithmic mean of the degree of preference of the 80 users according to the Saaty’s fundamental scale.

Table 2 Matrix of pairwise comparisons with respect to policy goals

The matrix in Table 2 is positive, mutual and consists of finite elements. For a matrix of 5 × 5 order, such as that in Table 2, RI = 1.12. The maximum eigenvalue associated with the matrix is calculated:

$$ {\uplambda}_{\mathrm{max}}=5.3185\ \mathrm{n}=5 $$
(17)
$$ C\mathrm{I}=\frac{5.3185-5}{4}=0.0796 $$
(18)
$$ C\mathrm{R}=\frac{0.0796}{1.12}=0.0711=7.11\%<10\% $$
(19)

Therefore, the matrix can be considered consistent.

The eigenvector associated with the maximum eigenvalue can now be calculated:

$$ {\mathrm{w}}_1=\left[0.0703;0.1403;0.8582;0.4194;0.2509\right] $$
(20)

The normalized vector is:

$$ {\mathrm{w}}_{1\mathrm{N}}=\left[0.0404;0.0807;0.4935;0.2412;0.1443\right] $$
(21)

which is the local weight vector with respect to the policy goals.

Finally, by aggregating all the global weights for each of the three alternatives, we obtain the final sorting vector, which indicates the priorities of the alternatives, i.e., the result of the decision problem.

$$ \mathrm{Cytological}\ \mathrm{Analysis}:0.0114+0.00042+0.00960+0.0105+0.00512+0.0228+0.0270+0.0080+0.0114++0.0258+0.01632+0.0077+0.00954+0.1774=0.334414; $$
(22)
$$ \mathrm{Biosensor}\ \mathrm{LOF}:0.00320+0.00320+0.00162+0.0288+0.00149+0.0256+0.0961+0.0961+0.0450+0.0046+0.00544+0.0018+0.0477=0.36065; $$
(23)
$$ \mathrm{Biosensor}\ \mathrm{LPG}:0.00270+0.00270+0.00376+0.0288+0.00149+0.0256+0.0811+0.1139+0.0041+0.00544+0.0022+0.0477+0.0503+0.0497=0.419; $$
(24)

The results of the static AHP revealed a slight preference for the biosensor LPG, i.e., the one with the fibre along the surface.

Once the global weights of the sub-criteria with respect to each parent policy are obtained, each is incorporated as a variable into the model to which they belong, such as the model of the technical aspects, which includes global sub-criterion weights for technological efficiency, reliability and safety technology. These global weights are then multiplied by the outputs of each model that is local to the weights of the three alternatives in each sub-criterion.

In accordance with the AHP model, first, the target (goal) of the survey is defined, i.e., to evaluate a new health technology to assay Tg in patients with differentiated thyroid cancer. Five main dimensions corresponding with the sub-dimensions were identified, which together compose the so-called dominance hierarchy. In particular, the following main dimensions (criteria) were chosen: technical, organizational, economical, clinical, and social, legal, and ethical aspects. Figure 1 (shown above) reports the overall hierarchy of both the criteria and sub-criteria.

Then, a questionnaire composed of two parts was developed. The first part of the questionnaire (Table 3) included 13 items, each semantically connected to dimensions of the quality of a service identified in the dominant hierarchy in Fig. 1. The respondents were asked to express their degree of agreement/disagreement with each statement by choosing one of the five answers provided by the Likert scale: strongly agree, agree, uncertain, disagree, and strongly disagree. The second part of the questionnaire (Table 4) related to the AHP and consisted of 13 comparisons, divided into 5 batteries, each containing some comparisons; the first battery regarded the pairwise comparison between the 5 main dimensions; each of them included a battery of pairwise comparisons between the three corresponding sub-dimensions.

Table 3 First part of the questionnaire based on the Likert methodology
Table 4 Second part of the questionnaire based on the Likert methodology

For each comparison, the respondents were asked which of the two parameters, in his own experience, is more important in determining the quality of the service, according to the Saaty’s scale [9]. The questionnaire was administered to a group of 80 experts in technologies for oncological treatment care at the National Hospital A.O.R.N “A. Cardarelli” of Naples.

Processing the questionnaire data based on the Likert scale

The first step was to evaluate the percentage of experts who provided a specific response based on their satisfaction with the quality of service provided. The frequency percentages of the responses are reported in Fig. 4. The obtained data revealed that the experts expressed relatively high satisfaction percentages (rates) for each item, and in particular those items relating to the effectiveness, the side effects and the respect for social aspects.

Fig. 4
figure4

Graph of processed Likert (Subcategories) data

Calculation of local and overall priorities

The eigenvalue method, as proposed by Saaty [9], was used to evaluate the local priorities. In particular, the Global weights of the sub-criteria with respect to their parent criterion are calculated through the AHP by building the matrix of pairwise comparison (Table 2) and then calculating the maximum eigenvalue of the matrix and of the related eigenvector which, once normalized, will represent the vector of the priority weights of the element of the hierarchy that is being considered.

Therefore, for the main categories, we have the following order of alternative variables:

  • Technical Aspects weight: 0.0404

  • Organizational Aspects weight: 0.08065

  • Clinical Aspects weight: 0.2413

  • Economic Aspects weight: 0.4933

  • Social, Legal, and Ethical Aspects weight: 0.1439

The next step was to calculate the overall weight of each sub-criterion by multiplying the sub-criterion priorities by the priority of their parent criterion [65]. For example, the weight of the sub-criterion “Technical Aspects” is 0.0404.

The matrix and weight analysis of the main categories and sub-categories are shown in Table 5.

Table 5 Analysis of the matrix and weights of the main categories and sub-categories

From these calculated values, which represent the absolute weights of each sub-criterion, the “hierarchy of needs” of the users could be defined (Fig. 5), the top priority could be deduced from the histogram. Particularly high importance was attributed to ROS, usefulness, respect for legal issues, effectiveness and clinical efficiency. In contrast, little importance was attributed to the technological safety, the respect for ethical principles, human resources and reliability.

Fig. 5
figure5

Graph of processed (AHP hierarchy of needs) data

Comparison and aggregation of the obtained data

The next step was to compare the obtained results and to combine the information produced these two techniques to create a practical tool that can facilitate the decision-making process, thus improve its perceived quality. The parameters that users evaluated as unsatisfactory are characterized by a low value from the Likert method and high values from the AHP.

The hierarchy of needs reveals that technological safety, human resources, and other parameters, such as side effects, usefulness and the respect for social issues, were highly valued with regard to customer satisfaction. Therefore, the experts’ perceptions regarding quality were higher when the degree of user satisfaction was evaluated than when the same item was evaluated by the Likert scale.

Low satisfaction with the clinical efficiency, the effectiveness, the usefulness and the ROS was observed. Therefore, satisfaction with these parameters must be improved to increase the quality of service.

This same reasoning can be extended to all the items, so users’ needs can be understood by applying the AHP, and the users’ satisfaction can be understood by applying the Likert method. In fact, the AHP results confirm a slight preference for the LPG biosensor.

Conclusions

A new health technology for the assay of Tg in patients with differentiated thyroid cancer was evaluated through two methodologies: the Likert scale and the AHP, to improve its service from an organizational point of view. The evaluation of a health technology is a clear example of a multicriteria decision problem that has a high multitude of interconnected variables. In particular, this choice involves a particularly serious application, and it requires evaluating either a traditional procedure such as cytological analysis or one of two optoelectronic systems, which have completely different characteristics and organization from the first.

In conclusion, this method can be used to increase the quality of a service while avoiding wasting time and costs. Instead, these resources can be invested in improving those service criteria for which users’ satisfaction is low, as revealed with the Likert method, and those that the users assign high importance, as revealed by the AHP.

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Abbreviations

AHP:

Analytic Hierarchy Process

CI:

Consistency index

CR:

Consistency ratio

RI:

Random consistency index

References

  1. 1.

    Xu H, Hipel K, Kilgour D, Chen Y. Combining strength and uncertainty for preferences in the graph model for conflict resolution with multiple decision makers. Theor Decis. 2009;69(4):497–521.

  2. 2.

    Pavan M, Todeschini R. Multi-criteria Decision Making Methods. Comprehensive Chemometrics. S Brown, B Walczak & R Tauler. 2009;1:591-629.

  3. 3.

    Bouyssou D. Building criteria: a prerequisite for MCDA. Readings in multiple criteria decision aid. Berlin, Heidelberg: Springer; 1990. p. 58-80.

  4. 4.

    Hwang C, Yoon K. Methods for multiple attribute decision making. Multiple attribute decision making. Berlin, Heidelberg: Springer; 1981. p. 58-191.

  5. 5.

    Stewart T. A critical survey on the status of multiple criteria decision making theory and practice. OMEGA Int J Manag Sci. 1992;20:569–86.

  6. 6.

    Hwang C, Paidy S, Yoon K, Masud A. Hwang, C.L., Paidy, S.R., Yoon, K., Masud, a.S.M. (1980), mathematical programming with multiple objectives: a tutorial, in “ computers and operations research ”, Vol. 7, pp. 5–31; 1980.

  7. 7.

    Goicoechea A, Hansen DR, Duckstein L. Multiobjective decision analysis with engineering and business applications. New York: Wiley; 1982.

  8. 8.

    Saaty TL. Decision Making for Leaders. Pittsburgh: RWS Publications; 1999.

  9. 9.

    Saaty TL. Multicriteria decision making - the analytic hierarchy process- planning, priority, setting, resource allocation. Pittsburgh: RWS Publications; 1988.

  10. 10.

    Saaty TL. Fundamentals of decision making and priority theory with the analytic hierarchy process, Analytic hierarchy process series, vol. 6; 1994b.

  11. 11.

    Saaty TL. Time dependent decision-making; dynamic priorities in AHP/ANP: generalizing from points to functions and from to complex variables. Math Comput Model. 2007;46:860–91.

  12. 12.

    Fiala P. An ANP/DNP analysis of economic elements in today’s world network economy. J Syst Sci Syst Eng. 2006;15:131–40.

  13. 13.

    Raharjo H, Xie M, Brombacher AC. On modeling dynamic priorities in the analytic hierarchy process using compositional data analysis. Eur J Oper Res. 2009;194:834–46.

  14. 14.

    Gao Y, X D, Zhang M. The application of dynamic priority of AHP on operation risk assessment of metro, Modeling Risk Management in Sustainable Construction; 2011. p. 65–72.

  15. 15.

    Shubo X, Bao L. The new dynamic priorities model and an analysis of China’s energy strategy for the future, Toward Interactive and Intelligent Decision Support Systems, vol. 2; 1987. p. 249–56.

  16. 16.

    González-Prida V, Viveros P, Barberá L, Crespo MA. Dynamic analytic hierachy process: AHP method adapted to a changing environment. J Manuf Technol Manag. 2014;25(4):457–75.

  17. 17.

    Benítez J, Delgado-Galván X, Izquierdo J, Pérez-García R. An approach to AHP decision in a dynamic context. Decis Support Syst. 2012;53(3):499–506.

  18. 18.

    Benítez J, Delgado-Galván X, Izquierdo J, Pérez-García R. Achieving matrix consistency in AHP through linearization. Appl Math Model. 2011;35(9):4449–57.

  19. 19.

    Ishizaka A, Labib A. Review of the main developments in the analytic hierarchy process. Expert Sys Appl. 2011;38(11):14336–45.

  20. 20.

    Hogarth R. Judgement and choice. 2nd ed; 1987. p. 177–84.

  21. 21.

    Berrah L, Mauris G, Montmain J. Monitoring the improvement of an overall industrial performance based on a Choquet integral aggregation. Omega. 2008;36(3):340–51.

  22. 22.

    Tervonen T, Hakonen H, Lahdelma R. Elevator planning with stochastic multicriteria acceptability analysis. Omega. 2008;36(3):352–62.

  23. 23.

    Gomes C, Nunes KRA, Xavier L, Cardoso R, Valle R. Multicriteria decision making applied to waste recycling in Brazil. Omega. 2008;36(3):395–404.

  24. 24.

    Bana e Costa A, Oliveira C, Vieira V. Prioritization of bridges and tunnels in earthquake risk mitigation using multicriteria decision analysis: application to Lisbon. Omega. 2008;36(3):442–50.

  25. 25.

    Rabelo L, Eskandari H, Shalan T, Helal M. Supporting simulation-based decision making with the use of AHP analysis, Proceedings of the 2005 winter simulation conference; 2005. p. 2042–51.

  26. 26.

    McNaught K. Influences and connections between system dynamics and decision analysis. In: Proceedings of the 21st International Conference of the system dynamics society, 2003.

  27. 27.

    Kivijärvi H, Tuominen M. Integrating AHP and dynamic simulation:experiences,conceptualizations and business experiments. In: ISAHP; 1999.

  28. 28.

    Liu H, Yeh Y, Huang J. Correlated analytic hierarchy process. Math Probl Eng. 2014;2014(961714):1-7.

  29. 29.

    Zhang Z, Lu W, Zhao Y, Song W. Development tendency analysis and evaluation of the water ecological carrying capacity in the Siping area of Jilin Province in China based on system dynamics and analytic hierarchy process. Ecol Model. 2014;275:9–21.

  30. 30.

    Suner A, ÇelikoğLu CC, Dicle O, Sökmen S. Sequential decision tree using the analytic hierarchy process for decision support in rectal cancer. Artif Intell Med. 2012;56(1):59–68.

  31. 31.

    Suner A, Karakülah G, Dicle O, Sökmen S, Çelikoğlu CC. corRECTreatment: a web-based decision support tool for rectal cancer treatment that uses the analytic hierarchy process and decision tree. Appl Clin Inform. 2015;6(1):56–74.

  32. 32.

    Suner A, Oruc OE, Buke C, Ozkaya HD, Kitapcioglu G. Evaluation of infectious diseases and clinical microbiology specialists’ preferences for hand hygiene: analysis using the multi-attribute utility theory and the analytic hierarchy process methods. BMC Med Inform Decis Mak. 2017;17:129.

  33. 33.

    Figueira J, Greco S, Ehrgott M. Multiple criteria decision analysis: state of the art surveys, vol. 78. New York: Springer Science & Business Media, Springer-Verlag; 2005. p. 73-112.

  34. 34.

    Banta D, Behney C, Andrulis D. Assessing medical technologies. Bull N Y Acad Med. 1978;54(1):113.

  35. 35.

    Danner M, Hummel J, Volz F, van Manen J, Wiegard B, Dintsios C, Bastian H, Gerber A, Ijzerman M. Integrating patients’ views into health technology assessment: analytic hierarchy process (AHP) as a method to elicit patient preferences. Int J Technol Assess Health Care. 2011;27(4):369–75.

  36. 36.

    Improta G, Fratini A, Triassi M. Health technology assessment: an essential approach to guide clinical governance choices on risk management. In: Risk Management for the Future - Theory and Cases; 2012.

  37. 37.

    Litsios S, Gladstone RJ. Mathematical models in health-planning research. Int J Syst Scie. 1972;3(3):313-23.

  38. 38.

    Improta G, Russo MA, Triassi M, Converso G, Murino T, Santillo LC. Use of the AHP methodology in system dynamics: modelling and simulation for health technology assessments to determine the correct prosthesis choice for hernia diseases. Math Biosci. 2018;299:19–27.

  39. 39.

    Battista, Hodge. The evolving paradigm of health technology assessment: reflections for the millennium. CMAJ. 1999;160(10):1464–7.

  40. 40.

    Favaretti, Cicchetti, Guarrera, Marchetti and Ricciardi, “Health technology assessment in Italy.,” International journal Technolical assessment health care, 2009.

  41. 41.

    Converso, D. Giacomo, Murino and Rea, “A System Dynamics Model for Bed Management Strategy in Health Care Units,” SOMET, 2015.

  42. 42.

    Converso, Ascione, Nardo D, Natale. An optimization model in health care processes by lean energy approach. In: International Society of Science and Applied Technologies; 2014.

  43. 43.

    G. Converso, G. Improta and M. Mignano, “A simulation approach for implementing of agile production logic for a hospital emergency unit,” Intelligent software methodologies, tools and techniques, 2015.

  44. 44.

    Improta G, Simone T, Bracale M. HTA (health Technol-ogy assessment): a means to reach governance goals and to guidehealth politics on the topic of clinical risk management. In: WorldCongress on Medical Physics and Biomedical Engineering; 2009.

  45. 45.

    Improta G. L’HTA (health technology assessment) per il management delle apparecchiature biomedicali di un’Azienda Ospedaliera: riallocazione, donazione ai paesi in via di sviluppo, ideazione e progettazione di una comunità virtuale; 2010.

  46. 46.

    Converso G, De Carlini R, Santillo L, Improta G. Project Management implementation for healthcare activities organization. Adv Comput Sci. 2012;8:436-43.

  47. 47.

    G. Improta, T. Simone and M. Bracale, “HTA (health technology assessment): a means to reach governance goals and to guide health politics on the topic of clinical risk management,” World Congress on Medical Physics and Biomedical Engineering, 2009.

  48. 48.

    G. Improta, “L’HTA (Health Technology Assessment) per il management delle apparecchiature biomedicali di un’Azienda Ospedaliera: riallocazione, donazione ai Paesi in via di sviluppo, ideazione e progettazione di una comunità virtuale,” Università degli Studi di Napoli Federico II,, 2010.

  49. 49.

    Improta G, Triass M, Guizz G, Santillo LC, Revetria R, Catania A, Cassettar L. An innovative contribution to health technology assessment. In: Modern Advances in Intelligent Systems and Tools; 2012.

  50. 50.

    P. Melillo, A. Delle Donne, G. Improta, S. Cozzolino and S. Bracale, “Assessment of patient satisfaction using an AHP model: an application to a service of pharmaceutical distribution,” Proceedings of the international symposium on the analytic hierarchy process, 2011.

  51. 51.

    Li A-T, Lin J-W. Constructing core competency indicators for clinical teachers in Taiwan: a qualitative analysis and an analytic hierarchy process. BMC Med Educ. 2014;14(1):1.

  52. 52.

    Schmidt K, Aumann I, Hollander I, Damm K, von der Schulenburg J-M. Applying the analytic hierarchy process in healthcare research: a systematic literature review and evaluation of reporting. BMC Med Inform Decis Mak. 2015;15(1):1.

  53. 53.

    Singh S, Dolan J, Centor R. Optimal management of adults with pharyngitis–a multi-criteria decision analysis. BMC Med Inform Decis Mak. 2006;6(1):1.

  54. 54.

    Lambooij M, MJ H. Differentiating innovation priorities among stakeholder in hospital care. BMC Med Inform Decis Mak. 2013;13(1):1.

  55. 55.

    Pecchia L, Martin J, Ragozzino A, Vanzanella C, Scognamiglio A, Mirarchi L. User needs elicitation via analytic hierarchy process (AHP). A case study on a computed tomography (CT) scanner. BMC Med Inform Decis Mak. 2013;13(1):1.

  56. 56.

    Aller M, Vargas I, Waibel S, Coderch J, Sánchez-Pérez I, Colomés L. A comprehensive analysis of patients’ perceptions of continuity of care and their associated factors. Int J Qual Health Care. 2013;25(3):291-9.

  57. 57.

    Bruijn-Geraets D, Daisy P, Eijk-Hustings V, Yvonne J, Vrijhoef H. Evaluating newly acquired authority of nurse practitioners and physician assistants for reserved medical procedures in the Netherlands: a study protocol. J Adv Nurs. 2014;70(11):2673–82.

  58. 58.

    Kersnik J. An evaluation of patient satisfaction with family practice care in Slovenia. Int J Qual Health Care. 2000;12(2):143–7.

  59. 59.

    Abbad G, Borges-Andrade J, Henriques Sallorenzo L. Self-assessment of training impact at work: validation of a measurement scale. Int J Psychol. 2004;38:277–84.

  60. 60.

    Bertram D. Likert Scales… are the meaning of life: CPSC 681–Topic Report. 2007. Available from: http://poincare.matf.bg.ac.rs/~kristina//topic-dane-likert.pdf. Accessed 15 June 2019.

  61. 61.

    Vaidya O, Kumar S. Analytic hierarchy process: an overview of applications. Eur J Oper Res. 2006;169:1–29.

  62. 62.

    Agha S. Evaluating and benchmarking non-governmental training program: an analytic hierarchy approach. Jordan J Mech Indust Eng. 2008;2:77–84.

  63. 63.

    Tonelli S. La qualità nelle organizzazioni sanitarie. Appunti per formazione e consulenza; 2007.

  64. 64.

    Summerfelt W, Meltzer H. Efficacy vs effectiveness in psychiatric research. Psychiatr Serv. 1998;49(6):834.

  65. 65.

    Saaty TL. Decision making with the analytic hierarchy process. Int J Serv Sci. 2008;1(1):83-98.

  66. 66.

    Sharma S, Pratap R. A case study of risks optimization using AHP method. Int J Sci Res Publ. 2013;3(10):1–6.

Download references

Acknowledgements

Not applicable.

Funding

Not applicable.

Author information

GI analysed data and drafted the manuscript. He also helped in the design of the study and critically revised the work for important intellectual content. MAR and AP contributed to the collection and interpretation of data. They were involved in drafting the manuscript. MT participated in the conception and design of the study and gave final approval of the version to be published. All authors read and approved the final manuscript and guarantee its accuracy and integrity.

Correspondence to Giovanni Improta.

Ethics declarations

Ethics approval and consent to participate

This study was judged exempt from ethics approval by the Ethical Committee of the University “Federico II” of Naples. In compliance with Italian legislation, ethics approval was judged not necessary for this study because none of the following procedures were performed:

  • interviews with patients or children

  • experimental tests

  • animal experiments

  • clinical trials in humans

  • work with animal or human tissues.

The questionnaires administered to the experts participating in the study were anonymous, and no personal information attributable to the subjects investigated was collected.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Improta, G., Perrone, A., Russo, M.A. et al. Health technology assessment (HTA) of optoelectronic biosensors for oncology by analytic hierarchy process (AHP) and Likert scale. BMC Med Res Methodol 19, 140 (2019) doi:10.1186/s12874-019-0775-z

Download citation

Keywords

  • Health technology assessment
  • AHP methodology
  • Multicriteria decision problems
  • Likert scale