Skip to main content

MedMinas project: design and use of mixed methods in the evaluation of pharmaceutical services in primary health care in Minas Gerais, Brazil

Abstract

Background

The main purposes of primary care-based pharmaceutical services (PHCPS) in Brazil are to provide free access to medicines and pharmaceutical care to patients. Several obstacles hinder achieving their goals; thus, MedMinas Project aimed to evaluate the PHCPS, the supply system, and the use of medicines. This paper reflects on our experience designing, planning, and conducting the project, describing the issues yielded in the field and lessons learned.

Methods

This work consists of a mixed-methods study conducted in Minas Gerais, Southeastern Brazil. We adopted the principles of Rapid Evaluation Methods, employing a multistage stratified sampling for the quantitative and a purposeful sampling for the qualitative components, respectively, and a documentary research. Data sources included individuals (patients and professionals), prescriptions, dispensed medicines, and policy documents collected between April and October 2019. The quantitative data described in this paper were analysed by descriptive statistics and the qualitative by Thematic Content Analysis.

Results

A total of 26 municipalities varying from 37,784 to 409,341 inhabitants were included. The field team spent, on average, 16 days in each location. We interviewed 1019 respondents, of which 127 were professionals and 892 patients. The participation rate varied from 92 to 100%, depending on the respondent subgroup. Most interviews lasted between 45 min and one hour. Fieldwork challenges included participants’ enrolment, field team, interview processes, and project budget. The participants provided positive feedback and five main themes emerged from the interview experience (self-awareness, sense of gratitude, research value, access to findings, and benefits of the research). Additionally, we collected copies of 1072 documents and 2070 pieces of data from prescriptions filled and medicines dispensed at the PCP.

Conclusion

We demonstrated the viability of conducting the MedMinas Project in an extensive geographic area within effective time frames that provided meaningful, high-quality data from multiple actors. The methods and lessons learned are valuable for researchers across various disciplines in similar urban settings in Brazil and other countries of low- and middle-income (LMIC).

Peer Review reports

Background

Strengthening primary health care has been a priority for the Brazilian government in the designing and implementation of policies and funding [1, 2]. Part of the country’s strategic policies refer to the Primary care-based pharmaceutical services – PHCPS program, launched in 1998 as an essential component of the National Medicines Policy [3, 4].

The PHCPS programme is part of the National Healthcare System in Brazil (Sistema Único de Saúde—SUS), tax-funded and universally accessible to all Brazilian citizens, free of charge. The Ministry of Health and States Health Secretariats have roles in coordinating and regulating policies and actions, and, as the health system is decentralized, program delivery is under the responsibility of the Brazilian municipalities (n = 5570). The local program network includes the Municipal Health Secretariats and public community pharmacies (PCP). PCP cover specific territories and populations, offering treatment to a range of acute and chronic conditions, such as infectious and parasitic diseases, hypertension, diabetes, and chronic respiratory conditions [1].

The PHCPS programme is structured into two main subsystems: the management subsystem and the pharmaceutical care subsystem. The main goals are to provide free access to quality and cost-effective medicines, ensure adequate dispensing practices, contribute to the safe and effective use of prescribed medicines by patients, and conduct health-promoting interventions both individually and collectively [5]. However, the programme is still underdeveloped in Brazil, even existing for more than 20 years.

The implementation and performance of the PHCPS are subjected to different interests and contexts, thus its outcomes vary significantly across the territories. Overall, management styles are usually guided by bureaucratic bias, jeopardizing the intended goals [6]. Several challenges persist, such as poor facilities’ infrastructure and organization [7,8,9], medicine shortages and stock-outs at PCP [10, 11], and inadequacies in medicine storage and inventory control [12]. Additionally, a large number of patients do not know how to take their prescribed treatment properly [13].

We proposed the MedMinas Project to contribute to overcoming these challenges. The main purpose was to evaluate the PHCPS programme delivery, the supply subsystem, and the use of medicines by patients, examining processes and contextual factors to understand how the programme operates and identify improvement opportunities.

The MedMinas Project began in 2018 and ended in 2021 and involved planning and designing, fieldwork preparation, and data collection, management, and analysis. This paper aims to describe the study design and methods and to reflect upon the fieldwork experiences and lessons learned while conducting this project.

Methods

Research framework

The development of the MedMinas research framework required a targeted literature review concerning primary care and pharmaceutical services: (1) Brazilian national and state-level policies, and both national and international guidelines [5, 14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30]; (2) relevant methods and findings from selected quantitative and qualitative research, considering Brazilian and international scenarios [3, 7, 31,32,33].

We adopted a system-thinking approach to include different actors, their interrelations, and PHCPS functions [34]. Thus, the next step was guided by defining the scope, concepts, and measurement domains for each essential dimension, covering aspects of the triad “structure–process–outcome”, proposed by Donabedian [35]. We included data related to professional evidence, such as practice experiences and workforce satisfaction. We also incorporated contextual evidence, such as local policies and guidelines and individual factors (e.g., socio-demographic, economic, geographical, environmental, and epidemiological variables), considering their importance for health service planning, management, delivery, and outcomes [30, 31].

The final framework is represented in Fig. 1. We incorporated three healthcare system organizational levels: 1) higher (Municipal Health Secretary); 2) intermediate (Director of Primary Health Care Services and Municipal Coordinator of Pharmaceutical Services); and 3) operational (pharmacist and dispensary assistant). The performance of the PHCPS system is viewed in terms of its outcomes (medicine availability, medicine safety, patient satisfaction) and in terms of related and underlying structures and processes that explain these outcomes and their interconnectivity, such as public policies and management of pharmaceutical systems and services, medicine supply system, and service delivery processes. Contextual factors involved variables related to patients and the environment.

Fig. 1
figure 1

MedMinas logic model. Organizational Level (Municipal Health Secretary); Intermediate (Director of Primary Health Care Services and Municipal Coordinator of Pharmaceutical Services); Operational (Pharmacist and Dispensary Assistant)

Study design and setting

The MedMinas Project has multifaceted research questions, both qualitative and quantitative. Therefore, we adopted a mixed-methods study design based on Rapid Evaluation Methods (REM) [36,37,38]. REM are useful to appraise the quality of health care services, identify problems, and guide managerial responses [39].

The study area was the State of Minas Gerais, located in Southeastern Brazil, the fourth largest State in the country, with a territorial extension of 586,528.293 km2. Minas Gerais was purposefully selected for this project for being the largest State concerning the number of municipalities (n = 853), the second-most populous (20,997,560), and the third by gross domestic product [40, 41]. The State is considered high in HDI (0.731) and ranks 9th among the 27 Brazilian states [42]. Minas Gerais is divided into 13 macro-regions (South, Central-South, Centre, Jequitinhonha, West, East, Southeast, North, Northwest, Eastern South, Northeast, South Triangle, North Triangle) and 76 micro-regions according to its Regionalisation Directing Plan [43]. Following the principles of decentralization of the health system, each of these regions has health pole municipalities, serving as healthcare references to other cities, i.e., they are considered healthcare standards in terms of equipment, human resources capabilities, and service provision to the other municipalities in the same region [21, 43].

The overall health profile of Minas Gerais shows a strong relative predominance of non-communicable diseases, such as cardiovascular conditions, cancer, and respiratory diseases, which are responsible for 66% of the disease burden, coexisting with infectious and parasitic diseases, road injuries, and interpersonal violence [44]. However, the mortality and morbidity profiles vary across State regions due to differences in the micro-political, economic, and socio-cultural environments [45].

The MedMinas project was designed under three premises: (1) fifty percent of the population of Minas Gerais (10,6 M inhabitants) live in medium to large-sized municipalities (between 35,000 and 900,000 inhabitants); (2) the healthcare management and organization model adopted in the State follows the Minas Gerais Regionalisation Directing Plan [43]; (3) evidence suggest suboptimal PHCPS performance in large municipalities, highlighting the need of specific assessment targeting these sites [46].

Information and data sources

Four types of information and data sources were included: (1) patients and health staff from PCP related to Primary Care, (2) health managers, (3) prescriptions and dispensed medicines, and (4) policy documents.

Sampling

Quantitative component

A multistage sampling technique was used to select the PCP, combining REM sampling recommendations of at least 20 health care facilities and 30 patients in each one, plus population size and geographic location criteria (Table 1). The first stage was selecting the macroregions of Minas Gerais, the second, the selection of municipalities within each macroregion, the third stage involved the selection of the PCP, and the fourth, patient selection in the facilities, as follows.

  • Stage 1. All 13 macroregions were included, a priori, in the sampling plan to ensure representativeness. The number of municipalities to be investigated in each region was estimated in proportion to their population. Percent fractions were rounded up to the nearest whole number, so the final sample size was 26 municipalities.

  • Stage 2. This stage comprised the selection of the municipalities within each macroregion. Eligible municipalities were those rated as health poles in the Minas Gerais Regionalisation Directing Plan [43] and with a population size ranging from 35,000 to 900,000 inhabitants (medium to large size-sized municipalities), totalling 68 municipalities. Formal invitations were sent, and those who agreed were included in the study, respecting the criteria established in stage 1.

  • Stage 3. This stage involved the selection of public community pharmacies (PCP). One PCP was selected from each municipality. The main advantage of this method is that experiences with similar methodologies have shown that treatment practices of an individual health care provider are similar to the set of providers with the same resources, so variations in the same services tend to be reduced within each municipality [47]. Thus, the selection of the PCP in municipalities with more than one facility was based on the proximity of the Municipal Health Secretary to reduce logistic costs.

  • Stage 4. The final stage was the selection of patients. According to REM, we randomly selected patients to be interviewed immediately after the contact with the PCP to assess the functioning of the health care facility from their perspective. To estimate the patient sample, we considered the number of PCP to be visited in each macroregion and the minimum number of 30 patients to be interviewed in each service [36]. A percentage of 20% was added to this sample to compensate for losses, totalling 936 patients (Table1). We included a random selection of patients aged 18 years or older who were patients from the PCP for at least six months and had their prescription and dispensed medicines in hand.

Table 1 Sampling process. MedMinas Project, 2018–2021

Qualitative component

A purposeful sample of two subgroups of health professionals was used to select key informants in each of the 26 municipalities included in the quantitative sample to capture potentially rich information [48]. We first undertook a desk review, mapping out managers responsible for outlining strategies and actions for Primary Care at the municipal level. Three levels of healthcare system managers were included: the Municipal Health Secretary (1), the Director/Coordinator of Primary Health Care Services (1), and the Municipal Coordinator of Pharmaceutical Services (1). The second group of professionals consisted of representatives of the PHCPS staff: a pharmacist (1) and a dispensary assistant (1). Thus, the estimated sample totalled five professionals per municipality, 130 considering all the 26 municipalities investigated. Considering the possibility of refusals, we sought to include at least two representatives of the subgroup of managers and two representatives of the PCP. Another municipality filling the correspondent inclusion criteria was selected in the case of refusals from more than one manager. If possible, in case of refusal of PCP staff, another facility from the same municipality was selected. Otherwise, the municipality was replaced.

Documentary research

In the Brazilian National Health System (SUS – acronym in Portuguese), some policies and instruments should be used for planning and management. Considering the relevance to primary care and PHCPS, several documents were selected for analysis and collected during the fieldwork: Municipal Health Plan (Plano Municipal de Saúde), Municipal Essential Medicines List (REMUME – Relação Municipal de Medicamentos Essenciais), collected at the Municipal Health Secretariat, and copies of prescriptions filled at PHCPS.

We also included the Annual Management Report to be extracted from the SARGSUS database (Sistema de Apoio à Construção do Relatório de Gestão). This database was developed by the Ministry of Health and contains, among other information, the results of PHCPS achieved during the execution of the Municipal Health Planning.

Preparation for data collection

Research team

The MedMinas research team comprised three groups of members: general coordination, research staff, and field team. General coordination was composed of four members, the principal investigator, the co-principal investigator, and two field coordinators. This team was responsible for, among other duties, assuring study protocol compliance, developing and maintaining records of all the study documentation, recruiting, training, and managing the field team, managing the data collection process, and storage of research products.

The research staff was composed of one administrator and five students (four graduates and one undergraduate). The administrator was responsible for maintaining diaries, arranging appointments, handling correspondence, and following up with target participants, while the students participated in developing part of the study documentation and data collection.

The field team comprised three statisticians, two field supervisors, three field support personnel, and twenty interviewers. This team was assembled by a third party hired by the general coordination and assigned to collect data and prepare field registers and databases.

Documents and materials

The general coordination and research staff developed the MedMinas logo representing the project’s visual identity to customize digital and printed documents and materials (e.g., slides, letters, certificates, banners). Communication guides and letters and electronic correspondence templates were developed to improve recruitment and stimulate participation in the study. We developed comprehensive and detailed manuals to guide the field team, informed consent to each participant profile, a graphic scale for collecting patient data, and leaflets and banners containing concise information about the project to support the recruitment process. Considering the diverse profile of the MedMinas target audience, we also prepared communication guidelines and letter and electronic correspondence templates. Furthermore, we prepared written information about the relevance of the research, its implications, and advances in science.

We also developed two health education materials to be distributed to participants during data collection: (1) a health calendar, designed to help personal management of healthcare appointments, containing a schedule card with relevant information about the Brazilian adult immunization programme; and (2) a booklet covering aspects of the correct use of medicines and information about medicine storage.

Instruments

The main purpose of the instruments was to operationalize concepts and domains from the study framework and explore multiple individual perspectives while enabling result triangulation. Accordingly, specific questionnaires were developed for the following respondent profiles: patients (1), dispensary assistants (1), pharmacists (1), Municipal Health Secretary and Directors of Primary Health Care Services (1), and the Municipal Coordinator of Pharmaceutical Services (1).

Instruments development was conducted by general coordination and involved four steps. The first step (1) consisted of listing the main variables of the study, searching the literature [49,50,51,52,53,54], and reviewing previously tested instruments used by our research group in similar investigations [7, 10, 13, 55, 56]. This step allowed us to focus on concepts that should be measured or operationalized during data collection, considering our target audience, characteristics, and potential limitations. Step (2) consisted of consulting a panel of specialists to discuss content, format, quantity, complexity, level of detail, time sequence (present, past, future), logical sequence of the questions, and type of question (tick box, date, Likert scale, date, short-free text and open-ended). Step (3) involved elaborating the preliminary version of the instruments. Five instruments were developed, formatted, and edited to be applied by electronic devices. The instruments ended with the following closing statement to collect participant feedback: “We have reached the end of our interview. We would like to thank you for your participation and ask you to register your last impressions or thoughts. Do you have any additional comments or suggestions?”.

A series of pretesting was conducted in Belo Horizonte, the state capital of Minas Gerais and headquarters of our research institution. The procedures involved the research staff, the field team, and academic and non-academic volunteers with a similar profile to the study population. We evaluated the performance of each instrument in terms of comprehensiveness, clarity, the flow of questions, timing, and the recording process. Step (4) consisted of obtaining the final version of the instruments. Domains and main topics explored in the instruments are presented in Table 2. The level of detail varied according to the job position in the healthcare system, health managers and health professionals.

Table 2 Questionnaires by domain, topic, and participant subgroups. MedMinas Project, 2018–2021

All instruments were formatted for tablet-based data collection using the software SurveyToGo©. A data validation process was conducted to verify data entry, review instructions, test skips logic, constrain responses when applicable, and evaluate potential errors and inconsistencies. Additionally, there was the last round workshop involving the research team to evaluate questions that needed rewording or that should be removed from the instruments.

Training

In order to ensure high-quality data collection, the general coordination prepared and delivered a 90-h training course for the field team, covering practical knowledge on field research, epidemiology, qualitative research, ethics, interviewing, and providing project-specific knowledge, especially regarding collecting accurate data on medicines and prescriptions.

The training course comprised a variety of classes, both in group and one-on-one lessons. Slide presentations, video classes, and a workbook were provided. Trainees were tested and granted certificates. Training materials were stored in a cloud service and thus accessible to all the research team members regardless of location.

During training, for ethical reasons and safety issues, interviewers were strongly encouraged to place project materials and electronic devices securely and out of sight in the project bag when not in use.

Enrolment

Two complementary strategies for enrolment were developed, one for municipalities and another for health managers and professionals.

First, a database containing relevant information from 68 eligible municipalities, such as contact details of the key informants and Municipal Health Secretariats and PCP, was prepared. To minimize selection bias, all the eligible municipalities, in the person of the Municipal Health Secretariat or a representative, were contacted by standardized telephone calls and formal written invitations (electronically and by mail). We obtained a low response rate using this strategy, so we asked the State Health Secretariat to formally present the project to managers and representatives of Municipal Health Secretariats.

Our presentation was delivered at the state-level collegiate instance of SUS, the Bipartite Inter-managers Commission. We then sent new invitations to the eligible municipalities, and those that agreed have provided letters of acceptance. Managers and health professionals from those municipalities were then contacted by telephone and email to be informed about the project details and confirm their individual acceptance to participate in the study. We included municipalities where at least two managers and two health professionals agreed to participate in the study.

The municipalities and participants entered the study according to a process of first-served basis that ended when the sample size quotient of 26 municipalities was full.

Data analysis

This paper describes the results of our experiences conducting the pilot study and data collection processes, reflecting upon the lessons learned. We used descriptive statistics for quantitative data. Thematic Content Analysis [48] was used to analyse respondent reflections upon the interview experience. We included all managers (n = 77), health professionals (n = 50), and a random sample of 10% of the patients (n = 83). Microsoft Excel® (Microsoft 365 MSO) was used for the quantitative and qualitative analysis.

Results

Pilot study

The pilot study was conducted from February to March 2019 and included a sample of patients and one health professional (n = 31) in five municipalities. This study had the purpose of increasing knowledge and assessing the practicalities of the field, highlighting aspects that could potentially compromise data collection [57]. We identified problems related to the field team, equipment, research participants, and venues.

The main problems regarding the field team were the interviewers’ lack of experience. Errors were committed when interviewing patients that did not fill all the study’s inclusion criteria; ignoring the ethical rule that requires participants to sign the informed consent before the interviews; taking blurry pictures, conducting interviews in noisy places, reading questions differently from the original text, and in entering wrong participant answers. Regarding equipment, we detected problems in recording and uploading the interviews. Some participants and study sites – especially the PCPs – offered extra challenges to data collection, such as asking for the research instruments in advance, trying to eavesdrop or interfere in other participant interviews, complaining about the presence of the field team. Some of the PCP did not have the minimal infrastructure for conducting research, i.e., room arrangements and furniture.

After the pilot study, another round of contact by telephone and email with the municipalities was conducted to clarify the study purposes and methods and arrange data collection on the venues. We also invested in a focused refresher course to upskill the field team. We took pictures of prescriptions and medicines during interviews to increase the quality of data collected from these items.

The technicalities were solved, and a protocol for field monitoring and auditing was developed by the general coordination team, including criteria for interview approvals and details on in-process quality verifications, such as listening to recorded interviews and database review.

Data collection

A total of 26 municipalities were included in the project, and none of them withdrew the acceptance after the study began (Fig. 2).

Fig. 2
figure 2

Map of the macroregions of Minas Gerais and the municipalities included in the MedMinas Project by population size

Data was collected from April to October 2019 and began by the municipalities located at the centre macroregion due to the proximity to the project’s leading institution based in Belo Horizonte, where the coordination team is based.

The field team was gradually sent, either in pairs or solo, to each of the 26 selected municipalities, following a standardized data collection plan and detailed daily schedule. Upon arrival in each municipality, the field team delivered introductory letters to health professionals at the selected facilities and health managers at the Municipal Health Secretariat, even though we had a formal agreement with all the municipalities and had exchanged comprehensive information about the project with health managers and professionals before going to the field. Only those that agreed to participate were interviewed at the workplace.

Most managers and professionals were fully adherent to the project. One Municipal Health Secretary refused to participate, and another asked to be replaced by the Municipal Director of Health Care, which is why the subgroup of “Coordinators of Primary Health Care” is over-represented (Table 3). Additionally, the Municipal Coordinators of Pharmaceutical Services of two municipalities were simultaneously responsible for the PCP investigated, reducing the number of pharmacists interviewed by 2.

Table 3  Respondent groups by the total number of participants, selected demographics, and participation rate and interview duration. MedMinas Project, 2018–2021

Patients were recruited at the PCP. Before dispensing, potential participants were approached by the interviewers, who introduced themselves, distributed the project leaflets, and informed about the general purpose of the study. Patients were approached again after dispensation. Those meeting the inclusion criteria were invited to participate while being fully informed about the study’s purpose, risks, benefits, and interview duration to make a conscious decision to accept or decline participation. These procedures were adopted to minimize selection bias and guarantee a probabilistic sample.

We interviewed 1019 respondents, of which 127 were professionals and 892 patients (Table 3). The participation rate in the study varied from 92 to 100%, according to the respondent subgroup. The mean number of interviews per municipality was 39 (SD ± 7). We achieved 97% of the maximum sample planned for professionals and 95.3% for patients. Most participants were female. The mean age varied from 35 to 53 years, depending on the participants' group. In general, patients were older than managers and health professionals. Most managers and health professionals have attained tertiary education, while the same rate among patients was minimal (8.2%).

We recorded almost 1000 h of interviews (Table 3), most of which lasted between 45 min and one hour. PHCPS coordinators went through the lengthier interview processes (mean time of 1 h 28 m).

The field team spent, on average, 16 days (SD ± 4.9; min, max: 8–24 days) in each location. Despite contacting municipalities ahead of visiting, data collection took more days than planned due to the non-availability of patients to participate in the study caused by medicine shortages at the PCP. In one municipality, patient recruitment was not feasible, so we lost patient data, while in another, we had to change data collection to another PCP facility because the patient flow was minimum.

We collected copies of 25 Essential Medicines Lists (REMUMEs), 26 Municipal Health Plans, 26 Annual Management Reports, and 995 prescriptions during the fieldwork, totaling 1072 documents. Additionally, we took 2070 pictures of prescriptions filled and medicines dispensed at the PCP.

Data monitoring and management

The field supervisor managed interviewers during the entire data collection process to ensure adherence to the study protocol, data quality, and completeness of the fieldwork. The team also inspected documents and images collected in the field. The coordination team oversaw the entire data collection process, assisting the field team in real-time and conducting in-process quality checks.

Additionally, we conducted a thorough auditing process for the first ten municipalities (n = 365) and a random sample of 10% of the other 16 municipalities (n = 70), totaling 435 interviews (42% of all interviews). The audit protocol included reading through all the questionnaires and listening to the correspondent audio-based records in detail to identify and correct errors or inconsistencies in data entry. These procedures were also used to evaluate interviewer performance, such as remaining neutral during interviews while establishing empathy and rapport with interviewees, asking questions correctly, and clarifying the questions.

During the auditing process, we found that around 24% of patient interviews contained minor errors and a few inaccuracies, especially regarding information collected from prescriptions and dispensed medicines. The interview software, for instance, was programmed to collect data from one prescription at a time, but some interviewers, especially at the beginning of the fieldwork, included information from more than one prescription together as if they were just one document. Another problem was data entry as “unreadable” or “missing”, when, in fact, we could identify the correspondent information in the prescription or in the dispensed medicine by checking photos taken during the fieldwork. We produced a report based on these problems and discussed them with the field team.

Data management included the storage in electronic folders and backup copies of all field products (questionnaires, documents, images, audio-records) organized by municipality and respondent profile. All field products were protected by passwords with access only to the coordination team.

Since MedMinas instruments contained both open-ended and closed-ended questions, we transcribed the interviews. These procedures began after completing the fieldwork due to a previous agreement with the third party assigned to collect data that had internal procedures to prepare and deliver the field products.

Fieldwork challenges, lessons learned, and participant feedback

MedMinas had complex field logistics addressing transportation, accommodation, field materials, and equipment. The most significant fieldwork challenges concerned: (1) municipality and participant enrolment; (2) field team; (3) interview processes; and (4) project budget. Table 4 summarises the occurrences we faced on the ground and the lessons learned from these challenges.

Table 4 Fieldwork challenges and lessons learned. MedMinas Project, 2018–2021

Regarding participant feedback, 39% of managers (n = 30), 60% of the health workers (n = 30), and 70% of patients (n = 57) informed that they did not want to add thoughts or impressions about the interview. However, even without further elaboration, several of them mentioned the interview covered everything they expected or declared “I am good”, “I cannot think of anything else right now”, “I have nothing else to add”.

On the other hand, most managers (n = 47; 61%), and around a third of health workers (n = 18; 36%) and patients (n = 26; 31%), broadly reflected on the experience of being interviewed. Five common major themes emerged from these groups: “self-awareness”, “sense of gratitude”, “research value”, “access to findings”, and “benefits of the research”. Numerous participants mentioned more than one theme.

Self-awareness was the predominant theme (n = 50, 46.3%), meaning that the process of being interviewed has encouraged respondents to think about issues that they had not previously thought about:

“I am glad that you came here to my city! The interview made us reflect upon many things, even about the work we've been developing here, right? We're so in this fire-drill mode that there is no time to stop and plan the work that has to be done” (Coordinator of Pharmaceutical Services, 7)

“This interview has already yielded results. I will set up a meeting with the Municipal Coordinator of Pharmaceutical Services to create our ‘Pharmacy and Therapeutics Committee’ and to update our municipal medicines list (sarg)… All the flaws I've mentioned here… Emm, this interview really gave me the opportunity to do something about it!” (Health Secretary, 16)

“I think that health professionals should be more valued, and more jobs should be offered to them.” (Patient, 315)

“I wish they had more medicines available. There are too much stock outs… physicians prescribe the most expensive medicines, and they [the pharmacy] don't have it!” (Patient, 164)

A “sense of gratitude” for the opportunity of being heard also emerged (n = 23; 21.3%):

“Thank you for picking us. We know the project only included a sample of the municipalities in the State, so we feel like we won an award… We needed to be included, to have the opportunity of giving our opinion!” (Coordinator of Pharmaceutical Services, 12)

“There are places where we feel abandoned. It's good to see someone looking out for us. Congratulations on the research initiative!” (Patient, 467)

The “research value”, i.e., the relevance of the study, was pointed out by the participants (n = 15; 13.9%):

“I really enjoyed this interview… I really like this type of academic work, with the aim of ‘diagnosing’ our health system… I'm sure it will bring about benefit to us managers… Research is a powerful instrument.” (Municipal Health Secretary, 24)

“Extremely valid to understand what people think to improve [the pharmacies]” (Patient, 373)

The “benefits of the research”, i.e., a hope that the research findings will improve PHCPS, was considered by the interviewees (n = 11; 10.2%):

“I wish the results of this interview could benefit people here, especially pharmacy users. There are many sick people in our neighbourhood, even bedridden…” (Patient, 255)

“I hope this study makes it possible, right, for us, to do some effective change, right, in the Pharmaceutical Services in the city (Health Secretary, 2)”

Another key theme was “access to findings”, i.e., the importance of the participants being informed about the research findings. However, this theme emanated only from managers and health professionals (n = 9; 8.3%).

“I think the work you started by doing this investigation, if the municipality could be informed of this data collection results, it will contribute to our planning or even to the necessary adjustments. The health field is very dynamic, the challenges we face every day, we need to adapt to managing. People here need this outlook… other managers need this ‘outside perception’ you bring to us…” (Coordinator of Pharmaceutical Services, 25)

Discussion

In Brazil, primary data collection is still essential for evaluating the performance of the healthcare system despite the existence of National Information Systems for more than 30 years [58]. Databases available in the country are remarkably limited, considering the need for information for planning, managing, and enhancing programmes such as the PHCPS.

We showed the viability of conducting a mixed-method study to provide relevant information on pharmaceutical services, medicine provision, and the use of medicine by patients, highlighting the diverse range of skills and attention to detail needed to take on complex projects of this nature.

Setting up this research project in one of the largest Brazilian States, Minas Gerais, proved to be a rich managerial experience. Considering this challenging research context, we knew from the beginning that the project would require unwavering preparation for data collection, and we would not obtain much knowledge from the literature. In fact, there is still no agreement on performance evaluation of primary care services [59] and even less information on PHCPS. Therefore, we developed the research tools, proposed the plan to achieve the goals, and established responsibilities and standards at an early stage.

The field logistics required much effort and was costly. We were benefited by obtaining an extra budget to complete data collection, but this is not a widespread reality in the research arena in Brazil, where there is a scarcity of potential funders for research focusing on the SUS, mostly coming from public sources [60].

We managed to ensure high-quality data despite challenges and unexpected on-the-spot difficulties from municipality and participant enrolment to the field staff, interview processes, and project budget. We believe the key element was balancing the efficiency of close monitoring with a degree of flexibility. Keeping the communication open and establishing connections and partnerships during the fieldwork helped improve the participation rate. The project adherence rates varied from 92 to 100% among the different groups of participants, which is very high. A nationwide study on PHCPS in Brazil, for example, reached participation rates of 61.5% for Municipal Health Managers, 84.5% for Coordinators of Pharmaceutical Services, 83.6% for health professionals responsible for dispensing medicines, and 97.8% for patients [61]. A study conducted in Minas Gerais reached participation rates of 85.6% for Coordinators of Pharmaceutical Services, 76.0% for health professionals responsible for dispensing medicines, and 92.4% for patients [46].

Another evidence that we could build trustworthy relationships on the field was the very positive feedback we had from participants. Many of them stated they were grateful for participating in the investigation, others recognizing that the experience of being interviewed has made them reflect on several issues related to PHCPS. Some participants happily indicated the relevance of the research, and some were hoping the results would help improve the PHCPS.

Some limitations of MedMinas must be recognized. A mixed-methods study, combining qualitative and quantitative approaches, is subject to selection and information biases and confounding. We minimized such interferences in all study phases. First, we developed a framework that considered PHCPS regarding its related underlying structures, processes, outcomes, contextual factors, and their interconnectivity. We operationalized the framework on multidimensional instruments, pre-tested and piloted, and applied them by a diligently trained field team that followed standardized procedures for data collection. We collected data from other sources, such as documents, prescriptions, and medicines dispensing. The characteristics of our patient sample are similar to the primary care population in Brazil [10, 62].

Conclusions

The methods and lessons learned from conducting the MedMinas Project are useful and valuable for several reasons. We adopted universally applicable techniques suitable for researchers across various disciplines related to Public Health, enabling us to rapidly evaluate the PHCPS in an extensive geographic area and within effective time frames. Additionally, the very positive rapport we built with study participants and our data monitoring and management process allowed us to access meaningful data from multiple actors that we could not obtain otherwise. Finally, our experience may be applied in similar urban settings in Brazil and other LMIC countries.

Availability of data and materials

Quantitative data sharing is not applicable to this article as no datasets were generated or analysed during the current study. The qualitative data that supported this study are not public, but are available from the corresponding author on reasonable request.

Abbreviations

HDI:

Human Development Index

LMIC:

Low- and Middle-Income Countries

MedMinas Project:

Evaluation of pharmaceutical services, medicines provision and use of medicines

PC:

Primary Care

PCP:

Public community pharmacies

PHCPS:

Pharmaceutical Services Based on Primary Care

REM:

Rapid Evaluation Methods

REMUME:

Municipal Essential Medicines List

SUS:

National Health System

References

  1. Paim J, Travassos C, Almeida C, Bahia L, Macinko J. The Brazilian health system: history, advances, and challenges. The Lancet. 2011;377(9779):1778–97.

    Article  Google Scholar 

  2. Fausto MCR, Bousquat A, Lima JG, Giovanella L, Almeida PF, Mendonça MHM, et al. Evaluation of Brazilian primary health care from the perspective of the users: accessible, continuous, and acceptable? J Ambul Care Manage. 2017;40(Suppl 2):S60-70.

    Article  Google Scholar 

  3. Bermudez J, Esher A, Osorio-de-Castro C, Vasconcelos D, Chaves G, Oliveira M, et al. Pharmaceutical services and comprehensiveness 30 years after the advent of Brazil’s Unified Health System. Cien Saude Colet. 2018;23:1937–49.

  4. Brasil. Ministério da Saúde (MS). Portaria nº 3.916, de 30 de outubro de 1998. Dispõe sobre a aprovação da Política Nacional de Medicamentos. 1998. https://bvsms.saude.gov.br/bvs/saudelegis/gm/1998/prt3916_30_10_1998.html. Accessed 21 Jan 2018.

  5. Brasil. Ministério da Saúde (MS). Secretaria de Ciência, Tecnologia e Insumos Estratégicos. Departamento de Assistência Farmacêutica e Insumos Estratégicos. Diretrizes para estruturação de farmácias no âmbito do Sistema Único de Saúde. Brasília: Ministério da Saúde. 2009. https://www.cff.org.br/userfiles/40%20-%20BRASIL_%20MINIST%c3%89RIO%20DA%20SA%c3%9aDE%202009%20Diretrizes%20para%20Estruturacao%20Farmacias%20no%20Ambito%20do%20SUS.pdf. Accessed 21 Jan 2018.

  6. Barreto JL, Guimarães MCL. Evaluation of decentralized management of basic pharmaceutical care in Bahia State. Brazil Cad Saude Publica. 2010;26(6):1207–20.

    Article  Google Scholar 

  7. Luz TCB, Costa MESS, Portes DS, Santos LBC, Sousa SRA, Luiza VL. Pharmaceutical services in primary health care: are pharmacists and users on the same page? Cien Saude Colet. 2017;22:2463–74.

    Article  Google Scholar 

  8. Leite SN, Manzini F, Alvares J, Guerra Júnior AA, Costa EA, Acurcio FA, et al. Infrastructure of pharmacies of the primary health care in the Brazilian unified health system: analysis of PNAUM - services data. Rev Saude Publica. 2017;51(Suppl 2):13s.

    PubMed  PubMed Central  Google Scholar 

  9. Rodrigues PS, Cruz MS, Tavares NUL. Avaliação da implantação do eixo estrutura do programa nacional de qualificação da assistência farmacêutica no SUS. Saúde debate. 2017;41 Spe:192–208.

    Article  Google Scholar 

  10. Bueno MAM, Simões TC, Luz TCB. Diferenças na disponibilidade de medicamentos prescritos na Atenção Primária: evidências do Projeto Prover. Cien Saude Colet 2021. http://www.cienciaesaudecoletiva.com.br/artigos/diferencas-na-disponibilidade-de-medicamentos-prescritos-na-atencao-primaria-evidencias-do-projeto-prover/17998.

  11. Nascimento RCRM, Álvares J, Guerra Júnior AA, Gomes IC, Costa EA, Leite SN, et al. Availability of essential medicines in primary health care of the Brazilian unified health system. Rev Saude Publica. 2017;51(Suppl 2):10s.

    PubMed  PubMed Central  Google Scholar 

  12. Costa EA, Araujo PS, Pereira MT, Souto AC, Souza GS, Guerra AAJ, et al. Technical issues and conservation conditions of medicines in the primary health care of the Brazilian unified health system. Rev Saude Publica. 2017;51(Suppl 2):12s.

    PubMed  PubMed Central  Google Scholar 

  13. Passagli LC, Cota BB, Simões TC, Luz TCB. Knowledge of prescribed drugs among primary care patients: findings from Prover Project. Int J Clin Pharm. 2021;43(5):1265–73. https://doi.org/10.1007/s11096-021-01246-x.

    Article  PubMed  Google Scholar 

  14. Brasil. Ministério da Saúde (MS). Secretaria de Atenção à Saúde. Departamento de Atenção Básica. Gestão da Atenção Básica. Brasília: Ministério da Saúde. 2015. 3 v.: il. (Retratos da Atenção Básica no Brasil – 2012, n. 2. https://bvsms.saude.gov.br/bvs/publicacoes/retratos_atencao_basica_gestao_atencao_n2_v2.pdf. Accessed 29 Jan 2018.

  15. Brasil. Ministério da Saúde (MS). Portaria nº 1.555, de 30 de julho de 2013. dispõe sobre as normas de financiamento e de execução do componente básico da assistência farmacêutica no âmbito do Sistema Único de Saúde (SUS). http://www.as.saude.ms.gov.br/wp-content/uploads/2016/05/PORTARIA-1555-2013.pdf. Accessed 29 Jan 2018.

  16. Brasil. Ministério da Saúde (MS). Secretaria de Atenção à Saúde. Departamento de Atenção Básica. Práticas integrativas e complementares: plantas medicinais e fitoterapia na Atenção Básica. Brasília: Ministério da Saúde. 2012. 156 p.: il. – (Série A. Normas e Manuais Técnicos) (Cadernos de Atenção Básica ; n. 31). https://bvsms.saude.gov.br/bvs/publicacoes/praticas_integrativas_complementares_plantas_medicinais_cab31.pdf. Accessed 7 Feb 2018.

  17. Brasil. Ministério da Saúde (MS). Portaria nº 2.488, de 21 de outubro de 2011. Aprova a Política Nacional de Atenção Básica, estabelecendo a revisão de diretrizes e normas para a organização da Atenção Básica, para a Estratégia Saúde da Família (ESF) e o Programa de Agentes Comunitários de Saúde (PACS). https://bvsms.saude.gov.br/bvs/saudelegis/gm/2011/prt2488_21_10_2011.html. Accessed 7 Feb 2018.

  18. Brasil. Ministério da Saúde (MS). Portaria nº 204, de 29 de janeiro de 2007. Regulamenta o financiamento e a transferência dos recursos federais para as ações e os serviços de saúde, na forma de blocos de financiamento, com o respectivo monitoramento e controle. 2007. https://bvsms.saude.gov.br/bvs/saudelegis/gm/2007/prt0204_29_01_2007_comp.html. Accessed 11 Feb 2018.

  19. Brasil. Ministério da Saúde (MS). Secretaria de Ciência, Tecnologia e Insumos Estratégicos. Departamento de Assistência Farmacêutica e Insumos Estratégicos. Aquisição de medicamentos para assistência farmacêutica no SUS: orientações básicas. Brasília: Ministério da Saúde. 2006. 56 p. – (Série A. Normas e Manuais Técnicos). http://www.ensp.fiocruz.br/portal-ensp/judicializacao/pdfs/284.pdf. Accessed 14 Feb 2018.

  20. Brasil. Ministério da Saúde (MS). Portaria nº 971, de 3 de maio de 2006. Aprova a Política Nacional de Práticas Integrativas e Complementares (PNPIC) no Sistema Único de Saúde. 2006. https://www.cff.org.br/userfiles/38%20-%20BRASIL_%20MINIST%C3%89RIO%20DA%20SA%C3%9ADE_%20Portaria%20n%C2%BA%20971,%20de%2003%20de%20maio%20de%202006_.pdf. Accessed 20 Feb 2018.

  21. Brasil. Ministério da Saúde (MS). Portaria nº 373, de 27 de fevereiro de 2002. Aprova a norma operacional da assistência à saúde do Sistema Único de Saúde. https://bvsms.saude.gov.br/bvs/saudelegis/gm/2002/prt0373_27_02_2002.html. Accessed 20 Feb 2018.

  22. Brasil. Ministério da Saúde (MS). Resolução nº 338, de 06 de maio de 2004. Aprova a Política nacional de assistência farmacêutica. https://bvsms.saude.gov.br/bvs/saudelegis/cns/2004/res0338_06_05_2004.html. Accessed 20 Feb 2018.

  23. Brasil. Ministério da Saúde (MS). Portaria nº 1.214, de 13 de junho de 2012. Institui o Programa Nacional de Qualificação da Assistência Farmacêutica no âmbito do Sistema Único de Saúde (QUALIFAR-SUS). https://bvsms.saude.gov.br/bvs/saudelegis/gm/2012/prt1214_13_06_2012.html. Accessed 20 Feb 2018.

  24. Brasil. Conselho Nacional de Secretários de Saúde. Assistência Farmacêutica no SUS/ Conselho Nacional de Secretários de Saúde. – Brasília : CONASS. 2007. 186 p. https://bvsms.saude.gov.br/bvs/publicacoes/colec_progestores_livro7.pdf. Accessed 22 Feb 2018.

  25. Brasil. Ministério da Saúde (MS). Secretaria de Ciência, Tecnologia e Insumos Estratégicos. Departamento de Assistência Farmacêutica e Insumos Estratégicos. Assistência farmacêutica na atenção básica: instruções técnicas para sua organização. 2.ed. – Brasília : Ministério da Saúde. 2006. 100 p.: il. – (Série A. Normas e Manuais Técnicos). http://www.ensp.fiocruz.br/portal-ensp/judicializacao/pdfs/283.pdf. Accessed 22 Feb 2018.

  26. Manzini F, Alencar LBO, Sales L, Bezerra MB, Campanha P, Macedo R, et al. O farmacêutico na assistência farmacêutica do SUS: diretrizes para ação. Brasília: Conselho Federal de Farmácia; 2015. 298 p. : il. https://www.cff.org.br/userfiles/file/livro.pdf. Accessed 22 Feb 2018.

  27. Organización Panamericana de la Salud (OPS). Servicios farmacéuticos basados en la atención primaria de salud. Documento de posición de la OPS/OMS. Washington, DC: OPS, 2013: Organizacion Pan Americana de la Salud/Orgaización Mundial de la Salud; 2013. (La Renovación de la Atención Primaria de Salud en las Américas, n.6). https://www.paho.org/hq/dmdocuments/2013/SerieRAPSANo6-2013.pdf. Accessed 26 Feb 2018.

  28. Minas Gerais. Secretaria de Estado de Saúde (SES-MG). Deliberação CIB-SUS/MG nº 2.416, de 17 de novembro de 2016. Aprova a Estratégia da Regionalização da Assistência Farmacêutica (ERAF) e as normas de financiamento do Componente Básico do Bloco da Assistência Farmacêutica (CBAF) no âmbito do SUS-MG. https://www.saude.mg.gov.br/images/documentos/Del2416-SUBPAS_SAF-Estrat%C3%A9giadaRegionaliza%C3%A7%C3%A3oCBAF.pdf. Accessed 26 Feb 2018.

  29. Minas Gerais. Tribunal de Contas do Estado de Minas Gerais (TCE-MG). Secretaria Executiva do Tribunal. Relatório Final de Auditoria Operacional. Assistência Farmacêutica: Programa Farmácia de Minas. 2013. https://tcnotas.tce.mg.gov.br/tcjuris/Nota/BuscarArquivo/250919. Accessed 26 Feb 2018.

  30. World Health Organization and World Bank Group. Monitoring progress towards universal health coverage at country and global levels: framework, measures and targets. Geneva: WHO. 2014. https://apps.who.int/iris/handle/10665/112824. Accessed 27 Feb 2018.

  31. Kringos D, Nuti S, Anastasy C, Barry M, Murauskiene L, Siciliani L, et al. Re-thinking performance assessment for primary care: opinion of the expert panel on effective ways of investing in health. Eur J Gen Pract. 2019;25(1):55–61.

    Article  Google Scholar 

  32. Aziz MM, Ji W, Masood I, Farooq M, Malik MZ, Chang J, Jiang M, et al. Patient satisfaction with community pharmacies services: a cross-sectional survey from Punjab; Pakistan. Int J Environ Res Public Health. 2018;15(12):2914.

    Article  Google Scholar 

  33. Jones LF, Owens R, Sallis A, Ashiru-Oredope D, Thornley T, Francis NA, et al. Qualitative study using interviews and focus groups to explore the current and potential for antimicrobial stewardship in community pharmacy informed by the theoretical domains framework. BMJ Open. 2018;8(12):e025101.

    Article  Google Scholar 

  34. World Health Organization (WHO). Systems Thinking for Health Systems Strengthening. Alliance for Health Policy and Systems Research, World Health Organization. Geneva: WHO. 2009. https://apps.who.int/iris/bitstream/handle/10665/44204/9789241563895_eng.pdf?sequence=1. Accessed 28 Feb 2018.

  35. Donabedian A. The quality of care. How can it be assessed? JAMA. 1988;260(12):1743−8.

  36. Anker M, Guidotti RJ, Orzeszyna S, Sapirie SA, Thuriaux MC. Rapid evaluation methods (REM) of health services performance: methodological observations. Bull World Health Organ. 1993;71(1):15–21.

    CAS  PubMed  PubMed Central  Google Scholar 

  37. Clark M. Rational Pharmaceutical Management Plus Program. Pharmaceutical management for malaria manual. Arlington, VA: Management Sciences for Health. 2004. https://www.msh.org/sites/default/files/rpmplus_pmm_2004_-manual_revised_may06.pdf. Accessed 28 Feb 2018.

  38. Management Sciences for Health (MSH). Family Planning Management Development. Using evaluation as a management tool. Fam Plan Manag. Spring 1997;6(1):1−18. https://www.msh.org/sites/default/files/v6_n1_en.pdf. Accessed 28 Feb 2018.

  39. McNall M, Foster-Fishman PG. Methods of rapid evaluation, assessment, and appraisal. Am J Eval. 2007;28(2):151–68. https://doi.org/10.1177/1098214007300895 casa_token=A91axwnjRa4AAAAA:L3UMwWCte27YYpeYO5jHejxFIEcAsa_L4iGWn-_gnuGuvIcbv-6ZfVGKY0HM0I5U9II21IVYLaaA9A. Accessed 11 Fev 2018.

    Article  Google Scholar 

  40. Instituto Brasileiro de Geografia e Estatística (IBGE). https://www.ibge.gov.br. Accessed 21 Fev 2018.

  41. Minas Gerais Investment and Foreign Trade Promotion Agency (INDI). https://www.indi.mg.gov.br/en/indi/about-indi/. Accessed 21 Fev 2018.

  42. Programa das Nações Unidas para o Desenvolvimento (PNUD). Atlas do desenvolvimento humano no Brasil 2013. http://www.pnud.org.br. Accessed 21 Fev 2018.

  43. Malachias I, Leles FAG, Pinto MAS. Plano diretor de regionalização da saúde de Minas Gerais. Belo Horizonte: Secretaria de Estado de Saúde de Minas Gerais. 2010. https://www.novaconcursos.com.br/arquivos-digitais/erratas/15885/20203/plano-diretor-regionalizacao.pdf. Accessed 26 Fev 2018.

  44. Brasil. Ministério da Saúde (MS). Departamento de Informática do Sistema Único de Saúde (DATASUS). http://www2.datasus.gov.br/DATASUS/index.php2021. Accessed 27 Fev 2018.

  45. Diniz JS. Necessidades em saúde e oferta da atenção primária nos municípios de Minas Gerais: uma análise sob a ótica da equidade em saúde. Belo Horizonte: Escola de Governo Professor Paulo Neves de Carvalho, Fundação João Pinheiro; 2014. http://www.bibliotecadigital.mg.gov.br/consulta/consultaDetalheDocumento.php?iCodDocumento=75131. Accessed 26 Fev 2018.

  46. Barbosa MM, Garcia MM, Nascimento RCRM, Reis EA, Guerra Júnior AA, Acurcio FA, et al. Infrastructure evaluation of pharmaceutical services in the national health system of Minas Gerais. Cien Saude Colet. 2017;22(8):2475–86.

    Article  Google Scholar 

  47. World Health Organization (WHO). Action Programme on Essential Drugs and Vaccines. (1993). How to investigate drug use in health facilities: selected drug use indicators. https://apps.who.int/iris/handle/10665/60519. Accessed 02 Mar 2018.

  48. Patton MQ. Qualitative research & evaluation methods: integrating theory and practice. 4th ed. Saint Paul: SAGE Publications Inc.; 2014.

    Google Scholar 

  49. Fröhlich SE, Dal Pizzol TS, Mengue SS. Instrument to evaluate the level of knowledge about prescription in primary care. Rev Saude Publica. 2010;44(6):1046–54.

    Article  Google Scholar 

  50. Grootaert C, Narayan D, Nyhan-Jones V, Woolcock M. Measuring social capital: an integrated questionnaire. World Bank Working Paper. 2004. https://openknowledge.worldbank.org/handle/10986/15033. Accessed 06 Mar 2018.

  51. Brasil. Pesquisa Nacional sobre Acesso, Utilização e Promoção do Uso Racional de Medicamentos no Brasil (PNAUM): Ministério da Saúde (MS). http://www.ufrgs.br/pnaum/documentos/questionarios-/PNAUM_Inq_Adulto.pdf/view. Accessed 06 Mar 2018.

  52. Instituto Brasileiro de Geografia e Estatística (IBGE). Pesquisa nacional de saúde. proposição do questionário da PNS 2013. https://www.pns.icict.fiocruz.br/. Accessed 07 Fev 2018.

  53. InstitutoBrasileiro de Geografia e Estatística (IBGE). Pesquisa nacional por amostra de domicílios. Rio de Janeiro: Manual do Suplemento: Saúde e Acesso à Internet; 2008.

    Google Scholar 

  54. Traverso ML, Salamano M, Botta C, Colautti M, Palchik V, Pérez B. Questionnaire to assess patient satisfaction with pharmaceutical care in Spanish language. Int J Qual Heal Care. 2007;19(4):217–24.

    Article  Google Scholar 

  55. Luz TCB, Osorio-de-Castro CGS, Luiza VL, Souza CM, Schall VT. Projeto Servir: avaliação do papel de serviços farmacêuticos no acesso a medicamentos em Divinópolis, Minas Gerais. Belo Horizonte: Grupo de Estudos Transdisciplinares em Educação em Saúde e Ambiente (GETESA/IRR/Fiocruz/MG); 2013.

    Google Scholar 

  56. Luz TCB, Simões TC, Cota BB. Avaliação do provimento de medicamentos na Atenção Básica em município do centro-oeste mineiro. Projeto Prover. Belo Horizonte: Grupo de Estudos Transdisciplinares em Educação em Saúde e Ambiente (GETESA/IRR/Fiocruz/MG); 2017.

  57. van Teijlingen E, Hundley V. The importance of pilot studies. Nurs Stand. 2002;16(40):33–6.

    Article  Google Scholar 

  58. Pinto LF, Freitas MPS, Figueiredo AWS. National information and population survey systems: selected contributions from the ministry of health and the IBGE for analysis of Brazilian state capitals over the past 30 years. Cien Saude Colet. 2018;23(6):1859–70.

    Article  Google Scholar 

  59. Noorihekmat S, Rahimi H, Mehrolhassani MH, Chashmyazdan M, Haghdoost AA, Tabatabaei SVA, et al. Frameworks of performance measurement in public health and primary care system: a scoping review and meta-synthesis. Int J Prev Med. 2020;11:165.

    PubMed  PubMed Central  Google Scholar 

  60. Silva RM, Caetano R, Silva AB, Guedes ACCM, Ribeiro GR, Santos DL, Paiva CCN. Perfil e financiamento da pesquisa em saúde desencadeada pela pandemia da COVID-19 no Brasil. Vigil sanit debate. 2020;8(2):28–38.

    Article  Google Scholar 

  61. Brasil. Ministério da Saúde (MS). Secretaria de Ciência, Tecnologia e Insumos Estratégicos. Componente Avaliação dos Serviços de Assistência Farmacêutica Básica: resultados. Brasília: Ministério da Saúde. 2016. 140 p.: il. – (Série Pnaum – Pesquisa Nacional sobre Acesso, Utilização e Promoção do Uso Racional de Medicamentos no Brasil; Caderno 4). https://bvsms.saude.gov.br/bvs/publicacoes/componente_avaliacao_assistencia_pnaum_caderno4.pdf. Accessed 07 Mar 2021.

  62. Guibu IA, Moraes JC, Guerra Júnior AA, Costa EA, Acurcio FA, Costa KS, et al. Main characteristics of patients of primary health care services in Brazil. Rev Saude Publica. 2017;51(Suppl 2):17s.

    PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

The authors would like to thank the Municipal and State Health Secretariats and their managers and health professionals, the Minas Gerais Council of Municipal Health Secretariats (COSEMS/MG), Research Development Foundation (FUNDEP), and Fiocruz Minas for supporting the development of the MedMinas Project. The authors express their sincere gratitude to all study participants for their cooperation during the study. We would also like to thank Victor Kiiti Tanaka da Anunciação for contributing to the Minas Gerais map preparation.

Funding

This study was supported by a grant from Minas Gerais State Research Support Foundation (FAPEMIG/Programa de Pesquisa para o SUS- PPSUS, Grant Number APQ-04124–17). Fiocruz Minas also supported this work. The funding bodies have no role in the design, data collection, analysis, and data interpretation, nor in writing the manuscript or deciding to report these data in a peer-reviewed journal.

Author information

Authors and Affiliations

Authors

Contributions

TCBL was responsible for funding acquisition, study conception, design, data analysis and interpretation, manuscript writing, and final manuscript review. NULT and EMS contributed to the discussion, and final manuscript review. AKSC and ICM supervised data collection and contributed to database preparation, data analysis, result discussion, and final manuscript review. BBC contributed to funding acquisition, discussion, and final manuscript review. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Tatiana Chama Borges Luz.

Ethics declarations

Ethics approval and consent to participate

The study was approved by the Ethics Committee of the Rene Rachou Institute of the Oswaldo Cruz Foundation (Fiocruz-Minas) (Reference: 2.682.759) in compliance with all legal and regulatory requirements from the National Commission for Research Ethics (Comissão Nacional de Ética em Pesquisa—CONEP). Prior approval of data collection was obtained from Minas Gerais State Health Secretariat and all Municipal Health Secretariats. All participants signed the informed consent form and gave the authorization to photograph prescriptions and dispensed medicines.

Consent for publication

Not applicable—no identifying data is present.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Luz, T.C.B., Tavares, N.U.L., de Castro, A.K.S. et al. MedMinas project: design and use of mixed methods in the evaluation of pharmaceutical services in primary health care in Minas Gerais, Brazil. BMC Med Res Methodol 22, 80 (2022). https://doi.org/10.1186/s12874-022-01568-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12874-022-01568-y

Keywords