Rev. Enferm. UFSM - REUFSM

Santa Maria, RS, v. 9 (Esp), e3, p. 1-19, 2019

DOI: 10.5902/2179769238739

ISSN 2179-7692

 

 

Submission: 6/27/2019    Acceptance: 11/18/2019    Publication 02/12/2019

Original Article

 

Analysis of the production of mixed method studies in the evaluation of mental health services

Análise da produção de estudos com métodos mistos na avaliação de serviços de saúde mental

Análisis de la producción de estudios con métodos mixtos en la evaluación de servicios de salud mental

 

Guilherme Emanuel Weiss PinheiroI

Angela Maria Pereira dos SantosI^I

Luciane Prado KantorskiIII

 

I Enfermeiro, Mestre em Ensino na Saúde pelo Programa de Pós-graduação em Ensino na Saúde da Universidade Federal de Ciências da Saúde de Porto Alegre, Professor EBTT no Colégio Politécnico da Universidade Federal de Santa Maria,  Santa Maria, RS, Brasil, guilhermeewp@politecnico.ufsm.br, ORCID: http://orcid.org/0000-0003-0069-7023

II Psicóloga, graduada em psicologia pela Universidade Franciscana, Estudante do curso técnico em enfermagem do Colégio Politécnico da Universidade Federal de Santa Maria, Santa Maria, RS, Brasil, angelacooplantio@hotmail.com, ORCID: http://orcid.org/0000-0003-0181-608X

III Enfermeira, Doutora em Enfermagem pela Universidade de São Paulo, Professora na Universidade Federal de Pelotas, Pelotas, RS, Brasil, kantorski@uol.com.br,

    ORCID: http://orcid.org/0000-0001-9726-3162

 

Abstract: Objective: To analyze the mixed method research designs produced on the evaluation of mental health services. Method: A document analysis study, conducted from April to June 2019, which analyzed the research designs of mixed methods of evaluation of mental health services, carried out between 2006 and 2016 by a nursing research group. Results: Three mix-method studies of evaluation of mental health services from the southern region of Brazil are presented, all developed from a similar integration dynamics. The integration steps, objectives, methodology and analytical axes are presented. Conclusions: Conducting mixed method studies has enabled the identification of integration and convergence points between quantitative and qualitative approaches. The potential of the methodology for evaluative studies is emphasized.

Descriptors: Research; Methods; Health assessment; Nursing

 

Resumo: Objetivo: analisar os desenhos de pesquisa de métodos mistos produzidos sobre avaliação de serviços de saúde mental. Método: estudo de análise documental, realizado nos meses de abril a junho de 2019, o qual analisou os desenhos de pesquisas de métodos mistos de avaliação de serviços de saúde mental, realizadas entre 2006 e 2016, por um grupo de pesquisa em enfermagem. Resultados: são apresentados três estudos de métodos mistos de avaliação de serviços de saúde mental da região sul do Brasil, todos desenvolvem-se a partir de uma dinâmica de integração similar. São apresentadas as etapas de integração, os objetivos, a metodologia e os eixos analíticos. Considerações finais: a realização de estudos de métodos mistos possibilitou a identificação de pontos de integração e convergência entre as abordagens quantitativas e qualitativas. Ressalta-se o potencial da metodologia para estudos avaliativos.

Descritores: Pesquisa; Métodos; Avaliação em saúde; Enfermagem

 

Resumen: Objetivo: analizar los diseños de investigación de métodos mixtos producidos en la evaluación de servicios de salud mental. Método: estudio de análisis de documentos, realizado de abril a junio de 2019 y en el que se analizaron los diseños de investigación de métodos mixtos de evaluación de servicios de salud mental prestados entre 2006 y 2016 por un grupo de investigación en enfermería. Resultados: se presentan tres estudios de métodos mixtos de evaluación de servicios de salud mental de la región sur de Brasil, todos desarrollados a partir de una dinámica de integración similar. Se presentan los pasos de integración, objetivos, metodología y ejes analíticos. Consideraciones finales: realizar estudios de métodos mixtos ha permitido identificar puntos de integración y convergencia entre enfoques cuantitativos y cualitativos. Se enfatiza el potencial de la metodología para estudios evaluativos.

Descriptores: Investigación; Métodos; Evaluación de salud; Enfermería

 

Introduction

Mixed research methods are an alternative to human and social sciences and health studies. In this context, research questions and problems are likely to be better answered and investigated. It is considered more than a method, being extended to a methodology that joins the quantitative and qualitative approaches around a research object.¹ In addition, it is a methodology that combines the methods, approaches, philosophy and orientation of the research project.¹ As fundamental aspects in the use of mixed methods, the following can be mentioned: collection analysis of quantitative and qualitative data; combination of the two types of data; integration of procedures in the same study; and structuring of procedures from philosophical and theoretical world views.²

The quantitative approach produces data such as numbers and indicators, and is analyzed from the metric and its expressions, being able to present fast and reliable information about the investigated object. As it uses open interviews, observation, among other techniques, the qualitative approach is intended to provide information about the speech of the interviewed or observed individuals, providing insight from other perspectives on the object studied and outlining subjective aspects of the researched phenomenon.¹

Known as a multi-method approach, the mixed method research is defined as a set of procedures for collecting, analyzing and combining quantitative and qualitative techniques around the same research design. The interaction between the approaches is considered one of the main assumptions, as it provides better possibilities for analysis.²

The quantitative and qualitative approaches maintain an epistemological and ontological debate. Thus, in the positivist view, the methods and techniques of the Social Sciences need to be closer to the Natural Sciences, while interpretivism holds that the Social and Natural Sciences constitute distinct approaches and, for that, they need specific methods and techniques.³ However, there is a difference between the Natural Sciences and the Social Sciences, which is the meaning proper to the latter and indispensable for the interpretation of social reality,³ since Social Sciences seek to deal with the interpretation of the actions of the subjects and reveal the subjective meanings in their analysis.³

Studies with mixed methods have advantages, among the highlighted ones are the integration between the qualitative and quantitative approaches and the possibility of taking the best out of each to answer the problem and the research questions listed.³ Thus, the mixture of approaches yields positive results, as each one contributes in its own way to a better research design, gathering complex results and understandings in the investigation.³

In a study published in an English journal, the top ten research resources with mixed methods in health systems were analyzed, noting that this is a third approach, increasingly popular and growing in health systems.⁴ This demonstrates that the quantitative approach allows knowing the extent of a given situation and also promote the understanding of the representativeness of the results. And the qualitative one works in the perspective of broadening the depth of understanding, offering different overviews of the concerned parties and providing justification for the performance of health systems. Thus, it is reinforced that the mixed methods in health research studies are increasing since, in this context it is necessary to address complex issues such as socio-cultural factors, which could not be sufficiently measured in isolation by a specific approach, being necessary the integration of approaches to understand the complexity of the phenomena.⁴

Some researchers have found that there is a gap in knowledge production in relation to mixed method research studies,^3,5-6 requiring more effort for their development.⁶ In this sense, it is argued that there is a need for integration and interaction between the quantitative and qualitative components of a study, being essential for some definitions to take shape and sustain themselves in the context in which they are investigated. This is important for expanding and strengthening the results of these studies, as well as answering research questions and producing high validity knowledge.⁵

A research group carried out mixed method studies from 2006 to 2016 in the area of mental health services evaluation, in order to develop evaluations of actions, programs and services within public health policies. There were three multi-centered matrix research projects that evaluated about forty health services in southern Brazil. As its research question, this manuscript presents the following: what are the contributions of the mixed research methods to evaluate mental health services? Thus, it aims to analyze the research designs of mixed methods produced on evaluation of mental health services.

 

Method

This is a document analysis study, conducted from April to June 2019. A documentary research may produce new knowledge through the creation of different ways of understanding the phenomena, of knowing how they relate and how they have been developed as objects of study.⁷ It is also believed that, for the development of studies in the various areas of knowledge, document analysis has its relevance preserved, because through this, researchers can collect, treat and analyze the sources of information.⁸ Thus, this study seeks to relate research designs that used mixed methods to recognize study objects in the field of health assessment, in particular the services of the psychosocial care network of southern Brazil and their contributions to the health and nursing field in mental health.

The documents analyzed were research reports of mixed methods of evaluation of mental health services, funded by the National Council for Scientific and Technological Development (Conselho Nacional de Desenvolvimento Científico e Tecnológico, CNPq), in the period from 2006 to 2016. The selection criteria were the following: having received funding from a funding agency via the Ministry of Science and Technology (Ministério da Ciência e Tecnologia, MCT) -CNPq/MS-SCTIE-DECIT-CT-Saúde - Edict 07/2005, and being a mixed method research.

These criteria are justified by the fact that the first call for funding for evaluative studies on mental health was published in 2015. Selection took place from the research studies selected by the aforementioned edict, and a group was chosen, the only one that carried out mixed method research studies. The period of this study is based on a 10-year longitudinal cut, in which there was funding for research in the area of mental health services evaluation.

A document analysis of the final reports of three studies using mixed research methods was performed, produced by a research group in nursing, mental health and collective health of a public university, certified by the CNPq's Brazilian Research Groups Directory, during 10 years. The studies analyzed complied with current ethical standards and were approved by the Research Ethics Committees.

 

Results

The three evaluation studies of mental health services in southern Brazil are presented, which followed the characteristics of explanatory or explanatory sequential mixed method design. The first study, named as Study 1 in Tables 1 and 2, was conducted from 2006 to 2008, in the three states of southern Brazil, with the purpose of evaluating the Psychosocial Care Centers (Centros de Atenção Psicossocial, CAPS). The second study, Study 2, was conducted from 2008 to 2010, in Rio Grande do Sul, with the purpose of evaluating the innovative experiences of composition of psychosocial care networks. And the third study, Study 3, was conducted from 2011 to 2016, in the three states of southern Brazil, being a second edition of Study 1.

The three studies presented a similar integration dynamics. The case studies of the qualitative stage were selected from the compilation of quantitative data and, together, presented better performance based on the legal norms of the Ministry of Health regarding psychosocial care services. In addition, the structure, process and result indicators were observed, compared to the legislations, thus, the choice of the municipalities occurred collectively and based on quantitative data.

Chart 1: Comparison of the objectives and data collection methodology of mixed method studies, 2019.

The studies worked with evaluation markers and from this, there was the integration between quantitative and qualitative markers. Even though these markers were generated based on instruments specific to each approach, data integration culminated in the discussion of the results. It is also important to highlight that the studies were built from the triangulation of data with interviews, document analysis, groups and observation. Based on this, the integration occurred when the qualitative stage offered elements for deepening and understanding gaps listed in the quantitative stage.

Chart 2: Comparison of the markers from mixed method studies, 2019.

Discussion

The analyzed studies present a differential in the construction of their designs, as they are mixed method research studies. Although each approach presents different techniques and methods, as well as independent theoretical bases in the construction of their designs, the mixed methods seek integration at different moments. The quantitative approach in these cases is based on epidemiology and positivism, and the qualitative approach is based on the fourth generation constructivist and responsive assessment and multiple case studies. However, the approaches converged to an integrative process, since the election of the qualitative stage scenario as in the discussion of the results.

The characteristics of positivism focus on singular reality (ontology), distance and impartiality (epistemology), a priori theory testing (methodology), and agreed definitions in the variables for writing the reports (rhetoric).² It also aims to test theories or to describe experiences by observation or measurement to control and predict the forces surrounding the object studied. It has a harsh view of pure reality that can be captured and predicted, so positivism uses deduction to confirm theories using methods of collecting and quantitative data analysis.⁹

Constructivism, in turn, focuses on multiple realities (ontology), in proximity to the object of study (epistemology), being inductive, since it comes from the views of the individuals and then reaches the standards (methodology) and builds its reports from an informal literary style (rhetoric). It is associated with the qualitative approach and works with a different world view. Once the participants present their understandings, the research was built based on individual perspectives for broad understandings.² From the subjectivity and reality experienced by the research subjects.

Mixed methods consist on the collection, analysis and integration of qualitative and quantitative data, which contributes to a better understanding of the research problem. It is not only about obtaining data, it is about integrating them.^2,5,10

Scholars in the field of methodology have emphasized that the integration between quantitative and qualitative approaches is a centerpiece in mixed method studies, being an intentional process. There is an independence of approaches to issues and research hypotheses that are both common. Researchers realize the benefits of mixed methods by observing that integration produces a whole greater than the sum of the individual contributions of qualitative and quantitative approaches.⁶

To exemplify how integration is possible and to facilitate didactically, the authors worked on the systematization of basic project designs, which could be exploratory sequentials, explanatory sequentials or explanatory and convergent.^4-6,10-11 In addition, embedded or nested designs can be added:^4-5,11 the transformer and the multiphase.^4-5

The exploratory sequential starts with the qualitative data collection and analysis phase that will base the next quantitative phase and may produce generalizations. The explanatory or explanatory sequential comes from a quantitative phase of data collection and analysis, which will proceed with the qualitative stage, seeking to explain the quantitative data. And the convergent comprises the quantitative and qualitative collections in near moments and proceeds with the integrated analysis, the data have different but complementary constitutions. Moreover, they emphasize that the integration of quantitative and qualitative approaches can occur in other ways: discussion of quantitative results with a qualitative approach; construction of quantitative instruments, based on qualitative results; merging of results; or incorporating one approach within another.^4-6,10-11

When it comes to integrating approaches to building mixed method designs, other designs called advanced designs can be considered, namely: multiple stages; intervention studies; case study; transformative research and participatory research; evaluation studies; or app/website/platform development.¹⁰

Thus, the analyzed studies used a mixed method approach, following the basic sequential explanatory or explanatory design, which aims to use the qualitative approach to help explain the results of the quantitative approach. Thus, the studies had their quantitative stages performed at a first moment and, after that, the results were the basis for choosing the services that would receive the qualitative stage, highlighting those with positive experiences of mental health care, from the quantitative data and analysis of ministerial regulations. The qualitative data, in turn, helped to explain the quantitative results and offered an in-depth study of the services highlighted in the first stage. At the end, there was data integration in the presentation of research reports.

It is necessary to keep discussing the data presented in Chart 1. Starting with the objectives of the studies, there is a combination of the evaluation process, in the quantitative study the evaluation happened through the conditions of the structure, the process and the result of the psychosocial care and through the use of some terms such as factors, causes, measures, determinants, correlations, trends, levels and magnitudes. And the qualitative evaluation culminates with the meanings, experiences and looks of the actors involved in psychosocial care. Thus, the mixed method research needs to combine quantitative and qualitative approaches, enabling the understanding of the individuals' experiences and the relations between the variables.¹⁰

By analyzing the objectives of the study approaches, they are convergent regarding the exploration of common aspects in the analysis axes: structure, process and result of the work developed in the services, but they also approach different aspects within each analysis axis, such as the following: physical infrastructure, material resources, human resources, technical resources, reference system, food and medicines (structure); adequacy of attention, territory, diversification and frequency of activities and assemblies (process); and satisfaction and individual characteristics (result). The objectives of quantitative studies come from concepts that have variables such as structure, process and result, which are defined objective variables. Those of qualitative ones come from a broader context, such as the evaluation of the quality of service, which is subjective and that each interviewee will base his or her place of speech and evaluate from their experiences of ambience, plasticity of service, insertion of the family, work organization, result of psychosocial attention, among others.

In the quantitative approach, the variables need to have established relationships and from this, being proved or refuted as initial hypotheses. Meanwhile, in the qualitative one, the ideas and basic definitions need to be identified, basing the study and the main awareness concepts on which the data will be built.¹²

Another aspect that can be highlighted in Chart 1 is the convergence between the study participants (health workers, users of mental health services and family) and the unit of analysis (the CAPS, in both stages). In the studies analyzed, the quantitative stage was performed with the same group of individuals in a random sample, which ensured equiprobability, since all the services had the same chance of being selected at the later stage.³

            At the end of this stage, services (units of analysis) were selected and, among the individuals who participated in the first stage and based on pre-established criteria, participants who would take part in the qualitative stage were selected. Regarding the unit of analysis it is necessary to define what the basic unit of research will be and, afterwards, it must be ensured that it is homogeneous.³ The integration in the mixed method research is emphasized, since the quantitative and qualitative sampling process ensures that the study individuals and the unit of analysis are the same.¹⁰

We highlight the differences in sampling between the studies regarding the number of services or municipalities and of individuals who participated in the research stages. In Study 1, in the quantitative stage there were 30 CAPS and 2,563 people interviewed, in the qualitative one, five case studies in five CAPS and 205 individuals interviewed. In Study 2, in the quantitative stage there were five municipalities and 615 people interviewed and, in the qualitative one, two municipalities and 77 individuals interviewed. And in Study 3, in the quantitative stage there were 308 CAPS and 3,405 individuals interviewed and, in the qualitative one, three CAPS and 111 individuals interviewed.

Thus, the qualitative approach comes from close observation, being a form of study that needs deepening and detail to build its research bases, orchestrated by intensive studies and with a small number of cases. The quantitative stage has the prerogative of working with larger samples, that is, more numerous so that their representativeness is valid in their method.¹³

In the same direction, examining the association between the variables can result in generalizations for a given population by using statistical inferences. Thus, the quantitative approach focuses its analysis on larger samples, but does not present an understanding of individuality. While the qualitative approach is endowed with detailed descriptions of the studied phenomena, presenting the context and being able to focus its analysis on the deepening of individuality, but its results are not generalizable.¹⁴

Regarding the instruments and techniques used in the different stages of the studies, it is possible to observe that in the quantitative approach the instruments were questionnaires with scales and other closed questions to evaluate, among other aspects, the structure, process and outcome of health services. While in the qualitative approach, other instruments were used, such as the semi-structured interview, with open questions, participant observation and the validation group. This made it possible to know and understand the cultural, economic, organizational, political and social particularities of the services studied from the qualitative approach, and integrating this the qualitative approach, which had the ability to measure the associations between the factors that made up the evaluation of the different aspects addressed.¹⁴

The use of different techniques or instruments is not chosen randomly, but rather based on the theoretical framework that each approach inherently carries. In this sense, there are discussions regarding the bases used in each approach; however, it is necessary to be aware that, when studying social reality, the subjectivity that permeates this medium needs to be valued; and, when working with quantitative indicators and variables, the positivist view becomes important and necessary for the results to be plausible to the object being investigated.³

What is going to characterize the analyzed research studies that count on a qualitative and a quantitative stage, in research studies with mixed methods, is precisely the capacity of integration within the research that goes from the objectives and the methods to the complementarity of the results. Some theorists point out some basic characteristics for integration to be guaranteed, namely: corroboration or convergence (the results of one approach collaborate with those of another); complementarity (one approach helps to improve/clarify the results of another); development (results from one approach inform results from the other – designing an instrument, delimiting a sampling); initiation (helps in discovering paradoxes and contradictions of the other); the expansion (depth and amplitude are expanded); and disagreement (when there is conflict between approaches).^10-11

Integration and convergence are characteristic of multi-method studies and the more convergent and integrated the results, the more consistent the research will be. Thus, integration aims, among other objectives, to enhance the information incorporated in a research, benefiting and improving the findings of the study.³

In Chart 2 it is possible to notice the approximations, differences and complementarities between the markers used in the analysis phase, when observing the structure evaluation, for example, in the quantitative approach, the physical infrastructure issue – number of rooms and size of space; the material resources, human resources – for example: number of professionals, workload of the professionals, and service characteristics, among others. With this, it is possible to verify if the structure is adequate or not and if it presents the minimum required by the current norms and laws of the mental health services. 

In the qualitative approach, the ambience is observed, regarding the structure, with subjective details of the individuals involved in these evaluation processes, that is, how the actors see the structure, how they feel in it, if they have privacy, if they feel welcomed, what are their perceptions and concerns. Thus, the quantitative approach acts on the objective variables in the evaluation process and the qualitative one on the subjective and intersubjective questions of the evaluation.¹² The examples of these evaluative studies contribute to demonstrate the ability to complement, integrate and strengthen the results presented by the different approaches by constituting a study of mixed methods.

From this observation it is possible to notice the connection between the approaches, that is, how the quantitative contributes to the qualitative, and vice versa. The main question in this connection is how this integration between approaches occurs so that the mixed method is consolidated as an alternative to research.^3,10,15

In the same vein, the place that the researcher occupies in each approach needs to be taken into account, especially when discussing the results of research in the area of health service evaluation. In the qualitative approach the researcher actively participates in the process, his/her values are incorporated as part of the research and, in the quantitative one, the researcher and his/her values remain away from the evaluated object.¹² This finding allows for data to be treated differently in each approach, as they are obtained differently. Thus, it reinforces the collaborative capacity of one approach with another, and vice versa.

In the experience of these studies, it can be noticed that the approaches subsidize and complement each other. The quantitative studies, performed first, presented the general picture of mental health services and allowed generalizations at the regional level, offering resources for the qualitative to be performed. From this, the quantitative studies were taken as basis and the best results in relation to the markers and the experiences of mental health care were chosen, based on objective criteria, so that the case studies and the fourth evaluation could be performed with a qualitative approach.

From this experience, from a quantitative study to support a qualitative study in the area of health services evaluation, it is important to highlight that health systems are becoming somewhat complex due to several factors (population increase, aging, illness, and budget restrictions, among others). Thus, research problems accompany the complexity of health system issues, challenging researchers to look for more comprehensive forms of research.¹¹ In this way, the mixed method research, which is multidimensional and broader than conventional approaches, contributes to seeking answers to the needs of health systems. Thus, the mixed method research allows researchers to explore complex phenomena in detail,¹¹ respecting the principles of each approach and aggregating to investigate the research object, according to the need.

In this sense, nursing is highlighted in the performance of these studies, since the studies analyzed in this article were coordinated by nurses and it can be seen that nursing concentrates management functions within health services. Corroborating this, a multi-center study points out that the nursing staff is the professional category that performs the most central functions in the health care network and, in the Family Health Strategy (FHS), these professionals are present in all the teams, impacting the quality of care provided.¹⁶

Endorsing this statement, we retrieve a study that verified the contribution of the use of health indicators in the everyday work of FHS nurse managers for health planning and evaluation.¹⁷ In this way, the integration of qualitative and quantitative indicators is important, even in the health practice, advancing beyond their applicability in the research field, offering the understanding of mixed methods to evaluate health indicators.

 

Final considerations

The use of mixed methods in research studies is innovative and has been gaining space in the health area. It is important to note that integration and collaboration within mixed method studies validate it as a powerful and necessary research methodology for health research. 

With this article it was possible to analyze the mixed method studies performed by a research group, presenting the points of integration and convergence between the approaches and discussing how each one favors the evaluation process. Thus, it was possible to demonstrate the main contributions of the integration of quantitative and qualitative approaches in the investigation of a research object, in this case: the evaluation of mental health services.

This study has limitations regarding the analysis of the reality of a specific research group in a specific field of service evaluation. However, it is interesting to enhance studies in this area so that mixed methods are recognized as a viable and promising health methodology.

In addition to the integration and complementation of the approaches, it was possible to verify the mixed and collaborative posture that this methodology offers, providing a complex and comprehensive look at the object to be studied, strengthening the results found. Moreover, these research studies presented contributions with the use of mixed methods to the health area, because the evaluation of the services contributes to the quality of care provided, to the consolidation of the health care network and to nursing care.

 

References

1. Creswell JW. Investigação qualitativa e projeto de pesquisa: escolhendo entre cinco abordagens. Porto Alegre (RS): Penso; 2014.

2. Creswell JW, Clark VLP. Pesquisa de métodos mistos. Porto Alegre (RS): Penso; 2013.

3. Paranhos R, Figueiredo Filho DB, Rocha EC, Silva Júnior JA, Freitas D. Uma introdução aos métodos mistos. Sociologias [Internet]. 2016 maio-ago [acesso em 2019 jun 20];42(18):384-411. Disponível em: http://www.scielo.br/scielo.php?pid=S1517-45222016000200384&script=sci_abstract&tlng=pt doi:10.1590/15174522-018004221

4. Ozawa S, Pongpirul K. 10 best resources on... mixed methods research in health systems. Health Policy Plan [Internet]. 2013 abr [acesso em 2019 jun 18];29(3):323-7. Disponível em: https://www.ncbi.nlm.nih.gov/pubmed/23564372 doi:10.1093/heapol/czt019

5. Schoonenboom J, Johnson RB. How to construct a mixed methods research design. Kolner Z Soz Sozpsychol [Internet]. 2017 [acesso em 2019 jun 18];69:107–31. Disponível em: https://link.springer.com/article/10.1007/s11577-017-0454-1 doi:10.1007/s11577-017-0454-1

6. Guetterman TC, Fetters MD, Creswel JW. Integrating quantitative and qualitative results in health science mixed methods research through joint displays. Ann Fam Med [Internet] 2015 nov-dez [acesso em 2019 jun 20];13(6):554-61. Disponível em: https://www.ncbi.nlm.nih.gov/pubmed/26553895 doi:10.1370/afm.1865

7. Sá-Silva JR, Almeida CD, Guindani JF. Pesquisa documental: pistas teóricas e metodológicas. Rev Bras Hist Ciênc Soc [Internet]. 2009 jan-jun [acesso em 2019 maio 11];1(1):1-15. Disponível em: https://www.rbhcs.com/rbhcs/article/view/6/pdf

8. Garcia Junior EF, Medeiros S, Augusta C. Análise documental: uma metodologia da pesquisa para a Ciência da Informação. Temática [Internet]. 2017 jul [acesso em 2019 maio 11];13(7):138-50. Disponível em: http://www.periodicos.ufpb.br/ojs2/index.php/tematica/article/view/35383/18042

9. Silva HCC, Siqueira AO, Araújo MAV, Dornelas JS. Sejamos pragmáticos: pesquisas em sistemas de informação com relevância e rigor. iSys Rev Bras Sist Inf [Internet]. 2017 dez [acesso em 2019 jun 24];10(4):66-79. Disponível em: http://www.seer.unirio.br/index.php/isys/article/view/6664/6252

10. Dal-Farra RA, Fetters MD. Recentes avanços nas pesquisas com métodos mistos: aplicações nas áreas de Educação e Ensino. Acta Sci [Internet]. 2017 maio-jun [acesso em 2019 jun 15];19(3):466-92. Disponível em: https://www.researchgate.net/publication/325090710_Recentes_avancos_nas_pesquisas_com_metodos_mistos_aplicacoes_nas_areas_de_Educacao_e_Ensino_Recent_advances_in_mixed_methods_research_Applications_in_the_fields_of_Education_and_Teaching

11. Halcomb E, Hickman L. Mixed methods research. Nurs Stand [Internet]. 2015 [acesso em 2019 maio 20];29(32):41-7. Disponível em: https://ro.uow.edu.au/cgi/viewcontent.cgi?article=3676&context=smhpapers doi:10.7748/ns.29.32.41.e8858

12. Sautu R. Todo es teoría: objetivos y métodos de investigación. Buenos Aires: Lumiere; 2005.

13. Cardano M. Métodos qualitativos para pesquisa em saúde. J Nurs Health [Internet]. 2017 set-dez [acesso em 2019 jun 18];7(3):e177306. Disponível em: https://periodicos.ufpel.edu.br/ojs2/index.php/enfermagem/article/view/12847/8061

14. Galvão MCB, Pluye P, Ricarte ILM. Métodos de pesquisa mistos e revisões de literatura mistas: conceitos, construção e critérios de avaliação. InCID Rev Ciênc Inf Doc [Internet]. 2017 set-fev [acesso em 2019 jun 2018];8(2):4-24. Disponível em: http://www.revistas.usp.br/incid/article/view/121879/133611 doi: 10.11606/issn.2178-2075.v8i2p4-24

15. Almalk S. Integrating quantitative and qualitative data in mixed methods research: challenges and benefits. J Educ Learn [Internet]. 2016 jul [acesso em 2019 jun 2018];5(3):288-96. Disponível em: https://files.eric.ed.gov/fulltext/EJ1110464.pdf doi:10.5539/jel.v5n3p288

16. Pires DEP, Machado RR, Soratto J, Schere MA, Gonçalves ASR, Trindade LL. Cargas de trabalho da enfermagem na saúde da família: implicações no acesso universal. Rev Latinoam Enferm [Internet]. 2016 mar [acesso em 2019 jun 21];24:e2677. Disponível em: http://www.scielo.br/pdf/rlae/v24/pt_0104-1169-rlae-0992-2682.pdf doi:10.1590/1518-8345.0992.2682

17. Paes LG, Bellato TMS, Machado BP, Lima SBS. O uso de indicadores como ferramenta de gestão na estratégia saúde da família. Rev Enferm UFSM [Internet]. 2015 jan-mar [acesso em 2019 jun 21];5(1):40-9. Disponível em: https://periodicos.ufsm.br/reufsm/article/view/14150/pdf doi:10.5902/2179769214150

 

Corresponding author:

Guilherme Emanuel Weiss Pinheiro

E-mail: guilhermeewp@politecnico.ufsm.br

Address: Avenida Roraima, n° 1000, Campus UFSM, Prédio 70 G, sala G321A, Bairro Camobi, Santa Maria - RS

ZIP CODE: 97105-900

 

 

Authorship Contributions

1 – Guilherme Emanuel Weiss Pinheiro

Conception and planning of the research, analysis and interpretation of data, writing and critical review.

 

Author 2 – Angela Maria Pereira dos Santos

Analysis and interpretation of data and critical review.

 

3 – Luciane Prado Kantorski

Conception and planning of the research, analysis and interpretation of data, writing and critical review.

 

 

 

How to cite this article

Pinheiro GEW, Santos AMP, Kantorski LP. Analysis of the production of mixed method studies in the evaluation of mental health services. Rev. Enferm. UFSM. 2019 [Accessed at: Year Month Day];vol.9 (Esp) e3: 1-19. DOI:https://doi.org/10.5902/2179769238707