Universidade Federal de Santa Maria

REGET, Santa Maria, v. 24, e32, 2020

DOI: https://doi.org/10.5902/2236117043409

Received: 08/04/2020 Accepted: 08/04/2020 Published: 13/05/2020

 

https://periodicos.ufsm.br/reget/article/viewFile/43409/html/221152

 

Environmental Management

 

The Role of Organizational and Ethical Factors in Developing Evidence-Based Strategic Decision Making in Hospitals of Tehran University of Medical Sciences in Iran

 

O Papel dos Fatores Organizacionais e éticos no Desenvolvimento da Tomada de Decisão Estratégica Baseada em Evidências em Hospitais da Universidade de Ciências Médicas de Teerã, no Irã

 

 

Samaneh SohrabiI

Saad MahfozpoorII

Shaqayeq VahdatIII

Mahmod MahmodiIV

Mahnaz RabiyiV

 

I Department of Health Services Management, South Tehran Branch, Islamic Azad University, Tehran, Iran - Samanehsohrabi96@gmail.com

II Associate Professor; Health Services Management, South Tehran Branch, Islamic Azad University, Tehran, Iran - smahfoozpour@yahoo.com

IIIAssistant Professor, Health Services Management, South Tehran Branch, Islamic Azad University, Tehran, Iran - Sha_vahdat@yahoo.com

IVProfessor, Department of Statistics, Research Branch, Islamic Azad University, Tehran, Iran - mahmoodim@tums.ac.ir

Assistant Professor, Department of Economics, South Tehran Branch, Islamic Azad University, Tehran, Iran - Dr_mahnaz_rabiei@azad.ac.ir

 

 

Abstract

The present study aims to investigate the role of organizational and ethical factors in developing evidence-based strategic decision making in hospitals. The present study was a cross-sectional and qualitative study. The statistical population of the study included all knowledgeable experts in the field of hospital management in Tehran as well as managers of selected hospitals of Tehran University of Medical Sciences. The research tools included semi-structured interviews and a researcher-made questionnaire. Structural equation method was used to analyze the data of the present study using SPSS and PLS software. The results of this study showed that environmental factors including indoor and outdoor environment as well as organizational factors including educational factors, job factors, factors related to organization considerations, factors related to organization strategies, factors related to decision making, factors related to characteristics The decision process and factors are related to the nature of the decision. The results of the present model showed that environmental and organizational factors influence the development of evidence-based strategic decision making in hospitals.

Keywords: Strategic decision making; Evidence; Hospital; Environment; Organization

 

 

Resumo

O presente estudo tem como objetivo investigar o papel de fatores organizacionais e éticos no desenvolvimento de decisões estratégicas baseadas em evidências em hospitais. O presente estudo foi um estudo transversal e qualitativo. A população estatística do estudo incluiu todos os especialistas no campo da gestão hospitalar em Teerã, bem como gerentes de hospitais selecionados da Universidade de Ciências Médicas de Teerã. As ferramentas de pesquisa incluíram entrevistas semiestruturadas e um questionário elaborado pelo pesquisador. O método da equação estrutural foi utilizado para analisar os dados do presente estudo utilizando os softwares SPSS e PLS. Os resultados deste estudo mostraram que fatores ambientais, incluindo ambiente interno e externo, bem como fatores organizacionais, incluindo fatores educacionais, fatores de trabalho, fatores relacionados às considerações da organização, fatores relacionados às estratégias da organização, fatores relacionados à tomada de decisão, fatores relacionados às características processo e fatores estão relacionados à natureza da decisão. Os resultados do presente modelo mostraram que fatores ambientais e organizacionais influenciam o desenvolvimento de decisões estratégicas baseadas em evidências em hospitais.

Palavras-chave: Tomada de decisão estratégica; Evidência; Hospital; Meio ambiente; Organização

 

 

1 Introduction

Organizations today face numerous opportunities and threats in the path of excellence. Some of them are challenging and managing them requires strategic decision making. Correct decision making can lead to major leaps in the direction of the organization's movements; however, its inaccuracy can slow the organization's progress in the contemporary accelerated world or even disrupt the organization (AHMADIAN, 2008). This is more important in some service organizations, including hospitals. Hospitals need regular coordination and planning because of their different stakeholders and groups. The presence of managers, staff, physicians, nurses, patients, insurers and suppliers in the hospital environment has made strategic decision-making an important issue in that field (DEMIR & OGURLOGLU, 2019).

The optimal management of hospitals as the largest centers of health care delivery has always been the focus of thinkers and policymakers in the related sector. Due to the limited investment resources and its long-term investment in the health sector, as well as factors such as the high cost of hospital construction, expensive equipment, and a lack of skilled human resources, it is extremely difficult to expand hospital facilities. Therefore, maximizing the use of available resources by utilizing top management strategies is inevitable (HARI et al., 2014). On the other hand, hospital administrators must adapt to theories and management practices as they come to grips with the increasing pressures of the internal and external environment and become familiar with new theories. Therefore, all stakeholders of the health system make agreement on the use of optimal management methods, but their disagreement occurs in choosing the best and most appropriate way with the structure and environmental conditions of the organizations in this sector (BRYSON, 2018). The best and most appropriate way of making environmental adjustments, and given the current dramatic developments, is to develop the organization in a way that can better adapt to environmental changes and practices, rather than having a passive state, be active and anticipate events such as economic, political, technological changes, before taking action (BUTANI & RIZZI, 2006).

One of the most important issues in strategic decision making is evidence-based strategic decision making. Evidence-based strategic decision-making means strategic decision-making based on the results of monitoring, measuring and analyzing data. This means that when decisions are based on traceable results and factual information, it is naturally easier to achieve the desired output (MAZUKA et al., 2019). Since 1990, following major developments in medical science, evidence improvement has been identified as a major issue in medicine. This has led to evidence-based strategic decision-making being identified as a necessity in medical science (YARBAR et al., 2015). The importance of managing information, flows, and programs in hospitals has led to the identification of evidence-based strategic management today as one of the important prerequisites for hospital success (KUITANOVIC et al., 2016).

Nowadays, the use of evidence-based decision making has been taken as a new approach to management in many countries. Evidence-based decision making, by providing the best available evidence and documents to decision makers and executives, leads to consciously decisions about programs, policies and projects (DAVIES, 2004). Evidence-based decision making is one of the newest approaches to public decision making. This approach essentially employs research evidence in policy makings. In recent years, evidence-based strategic decision-making has been the focus of many governments and many efforts have been made to take advantage of this new approach to decision-making. Studies show that the common terms of evidence-based strategic decision making are still vague and incomprehensible to managers and those who are directly involved. It is not clear how they use evidence in their strategic decisions (CAMPBELL et al., 2007). In spite of numerous studies, the evidence-based strategic decision-making process has not yet been addressed (SINGH et al., 2018). The existing literature on evidence-based strategic decision-making within the country faces severe poverty both theoretically and empirically. In recent years, this approach has attracted the attention of many theorists of science and administration. But today's research on evidence-based strategic decision-making has focused heavily on theory, and actions to this end appear to be necessary (REAY et al., 2009). In spite of much research on evidence-based medicine, research on evidence-based decision making in health care management and its related outcomes is scarce (CHO ET AL., 2018). So we are also faced with a general lack of transparency in defining what is called evidence-based decision making in health care management. Evidence-based decision making is rooted in knowledge. In other words, knowledge is the root of evidence-based decision making. In fact, knowledge is the means of decision making. Evidence-based decision making is the scientific and research-oriented method of decision making. This approach can provide managers with good scientific justification, but unfortunately, this approach still does not have the necessary impact on the decision making environment in Iran. Many of the justifications for public decisions raised do not yield the desired results. Even the best recommendations may not be adopted or lead to a decision. This has resulted in decisions that are largely ineffective and unable to solve general problems (EBRAHIMI et al., 2014). Several investigations have examined the factors that influence the development of evidence-based strategic decision makings. TURMELL et al. (2017) found in their research that management factors play an important role in improving evidence-based strategic decision making. In a research, RWANGARD (2012) found that creating sufficient background for more scientific and evidence-based decision making is felt by all hospital managers. In their research, Ikeda et al. (2018) found that some structural issues, including the characteristics of the organization's individuals and the generality of the decision-making process in organizations, are one of the important issues in developing and expanding evidence-based strategic decision making. SPERTUS (2018) found in his research that stakeholder pressure plays an important and fundamental role in improving and expanding evidence-based strategic decision making in organizations. WERDIN et al. (2009) identified that management, coordination, and training in hospital settings are important and key factors for improving evidence-based strategic management. According to research conducted in this field, it is clear that there is a lack of research on the factors that influence the development of evidence-based strategic decision making in hospitals, which has hampered strategic decision making in these settings. On the other hand, the environment and the organization have always been instrumental in promoting and improving some aspects of the organization, including evidence-based strategic decision making. The lack of necessary insights into the role of environmental and organizational factors in developing and deploying evidence-based strategic decision-making has led to a lack of planning to improve the environment and the organization in order to promote evidence-based strategic decision-making in hospitals. This issue made the present study to design and execute environmental and organizational factors on the development of evidence-based strategic decision making in Tehran hospitals. Therefore, the main question of the present study is what are the environmental and organizational factors on the development of evidence-based strategic decision making in Tehran hospitals?

 

 

2 Methodology

This research is a combination exploratory research conducted in the field. The first group consisted of academic experts whose data were analyzed by qualitative methods and the second group was the managers of selected hospitals of Medical Sciences Faculty of Tehran University. Managers are at three senior, middle and operational levels, or, in other words, the Chief, Chief Executive Officer, Director of Administration and Hospital Matrons, who have worked in the spring of 2018. 544 of them were working in 136 hospitals in Tehran. The obtained data were used to validate the model of evidence-based strategic decision making using quantitative methods. We selected samples for the first community of academic experts using the snowball sampling method. A total of 7 individuals were interviewed in three stages. For the second population, we used cluster (two-step) sampling method. Cochran formula was used to determine sample size. The sample size was 225 individuals. Thus, in the first phase, 75 hospitals were selected from 136 hospitals randomly and in this hospital, managers of selected hospitals were randomly selected to complete 225 samples.

In the present study, combined method was used to collect the data. During library method, research tools, tables, and forms have been used to exploit resources such as books and articles, types of publications, and statistics to produce and arrange research literature, as well as identify effective variables and indicators. In descriptive research, descriptive information is usually collected through questionnaires, interviews, or observations. In the present study, considering the subject, its purpose and also considering the type of research method, the most important tool for collecting information was questionnaire and interview. According to the mentioned methods, two research tools (questionnaires) were used for conducting the research. The first questionnaire is a qualitative questionnaire (Delphi) which was completed by experts in two different versions and the second is a Likert-type questionnaire completed by the managers of selected hospitals in Tehran. We will discuss about both questionnaires in details.

A qualitative questionnaire was designed for experts in two different versions. The first is a two-part series exploring the environmental and organizational factors extracted from library studies in the first section, and the Likert spectrum has been asked about the effectiveness of evidence-based strategic decision making. In other words, environmental and organizational influencing factors on the evidence-based strategic decision making are addressed. In the second part, it is asked to identify the environmental and organizational factors and components that influence evidence-based strategic decision making. The environmental and organizational factors and components extracted from the qualitative section are presented in Table 1.

 

Table 1 - Results from the qualitative section

Factors

Questions

Environmental factors

Includes internal and external environments

4

Organizational factors

Includes training factors, job factors, factors related to organization considerations, factors related to organization strategies, factors related to decision making, factors related to decision process characteristics, factors related to the nature of decision making, and factors related to the characteristics of senior managers.

18

 

The quantitative research questionnaire is designed in three parts. The first part is the demographics of the respondents, consisting of 5 questions. The second part is a researcher-made questionnaire that measures the components extracted from a qualitative analysis (environmental and organizational factors).

The third part of the questionnaire deals with the measurement of strategic decision making. The questionnaire includes seven dimensions of quality of strategic decision making, accuracy, precision, timeliness, being economical, easiness, security and following-up of decision making. The questionnaire consists of 23 questions and is designed in the form of a five-point Likert scale. The reliability of the questionnaire in Akhbarizadeh (2015) was 0.86 which indicates the desirability of the questionnaires reliability. Data analysis was performed in two parts: descriptive statistics and analytical or inferential statistics. For this purpose, Smart-PLS and SPSS 22 software were used.

 

 

3 Results

The descriptive results of the gender of the sample showed that 64.4% of the respondents are female and 35.6% are male. The results related to the educational level of the sample showed that the highest frequency of (42.2%) was bachelor's degree (95 individuals). Postgraduate, PhD, General Practitioner and Specialist degrees are 34, 25, 51 and 20, making up 15.1, 11.1, 22.7 and 8.9% of the total sample, respectively. The results of marital status of the sample showed that 82.2% of the respondents are married and 17.8% are single. Also, the results of the work experience of the sample showed that the highest record was for the group of 6 to 10 years old, with 62 persons being 27.56%. The results of the age group of the sample showed that the highest frequency was in the age group of 45-35, with a frequency of 84 persons, 37.33% of the sample, and the age group of 25-35, followed by 72 people made up 32.0% of the sample. Structural equation method was used to investigate the research objectives. Figure 1 shows the model presented in the present study.

 

Figure 1 - Standard Coefficients Research Model

https://periodicos.ufsm.br/reget/article/viewFile/43409/html/221153

 

Figure 2 - Research model in t mode

https://periodicos.ufsm.br/reget/article/viewFile/43409/html/221154

 

Figure 1: Path coefficient between independent and dependent variables and within parentheses Significant coefficient

Information about the research model is summarized in Table 2.

 

Table 2 - Path coefficient and significance of the relationship between independent and dependent variables

Relationships

Impact

t

Environmental factors on The correctness of decision making

0.384

2.952

Environmental factors on The accuracy of decision making

0.707

3.850

Environmental factors on Timeliness of decision making

0.335

1.061

Environmental factors on Economical being of decision making

0.725

4.484

Environmental factors on Easiness of decision making

0.488

3.220

Environmental factors on Security of decision making

0.312

1.946

Environmental factors on Follow up on strategic decision making

0.820

6.420

Organizational factors on The correctness of decision making

0.498

3.734

Organizational factors on The accuracy of decision making

0.110

0.545

Organizational factors on Timeliness of decision making

0.373

1.270

Organizational factors on Economics of decision making

0.090

0.550

Organizational factors on Ease of decision making

0.357

2.327

Organizational factors on Security of decision making

0.546

3.636

Organizational factors on Follow up on strategic decision making

0.114

0.829

 

As it can be seen, all path coefficients are positive. On the other hand, the criterion for meaningful relationships is t. If the t value is greater than 1.96, it can be stated that the relationships are significant. The results showed that the effect of environmental factors on decision correctness, decision accuracy, being economical, easiness of decision making and strategic decision-making due to having a t value greater than 1.96 was significant. The results also showed that the effect of organizational factors on accuracy decision making, easiness of decision making and security due to having a t-value greater than 1.96 was significant.

Structural equation modeling with a PLS approach were used to evaluate the suitability of fitting indices in the present research model. Table 3 shows the results of the fitting indices of the above model.

 

Table 3 - Fitting indices of the research model

 

Environmental factors

Organizational factors

Correctness

Accuracy

Timeliness

Being economical

Easiness

Security

Following up

Cronbach's alpha

0.83

0.85

0.87

0.88

0.89

0.82

0.84

0.84

0.85

Combined reliability

0.85

0.87

0.89

0.89

0.94

0.93

0.89

0.92

0.90

Convergent validity

0.62

0.62

0.63

0.63

0.60

0.72

0.71

0.72

0.70

Gof

-

-

0.44

0.40

0.33

0.38

0.35

0.40

0.49

 

The results of the fitting indices showed that the value of Cronbach's alpha and the combined reliability of the variables were higher than 0.7 in all factors, indicating a good reliability of the model. The results also showed that the convergent validity of all factors was above the criterion level of 0.4 which indicates that the convergent validity of the research is appropriate. Also, according to Figure 1, all factor loadings related to the research factors are at an appropriate level, indicating that this criterion is appropriate. The GOF also indicates that the index is appropriate. It was found that the model presented in the present study has good fitness.

 

4 Discussion and Conclusion

The present study showed that environmental factors including internal and external environments as well as organizational factors including educational and job factors, factors related to organization considerations, factors related to organization strategies, factors related to decision making, factors related to process characteristics, factors related to the nature of decision-making, and attributes of senior management characteristics influence some aspects of evidence-based strategic decision-making in hospitals. The identified factors indicate that in order to improve evidence-based strategic decision-making in hospitals in Tehran, various issues need to be addressed. In other words, different areas are involved in strategic decision making based on evidence in Tehran hospitals, which indicates the necessity of multi-dimensional approaches. In a research, IKEDA et al. (2018) indicated that multi-dimensional planning in different organizational domains is needed to improve evidence-based strategic decision making. CHOW et al. (2018) also found that in order to enhance the status of evidence-based strategic decision-making, different areas need to be considered and organizational managers consider different dimensions. Therefore, hospital managers in Tehran must take one-dimensional view of strategic decision-making away in hospitals and examine all aspects of organizational development to enhance evidence-based strategic decision making.

The results of the present study showed that environmental factors have a significant impact on decision making correctness, decision accuracy, economical decisions, easiness of decision making and strategic decision-making. CHOW et al. (2018) and WERDIN et al. (2009) in their research concluded that in order to implement evidence-based strategic decision making, environmental issues including the internal environment of organizations play a decisive role. SPERTUS (2018) also identified that the internal environment should be considered by the managers of organizations because it is one of the basic structural needed in evidence-based strategic decision making. It seems that development of evidence-based strategic decision-making in hospitals in Tehran requires attention to environmental issues. This is due to the decisive role of the environment, especially the internal environment of organizations. In other words, making decisions and doing activities in organizations, requires the integration of internal environment as one of the most important and essential steps. Having appropriate environment, one can expect that hospitals in Tehran take evidence-based decision making in their priority.

The present study also found that organizational factors such as accuracy, correctness, easiness and security of decision making have significant impact on evidence-based decision making. IKEDA et al. (2018) identified that educational issues play an important role in the development of evidence-based strategic decision making. SPERTUS (2018) also found that training issues are one of the most important strategies for developing effective factors for evidence-based strategic decision making and its implementation in organizations. Therefore, it can be stated that the results of the present study are inconsistent with those of IKEDA et al. (2018) and SPERTUS (2018). It seems that substantial differences in some of the cultural, social, and structural aspects of Tehran's hospitals and the environments studied in this study appear to have caused this discrepancy. Although educational factors have been identified in various studies to improve the development of evidence-based strategic decision making, but in the present study this issue has not been identified as a priority. It seems that in the current environment of Tehran hospitals, educational issues do not develop evidence-based strategic decision-making. This may be due to the disadvantage of some of the more important aspects such as environmental, occupational and ethical issues in hospital settings in Tehran. In other words, for the development of evidence-based strategic decision-making, different situations are needed which, given the status of hospitals in Tehran, educational factors are less important because of the necessity of some other factors.

According to the results of this study, it is suggested to pay more attention to indoor dimensions in order to develop strategic evidence-based decision making in Tehran hospitals. To this end, hospital managers in Tehran monitor the conditions for the development of evidence-based strategic decision-making by regularly reviewing and evaluating the hospital environment. It is also recommended that hospitals, with appropriate facilities for data analysis, provide the infrastructure needed to develop evidence-based strategic decision-making. On the other hand, it is suggested that by combining evidence-based strategic decision-making goals with hospital job descriptions, it will be possible to incorporate all elements of hospitals with evidence-based strategic decision-making.

 

References

Ahmadin A. Strategic Decision Making in a Nongovernmental Organization, Fifth International Management Conference, Tehran, Research Group, 2008.

 

Bottani E, Rizzi A. Strategic management of logistics service: A fuzzy QFD approach. International journal of production economics. 11(3), 585-99, 2006.

 

Bryson JM. Strategic planning for public and nonprofit organizations: A guide to strengthening and sustaining organizational achievement. John Wiley & Sons. 14(4),66-79, 2018.

 

Campbell S, Benita S, Coates E, Davies P, Penn, G. Analysis for policy: evidence-based policy in practice. Government Social Research Unit, HM Treasury, 2007.

 

Chow N, Gallo L, Busse JW. Evidence-based medicine and precision medicine: Complementary approaches to clinical decision-making. Precision Clinical Medicine. 17(2), 60-64, 2018.

 

Cvitanovic C, McDonald J, Hobday AJ. From science to action: principles for undertaking environmental research that enables knowledge exchange and evidence-based decision-making. Journal of Environmental Management. 18(3), 864-74m 2016.

 

Demir IB, Ugurluoglu O. Evaluation of the Use of Strategic Management Tools by Hospital Executives in Turkey. Journal of Health Management. 21, 76-88, 2019.

 

Ebrahimi, Abbas; Danaei Fard, Hassan; Azar, Adel; Forouzandeh, Lotfollah. Designing an Evidence-Based Policy Model in Iranian Health System (Qualitative Research), Public Management Research, 7(24), 5-24, 2014.

 

Hari M, Nasr Esfahani F, Ferdowsi M. Prioritizing Hospital Strategies and Isfahan Sina Heart Center by Analytical Hierarchy Process, 5 (2), 53-43, 2014.

 

Ikeda AK, Hong P, Ishman SL, Joe SA, Randolph GW, Shin JJ. Evidence-based medicine in otolaryngology part 7: introduction to shared decision making. OtolaryngologyHead and Neck Surgery. 158(4), 586-93, 2018.

 

Mazzucca S, Parks RG, Tabak RG, Allen P, Dobbins M, Stamatakis KA, Brownson RC. Assessing Organizational Supports for Evidence-Based Decision Making in Local Public Health Departments in the United States: Development and Psychometric Properties of a New Measure. Journal of Public Health Management and Practice. 12, 134-151, 2019.

 

Ravangard R, Keshtkaran V, Nickname S, Yousefi A, Heidari A. Decision-making practices of directors of public and private hospitals in Shiraz. Hospital, 12(3), 39-45, 2012.

 

Reay T, Berta W, Kohn MK. What's the evidence on evidence-based management? Academy of Management Perspectives. 23(4), 5-18, 2009.

 

Singh S, Zieman S, Go AS, Fortmann SP, Wenger NK, Fleg JL, Radziszewska B, Stone NJ, Zoungas S, Gurwitz JH. Statins for primary prevention in older adults moving toward evidence based decision making. Journal of the American Geriatrics Society. 66(11), 2188-96, 2018.

 

Spertus JA. Understanding How Patients Fare: Insights into the Health Status Patterns of Patients with Coronary Disease and the Future of Evidence-Based Shared Medical Decision-Making, management decision, 14, 89-101, 2017.

 

Turmell JW, Coke L, Catinella R, Hosford T, Majeski A. Alarm fatigue: use of an evidence-based alarm management strategy. Journal of nursing care quality. 1, 32(1), 47-54, 2017.

 

Werdin F, Tennenhaus M, Schaller HE, Rennekampff HO. Evidence-based management strategies for treatment of chronic wounds. Eplasty, 9, 56-66, 2009.

 

Yarber L, Brownson CA, Jacob RR, Baker EA, Jones E, Baumann C, Deshpande AD, Gillespie KN, Scharff DP, Brownson RC. Evaluating a train-the-trainer approach for improving capacity for evidence-based decision making in public health. BMC health services research. 15(1), 547-559, 2015.

 



Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

DEAR AUTHORS,

PLEASE, CHECK CAREFULLY BEFORE YOUR SUBMISSION:

- IF ALL AUTHORS "METADATA" (ORCID, LINK TO LATTES, SHORT BIOGRAPHY, AFFILIATION) WERE ADDED,

- THE CORRECT IDIOM YOUR SECTION,

- IF THE HIGHLIGHTS WERE ADDED,

- IF THE GRAPHIC ABSTRACTS WAS ADDED,

- IF THE REVIEWERS INDICATION WAS DONE,

- IF THE REFERENCES FORMAT ARE CORRECT(ABNT)

- IF THE RESOLUTION YOUR FIGURES (600 DPI) ARE SUITABLE

*******************************

PREZADOS AUTORES,

POR FAVOR, VERIFIQUE ATENTAMENTE ANTES DA SUBMISSÃO: 
- SE OS METADADOS (ORCID, LINK PRO LATTES, CURTA BIOGRAFIA E AFILIAÇÃO) DE "TODOS" OS AUTORES FORAM ADICIONADOS, 
- IDIOMA, 
- SE OS HIGHLIHTS FORAM ADICIONADOS, 
- SE O GRAPHICAL ABSTRACT FOI ADICIONADO, 
- SE A INDICAÇÃO DOS REVISORES FOI FEITA, 
- SE O FORMATO DAS REFERÊNCIAS ESTÁ ADEQUADO (ABNT) 
- SE A RESOLUÇÃO SUAS FIGURAS (600 DPI) ESTÁ ADEQUADA.

 

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

 

**************************************************

WE ARE ON FACEBOOK! (You are our guest!)

 

Digital Object Identifier (DOI): 10.5902/22361170

Contact: reget.ufsm@gmail.com

...................................................................................


Accesses since 19/06/2012

...................................................................................

Sponsors: