Resumen: Objetivos: evaluar la dimensionalidad y la consistencia interna de la escala de creencias; verificar la existencia de relaciones entre variables criterio y variables creencias; investigar variables predictoras de creencias en el proceso de evaluación de acciones educativas del equipo de salud. Método: estudio cuantitativo correlacional, desarrollado en 14 instituciones de salud, con la participación de 992 trabajadores, en el cual se aplicó escala Likert, compuesta por 31 artículos relacionados a las creencias del equipo en el proceso de evaluación de entrenamientos. La dimensionalidad/estructura interna de la escala fue realizada por medio de análisis factoriales exploratorios; también se aplicó análisis de regresión múltiple estándar, pruebas t de Student, ANOVA y análisis post hoc. Resultados: encontrados tres factores; las variables predictoras fueron "número de entrenamientos en los 12 meses", "participación en entrenamiento en el momento de la investigación" y "edad". Conclusiones: la escala presentó consistencia interna y hubo identificación de correlaciones y variables predictoras.

Descriptores: Desarrollo de personal; Educación continuada; Evaluación educacional

Introduction

One of the advances in research on Training, Development and Education (TD&E) concerns the identification of factors that influence its effectiveness. For this reason, professionals working in people management need to improve their knowledge about how people deal with empowerment actions and what can be done to enhance their effect.¹

In recent years, researchers have been increasingly interested in investigating the relationship between trainee actions and the transfer of training, which can be defined as the degree to which participants apply their knowledge, skills and attitudes acquired in the educational action, thus affecting the degree to which what has been learned is transferred to the work context. In other words, their conduct is due to existing beliefs at various levels.¹

To believe is an inherent condition of the human race; it is a feeling, a subjectivity, the sense of conviction has the same epistemological root of the word opinion or doxa in Greek. Therefore, to believe is to take something to be certain truth, a conviction for which there is no doubt or uncertainty, and there is no need for social or scientific approval. It is, therefore, a conclusion arising from an internal reflective process of free observation about a fact, event, or rule. Based on this premise, a belief can be shared by an individual, or a group of people, transposing time, languages, cultures, environment, and social rules.²

Beliefs are basic cognitive structures upon which actions are based and are therefore essential to promote change in already established attitudes. In health work, this change requires a continuous effort and well-structured educational processes, enabling in practice the principles of continuing education in health.^2-3

Educational actions directed at health workers are important to promote the quality and safety of care; for this, it is advisable to determine whether these actions are achieving their objectives, which should be achieved through systematic evaluation.^3-4

In this way, it is essential to know the results of educational actions, and evaluation is the process indicated for this. However, the evaluation of results from training programs are the least developed aspects in the education proposals and, while their importance is acknowledged, their effective realization and allocation of resources is secondary.^4-5

There are still few studies that address the professionals' beliefs in the training assessment process, with this being one of the least developed stages in TD&E systems. Considering such evaluation as the end of a cycle and beginning of another, the investigation of the professionals’ beliefs in this stage adds information pertinent to the expected effectiveness in the educational process and whether the results obtained are subsequently applied in the workplace.⁵

Thus, the object of this study is to investigate the beliefs of health professionals in the evaluation process of training and institutional development.

In addition, there are distinct mechanisms of cognitive functioning based on individual beliefs about the adjustment to the world of work, which motivates the understanding of associations between variables.⁶ Assuming also that beliefs come from people's experiences and life events, the question was asked whether variables such as gender, age and academic background are related to the beliefs of individuals in the evaluation process.

Inspection of the relationships between these variables could support the promotion of managerial strategies for the refinement of the entire process.

Thus, the objectives of this study were: to evaluate the dimensionality and internal consistency of the belief scale; to verify the existence of relations between the criterion variables and belief variables and to investigate predictive variables of beliefs in the evaluation process of the health team’s educational actions.

Method

This study integrates the research “Beliefs of the health team in institutional educational actions”, which received financial support from CNPq-Edital Universal 2011/13.

With a cross-sectional cut and a correlational design, the present study was developed in 14 health institutions in the city of São Paulo and Sorocaba, both in São Paulo State, chosen considering the practical teaching link with the University of São Paulo (USP) Nursing School and with members of the Research Group registered at the National Council for Scientific and Technological Development (CNPq) – “Nucleus of studies and research on psychosocial aspects of teaching and management in nursing and health” (NEPAPEGES) which performed this study.

Of the 14 participating institutions, eight represented primary care (19.7%); four tertiary care (50.3%) and two secondary care (30.0%); nine are from the public health network and five in the private sector; all have structured human resources or education services, provide regular courses and formal reaction and learning assessments, and four of which also employ training impact assessment.

The population comprised 3539 health workers participating in educational programs of the institutions, including professionals from the middle and higher levels in the areas of nursing, medicine, pharmacy, nutrition, odontology and others. The sample was random, proportional to the size of the strata, per unit of work and function, using the Proportional Stratified Sampling method.⁷ Estimation was also used considering the finite population, with the 5% significance level and 2% maximum relative error above or below the estimated mean score.⁸ The sample resulted in 992 participants, distributed per institution as shown in Table 1.

Table 1- Distribution of the participants according to Institution, São Paulo/SP, Brazil, 2014. (N=992)

Institution	Frequency	%
1	19	1.91
2	11	1.10
3	22	2.21
4	15	1.51
5	27	2.72
6	3	0.30
7	5	0.50
8	85	8.57
9	240	24.19
10	95	9.58
11	101	10.18
12	134	13.51
13	79	7.96
14	156	15.73
Total valid	992	100

The instrument used the Beliefs Scale in the Process of Training Evaluation (BSPE), as constructed in an original study⁹, which verified the evidence of content validity for its items.¹⁰ On this occasion, the instrument was not applied, therefore, it was not submitted to statistical analysis. It was used for the first time in the main research, from which the present study originated and in which the first analyses were performed to verify evidence of internal consistency and dimensionality.¹¹

Originally constituted by 31 items related to the health team beliefs regarding the training evaluation process, answered on a Five-point Likert scale: Strongly Disagree, Disagree, Neither Agree nor Disagree, Agree, and Strongly Agree. Thus, the participants were able to express favorable and unfavorable beliefs regarding the system of training and development implemented in these institutions. In addition to the BSPE items, the data collection protocol presented questions of sociodemographic characterization, as well as questions about participation in trainings, namely: age; sex; qualification and time qualified; unit, time and work shift; work function and time in work function; number of participations in training and number of hours of training in the last 12 months and if they were participating in training at the time of data collection.

The data collection protocol was distributed by the researchers to study participants and collected approximately 10 days later. The data collection extended throughout years 2012 and 2013, due to the diversity and availability of each participating institution.

Regarding data analysis procedures, the internal dimensionality/structure of the instrument was evaluated first by means of exploratory factorial analysis (EFA), performed using the Exploratory Structural Equation Modeling (ESEM)¹² method in Mplus 7 software. The indication for number of factors to be extracted was given by results of parallel analyses.^13,14 The factorial analyses were performed with the polychoric correlation matrix as information source, the robust weighted least square mean and variance method and GEOMIN oblique rotation. Saturation ≥ 0.40 was used as criterion for the maintenance of items in the factors. The following indexes of goodness of fit for the factorial model were also defined a priori: Comparative Fit Index (CFI), values ≥0.90 indicate an acceptable fit; The root mean square error of approximation (RMSEA) and its standardized version (SRMR), according to which values ≤ 0.06 indicate fit, with a maximum acceptable limit of 0.08¹⁵. For each retained dimension, internal consistency coefficients, Cronbach's alpha, were calculated adopting values ≥ 0.60 as satisfactory for initial/exploratory studies.¹⁵

Then, using Statistical Package for Social Sciences 23® (SPSS), scores were created to represent the mean of the answers given to items that were grouped in the dimensions/factors given by the measuring instrument. Once the assumptions of the regression analyzes were verified, these scores were used as criterion variables in standard multiple regression analyses, thus reproducing the procedures adopted in a similar study.¹⁶The following predictive variables were defined: age, time qualified, time in unit, number and hours of training in the last 12 months and participation in training at the time of the survey. Due to the plurality of the variables training, schooling and function, these were transformed into dummy variables to make the regression analysis feasible.

Finally, in order to evaluate the existence of significant differences in the scores of the factors of the belief scale according to the sample characteristics also using SPSS, two types of statistical tests were performed: Student's t test for comparison of two groups and ANOVA for comparison of three or more groups. To better understand the differences identified through ANOVA, post hoc analyses were used. In those cases where a homoscedastic pattern was observed among the variables under comparison, the Tukey's honestly significant difference test for multiple comparisons was used. While in those cases in which the variance between the groups was not homogeneous, Dunnett’s t3¹⁷ comparison test was used. The 5% significance level was adopted for all tests.

The research project was approved in 2011 by the Research Ethics Committee (REC) of the University of São Paulo Nursing School - Proponent institution, under protocol nº 1043/2011 and by the REC of the participating institutions. The research was performed in compliance to the ethical standards required, with explanations given in the term of free and informed consent on the objectives of the study and methods employed, voluntary participation, absence of costs and damages, respect for the principles of confidentiality and secrecy, and right of refusal at any time without loss to the participant.

Results

Regarding gender of the participants, 80.2% (n = 796) were female, 19.4% (n = 192) male and 0.4% (n = 4) did not answer. Their age ranged from 18 to 70 years (mean = 37 years, SD = 8.98); the prevalent age group was 41 years or older (30.1%, n = 299), followed by up to 30 years (26.5%, n = 263). Study participants aged over 31 years accounted for 70% (n = 695). The grouping of the educational qualification variable was performed by level of schooling and resulted in 49.5% (n = 491) of secondary level; 38.8% (n = 385) of higher level; 3.8% (n = 38) elementary school; 0.9% (n = 9) graduate; 7.0% (n = 69) did not respond. The distribution of participants according to time qualified ranged from 0.5 to 40 years (mean = 9.2, SD = 7.6), with prevalence in the range of qualified for up to five years (26.1%; n = 259)⁷.

Regarding work shifts, the distribution was as follows: morning (29.0%, n = 288); night (28.6%; n = 284); integral period (20.9%, n = 207) and afternoon (16.9%, n = 168). As for the role of participants, nursing technician predominated (41.3%, n = 410), followed by nurses (23.7%, n = 235); nursing assistant (9.5%; n = 94); physical educator (6.1%; n = 61); head nurse (5.54%, n = 55); community health agent (4.3%; n = 43); physician (1.4%; n = 14); nutritionist (1.4%; n = 14); the other functions such as psychologist, pharmacist, dentist, physiotherapist, operational technician and pharmacy corresponded to less than 1.0% each.

Due to the preponderance of nursing staff and also because this area traditionally develops structured educational actions, the following grouping was realized: nursing staff (80.0%, n = 794) and other functions (17.0%; n = 169); data lost 3.0% (n = 29). The functions with and without a managerial position were also grouped within the nursing team, of which 5.5% (n = 55) of nurses in leadership or head positions and 74.5% (n = 739) without these positions (nurses, technicians and nursing assistants), 20.0% (n = 198) of the data were lost.

The number of trainings undertaken by the participants in the last 12 months ranged from 0 (zero) to 60 (mean = 5.7, SD = 5.9). The range of up to 60, which at first seems excessive, is due to Institution 10 considering all the educational actions carried out in the daily tasks to be training. For this variable, the participants were divided into the following groups: no training (6.7%; n = 66); between one and two trainings (22.9%, n = 227); three to four (13.5%, n = 134); five to ten (20.3%, n = 201); 11 or more (15.3%, n = 152); data lost/without information (21.3%, n = 212). For the amount of training hours in the last 12 months, the range was 0 (zero) to 264 hours (mean = 15.8, SD = 19.8), with a more frequent interval of up to 6 hours of training (24.0%, n = 238). At the time of data collection, 65.2% (n = 647) did not participate in training, 28.1% (n = 279) participated and 6.7% (n = 66) did not answer.

Analyses of the dimensionality of the scale indicated the solution of three factors as the one that produced the best theoretical and empirical fit. The goodness of fit index for the factorial model obtained for this three-factor solution were the following: CFI = 0.91; RMSEA = 0.057 [CI 90%: 0.054 - 0.060]; SRMR = 0.048. This solution was the one that presented the lowest number of items with crossed loads. Considering the criterion of maintenance of items, those whose factor loads were ≥ 0.40, eight items of the instrument (items 05, 09, 15, 19, 24, 25, 28 and 30) were excluded, leaving the final version comprising 23 items, distributed in three factors, denominated as⁷:

Factor 1. Purpose and results of the evaluation process with 16 items; amplitude of factorial loads from 0.40 to 0.78; Cronbach's alpha = 0.82. Expresses beliefs favorable to the techniques and results of reaction and learning evaluations used in the institutions;

Factor 2. Implementation perspectives of the impact assessment with 3 items; amplitude of the factorial loads of 0.50 to 0.74; Cronbach's alpha = 0.61. Describes beliefs favorable to the evaluations made in the daily tasks by the management, for the performance and application of that learned in professional practice;

Factor 3. Individual interferences in the evaluation measures with 04 items; amplitude of factorial loads from 0.50 to 0.70; Cronbach's alpha = 0.60. Summing unfavorable beliefs to the evaluation process, such as lack of commitment, lack of attention and sincerity in the appraisal of the evaluation criteria.

In the procedure for creating factor scores, items representing Factor 3, which evaluate negative beliefs, were reversed so that the interpretation of the factors could be in the same direction as the other factors, that is, the higher the score, the greater the agreement with the item and stronger beliefs. The means obtained for the BSPE factors, considering the total sample, were as follows: Factor 1 = 4.00; Factor 2 = 3.902 and Factor 3 = 3.98, suggesting beliefs favorable to the evaluation processes.

For the regression analysis, considering that the elevated figure of data lost in some predictive variables was high, the listwise deletion treatment was chosen as the one capable of producing safer and more adequate results. Thus 450 cases, with complete information for all the variables that entered the equation, were maintained for analysis. Three regressions were performed, each of them having as criterion variable one of the three factors measured by BSPE. In general the basic assumptions of multiple regression were respected, however small deviations from normality were observed in the distribution of the criterion variables. The relevance of applying transformations to these variables was tested, but this strategy did not contribute to an improvement in the regression model, so the results are reported considering the gross variables, as presented in Table 2.

Table 2 - Standard multiple regression analyses for beliefs in the evaluation process, São Paulo/SP, Brazil, 2014. (n = 450)

Predictive Variables	Criterion variables
	Factor 1- Purpose and results of the evaluation process			Factor 2 – Perspectives for implementation of the impact evaluation			Factor 3 – Individual interferences in the evaluation measures
	B	β	sr²	B	β	sr²	B	β	sr²
A	0.01	0.04	0.03	0.01*	0.14	0.12	-0.01	-0.14	-0.11
B	-0.01	-0.12	-0.09	0.01	0.03	0.02	0.02	0.21	0.15
C	-0.01	-0.05	-0.04	-0.01	-0.05	-0.04	-0.01	-0.12	-0.09
D	0.01*	0.13	0.11	-0.01	-0.01	-0.01	0.01	0.01	0.01
E	-0.01	-0.07	-0.06	-0.01	-0.07	-0.07	0.01	-0.01	-0.01
F	-0.09**	-0.11	-0.11	0.07	0.06	0.06	0.04	0.03	0.03
Summary	m = 3.99; SD=0.85; [F = 3.375; p ≤ 0.01]; R ²= 0.044; R²adjusted = 0.031			m = 3.91; SD=0.58; [F = 2.100; p ≤ 0.05]; R²= 0.028; R²adjusted = 0.015.			m = 1.95; SD=0.65; [F = 1.388; p ≤ 0.218]; R²= 0.018; R²adjusted = 0.005

*p<0.01; **p<0.05

Legend for predictive variables: A = Age in years; B = Formation time; C = Time in unit (years); D = Number of trainings performed in the last 12 months; E = Hours of training in the last 12 months; F = Participation in a training at the time of collection.

According to results presented in Table 2, the variables “number of trainings participated in over the last 12 months” and if “participating in training at the time of the research” were able to generate significant regression weights for the prediction of beliefs of Factor 1, explaining 3.1% of the variations in the beliefs scores measured in this factor. For Factor 2, the “age” variable was the only one capable of generating significant regression weights, with an explanation of 1.5% of the variations in these beliefs. Regarding Factor 3, none of the on-screen variables were able to produce significant predictive results in the regression model.

Finally, group comparison analyses revealed that participants differ significantly in terms of their beliefs in various group configurations. Regarding gender, male participants (mean = 4.06) presented a significantly higher score in Factor 1 - Belief in the purpose and results of the evaluation process (t = -1.965; gl = 976; p ≤ 0.05) in comparison with the female group (mean = 3.99). In Factor 2 - Implementation perspectives for the impact assessment (t = 1.925; gl = 975; p ≤ 0.05), the female sex (mean = 3.92) presented a significantly higher mean than the male group (mean = 3.82). No significant difference was observed for Factor 3. However, the difference in the size of the groups is emphasized. In relation to age group, the youngest group, up to 30 years old (mean = 4.07), presented significantly higher mean in Factor 3 - Individual interferences in the evaluation measures [F (3.944) = 3.368, p ≤ 0.018] in comparison with the older age group, 41 years or older (mean = 3.91). No significant differences were observed among other age groups.

Regarding training, Factor 3 - Individual interferences in the assessment measures detected significant differences between the groups [F (3.910) = 4.784, p ≤ 0.003]. The group with postgraduate studies (mean = 4.39) presented a significantly higher mean in comparison with the group with elementary level (m = 3.74). There were no differences between the other groups. For the time qualified variable, no significant differences were found for any of the factors.

The group that was participating in training at the time of the research (m = 4.06) presented a significantly higher score in Factor 1 (t = 2.526; gl = 918; p ≤ 0.015) in relation to the group that was not participating (m = 3.99). There was no significant difference for Factors 2 and 3.

In the grouping of functions, nursing team group and group of other areas, we observed that the nursing group (M = 4.02) had a significantly higher score in Factor 1 (t = 2.387; gl = 952; p ≤ 0.017) in the comparison with groups that perform functions in other areas (M = 3.94). In Factor 2, there was no significant difference between groups. In Factor 3, the nursing group (M = 4.02) presented a significantly higher score (t = 3.998; gl = 952; p ≤ 0.001) when compared to the groups that perform functions in other areas (M = 3.80). In the nursing team divided into two groups for leadership positions and other functions, no significant differences were observed for any of the factors.

In the comparison of participants divided according to work shift, significant differences were observed for scores in Factor 1 [F (3.935) = 6.488, p ≤ 0.001] and Factor 3 [F (3.935) = 5.061, p ≤ 0.002]. The full-time group scored significantly higher in Factor 1, compared to the morning (mean = 4.05) and night (mean = 4.03) groups. In Factor 3, the group that worked nights (mean = 3.88) presented a significantly lower mean in comparison with the afternoon (mean = 4.06) and morning (mean = 4.06) groups.

When participants were compared according to the level of health care in which they acted as a priority, significant differences were observed for all three factors. In Factor 1 [F (2.979) = 5.418, p ≤ 0.005], the primary care group (mean = 3.93) presented a significantly lower mean in comparison with the tertiary care group (mean = 4.04). In Factor 2 [F (2.977) = 3.730, p ≤ 0.024], the primary care group (mean = 3.84) presented scores significantly lower than the secondary care groups (mean = 3.98). Finally, in Factor 3 (F (2.979) = 3.369, p ≤ 0.035), the primary care group (mean = 3.88) presented a significantly lower mean in comparison with the secondary care groups (mean = 4.02).

Discussion

No studies were found in the literature that approached beliefs in the process of training and development evaluation, nor did they focus on the predictive character or associate the criterion variables. For this reason, the findings in this study are discussed in light of research that approaches belief and evaluation constructs and portrays the area of education.

Accordingly, the study¹⁶ about beliefs in the Training, Development and Education System (TD&E) is taken for discussion, as it considered the very meaning of beliefs, in the sense of mental representations arising from experiences that could explain some belief results, such as professional experience, job position and educational level.

Predicting the number of trainings and participating in training on beliefs in the purpose and results of the evaluation ratifies that experience underpins belief. In other words, professionals with more personal experience are better acquainted with education and evaluation actions and, therefore, evaluate those actions more appropriately.

On the other hand, another study indicated that beliefs in results and training processes did not present significant differences between those who participated in training and those who did not participate, suggesting that participation in educational actions did not lead to an increase in beliefs about the results of the training process.¹⁶

Experience is also represented in the prediction of age on beliefs in the feasibility of using impact evaluation.

On the other hand, younger individuals present less negative interference, such as inattention and lack of interest in the evaluation, suggesting, in this case, that greater experience resulted in an accumulation of stigmas.

In this study, as well as in research on beliefs in the training system, the results showed that older people who have more work experience tend to have more positive beliefs³, highlighting that evaluation is a subsystem of this. Therefore, less experience suggests a lack of understanding regarding the importance of evaluation.

Differences observed in beliefs between the sexes should be analyzed with care, since the female group was much larger than the male counterpart.

The fact that individuals with a higher educational level present less negative interference in the assessments indicates, once again, that experience can lead to greater seriousness in its execution. Experience is understood here by the greater frequency with which these professionals are submitted to evaluations and who are thereby more prepared for it.

With this, the higher the level of schooling, the greater the understanding of the importance of evaluation and its potential benefits to practice. On the other hand, a study showed that the perception of impact of training on practice is greater in individuals with lower levels of education, which can be attributed to the fact that they have less possibilities, outside the institution, to see their learning needs attended.¹⁸

Older and more experienced professionals have more positive beliefs in impact assessment. If, on the one hand, less experience suggests a lack of understanding of the importance of evaluation³, on the other, consideration should be given to the need to improve training and assessment strategies to reinforce beliefs among younger individuals. Thus, strategies related to active learning should be promoted that stimulate the participation and criticality of individuals, considering the technological incorporation inherent in the new generations.¹⁹

Regarding the professional classes, in the current research, nursing appears with the highest mean beliefs in the TD&E evaluation process, and this is precisely the professional class that also presents the highest mean of negative opinions in the TD&E evaluation process. This finding may be related to the fact that the nursing team, traditionally, in health institutions, has a structured education service, which affords them greater experience and, supposedly, greater critical capacity, as seen in this study, with greater belief in the purpose and results of the evaluation, imprinting fewer negative feelings.

The fact that night shift professionals present lower belief scores can be understood by the characteristics of their work in hospital institutions, in that the work burden caused by night shifts directly interferes with the development of knowledge, skills and attitudes towards TD&E actions and the consequent evaluation of these actions. Thus, the literature reinforces that educational activities after the night shift show little productive learning due to fatigue among the trainees; however, at the beginning of the shift, participation is low due to the dynamics of the units.²⁰

Emphasizing the long existing dilemma of health institutions on how to train individuals working at night, the suggested action is that training takes place during working hours, thereby avoiding post-shift education actions that may lead to decreased interest and learning.

Secondary and tertiary care levels with a predominant biomedical model include in their scope of work those processes that involve technical procedures and the use of highly complex technological apparatus, which demands more hours of formal educational actions with a technical focus, and thus greater familiarity with the training and evaluation process.

Primary care, in turn, transcends formal training for educational actions, which analyze the social context of health and propose agreements to solve the problem situations during daily work. In this form of education, formal evaluation is not present determining less experience of the professionals in this level of assistance.

In compensation, studies developed with the support of the Ministry of Health, following implementation of the National Policy on Permanent Education in Health, reported that the search for insertion of educational and training actions in primary care exists and points to difficulties related to infrastructure problems and not to the lack of adhesion or interest among health professionals.^21-22

Despite the favorable beliefs presented in this study, showing the fertility to increase new evaluation strategies, one can ignore obstacles related to the training process in the health institutions in general, because there is a lack of focus in the formal training system wherein its three main components - evaluation of needs, planning and execution, and evaluation of results – do not present the connection and interrelationship necessary to ensure effectiveness of the educational action.^23-24

This fact, in practice, points to the existence of weaknesses in the three factors studied - that is, the purpose and results of the evaluation process, perspectives for the implementation of the impact assessment, and individual interferences in the evaluation measures.

This result could imply two questions: although they are not fully realized, professionals appreciate and believe in educational actions to improve their practices. The second question, albeit still little known and divulged, is the evaluation of results and more specifically, the impact of educational actions in practice.

Historically, institutions incorporate into their training processes, an evaluation of immediate results from the educational action, with knowledge tests before and after the training and evaluation of satisfaction afterwards. However, actions to monitor impact and transference to practice are rarer.^16,18

There are few Brazilian studies that relate learning and the impact of training in health institutions. However, it is only through evaluation of the training that it is possible to determine if the TD&E programs are being effective, what aspects facilitate or impair the performance of the trainees involved, and even to what extent these actions offer a financial return for the organizations. Therefore, research and reflection are necessary to evaluate the transfer of learning acquired in training as the real benefit achieved by the trainees and by the organization.¹

Studies that problematize the actions of TD&E for the nursing team report, as the main factors that impede the accomplishment of effective capacitation, the inconstancy of the process, with realization disconnected from the needs of individuals and teams; absence of professionals with the profile of educators; lack of time dimension associated or not to work overload and distancing of the sector that plans and executes the educational program from the care sectors that receive it.^1,3,19

The literature addresses the importance of educational actions, also, for newly admitted professionals, for whom the training should allow the new worker to learn the protocols and guidelines for care and organization, new technologies and incorporate the culture of the institution.^19,25 Since knowledge of the culture can change the patterns of belief.

In addition, although training is considered effective, institutions should be concerned with developing behavioral and outcome assessment strategies as a means to improve the diagnosis of learning needs.²⁴ In this sense, a regular practice of assessing professional performance contributes as a managerial tool for evaluating the results of educational actions.

Performance evaluation demands consideration of the punitive culture that also tends to be present in education evaluations. Participation in training is a normal act for workers, however, being submitted to a knowledge test before the educational action, is not so common and could cause anxiety, thereby distorting the sense of evaluation of learning from the perspective of advancement in the teaching process.

Improvement of the evaluation process, especially in the practical evaluation and dissemination of results, is required to remove a punitive character and meritocracy from the evaluation, therefore, a dialectical culture should be constructed in which the concepts and purposes of the evaluation are discussed with clarity and transparency among the people involved.

This advance will not be immediate, and in addition, interpersonal relations and the still rigid hierarchy in health institutions include relations of power, a non-technical element, but, ethico-political. This signifies that, in addition to the technical options, the evaluation process is a political option, depending on the management model adopted.

Although complex, and even before the evaluation, it is necessary to reflect on the teaching-learning process, aiming at meaningful learning, through active pedagogical strategies.

The active methodologies based on constructivism, institute collaborative positions of teaching and learning and imply new roles; the teacher becomes the mediator and the student assumes the role of antagonist in the process.²⁶

Since epistemologically, belief portrays the subjective component of knowledge and is considered a conviction,²⁷ strengthening beliefs through continuous educational actions will also expand the incentive to participate in educational events and more fecundity to evaluate these actions. Therefore, positive experiences in education and evaluation will reduce the fear of a punitive character and promote its nature of growth.

Conclusions

The instrument used presented dimensionality with theoretical and empirical adjustment of three factors: 1. Individual interferences in the evaluation measures, with Cronbach's Alpha of 0.82; 0.61 and 0.60, respectively, which demonstrates internal consistency.

The influence of criterion variables on the beliefs variables with significant difference found were: Sex - Factors 1 and 2; Age group - Factor 3; Training - Factor 3; participating in training at the time of the research - Factor 1; Grouped function - Factors 1 and 3; Work shift - Factors 1 and 3; Health care level - Factors 1, 2 and 3.

The multiple regression analysis indicated as predictive variables: Factor 1: number of trainings in the last 12 months; participation in training at the time of the research; In Factor 2: age. In Factor 3, there was no predictive result.

Despite the sufficient number of participants for the association analyses, the results were interpreted with caution, since the groups were very different in the number of respondents, which can be considered as a limitation of the study.

Knowledge of beliefs that refer to behavioral and cognitive states related to experience is important to clarify the direction of efforts to particular populations and the achievement of actions to turn negative beliefs into favorable beliefs.

The results of this study outline a scenario favorable to the implementation of improvements in the process of evaluation of educational actions in the health area, since the professionals demonstrated they believe in the practical implications of the evaluation process.

Because it is a little studied theme, there is a need for studies that include intervenient variables to educational and evaluative actions, such as beliefs in the support to the transference of learning, prospecting the individual and organizational impact.

This study clearly ratified how experience influences belief, predictions according to the number of trainings and participating in training exemplify such evidence.

Likewise, it is important to offer more opportunities for educational actions followed by evaluation to broaden the experience, especially of night shift professionals, primary care workers and those working in areas other than nursing. Interprofessional education can contribute to improving educational and evaluative practice.

It is indispensable to affirm, however, that in order to achieve positive beliefs, the learning experience must be significant and the evaluation be recognized as a profitable means of promoting qualitative improvement.

References

1. Balarin CS, Zerbini T, Martins LB. A relação entre suporte à aprendizagem e impacto de treinamento no trabalho. REAd Rev Eletrônica Adm [Internet]. 2014 [acesso em 2017 jun 15]; 2(2):341-70. Disponível em: http://www.scielo.br/scielo.php?pid=S1413-23112014000200341&script=sci_abstract&tlng=pt.

2. Brant SRC, Borges-Andrade JE. Crenças no contexto do trabalho: características da pesquisa nacional e estrangeira. Rev Psicol Organ Trab [Internet]. 2014 [acesso em 2017 jun 15];14(3):292-302. Disponível em: http://pepsic.bvsalud.org/scielo.php?script=sci_arttext&pid=S1984-66572014000300005.

3. Bastos LFL, Ciampone MHT, Mira VL. Avaliação de suporte à transferência e impacto de treinamento no trabalho dos enfermeiros. Rev Latinoam Enferm [Internet]. 2013 [acesso em 2017 abr 03];21(6):1274-81. Disponível em: http://www.scielo.br/pdf/rlae/v21n6/pt_0104-1169-rlae-21-06-01274.pdf.

4. Ferreira RR, Abbad GS. Training needs assessment: where we are and where we should go. BAR Braz Adm Rev [Internet]. 2013 [acesso em 2017 abr 03];10(1):77-99. Disponível em: http://www.scielo.br/scielo.php?script=sci_arttext&pid=S1807-76922013000100006.

5. Otrenti E, Mira VL, Bucchi SM, Borges-Andrade JE. Evaluation of formal educational processes for healthcare professionals. Invest Educ Enferm [Internet]. 2014 [acesso em 2017 maio 24];32(1):103-11. Disponível em: https://www.ncbi.nlm.nih.gov/pubmed/25229909.

6. Frank DH, Wertenbroch K, Maddux WW. Performance pay or redistribution? Cultural differences in just-world beliefs and preferences for wage inequality. Organ Behav Hum Decis Process [Internet]. 2015 [acesso em 2017 mar 05];130:160–70. Disponível em: http://www.sciencedirect.com/science/article/pii/S074959781500028X.

7. Bolfarine H, Bussab WO. Elementos de Amostragem. Rio de Janeiro: Blucher; 2017. Amostragem aleatória simples; p. 61-90.

8. Levy PS, Lemeshow S. Sampling of populations: methods and applications. 4th ed. USA: Wiley-Blackwell; 2009.

9. Mira VL. Avaliação de programas de treinamento e desenvolvimento da equipe de enfermagem de dois hospitais do município de São Paulo [livre-docência]. São Paulo: Escola de Enfermagem da Universidade de São Paulo; 2010. 225 p.

10. American Educational Research Association. American Psychological Association. National Council on Measurement in Education. Standards for educational and psychological testing. Washington (DC); 2014.

11. Mira VL. Relatório de pesquisa apresentado e aprovado pelo CNPq: crenças da equipe de saúde nas ações educativas institucionais. São Paulo; 2014.

12. Aparuhov T, Muthen LK, Muthen BO. Exploratory structural equation modeling. Struct Equ Model Multidiscip J. 2009;16(3):397-438.

13. Horn JL. A rationale and technique for estimating the number of factors in factor analysis. Psychometrika [Internet]. 1965 [acesso em 2015 mar 11];30(1):179-85. Disponível em: http://cda.psych.uiuc.edu/psychometrika_johnson/horn_citation_classic_number_of_factors.pdf.

14. Damasio BF. Uso da análise fatorial exploratória em psicologia. Aval Psico [Internet]. 2012 [acesso em 2015 abr 25];11(2):213-28. Disponível em: http://pepsic.bvsalud.org/pdf/avp/v11n2/v11n2a07.pdf.

15. Kline RB. Principles and practice of Structural Equation Modeling. 3th. The Guilford Press: New York, London; 2011.

16. Lopes JM, Mourão L. Crenças acerca do sistema de treinamento: a predição de variáveis pessoais e funcionais. Estud Psicol [Internet]. 2010 [acesso em 2017 abr 22];27(2):197-206. Disponível em: http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0103-166X2010000200007.

17. Dancey CP, Reidy J. Statistics without maths for psychology. 3th ed. New Jersey: Pearson Education International; 2006.

18. Flores GE, Oliveira DLL, Zocche DAA. Educação permanente no contexto hospitalar: a experiência que ressignifica o cuidado em enfermagem. Trab Educ Saúde [Internet]. 2016 [acesso em 2017 mai 23];14(2):487-504. Disponível em: http://www.scielo.br/scielo.php?script=sci_arttext&pid=S1981-77462016000200487.

19. Cardoso RB, Ferreira BJ, Martins WA, Paludeto SB. Programa de educação permanente para o uso do prontuário eletrônico do paciente na enfermagem. J Health Inform [Internet]. 2017 [acesso em 2017 set 08]; 9(1):25-30. Disponível em: http://www.jhi-sbis.saude.ws/ojs-jhi/index.php/jhi-sbis/article/view/429/302.

20. Oliveira B, Martino MMF. Análise das funções cognitivas e sono na equipe de enfermagem nos turnos diurno e noturno. Rev Gaúch Enferm [Internet]. 2013 [acesso em 2017 jun 15];34(1):30-6. Disponível em: http://www.scielo.br/scielo.php?script=sci_arttext&pid=S1983-14472013000100004.

21. Lemos CLS. Educação Permanente em Saúde no Brasil: educação ou gerenciamento permanente. Ciênc Saúde Colet [Internet]. 2016 [acesso em 2017 ago 13];21(3):913-22. Disponível em: http://www.scielo.br/scielo.php?pid=S1413-81232016000300913&script=sci_abstract&tlng=pt.

22. Pinto HA, Ferla AA, Ceccim RB, Florêncio AR, Matos IB, Barbosa MG et al. Atenção básica e educação permanente em saúde: cenário apontado pelo Programa Nacional de Melhoria do Acesso e da Qualidade da Atenção Básica (PMAQ-AB). Divulg Saúde Debate [Internet]. 2014 [acesso em 2017 set 05];(51):145-60. Disponível em: http://cebes.org.br/site/wp-content/uploads/2014/12/Divulgacao-51.pdf.

23. Abbad GS, Mourão L. Avaliação de necessidades de TD&E: proposição de um novo modelo. Rev Adm Mackenzie [Internet]. 2012 [acesso em 2017 set 23];13(6):107-37. Disponível em: https://www.redalyc.org/html/1954/195424913006/.

24. Mourão L, Gondim SMG, Macêdo MEC, Luna AF. Avaliação de necessidades de treinamento como preditora do impacto do treinamento no trabalho. Rev Psicol Organ Trab [Internet]. 2013 [acesso em 2017 maio 23];13(2):203-16. Disponível em: http://pepsic.bvsalud.org/scielo.php?script=sci_arttext&pid=S1984-66572013000200008.

25. Sapatini, TF, Gasparino RC, Polli L, Oliveira AS. Avaliação de um programa admissional para a equipe de enfermagem. Esc Anna Nery Rev Enferm [Internet]. 2016 [acesso em 2017 jun 22];20(3). Disponível em: http://www.redalyc.org/articulo.oa?id=127745807010.

26. Francisco THA, Vefago YB, Ramos AM, Souza IR. Para além das metodologias ativas: uma reflexão sobre a indução da qualidade em cursos de Administração. Rev Bras Ensino Super [Internet]. 2017 [acesso em 2018 abr. 27];3(2):38-57. Disponível em: https://seer.imed.edu.br/index.php/REBES/article/view/2024. doi: https://doi.org/10.18256/2447-3944.2017.v3i2.2024.

27. Gerhardt TE, Silveira DT [organizadoras]. Métodos de pesquisa. Coordenado pela Universidade Aberta do Brasil – UAB/UFRGS e pelo Curso de Graduação Tecnológica – Planejamento e Gestão para o Desenvolvimento Rural da SEAD/UFRGS. – Porto Alegre: Ed. UFRGS; 2009. Disponível em: http://www.ufrgs.br/cursopgdr/downloadsSerie/derad005.pdf.

Corresponding author: Gabriela Cristina da Costa Gonçalves

E-mail: gabriela.goncalves@usp.br

Endereço: Rua Agostinho Ferreira Campos, 388, Bairro Cidade da Saúde, Itapevi- SP Brazil

CEP: 06693-120

Contributions by the Authors

1 – Gabriela Cristina da Costa Gonçalves

Interpretação dos dados, redação e revisão crítica

2 – Daniela Miori Pascon

Design and planning of the research project, data collection and critical review

3 – Melissa Messias

Design and planning of the research project, obtaining and interpreting data and redaction

4 – Patrícia Tavares dos Santos

Design and planning of the research project and data collection

5 – Everson Cristiano de Abreu Meireles

Data analysis and critical review

6 – Vera Lucia Mira

Design and planning of the research project, redaction and critical review

How to site this article

Gonçalves GCC, Pascon DM, Messias M, Santos PT, Meireles ECA, Mira VL. Predictors of beliefs in the evaluation process of educational actions to the health team. Rev. Enferm. UFSM. 2019 [Acess at: 2019 jun 15];vol ex:1-24. DOI:https://doi.org/10.5902/2179769229250