Learning, Intention to Transfer and Transfer in Executive Education

AN0095876768;bpu01jun.14;2018Jul09.10:21;v2.2.500

Learning, intention to transfer and transfer in executive education. 

The main aim of this study was to understand the relationship between self‐reported ‘intention’ to transfer and self‐reported ‘actual’ transfer within a population of middle and senior executives on an executive education programme. A secondary objective was to consider the relationship between these qualitative self‐reports of intent and actual transfer and a quantitative measure of knowledge and skill acquisition immediately post programme. The focus is on ‘what’ is transferred as well as changes in behaviour rather than on identifying and measuring the strength of factors associated with the act of transfer. Practical tools as well as frameworks and ideas designed to foster a change of mindset were mentioned most frequently in terms of both transfer and intention to transfer, thus there was found to be a similar relationship between those aspects of a leadership development programme that participants have already transferred back in to the workplace and those they intend to transfer. In consideration of the secondary objective, the themes that received the largest number of nominations for transfer of learning were also comparable to the skills that gained the highest learning change scores on the questionnaire. It is postulated that the types of learning material used in the executive development programme adhere to four basic principles of learning: active engagement, limited but some prior knowledge, clear applicability back in to the workplace and the opportunity for repeated practice. The implications for programme design to maximize transfer of learning for middle and senior executives, focusing on these four principles, are discussed, along with the importance of executives autonomy in their working environment to facilitate the transfer (or intention to transfer) learning.

According to Kelloway and Barling ([32] ), many practitioners feel that organizations should spend more resources on training their employees, and this is echoed by researchers who note that investment in human capital has been regarded as key for competitive advantage (e.g. Adler & Kwon, [1] ; Bontis, [8] ; Storberg‐Walker, [48] ). Given the strategic importance of this investment, and the financial implications (e.g. Merritt, [41] examined over 100 companies across 20 countries and found that $210 million was spent training 21,000 employees in a variety of executive education programmes) it is not surprising that a critical issue for those individuals responsible for commissioning both internal and external training programmes is the notion of learning transfer.

Learning transfer is defined by Ford and Weissbein ([21] ) as the application, generalizability and maintenance of new knowledge and skills to a wide variety of managerial situations, and fundamentally examines the extent to which course content is transferred or applied by participants back in their workplace. Estimates of the extent of transfer range from a pessimistic 10 per cent (Georgenson, [25] ) to approximately 50 per cent (Saks & Belcourt, [46] ) with a significant number of studies raising concerns regarding the level of transfer of learning across executive education programmes (e.g. Baldwin & Ford, [2] ; Burke & Hutchins, [11] , Cheng & Hampson, [13] ).

The Kirkpatrick model

In 1967, Kirkpatrick developed a four level model of training evaluation. Level one (reactions) relates to participants immediate response in terms of their feelings about a programme, declarative knowledge and skill acquisition form part of level two (learning), level three (behaviour) measures learning transfer and level four (results) focuses on reaching end goals such as achieving results, reducing costs, staff turnover etc. The model has subsequently been developed and built upon (including a fifth element, Return on Investment or ROI, Philips, [42] ) yet still provides a useful framework for understanding the different aspects of the learning and transfer process, particularly as the model continues to retain prominence with the learning and development functions in organizations. This may be partly due to the simplicity of the model, allowing for a straightforward understanding of the transfer process in complex organizations, and partly because whilst other models have built upon and extended the initial concept, the fundamentals of the four levels of evaluation remain valid today. Due to the continued adoption of this model within organizations as the method of measuring and explaining transfer of learning in both internal and external training provision, this paper will use the Kirkpatrick ([34] ) framework to explain and understand the transfer of learning process within the study.

Barriers and facilitators of transfer (Kirkpatrick level three)

Given the financial investment in executive education programmes, and the importance of developing competent managers [particularly at a time when there are fewer workers entering the labour market and a significant number of managerial positions are held by baby boomers who are close to retirement (Calo, 2008; Lee, [36] )], the focus of more recent research examining transfer of learning has been on determining:

the barriers and facilitators to the transfer process (classified by Cheng & Hampson, [13] in a review paper as individual, job/career, and situational variables with training outcome variables mediating the previous three and the transfer process);

the relative importance of each of these, and more recently;

the relationship between each within a holistic framework of transfer.

Whilst reviews of the literature on transfer variables are provided in detail by Baldwin and Ford ([2] ), Cheng and Ho ([14] ), Cheng and Hampson ([13] ), Colquitt et al. ([15] ) and Belling et al. ([5] ) amongst others, the most reliable predictors of transfer have been found to include locus of control, conscientiousness and anxiety for Cheng and Hampson's ([13] ) individual dimension (similar to Baldwin & Ford, [2] ‘trainee characteristics’) (e.g. Barrich & Mount, [3] ) and transfer climate including peer and line‐manager support systems and intervention strategies such as goal setting and relapse prevention training (based upon Marx, [38] ) as situational variables (e.g. Burke & Baldwin, [10] , Facteau et al., [20] ; Gist et al., [27] ; Rouiller & Goldstein, [44] ; Xiao, [54] ). Aspects of Baldwin and Ford's ([2] ) ‘work environment’ characteristics are included within this situational variable classification. Job and career factors, however, have been studied less frequently and the results have been less consistent. For example, Tannenbaum et al. ([49] ) found a positive relationship between organizational commitment and motivation to transfer whereas Facteau et al. ([20] ) failed to find such a relationship. The final set of variables classified by Cheng and Hampson ([13] ) as training outcomes, seen to mediate between the individual, job and situational variables and the transfer process, include declarative knowledge, skill acquisition and post‐training self‐efficacy, which have all been shown to predict transfer behaviour (e.g. Ford et al. [23] ; Tannenbaum et al., [49] ).

Measuring Kirkpatrick level two

Whilst there may be a significant literature examining the process of transfer (Kirkpatrick, [34] level three) and the barriers and facilitators of this process including the mediating role of aspects of Kirkpatrick ([34] level two) such as declarative knowledge and skill acquisition, it is often with small sample sizes when using executive education samples (e.g. Kets de Vries et al., [33] ), and frequently focusing on training rather than executive education programmes (e.g. Ford et al. [23] ; Machin & Fogarty, 2003; Mathieu et al., [39] ; Tannenbaum et al. [49] ; Velada et al. [51] ; Warr et al., [53] ).

Measuring learning (Kirkpatrick's level two) such as declarative knowledge, procedural knowledge and skill acquisition is fundamental to understanding transfer because, as argued by Brown and McCracken ([9] ), it is not possible to see transfer without initially considering the context, that is, what is being learned. Whilst the focus of the level three transfer research is on the process (barriers and facilitators, for example), the literature on level two learning pays attention to both how the learning is gained (process) and the nature of the material learnt (content) although, once again, with a focus on non‐executive education training programmes. Those studies that have used executive education often have small sample sizes. For example, Culpin and Scott ([17] ) examined whether ‘hard’ skills such as strategy could be developed and learnt by using a case study approach traditionally used to develop transferable or ‘soft’ skills. They found that this methodology did indeed provide significant learning (as measured by pre‐ versus post‐programme change) but they note the caveat of a small sample size (n = 19).

The importance of understanding what is learnt

There is a large body of previous research examining the acquisition of knowledge and skills immediately post‐programme (learning) in terms of content and what that means for programme design and participant learning and development, but there is very little focus on content per se in the transfer literature other than as a dependent variable (thus not the variable of interest) in studies examining effective transfer processes. Whilst understanding the transfer process, and the barriers and facilitators to this, is key to improving the transfer of learning back in to the workplace, it is also critical that firstly, transfer research examines in more depth what is actually transferred not just how it is transferred and secondly, that the relationship between what is learnt immediately post‐programme (the acquisition of knowledge skills and behaviours in Kirkpatrick's level two) and what is transferred (the use of this acquired knowledge and skill set in the workplace) is scrutinized. Often research that does consider the content transferred does not examine the immediate post‐programme learning to understand the nature of the relationship between the two (e,g. Kets de Vries et al., [33] ). This is particularly important given that Blanchard et al. ([6] ) notes that organizations prefer to measure the first two levels of the Kirkpatrick ([34] ) model than transfer or results (levels three and four). If the knowledge and skills being acquired immediately post‐programme are similar to those which are subsequently transferred, focusing solely on levels one and two to evaluate programme success is of less concern than if there is little relationship in terms of content between immediate learning and transfer once back at work.

Research rationale

This research was designed to investigate whether a series of interventions during a large multi‐national executive education programme (over 400 participants) would lead to:

a classroom experience that both results in changes in learning [with learning outcomes defined using Kraiger et al.'s ([35] ) typology of learning as a multi‐dimensional construct in terms of skills, affect (behaviour) and knowledge] immediately post‐course and

facilitates the transfer of this learning back in to the organization, and

whether the key learning ‘areas’ immediately post‐programme and those after one month back in the organization are the same or different.

As Warr et al. ([53] ) note, there have been inconsistencies in studies that have examined Kirkpatrick ([34] ) level two outcomes. Some researchers assess changes from before to after a programme (e.g. Basadur et al., [4] ) whereas others measure post‐programme attainment scores alone (e.g. Davis & Mount, [18] ; Warr & Bunce, [52] ). This latter approach is common in non‐executive education training courses when attaining a specific level of competency is more important than measuring changes in learning (Sacklett & Mullen, [45] ). In this research, self‐reported changes in learning will be measured using change or gain scores from pre‐ (time 1) to post‐ (time 2) programme. This allows a measure of change in learning for each participant, thus controlling for individual knowledge and skills in specific areas prior to any learning on the programme. Self‐report will also be used to measure transfer once participants are back in the organization. Warr et al. ([53] ) note that whilst not common, this approach is used when there is a methodological necessity. Unlike other research on the transfer of learning, this research will ask participants to report not only what they have transferred but also what they intend to transfer (hence the need for self‐report). Intention to transfer has not yet been studied within the executive education learning literature and so the relationship between transfer and intention to transfer is as yet unknown (although it may be hypothesized that knowledge, skills and behaviours that have not been transferred but are still intended to be transferred will be related to transfer barriers such as opportunity to perform what they have learnt (Ford et al., [22] ; Quinones, [43] ) and transfer climate (Mathieu et al., [39] ).

The current study aims to examine three research questions, with the second and third of most interest given the sparse nature of research directly focused on these aspects:

What are the characteristics of the skills, behaviour and knowledge that participants self‐assess as having improved immediately post‐programme?

What are the most frequently cited skills, behaviours and knowledge that participants claim they have transferred back or intend to transfer back in to the workplace, and do these two differ?

What is the relationship between those skills, behaviours and knowledge that have improved immediately post‐programme and those that have been transferred back in to the workplace?

Method

Participants

Four hundred three individuals, in five cohorts, from a German IT company, took part in a leadership development programme, facilitated by Ashridge Business School. 67 of the individuals were female, 331 were male and 5 declined to answer. The mean age of the participants was 41.9 (n = 323) with a range of between 24.4 years and 59.8 years, and the average length of time the individuals had worked in their organization was 9.3 years (n = 320) with a range of between 0.1 and 38.8 years. Cohort one had 50 participants, cohort two 80 participants, cohort three 79 participants, cohort four 89 participants and cohort five 105 participants.

Programme design

The training programme was initiated with the purpose of supporting a major change project within the organization. The project aimed to transform the company from autonomously operating subsidiaries in ten different European countries, including five Central Eastern European nations, to having an organizational view as one international company irrespective of the legal division into individual entities. This shift in the organizational paradigm led to necessary changes in leadership approach, particularly around multi‐cultural teams, remote leadership and change. The programme lasted four days and covered a variety of topics relating to these core leadership issues. The five cohorts all completed the programme during the same calendar year with cohort one in March, cohort two in April, cohort three in May, cohort four in June and the final cohort in September.

Each training cohort lasted four working days and consisted of a mix of large group events and parallel workshops of up to 24 participants. The opening large group event on the first day of each training group addressed all participants (up to 130 participants at any one time) from various hierarchical levels from across Europe (mainly Germany, Scandinavia, UK, Central and Eastern Europe). The purpose of the large group event was to initiate the connection of people and to offer taster sessions on the topic areas being introduced later in the programme. The two and a half day parallel workshops were designed to meet the specific needs of the different hierarchical levels with the aim of elaborating on the core topic areas by examining issues such as communication and collaboration skills, leading change, working across boundaries and building trusting relationships.

One‐way and two‐way communication exercises were the main pedagogic approach during the programme, and this methodology was used to introduce participants to the core issues of multi‐cultural teams, remote leadership and change utilizing tools such as:

(a) The four rooms of change (Janssen, )

This is a diagnostic tool used to explore change in organizations through the metaphor of a house with four rooms (contentment, denial, confusion and renewal). Each of the rooms is deemed to represent a different state of readiness for change.

(b) Organizational storytelling

This is a method which elicits the culture and tacit knowledge of the organization through storytelling. Within this programme it was used as a generic approach to promote conversation and dialogue, helping to create a picture of the desired future of the client organization.

(c) Trust criteria

The 10 trust criteria presented on the programme were based on the work of WorldWork Ltd, whose definition of trust comprises criteria such as competence, openness with information and inclusion.

Within each session during the programme, participants were introduced to a variety of frameworks and tools, with a focus on organizational application. This was followed by the identification and discussion of the relevance and meaning of the techniques to the individual, thus showing applicability from both the organization and the individualistic perspective. Finally, each session closed with a review of the key learning points. To maximize the retention of knowledge and to support the transfer of learning, along with providing an additional networking opportunity, the morning session of day three also provided a learning review where participants within one small group were required to present their own learning to a different small group. This process facilitated the deeper understanding of the material they had been introduced to, along with increasing repetition of the key material. This repetition was once more addressed in the closing event on day four. The closing large group event held in the afternoon of day four summarized the learning across the various parallel workshops and helped participants engage in a dialogue in which they verbally committed to specific actions once back in the workplace.

This verbal commitment to apply the frameworks and tools was examined more fully by an email forwarded by the managing directors one month after the participants had completed the programme. Participants were asked the question ‘What have you done as a consequence of your participation on the training, and if you have not started yet, what do you intend to do?’ and responses were sent directly to the managing directors. [1]

Procedure

On the first morning of the first day (prior to the formal start of the programme), participants were requested to complete a short questionnaire. The same questionnaire was administered to participants after the close of the programme on day four and a total of 338 individuals completed the pre‐ and post‐programme questionnaire. For a copy of the questionnaire refer to Appendix Table [NaN] . One month after the end of each cohort an email was sent to all participants by the company's managing directors. In this email the managing director voiced his appreciation to all participants for having attended the programme and asked one important question; all participants were asked ‘What have you done as a consequence of your participation on the training, and if you have not started yet, what do you intend to do?’ Individuals were requested to email their responses directly to the managing director himself, thus, as noted in a previous footnote, removing anonymity of participants. One hundred thirty‐one individual responses were received and collated by independent researchers. This data set was used to obtain information on (1) what learning had been, or (2) what learning intended to be, applied within the organization.

Measures

The questionnaire used to examine post programme learning was anonymous [2] and comprised 16 questions focusing on the four key areas of development addressed within the programme – leadership, multi‐cultural teams, remote working and leading change. Five questions related to leadership skills generally (e.g. ‘I know how to listen effectively’) and more specific leadership competencies within the company (e.g. ‘I understand how the company's leadership model relates to my leadership behaviour’). Four examined leadership within the specific context of remote working (e.g. ‘I understand the challenges of working at a distance’). Four questions focused on leading multi‐cultural teams (e.g. ‘I know what it takes to create a climate of trust in a team working across national boundaries’) and the final three questions related to leading change (e.g. ‘I know how to support my team through change processes’). Participants were required to consider each question and respond on a 5‐point Likert scale, with 1 being ‘Not at all true’ and 5 being ‘Completely true’. This data set was used to assess the participants' self‐evaluation of their change in learning during the course of the training itself.

The questionnaire had excellent internal reliability, with a Cronbach's alpha coefficient reported of.895 and there were no corrected item‐total correlations of less than.3.

Results and discussion

A total of 338 individuals completed the pre‐ and post‐programme questionnaire, an 84 per cent response rate. The 16 questions from the questionnaire (mean across pre‐ and post‐programme data) were subjected to a principal components analysis (PCA) using SPSS. PCA was deemed appropriate as inspection of the correlation matrix revealed a number of correlations greater than 0.3, the Kaiser‐Meyer‐Olkin Measure of Sampling Adequacy was 0.93, above the recommended value of 0.6 (Kaiser, [31] ) and the Bartlett's Test of Sphericity was statistically significant (p = 0.001).

PCA revealed the presence of three factors with Eigenvalues above 1, explaining 56.01 per cent of the variance in total (41.70 per cent for factor 1, 7.89 per cent for factor 2 and 6.42 per cent for factor 3). After inspection of the Scree plot it was decided to retain the three factors and a Varimax rotation was performed. The rotated solution can be seen in Table [NaN] with corresponding factor labels. The three factor solution accounts for 56.01 per cent of variance, with factor 1 contributing 23.32 per cent, factor 2 contributing 20.14 per cent and factor 3 contributing 12.55 per cent.

Varimax rotation of the three factor solution for questionnaire items (with questionnaire ‘topic’ in parentheses)

Question	Factor 1	Factor 2	Factor 3
Communicating change	Inter‐cultural working	Company specific leadership
I know why it is important to involve my team in the change process (Leading Change)	.736
I understand my team members emotional reaction to change (Leading Change)	.722
I see the need to change the way we communicate with each other (General Leadership)	.643
I understand the challenges of working at a distance (Leading Remotely)	.627
I know how to support my team through changes processes (Leading Change)	.584
I know how to listen effectively (General Leadership)	.575
I am aware of how my views affect my communication with people in the organization (Multicultural Teams)	.545
I know how to help my team agree on an explicit code of conduct for working together (Multicultural Teams)	.532
I know how my colleagues in other countries interpret my communication with them (Leading Remotely)		.812
I know how to adapt my communication style when dealing with colleagues based in other countries (Leading Remotely)		.756
I know how to build effective cross‐cultural relationships (Multicultural Teams)		.648
I know what it takes to create a climate of trust in a team working across national boundaries (Multicultural Teams)		.630
I know how to make best use of the communication channels available to me (Leading Remotely)		.441
I know how to communicate effectively with head office (General Leadership)			.835
I understand how the company's leadership model relates to my leadership behaviour (General Leadership)			.676
I understand how to lead within the specific situation of our company (General Leadership)			.439

The questions were grouped together according to the PCA, and mean scores for the three new factors were calculated; communicating change, inter‐cultural working and company‐specific leadership. Table [NaN] illustrates the mean scores pre‐ and post‐programme, along with the magnitude of the difference, for each of the three factors. Table [NaN] in the Appendix provides the same data for each individual question.

Means table (and SD) for the 16 questions and the three factors

Factor	Pre‐programme	Post‐programme	Size of change
Communicating change (factor 1)	3.69 (0.57)	4.13 (0.56)	0.44
Inter‐cultural working (factor 2)	3.07 (0.70)	3.77 (0.70)	0.70
Company‐specific leadership (factor 3)	3.28 (0.68)	3.57 (0.64)	0.29

1 1 = Not True – 5 = Completely True.

A series of independent [3] t‐tests were conducted, firstly to examine the difference in mean scores for the three factors pre‐ and post‐programme. For Communicating Change, with equal variances not assumed, there was a significant increase in scores post‐programme (t = −10.120, df = 654.681, p = 0.001). For Inter‐Cultural Working, with equal variances not assumed, there was a significant increase in scores post‐programme (t = −12.938, df = 664.627, p = 0.001). For Company‐Specific Leadership, with equal variances assumed, there was a significant increase in scores post‐programme (t = −5.789, df = 662, p = 0.001).

A second set of t‐tests were conducted to examine the difference in mean scores for each question pre‐ and post‐programme. Given the multiple comparisons a Bonferroni correction was applied and a new alpha level of 0.05/16 = 0.003 was adopted. All comparisons were significant (at the 0.001 level) other than ‘I know how to communicate effectively with head office’ which failed to reach significance (t = 0.263, df = 668, p = 0.793).

Finally, an a priori content analysis was conducted on all of the responses to the emailed question ‘What have you done as a consequence of your participation on the programme and if you have not started yet, what do you intend to do?’. The 1st order coding, a line‐by‐line micro analysis of the data, was initially completed, followed by 2nd order coding, with items relating to similar topics organized into categories. Finally, axial coding was completed with the researchers re‐examining the text, with the data considered in terms of the categories developed. This process of coding produced a list of 10 categories of behavioural response; Increased Networking, Use of the Change House Model, Increased English Skills, Use of the Team Charter, Use of Coaching Skills (including GROW), Adapting Communication Style, Improving Team Communication, Improving Listening Skills (including The Ladder of Inference), Increased use of Storytelling and Sharing the Vision and Increased Trust. Examples of emailed responses from participants to illustrate each of the 10 categories can be seen in Table [NaN] .

Examples of email responses for the 10 categories

Category	Example email responses
Adapting communication style	‘I have promised myself to talk more and email less in the future’. ‘I am contacting colleagues personally rather than using mail. Walk, talk, mail is a simple message that has significant impact’.
Improving team communication	‘To seek confirmation that we all have the same understanding of something and that silence does not mean acceptance’. ‘In team work we have processed the current needs and expectations of the employees..and have worked out possible ways to regain confidence, vision, mission and motivation’. ‘I plan to meet my staff, if possible, personally and regularly’.
Increased networking	‘I will continue my networking process with the new tool office communicator’. ‘I have extended my professional network with a lot of colleagues’.
Use of the four rooms of change	‘I have asked colleagues to anonymously place a pin where they believe they are in the house of change’. ‘I have used the change model to train and let my people know about how we react to changes in general, but also in specific situations’.
Use of story telling	‘As a first action after the workshop I told the staff my story’.
Use of coaching skills	‘I have started to use more coaching’. ‘I have not started with detailed activities yet but I think about coaching on an individual basis to accompany them during the upcoming major changes’.
Improving listening skills	‘Listen to the assumptions and intentions in communications more than before’.
Improved English skills	‘I have been trying to make others aware of the importance of improving their English’. ‘Helping to further develop English language skills where team members are uncertain’.
Use of team charter	‘I have started to prepare for a team charter to try to find out how the team wants to communicate when being remote, what values and dimensions are currently most important and how we should build on these together’.
Increased trust	‘Focusing on building trust’. ‘Remote leadership relies on trust..to build trust’

Table [NaN] shows the mean (across the four researchers) number of emails specifically noting one of the responses. A Krippendorf Alpha was calculated for multiple raters and the alpha of 0.79 demonstrates good inter‐rater reliability.

Means table for the number of email responses (across researchers) for each identified category

Behaviour	‘Have done’	‘Will do’
Adapting communication style	31.75	10.00
Improving team communication	28.00	15.5
Increased networking	28.00	5.00
Use of change house	26.00	20.25
Use of story telling	17.75	7.75
Use of coaching skills	14.50	4.00
Improving listening skills	10.50	5.75
Improved English skills	6.25	2.25
Use of team charter	6.00	5.50
Increased trust	3.75	4.75

Characteristics of the skills, behaviour and knowledge improved immediately ...

Whilst there were significant positive changes in learning for all three of the factors post‐programme (according to increases in mean feedback scores post programme), intercultural Working, showed the greatest increase in self‐reported learning despite less time being devoted to topics falling under this broad heading during the four days of training. In terms of learning design this is a critical point; participants self‐report the greatest increase in learning in an area of the programme with the least input. One possible explanation is that individuals may have had less knowledge of the concepts and ideas around inter‐cultural working prior to the start of the programme than in the other two areas, and thus found it easier to improve. This explanation is supported by the pre‐programme scores which were lowest for this factor. What is important to note here, however, is that participants did show some prior knowledge of the topic area, and also appreciated the importance of the topic by virtue of the fact that the programme had been initiated as a response to a global change initiative. In this research, participants were engaged with the theme prior to the course, had some knowledge of the issues and difficulties around the topic, but may not have had the appropriate skills and tools to address these issues. A second, not mutually exclusive explanation is that just by bringing a diverse group of managers from many subsidiaries together for a joint training event may provide an opportunity to implicitly demonstrate the skills needed for successful collaboration in multi‐cultural arenas. Both small and large group sessions (regardless of content) will have resulted in a strong collective experience with participants emerging with a stronger sense of their ability to forge inter‐cultural relationships within the organizations management community. An interest in the topic area, limited but pertinent prior knowledge, plus explicit and implicit learning on the topic is likely, therefore, to have created a perfect learning environment.

During the needs analysis design phase of the programme, four topics were deemed to be pertinent for participants, multi‐cultural teams, leading remotely, leading change and general leadership, and it was those four that determined the initial question grouping on the pre‐ and post‐programme questionnaire. However, the three factor solution may more closely correspond to the participant's mindset in regards to their current work challenges. It may also be that the division in to four categories was a subtle and possibly more theoretical difference that participants, focusing on practical and tangible tools and techniques, did not need to appreciate.

In terms of the typology defined by Kraiger et al. ([35] ), participants in this study self‐reported greater changes for the skill‐based learning than knowledge or affect (behaviour). A potential explanation of this finding may be that participants more actively engaged when they felt that a certain exercise would equip them with skills which they could apply to tackle their real workplace issues (likelihood of transfer), and subsequently felt reasonably confident that the learned skills were of benefit or would be in the future when they filled out the post programme questionnaire. The work relating to fidelity as a design principle to maximize learning transfer supports these findings. Fidelity, defined as the similarity between the skills taught during training and those that exist on the job (Machin & Fogarty, [37] ), is known to be an important factor for transfer, or short‐term learning (Hesketh, [28] , [29] ) even if the similarity is only perceived (Gick & Holyoak, [26] ). However, this current research must consider that when assessing learning changes via a questionnaire, participants may find it easier to judge change when considering skills (easily observable, even to the self) than when judging understanding. In addition, they may also be more susceptible to self‐report biases. Whilst executive education courses may be an excellent vehicle for improving skills and tool use, particularly when the advantages to the individual back in the organization of mastering such skills are transparent, the benefit of such courses to improve deeper understanding of theory and concepts should not be ruled out. Instead, other methodologies for eliciting learning changes should be employed, such as observation in the workplace and structured interviews post programme.

Most frequently cited skills, behaviours and knowledge transferred or intended to transfer ...

Whilst the quantitative data allowed a measure of immediate post programme learning, the key research focus of this study was on the qualitative data, which provided a unique insight in to transfer and intention to transfer once back in the organization, albeit with the caveat of a potential response bias. The three categories that ranked highest in terms of frequency of mention comprised practical tools as well as frameworks and ideas designed to foster a change of mindset.

The nature of a residential development programme such as that utilized within this research means that there were significant opportunities outside of the ‘official’ programme to engage in the practice of differing communication styles, team communication and networking and to measure the effectiveness of such practices. The practical nature of these skills, the fact that there was an obvious need within the organization to develop these skills (and thus active engagement by individuals) the building upon existing knowledge (even if this existing knowledge is that prior to the course, cross‐cultural communication was ‘poor’ or ‘problematic’), the explicit opportunity to develop these skills within the programme (and monitor and measure the effectiveness of this) and the implicit learning outside of the core programme are all aspects, firstly of excellent programme design, and secondly conditions for facilitating transfer. The explicit and repeated opportunity to develop skills within the programme and the implicit opportunity to do so outside of ‘timetabled’ hours demonstrate the effectiveness of two principles of training design in relation to transfer included within the Baldwin and Ford ([2] ) model. The first is known as overlearning, that is the repeated exposure and learning of a task beyond initial successful performance (Machin & Fogarty, [37] ) which has shown to improve retention of material over longer periods (Baldwin & Ford, [2] ; McGehee & Thayer, [40] ). The second, related, principle is that of varied practice whereby the learning of new knowledge and skills is disseminated though a variety of different methodologies (Machin & Fogarty, [37] ) and this has also shown to facilitate transfer (Gick & Holyoak, [26] ). Finally, the opportunity given to participants to measure the effectiveness of their learning during the residential programme, and the subsequent transfer of the knowledge and skill sets which had the most opportunity for feedback during the programme supports the notion of the importance of self‐control cues. This relates to direct feedback on whether participants are conducting a task or practicing a skill correctly and has been shown to facilitate transfer, particularly for individuals with low self‐efficacy (Gist et al., [27] ).

Increased trust was the area that received the least number of mentions. In the same way that understanding a topic or concept is difficult to self‐rate in a questionnaire, it is possible that individuals may find it challenging to think of specific examples of when they have noted a tangible increase in trust. In addition, considering the transfer principles seen as key to improving transfer noted above, how can you practice increasing trust within a four day programme, how can you create a need for active engagement in the topic and how can you build this both implicitly and explicitly within the design of the training? All of these are possible, but significantly more difficult to do, and are unlikely to have the same transfer success, as skill development.

One final aspect of training design that is seen as critical to transfer is that of the transfer climate (Baldwin & Ford, [2] work climate), one dimension of which is a positive work climate. Rouiller and Goldstein ([44] ) note that a positive work climate can predict transfer independent of the amount that trainees have learnt on a programme and Blume et al. ([7] ) found that a positive work climate (where learning is seen as being valued and supported through policies, and where there is encouragement to apply learning) had the strongest relationship with transfer compared with other elements of the work environment. Whilst the work environment was not directly measured within this research, it could be argued that (a) funding a large scale development programme, (b) attendance at the programme by Managing Directors and (c) the email one month post programme from Managing Directors to enquire about learning transfer, suggests a very strong positive work climate.

A second aspect of the transfer climate, known to be critical in successful transfer, is the importance of top management support (Burke & Baldwin, [10] ; Seyler et al., [47] ; Tracey et al., [50] ; Foxon, [24] ). Foxon ([24] ) found that anticipation of and actual top management support of the learning intervention was more important in improving transfer than other methods such as action planning. In this research an important and explicit aspect of the programme design was the support of the top management. This was demonstrated in a number of ways but those particularly pertinent were the introduction to the programme on Day 1 by the Managing Director, and the email from the Managing Director requesting examples of transfer and intention to transfer. Not only is it likely that this directive from the most senior management in the organization improved response rates for the current study (particularly given the lack of anonymity for participant emailed responses), it is also hypothesized that this involvement in the programme by senior leaders improved the transfer climate within the organization and subsequently contributed to both actual and intended transfer, supporting many studies in the literature on the importance of management support (e.g. Cromwell & Kolb, [16] ; Facteau et al., [20] ).

In relation to intention to transfer, other than for networking, [4] the frequency of responses for ‘will do’ is remarkably similar to the frequency of responses for ‘have done’. This relates to the work of Ford and Weissbein ([21] ) who found that self‐reports of intentions to use trained knowledge and skills are a factor relating to training transfer, in that those individuals who self‐report an intention to transfer are more likely to subsequently do so. The current research builds upon this by suggesting that not only is there a relationship between intention to and transfer in terms of process, but that the content is also related. Those skills, knowledge and behaviours that have been transferred are also those that participants still intend to transfer if they have not already done so. The reasons for having not yet been able to apply the learning back in the workplace could be hypothesized to relate to transfer barriers such as opportunity to perform what they have learnt (Ford et al., [22] ; Quinones, [43] ) and transfer climate relating specifically to peer and direct line‐manager support (Burke & Baldwin, [10] ; Mathieu et al., [39] ; Seyler et al., [47] ; Tracey et al., [50] ) rather than inherent difficulty in transferring the material. If the latter were to be true it would be expected that there would be a difference between those skills, knowledge and behaviours listed in the ‘have done’ column and those in the ‘will do’ column. Fundamentally, therefore, there seems to be little difference between learning transfer and intention to transfer in terms of content, and that transfer barriers may be one reason why intentions rather than actions still exist one month post‐programme.

Relationship between skills, behaviour and knowledge improved immediately ...

The three core themes that received the largest number of nominations for transfer of learning were comparable to the skills that gained the highest learning change scores on the questionnaire. The themes of communication style, team communication and networking are strikingly similar to the transfer noted around adapting communication style, improving team communication and increased networking. What is particularly important here is that the emailed responses measuring transfer of learning were free text. Participants were not given categories to respond to, they wrote in a free format and yet almost identical categories emerge. In addition, both ‘types’ of learning material adhere to four basic key learning principles; active engagement, limited but some prior knowledge, clear applicability back in the workplace and opportunity for repeated practice which encompass a number of the training design factors shown to facilitate transfer of learning discussed above (e.g. overlearning, varied practice, fidelity and self‐control cues). Blanchard et al. ([6] ) noted that organizations tend to favour measuring level one and level two in the Kirkpatrick ([34] ) model as level three (and four) are seen to be difficult and expensive to measure. This research suggests that with excellent programme design, ensuring, as far as possible, the protection of facilitators to transfer in relation to training design variables, the material (content) reported to improve most significantly immediately post‐programme is very similar to that which is subsequently transferred. It may be postulated, therefore, that through utilizing exceptional programme design and maximizing the transfer facilitators at this stage, measurement of level three outcomes may not be critical given the close relationship between level two and level three self‐reports in the current study. However, future research should seek to understand this relationship in terms of transfer of content (rather than process) more closely before the importance of measuring level three outcomes should be reduced.

Conclusion

This research sought to examine the nature of material acquired during a four day executive education programme, the material subsequently self‐reported as either having been transferred or with an intention to transfer, and the relationship between these two. Using a large sample (with the caveat that anonymity of pre‐ and post‐programme questionnaires meant that independent t‐tests were conducted) it was found that using the typology of learning from Kraiger et al. ([35] ) it was skills, rather than knowledge or affect (behaviour) that received the greatest gain post‐programme, and these skills were also those most frequently cited as having been transferred or with an intention to transfer. Future research should increase the post‐programme gap from one month to six months to examine the relationship between immediate, one month and six month transfer and intention to transfer. The use of interviews and/or observations in the workplace should also be employed to avoid the inherent issues with self‐assessment questionnaires and finally, research should also seek to understand the relationship between transfer and intention to transfer in terms of transfer of content (rather than process) more closely.

Appendix Appendix A1

Means table (and SD) for the 16 questions

Question	Pre‐programme	Post‐programme	Size of change
I know how to build effective cross‐cultural relationships (Multicultural Teams)	2.99 (0.94)	3.80 (1.94)	0.81
I know what it takes to create a climate of trust in a team working across national boundaries (Multicultural Teams)	3.28 (0.97)	3.98 (0.82)	0.70
I know how my colleagues in other countries interpret my communication with them (Leading Remotely)	2.68 (0.90)	3.34 (0.85)	0.66
I know how to make best use of the communication channels available to me (Leading Remotely)	3.29 (0.81)	3.95 (0.72)	0.66
I know how to support my team through changes processes (Leading Change)	3.31 (0.85)	3.95 (0.76)	0.64
I know how to adapt my communication style when dealing with colleagues based in other countries (Leading Remotely)	3.14 (0.95)	3.78 (0.80)	0.64
I know how to help my team agree on an explicit code of conduct for working together (Multicultural Teams)	3.28 (0.86)	3.90 (0.75)	0.62
I understand how to lead within the specific situation of our company (General Leadership)	3.19 (0.83)	3.72 (0.81)	0.53
I am aware of how my views affect my communication with people in the organization (Multicultural Teams)	3.43 (0.85)	3.95 (0.76)	0.52
I understand my team members emotional reaction to change (Leading Change)	3.77 (0.93)	4.19 (0.82)	0.42
I know how to listen effectively (General Leadership)	3.78 (0.81)	4.18 (0.70)	0.40
I know why it is important to involve my team in the change process (Leading Change)	4.03 (0.90)	4.41 (0.74)	0.38
I understand the challenges of working at a distance (Leading Remotely)	3.97 (0.89)	4.35 (0.78)	0.38
I understand how the company's leadership model relates to my leadership behaviour (General Leadership)	3.27 (0.93)	3.64 (0.79)	0.37
I see the need to change the way we communicate with each other (General Leadership)	3.89 (0.99)	4.15 (0.87)	0.26
I know how to communicate effectively with head office (General Leadership)	3.39 (0.95)	3.37 (0.86)	‐0.02

2 1 = Not True – 5 = Completely True.

By Victoria Culpin; Timm Eichenberg; Ian Hayward and Priya Abraham