Enhancing effectiveness of online training

program through assessment of

participant engagement index
Shrikant Prabhakar Wavre and Sunaina Kuknor

Introduction Shrikant Prabhakar Wavre

is based at Symbiosis
Learning strategies have been an integral part of talent development, but the recent pandemic International University,
has altered these strategies by accelerating the adoption of technology for flexibility. Learning Pune, India. Sunaina
and Development (L&D) must maintain momentum and capitalize on the potential benefits of Kuknor is based at
technological advances in this technologically advanced age. Organizations must accept Symbiosis Institute of
advanced learning approaches and accelerate progress. Virtual platforms demand an Business Management
enhanced engagement-ownership approach for meeting training goals. Understanding Pune, Symbiosis
learners engagement is key to the success of online training program (OTP) effectiveness. International University,
Pune, India.
Recent studies (Johnson et al., 2018) have focused on the adult learners OTP engagement
gaps. Mikołajczyk (2022) noticed decreased employee engagement and increased fatigue
with the use of multiple, online development initiatives. Many researchers stressed the need
for a study of learners ownership for engagement in OTP. Franklin and Harrington (2019)
stated that it is essential for students to take ownership of the learning process to ensure
future success.
Bloom’s taxonomy highlighted the importance of six levels of learning engagement. Yang
et al.’s (2018) studies highlighted learner engagement theories. Active learning was a
stronger predictor than instructor accessibility on learning outcomes (Wang et al., 2022).
Thus, it is imperative to measure learner engagement by acknowledging and appreciating
learners responses, emotions and reactions. Further, evaluation pressure motivates
participants to learn in OTP. Kuknor and Bhattacharya (2022) found that the inclusion of
diverse group to meaningful engagement leads to learning. Hence, the authors argue that
interventions are needed to enhance ownership-driven learning engagement in OTP
through active and positive participation.

Participant engagement index (EI): a mediator for effective OTP

Yang et al. (2018) found that most factors that have proved effective in traditional settings
also apply to online learning environments; current research tends to repeat itself if seen the
lens of topics and results. Following current trends, they proposed new directions for
research on engagement in online learning. Research (Mikołajczyk, 2022; Johnson et al.,
2018) indicates that OTP reduces social interaction, engagement and human interaction
based collaborative learning opportunities. Yang et al. (2018) suggested to consistently
look at antecedents, engagement and outcomes of engagement in a dynamic way and also
launch more in-depth investigations into participants cognitive and behavioral engagement
using learning analytic or eye-tracking devices.

DOI 10.1108/DLO-01-2023-0031 j
© Emerald Publishing Limited, ISSN 1477-7282 DEVELOPMENT AND LEARNING IN ORGANIZATIONS: AN INTERNATIONAL JOURNAL j
 Accordingly, the literature study (45 studies list provided in supplementary table)
suggested assessing participant EI to enhance collaborative learning engagement in an
OTP using an observation tool addressing these research questions:
RQ1. What attributes influence and enhance active and positive participant engagement
for collaborative learning?
RQ2. How can these attributes be grouped into measurable factors or themes in order to
propose an OTP-EI?

Methodology
Objective
To develop a facilitator-led OTP-EI.

Approach
In this qualitative study, 18 one-on-one (test-group) interactions of 45 minutes each covering
seven open-ended questions with a semi-structured interview tool explored learner
engagement expectations (referring to learning engagement theory with cognitive,
behavioral, and emotional dimensions (Yang et al., 2018)) for collaborative learning and
ownership. Conversations were journaled.

Content analysis
Six core-themes of EI (Column-C) were sorted from 17 sub-themes (Column-A) generated
through content analysis (thematic coding by grouping attributes with similarity patterns of
ownership and collaborative learning) of learners engagement-focused qualities described
by the test-group (Table 1). For validation, the identified attributes from the test-group were
mapped against existing literature (Column-B) as a triangulation, and finally, a 60-minute
focus group discussion was held among three HRD and three L&D professionals.

Test-group
With an emphasis on corporate OTP, seven L&D experts in behavioral, functional, and technical
training domains and 11 corporate experts with 4–25 years of experience in manufacturing and/
or service industries participated out of 23 invited, reaching data saturation at 15.

Findings and discussions

For L&D professionals, HRD and functional managers, the paper proposes a six-factor
evaluation by using participant EI for OTP. The trainer should assess this by bringing in a
special observer or by analyzing participation trends during OTP. The challenge of evaluating
more than 20 participants can be solved by delegating additional observers. Each factor can
be rated using a simple scale. The study linked the identified attributes to training engagement
theories (cognitive, emotional, behavioral, intrinsic and extrinsic motivational variables) and
Arghode and Wang’s (2018) study on engagement’s meaning and techniques (table-I). The
results offer a competency and attribute-based assessment of engagement, which is essential
for the conscious positive alignment of learner and trainer to achieve OTP goals.
The study recommends using EI for OTP considering the positive correlation between
training engagement and effectiveness. Competitive engagement scores can boost
collaboration and peer learning. Collaborative interactions, smart questions, learning
partners, chat boxes for sharing and encouraging, etc. become crucial norms for bridging
the OTP vacuum. The proposed EI is similar to academic monitoring, however it helps
workplace adult learners be accountable. An academagogical framework with appropriate
adaptations can make corporate training more successful, outcome-oriented, and behavior-
focused by enhancing active-learning engagement. Thus, as progress towards a body of
knowledge, adding a measurement framework to existing learning engagement theories.

j DEVELOPMENT AND LEARNING IN ORGANIZATIONS: AN INTERNATIONAL JOURNAL j

 Table 1 Key findings – EI factors
B. Factors stated in literature review (Arghode and Wang, 2016)
A. Key attributes identified from test group Create a
(content analysis based on behavioral Cognitive, Behavioral, Intrinsic, Make Training Show Passion Demonstrate Focus on learning-conducive
attributes: 17 subthemes) Emotional Extrinsic Relevant and Energy competency application environment

Relevance of doubts l l l l
Rational & context of question l l l l

Purpose behind query l l l l

Checks, seeks clarity l l l l l

Highlights practical challenges l l l l l

Responds to facilitators’ questions l l l l
Results demonstration l l l l

Acknowledges mistakes with relevant challenges l l l l l

Identifies of gaps l l l l l l l

Shares practical examples l l l l l l l

Highlights logical, practical challenges in concept application l l l l l l
Poses imaginary practical situations for concept application l l l l l l l

Respect professional discipline l l

Adheres to the facilitator’s instructions l l

Encourages other participants l

Active participation in group activities l
Provide team opportunity to speak, participate l

Source: Author’s own compilation

(continued)

j DEVELOPMENT AND LEARNING IN ORGANIZATIONS: AN INTERNATIONAL JOURNAL j

 Table 1
A. Key attributes identified from test group B. Factors stated in literature review (Arghode and Wang, 2016) C. Six assessment factors for
(content analysis based on behavioral Showing Evoking Having engagement index (EI)
attributes: 17 subthemes) Utilize interpersonal skills care emotions empathy (Core Themes)

Relevance of doubts Quality of Questions

Rational & context of question
Purpose behind query
Checks, seeks clarity l Attention to Details

j DEVELOPMENT AND LEARNING IN ORGANIZATIONS: AN INTERNATIONAL JOURNAL j

Highlights practical challenges
Responds to facilitators’ questions
Results demonstration Quality of Output During Activities
Acknowledges mistakes with relevant challenges
Identifies of gaps
Shares practical examples Connecting to Practical Examples
Highlights logical, practical challenges in concept application
Poses imaginary practical situations for concept application
Respect professional discipline l l Professional Approach
Adheres to the facilitator’s instructions l l
Encourages other participants l l l Team Dynamics
Active participation in group activities l l l
Provide team opportunity to speak, participate l l l l
The framework can help L&D professionals construct engaging, qualitative training.
Engagement increases information exchange and meaningful participation. If used for
critical online meetings and organized with an emphasis on EI for optimal output, the
framework may help the emerging work-from-home trend. It will also help HRD and
functional managers make key strategic decisions by identifying each EI-factor’s correlation
with other evaluation criteria. Furthermore, comparing session-wide EI scores may help
assess facilitator performance. Socially, the proposed framework can help the education
sector improve student learning engagement.

Conclusion
Virtual learning provides limited collaborative learning opportunities. Learning is facilitated
Keywords:
through interpersonal interactions, and these are amplified when structured engagement Training evaluation,
acts as a moderating factor in OTP. The proposed six EI-factors enable collaborative Trainers,
learning engagement with conscious ownership. Learning engagement,
HRD managers,
This research provides a tool for gauging key measurable EI-factors influencing positive Online training
and active engagement in OTP. The study provides a research opportunity for empirical engagement,
validation of the proposed framework to add to the body of knowledge. Training effectives

References
Arghode, V. and Wang, J. (2016), “Exploring trainers’ engaging instructional practices: a collective case study”,
European Journal of Training and Development, Vol. 40 No. 2, pp. 111-127, doi: 10.1108/EJTD-04-2015-0033.
Franklin, H. and Harrington, I. (2019), “A review into effective classroom management and strategies for
student engagement: teacher and student roles in today’s classrooms”, Journal of Education and
Training Studies, Vol. 7 No. 12, pp. 1-12, doi: 10.11114/jets.v7i12.4491.
Johnson, E., Morwane, R., Dada, S., Pretorius, G. and Lotries, M. (2018), “Adult learners perspectives on
their engagement in a hybrid learning postgraduate programme”, The Journal of Continuing Higher
Education, Vol. 66 No. 2, pp. 88-105, doi: 10.1080/07377363.2018.1469071.
Kuknor, S.C. and Bhattacharya, S. (2022), “Inclusive leadership: new age leadership to foster
organizational inclusion”, European Journal of Training and Development, Vol. 46 No. 9, pp. 771-779, doi:
10.1108/EJTD-07-2019-0132.
Mikołajczyk, K. (2022), “Changes in the approach to employee development in organisations as a result
of the COVID-19 pandemic”, European Journal of Training and Development, Vol. 46 Nos 5/6,
pp. 544-562, doi: 10.1108/EJTD-12-2020-0171.
Wang, X., Lin, W., Jiang, Y., Wu, Y., Liu, Y. and Zhou, W.A. (2022), “Active learning and instructor accessibility
in online talent training: a field experiment in China during COVID-19”, Development and Learning in
Organizations: An International Journal, Vol. 36 No. 1, pp. 14-16, doi: 10.1108/DLO-02-2021-0023.
Yang, D., Levonen, J.M. and Niemi, H. (2018), “Online learning engagement: critical factors and research
evidence from literature”, Themes in eLearning, Vol. 11 No. 1, pp. 1-8.

Supplementary material
Supplementary material can be found online.

Corresponding author
Shrikant Prabhakar Wavre can be contacted at: spw815@gmail.com

For instructions on how to order reprints of this article, please visit our website:
www.emeraldgrouppublishing.com/licensing/reprints.htm
Or contact us for further details: permissions@emeraldinsight.com

j DEVELOPMENT AND LEARNING IN ORGANIZATIONS: AN INTERNATIONAL JOURNAL j