Accepted: 30 January 2017

DOI: 10.1111/eje.12265

ORIGINAL ARTICLE

Formative evaluation of a teledentistry training programme for

oral health professionals

K. K. McFarland1 | P. Nayar2,3 | A. Chandak2 | N. Gupta2

1
Creighton University, Omaha, NE, USA
2
Department of Health Services Research
& Administration, University of Nebraska
Medical Center, Omaha, NE, USA
3
Department of Oral Biology, College of
Dentistry, University of Nebraska Medical
Center, Omaha, NE, USA
Correspondence
Preethy Nayar, University of Nebraska Medical
Center, Omaha, NE, USA.
Email: pnayar@unmc.edu
Funding information
Health Resources and Services Administration,
Grant/Award Number: # D85HP20046
Abstract
Introduction: The objective of this study was to conduct a formative evaluation of a
teledentistry (TD) programme that was developed for a predominantly rural state in
the Midwestern United States.
Materials and Methods: Formative evaluation data were collected on programme ac-
tivities from the TD programme records. In addition, the effectiveness of the TD train-
ing programme was evaluated using a self-­
administered paper-­
based survey
administered to the participants, immediately following completion of the training ac-
tivity. Ninety-­three dental students, oral health and other health professionals partici-
pated in the TD training programme.
Results: Overall, the trainees rated the TD training programme highly, with regard to
the content, format and skills improvement. The evaluation also demonstrated a posi-
tive change in all trainees’ attitudes following the training sessions, with most trainees
acknowledging a positive impact of the training on their knowledge and competency.
Discussion and Conclusions: We identified challenges in the development of the TD
programme and in expanding access to oral health care for rural communities.
Challenges included reimbursement and a limited interest amongst established dental
offices. Dental schools can play an important role in preparing both dental health pro-
fessionals and other health professionals in the use of TD by providing training and
oral health expertise. The use of TD by non-­dental providers for consultation, referral
and disease management has the potential to improve oral health outcomes, particu-
larly for rural and underserved populations. Evaluation data provide critical feedback
to programme planners and administrators.

KEYWORDS
evaluation, teledentistry

1 |  INTRODUCTION
Teledentistry (TD) is the use of health information technology and
telecommunications for oral care, consultation, education and pub-
1
lic awareness with the broad goal of improving oral health. TD has
produced a dramatic transformation in oral health care, particularly
in rural and remote areas where there are no practicing dentists.2-4
Experienced dentists have used TD to aid their younger colleagues
5
in the identification of root canal orifices, to provide consultations
and support for dental trauma when the dentist is unavailable,6 to
diagnose dental caries amongst juvenile offenders7 and to provide
diagnostic support in primary healthcare clinics where an oral medi-
cine specialist is unavailable.8 Moreover, TD also provides for screen-
ing children in inner-­city elementary schools and childcare centres9 as
well as increasing interceptive orthodontic services to disadvantaged
children with real-­time supervision from an orthodontist.10 The bene-
fits of TD include reduced costs of care,11 increased quality of care,11
peer contact and specialist support for the dental practitioners,11

Eur J Dent Educ. 2017;1–6. wileyonlinelibrary.com/journal/eje © 2017 John Wiley & Sons A/S.  |  1
Published by John Wiley & Sons Ltd
 |
2       MCFARLAND et al.
access to service in rural areas,12,13 decreases in travel for patients and
their families to avail the necessary care,12 and reduced inappropriate
referrals to specialists.14,15
Nebraska is a predominantly rural state with approximately 49 of
the 93 counties designated as shortage areas by the Governor’s Rural
Health Commission. To incorporate TD into dental education, as well as
to provide a statewide network of oral health expertise, the College of
Dentistry (COD) at the University of Nebraska Medical Center (UNMC)
formally established a TD programme between 2010 and 2015. The
development of the TD network included a variety of community-­
based facilities across the state where access to care was limited and
dental students currently go to training as a part of their extramural
rotations. Organisationally, the TD network has a hub and spoke struc-
ture with the hub being the UNMC COD at Lincoln, the state’s capital,
and the spokes being the eight TD sites across Nebraska. The eight
TD sites include Federally Qualified Health Centers (FQHCs), rural pri-
vate practices, local health departments, an Indian Health Service (IHS)
clinic and a children’s hospital (Figure 1). The overall goal of the TD
programme is to expand access to oral health expertise for rural and
underserved populations in the region and train health professionals in
the use of TD. The aim of this study was to conduct a formative evalua-
tion of the UNMC TD programme and to evaluate the effectiveness of
the TD training programme for oral health and other health profession-
als including nurses and social workers.
1.1 | Programme description and activities
In 2010, faculty at the COD identified the TD training programme
learning objectives and developed telehealth training materials. The
learning objectives were as follows: (i) participants will be able to
define telehealth (TH), (ii) participants will be able to enumerate at
least three applications for TH services, (iii) participants will compre-
hend mainstream TH applications in modern American culture, (iv)
participants will be able to define TD, (v) participants will be able to
identify various types of technology used for TD consultations, (vi)
participants will learn how to schedule a TD consultation, and (vii) par-
ticipants will learn how to conduct a TD consultation using the Vidyo
(Vidyo Inc., New Jersey, USA) teleconferencing system, a computer
and an intra-­oral camera. The training programme materials included
information on conducting TD consultations, power point and video
clips on how to conduct TD consultations, guides to facilitating TD
training and record keeping, and TD training evaluation forms.
In 2010, UNMC faculty and staff trained oral health profession-
als in six TH sites: (Gering, Chadron, Columbus, Hastings, Omaha and
Lincoln) and provided TD equipment to these sites. The TD trainers
visited each of the six TH sites and provided in-­person TD training
for all participants who were students, faculty, staff and other inter-
ested personnel at the facility. The rationale for including non-­dental
personnel in the TD training was twofold: (i) non-­dental personnel are
more likely to need access to oral health expertise than are dental per-
sonnel in remote and underserved areas, and (ii) the COD’s previous
TD pilot project determined that interest amongst rural dentists to
incorporate TD into their practice was limited, because they simply
F I G U R E   1   Teledentistry sites in Nebraska. Nebraska TD network
with a hub and spoke structure, with the hub at Lincoln and the
spokes at eight teledentistry sites across the state
would refer a patient for specialty care if they felt it was indicated.
Therefore, developing partnerships with non-­dental personnel for TD
consults was determined to be important.
In 2011, the TD programme staff developed Web-­based TD train-
ing modules on topics such as scheduling and conducting consulta-
tions and documenting and billing for TD consults. A TD webpage was
also developed that included TD frequently asked questions, a map of
the TD sites (Figure 1), TD community grand rounds and TD resources
and scheduling information. TD training and equipment were pro-
vided for the health professionals located in the four remaining TH
sites at North Platte, Burwell, Macy and Norfolk. TD Grand Rounds
which were real-­time patient consultations with remote sites and rural
providers seeking COD faculty expertise were conducted. The Grand
Rounds included consultations for patients undergoing head and neck
radiation treatment, dental reconstruction for a previous gunshot
wound to the jaw and the oral health management of an adolescent
with ectodermal dysplasia.
In 2012, the TD equipment was updated because several of the
sites had large TH carts that were no longer compliant with the Food
and Drug Administration (FDA) rules and regulations regarding TH
encryption and Health Insurance Portability and Accountability Act
(HIPAA) rules regarding privacy. Therefore, the carts were replaced
with TH compliant laptop computers.
In 2013, an instructional technologist for TH was hired and a
statewide TD training conference was provided for dental students,
faculty and communities who were utilising or interested in utilising
TD services. The university also implemented the use of Vidyo video-
conferencing software for TH consultations college-­wide, as well as at
the university hospital. The Vidyo software was installed on desktop
computers and Microsoft Surface Pro tablets. A total of six TD consul-
tations were completed between July and September, 2013. In 2014,
36 TD consultations were conducted, including 34 TD consultations
with school nurses. In 2015, five TD consultations were completed
between January and May.
2 | METHODS
The effectiveness of the TD training programme was self-­evaluated
by the participants, using a paper-­based survey administered immedi-
ately following completion of the in-­person, training activity at the six
TH training sites (Gering, Chadron, Columbus, Hastings, Omaha and
MCFARLAND et al. |
      3

Lincoln). Institutional review board approval was obtained for the study.

93 (100.0)

93 (100.0)

93 (100.0)
93 (100.0)
93 (100.0)

93 (100.0)

93 (100.0)
The survey questions addressed the format, content and relevance of

Total
the training and the participants’ ratings of the effectiveness of the

n (%)
training. All ratings of the training were assessed using a six-­point Likert
scale from 0 to 5 (0=Don’t Know; 1=Strongly Disagree; 2=Disagree;
3=Neutral; 4=Agree; 5=Strongly Agree). The survey data were descrip-
Missing

5 (5.4)

3 (3.2)

4 (4.3)
3 (3.2)
5 (5.4)

2 (2.2)

3 (3.2)
tively summarised. Further, differences in the evaluation ratings for dif-
n (%)

ferent types of survey respondents (dentists, dental students, dental

hygienists/assistant and other respondents) were assessed using the
Kruskal-­Wallis test. All analyses were conducted using SAS 9.4 software.
Strongly agree

23 (24.7)

20 (21.5)

19 (20.4)
23 (24.7)
13 (14.0)

18 (19.4)

21 (22.6)
3 | RESULTS
n (%)

Ninety-­three dental students, oral health and other health profession-

als participated in the in-­person TD training programme. One-­third
31 (33.3)

39 (41.9)

41 (44.1)
49 (52.7)
41 (44.1)

39 (41.9)

44 (47.3)

of the participants (n=34; 36.6%) were fourth-­year (D4) dental stu-

Agree
n (%)

dents, 19 (20.4%) were dentists, 11 (11.8%) were dental assistants,

six (6.5%) were dental hygienists, one (1.1%) was a D3 dental student,
and 21 (21.5%) identified themselves as other health professionals.
25 (26.9)

25 (26.9)

19 (20.4)
10 (10.8)
25 (26.9)

22 (23.7)

18 (19.4)
Neutral

Those who identified as other health professionals listed themselves

n (%)

as office manager, paediatric dental resident, nursing student, health

planner, infection prevention, LPN, preceptor, public health consult-
ant, public health nurse, registered nurse, social worker and specialty
Disagree

clinic receptionist.
3 (3.2)

4 (4.3)

6 (6.5)
5 (5.4)
5 (5.4)

8 (8.6)

5 (5.4)

About half of the participants (n=54; 58%) either strongly agreed

n (%)

or agreed that the competencies to be gained through the training

were clearly identified in the syllabus. About two-­thirds of the partic-
ipants (n=59; 63.4%) strongly agreed or agreed that the training was
Strongly disagree

effective in helping them gain the competencies and that the training
was relevant to their work. Further, the majority of the participants
(n=72; 77.4%) strongly agreed or agreed that the training increased
2 (2.2)

1 (1.1)

3 (3.2)
3 (3.2)
3 (3.2)

4 (4.3)

2 (2.2)
n (%)

their knowledge of the subject. With respect to the training materials,

over half of the participants (n=54; 58.1%) strongly agreed or agreed
that “the readings and/or handouts added to their understanding of
Don’t know

the topic,” that “the amount of knowledge gained in the training was
worth the effort put forth” (n=57; 61.3%) and that they “gained knowl-
4 (4.3)

1 (1.1)

1 (1.1)
0 (0.0)
1 (1.1)

0 (0.0)

0 (0.0)
n (%)

edge that they can use in the future” (n=65; 69.9%). With respect to
the amount of time devoted to the training, about two-­thirds of the
T A B L E   1   Evaluation of the teledentistry training

The amount of knowledge I gained in this course was

participants (n=61; 65.6%) responded that it was “about right.” With

The training increased my knowledge of the subject
The competencies to be gained through this course

I gained knowledge in this training that I can use in

respect to the pace of the training, the majority of the participants

The training was effective in helping me gain the

(n=65; 69.9%) responded that it was “about right.” Further, the major-
The readings and/or handouts added to my

ity of the participants (n=72; 77.4%) responded that they devoted an

were clearly identified in the syllabus

The training was relevant for my work

understanding of the topics covered

average between 0 and 3 hours per week to this training. Table 1 sum-
marises the findings of the evaluation of the TD training programme.
With respect to their overall opinion of the TD training programme,
worth the effort I put forth

13 participants (14.0%) rated it as excellent, about one-­fourth (n=22;

23.7%) as very good, 34 (36.6%) as good, 17 (18.3%) as fair, whilst four
Training component

participants (4.3%) rated it as poor (Figure 2).

competencies

The mean rating for training effectiveness in helping to gain

the future

competencies differed significantly between participants, with den-

tal hygienists/assistants rating this highest (4.41; P-­value <.01). The
mean rating for relevance of training to the participant’s work differed
 |
4       MCFARLAND et al.

Overall opinion of the training programme

(n= 93)
40.0%
34, 36.6%
35.0%

30.0%

25.0% 22, 23.7%

20.0% 17, 18.3%

13, 14.0%
15.0%

10.0%

3, 3.2% 4, 4.3%
5.0%

0.0% F I G U R E   2   Participants’ Overall Opinion of

Missing Excellent Very Good Good Fair Poor the Teledentistry Training Programme (n, %)

significantly, with dentists rating this highest (4.06; P-­value <.05). The
mean rating for the “training increasing the participant’s knowledge”
differed significantly, with dental hygienists/assistants rating this the
highest (4.47; P-­value <.01). The mean rating for “the readings and/or
handouts adding to the understanding of the topic” also differed sig-
nificantly, again with the dental hygienists/assistants rating it the high-
est (4.24; P-­value <.01). Similarly, the mean rating for “the amount of
knowledge gained being worth the effort put forth by the participant”
differed significantly with the dental hygienists/assistants having the
highest mean rating (4.41; P-­value <.0001). This finding was similar
to the rating for “gaining knowledge that can be used in the future
by the participant,” with the highest ratings by the dental hygienists/
assistants (4.41; P-­value <.001). There was no significant difference
in the mean rating for “training competencies being clearly identified
in the syllabus” between dentists, dental students, dental hygienists/
assistants and other professionals (Figure 3).

Mean Ratings of Course Components (n=93)

Dentists (n=19) Dental Students (n=35) Dental Hygienists/ Assistants (n=17) Other Health Care Professionals (n=21)

5
4.4 4.5 4.4 4.4
4.5 4.2
4.1 4.0 4.1 4.0 4.1 4.0 4.0 4.1
3.9 3.8 3.9
4 3.8
3.5 3.4 3.5 3.5 3.6 3.6
3.4 3.4 3.5
3.5 3.3
3.0
Mean Rating

3
2.5
2
1.5
1
0.5
0
The competencies The training was The training was The training The readings The amount of I gained
to be gained effective in relevant for my increased my and/or handouts knowledge I knowledge in this
through this helping me gain work* knowledge of the added to my gained in this training that I can
course were the subject** understanding of course was worth use in the
clearly identified competencies** the topics the effort I put future***
in the syllabus covered** forth***
*p<0.05; **p<0.01; ***p<0.001

F I G U R E   3   Participant Differences in Evaluation of the Teledentistry Programme. Mean ratings indicate the means of the six-­point Likert
scale from 0 to 5 (0=Don’t Know; 1=Strongly Disagree; 2=Disagree; 3=Neutral; 4=Agree; 5=Strongly Agree). There were statistically significant
differences in the mean ratings of training effectiveness in helping to gain the competencies (P<0.01), training relevance for work (P<0.05),
training led to increase in knowledge of subject (P<0.01), readings/handouts adding to the understanding of the topics covered (P<0.01), amount
of knowledge gained being with the effort put forth (P<0.0001) and gained knowledge in training that can be used in future (P<0.01)
MCFARLAND et al.
4 | DISCUSSION
We evaluated a TD training programme aimed at training dental stu-
dents, oral health and other health professionals in a predominantly
rural Midwestern state. Overall, the trainees who participated in
the person training sessions rated the training programme high with
regard to the content, format and skill improvement.
Cooper et al.16 in a previous study had found that dental hygiene
students had a positive change in their knowledge, after completion
of a course on TD. The dental hygiene students in that study also
demonstrated significant improvement in their knowledge about the
effectiveness of TD in identifying dental needs in underserved areas.
Our evaluation findings likewise demonstrated a positive change in all
trainees’ attitudes following the training sessions, with the majority
of trainees acknowledging a positive impact on their knowledge and
competency. To address concerns about reliability and to limit recall
bias, the evaluation data were collected immediately following the
training programme.
During the period of the study, the TD training programme content
evolved to include new equipment, software and training tools. These
changes in training programme content were required to adhere to fed-
eral regulations regarding TH. Although there were several challenges
faced by programme staff, particularly with regard to increasing the
use of TD consultations, these challenges were met, by training non-­
dental personnel to use TD to assist in addressing the demand for oral
health expertise. Overall, the number of TD consultations completed
increased from 6 in the first year following implementation of the pro-
gramme to 36 in the third year following implementation. This sixfold
increase was made possible due to a pilot programme conducted in
collaboration with area schools that trained school nurses in the use
of TD. School nurse-­initiated TD consultations were conducted for
a variety of reasons including case management, oral pathology and
oral health status evaluation. School nurses are not eligible to be reim-
bursed by the State Medicaid programme. However, they benefit by
knowing that they have facilitated timely dental treatment for a child
who otherwise may never receive it. For example, children requiring
Operating Room (OR) care did not waste time and resources going to
see a general dentist.
The incorporation of TD training and TD services in community-­
based settings by non-­dental personnel has the potential to increase
access to oral health care, to improve oral health outcomes and to
educate families about the importance of good oral health.
The state of Nebraska has a well-­established telehealth network.
The statewide TH network provided the backbone for the develop-
ment of the TD network. Therefore, other dental schools or states
may or may not be able to replicate a similar system depending on
resources available. The development of the TD programme at the
COD was supported by a federal grant from the Health Resources
and Services Administration (HRSA). The ability to leverage state and
federal resources was a significant factor in the development of the
TD network. Recent changes in the state regulations regarding school
health now require school nurses to provide oral, vision and hearing
screening annually (kindergarten through 4th, 7th and 10th grade).
|
      5
These new state regulations may have added to the school nurses’
interest in TD training.
One of the challenges that remains in the development of TD
programmes is the issue of reimbursement. The State Medicaid pro-
gramme provides only limited reimbursement that may not be suf-
ficiently attractive for participating dentists to make a significant
commitment to adding this technology to their routine practice of
dentistry. Therefore, reimbursement will likely be a challenge to the
widespread adoption and use of TD.
5 | CONCLUSIONS
The success of a teledentistry programme requires effective training
of oral health and other health professionals such as school nurses.
Dental schools can play an important role in preparing both dental
health professionals and other health professionals in the use of TD
by providing training and oral health expertise. The use of TD by non-­
dental providers for consultation, referral and disease management
has the potential to improve oral health outcomes, particularly for
rural and underserved populations. Evaluation data provide critical
feedback to programme planners and administrators.
AC KNOW L ED G EM ENTS
This publication was made possible by Grant # D85HP20046 from the
Health Resources and Services Administration. Its contents are solely
the responsibility of the authors and do not necessarily represent the
official views of the Health Resources and Services Administration,
Bureau of Health Workforce or the Department of Health and Human
Services.
CO NFL I C T O F I NT ER ES T
The authors have no conflict of interest to disclose.
REFERENCES
1. Daniel SJ, Kumar S. Teledentistry: a key component in access to care.
J Evid Based Dent Pract. 2014;14:201‐208.
2. Bauer JC, Brown WT. The digital transformation of oral health
care: teledentistry and electronic commerce. J Am Dent Assoc.
2001;132:204‐209.
3. Sanchez Dils E, Lefebvre C, Abeyta K. Teledentistry in the United States:
a new horizon of dental care. Int J Dental Hygiene. 2004;2:161‐164.
4. Glassman P, Helgeson M, Kattlove J. Using telehealth technologies
to improve oral health for vulnerable and underserved populations.
J Calif Dent Assoc. 2012;40:579‐585.
5. Brullmann D, Schmidtmann I, Warzecha K, d’Hoedt B. Recognition of
root canal orifices at a distance -­a preliminary study of teledentistry.
J Telemed Telecare. 2011;17:154‐157.
6. Lienert N, Zitzmann NU, Filippi A, Weiger R, Krastl G. Teledental con-
sultations related to trauma in a Swiss telemedical center: a retro-
spective survey. Dent Traumatol. 2010;26:223‐227.
7. Morosini IAC, de Oliveira DC, Ferreira FM, Fraiz FC, Torres-Pereira
CC. Performance of distant diagnosis of dental caries by teledentistry
in juvenile offenders. Telemed e-­Health. 2014;20:584‐589.
 |
6       MCFARLAND et al.
8. Torres-Pereira C, Possebon RS, Simoes A, et al. Email for distance di-
agnosis of oral diseases: a preliminary study of teledentistry. J Telemed
Telecare. 2008;14:435‐438.
9. Kopycka-Kedzierawski DT, Billings RJ. Teledentistry in inner-­ city
child-­care centres. J Telemed Telecare. 2006;12:176‐181.
10. Berndt J, Leone P, King G. Using teledentistry to provide interceptive
orthodontic services to disadvantaged children. Am J Orthod Dentofac
Orthop. 2008;134:700‐706.
11. Bhambal A, Saxena S, Balsaraf SV. Teledentistry: potentials unex-
plored. J Int Oral Health. 2010;2:1‐6.
12. Clarke K, Marino R, Hopcraft M, McCullough M, Manton D. Paediatric
teledentistry: delivering oral health services to rural and regional
children. 2014. Available at https://minerva-access.unimelb.edu.
au/bitstream/handle/11343/44085/Paediatric-Teledentisty.pdf?se
quence=1 (Accessed 20 May 2016).
13. Fricton J, Chen H. Using teledentistry to improve access to dental care
for the underserved. Dent Clin North Am. 2009;53:537‐548.
14. Cook J, Edwards J, Mullings C, Stephens C. Dentists’ opinions of
an online orthodontic advice service. J Telemed Telecare. 2001;7:
334‐337.
15. Mandall N, O’Brien K, Brady J, Worthington H, Harvey L. Teledentistry
for screening new patient orthodontic referrals. Part 1: a randomised
controlled trial. Br Dent J. 2005;199:659‐662.
16. Cooper BR, Engeswick LM. Knowledge, attitudes, and confidence lev-
els of dental hygiene Students regarding teledentistry. Am Dent Hyg
Assoc. 2007;81:114.
How to cite this article: McFarland KK, Nayar P, Chandak A,
Gupta N. Formative evaluation of a teledentistry training
programme for oral health professionals. Eur J Dent Educ.
2017;00:1–6.