See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/2653699

The Calculator Project - Formal Reasoning about Programs

Article · March 1995

DOI: 10.1109/SEDC.1994.475332 · Source: CiteSeer

CITATIONS READS
5 2,277

5 authors, including:

Steve Reeves Richard Bornat

The University of Waikato Middlesex University, UK
150 PUBLICATIONS   921 CITATIONS    97 PUBLICATIONS   1,213 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Weak memory logic View project

All content following this page was uploaded by Steve Reeves on 02 November 2012.

The user has requested enhancement of the downloaded file.

To appear in IEEE Proceedings of SRIG-ET’94.
The Calculator Project - Formal Reasoning about Programs
Steve Reeves Doug Goldson
Department of Department of
Computer Science Computer Science
University of Waikato Massey University
New Zealand New Zealand
Abstract
This paper describes The Calculator Project, which was a
three-year joint research project between the Centre for
Information Technology in Education at The Open
University, U.K. (Pat Fung, Tim O’Shea) and the
Department of Computer Science, QMW, University of
London, U.K. (Richard Bornat, Doug Goldson, Mike
Hopkins, Steve Reeves). The project was funded by the
U.K. Joint Council Initiative in Cognitive Science and
Human-Computer Interaction.
The central aim of the project was to test the hypothesis
that providing so-called calculators would improve
students’ performance in those parts of the undergraduate
first-year that relied on formal reasoning skills.
1: Introduction
Computer Science undergraduates find “formal
methods”, i.e. the use of mathematical notations and
methods for supporting the development of computer
software and systems, difficult and in particular they find
the part of formal methods that demands reasoning about
their programs the most difficult.
The work reported in this paper set out to see what
factors affected these difficulties and also to test the
hypothesis that providing so-called calculators would ease
the burden that formal reasoning imposes.
In order to gather these data the project team used many
different methods: questionnaires, interviews, test and
exam results, logging of system use and monitoring of
teacher and student discussion on electronic bulletin-
boards.
As we shall see, this data gathering exercise allowed us
show some statistically significant results concerning the
experience of students prior to enrolling on the courses. It
also, equally valuably, produced much anecdotal evidence
about how courses should be run.
The plan of this paper is as follows: in the next section
we introduce and discuss the term “calculators”; we then
go on to introduce the main hypothesis of the
experimental work of the project and the design of the
Pat Fung Mike Hopkins
Tim O’Shea Richard Bornat
CITE Department of
The Open University Computer Science
Milton Keynes QMW, London
UK UK
experiments that we conducted; next we introduce and
discuss two of the calculators used during the project;
following that we present and discuss the results that we
have, so far, obtained; finally we suggest where a follow-
on project might lead and suggest some improvements to
our work.
2: Calculators?
A calculator, for us, is a self-contained, robust, simple
to use software device which supports working in some
area analogous to the way the familiar, hand-held,
electronic calculator supports working in arithmetic. The
traditional calculator supports the users in arithmetic tasks
by relieving them of the burden of carrying out the
arithmetic operations, allowing them to concentrate on the
substance of the problem at hand. The aim of the
calculators developed or borrowed in this project was to
support learning to program in Miranda (“Miranda” is a
trademark of Research Software Ltd.) and the
understanding and use of the language of first-order logic.
An important design feature of a calculator should be
that it can be used without a great deal of explanation and
instruction. A second feature should be that it provides a
way for students to build-up a reliable model of the way in
which a calculation takes place. They should come to
understand how it works and what rules it follows.
This means that we actually have two goals for our
calculators: they should support the student in the task of
building an understanding of the activities that the
calculator helps to carry out; they should support the
student in the more mechanical and book-keeping parts of
their tasks, once learned properly, so that the tasks they
can tackle become progressively harder.
When we first developed our ideas for this project we
had also hoped that the calculators would be thought so
much fun to use that students would use them
spontaneously and perhaps play with them. This turned
out to be a very naïve idea on our behalf - our students
were too busy and so oriented towards doing what they
must do to choose to do something viewed as extra work.

3: Project hypothesis and design
In the project we had a simple hypothesis - providing
calculators for first-year undergraduate students in
Computer Science would improve their performance on
courses that involved formal reasoning, which students
typically find hard, to say the least [1,2].
The project had two parallel and interconnecting
streams of work: one (based at QMW) scoured the world
for already existing calculators, evaluated them and,
because for functional programming no suitable
calculators existed, developed one of its own; the other
stream (based at the OU) carried out an educational
evaluation by comparing students who had used the
calculators with those who had not - the comparison was
done using a mixture of interviews, questionnaires,
consideration of exam, test and coursework results and
monitoring of the electronic bulletin boards that students
and tutors on the courses concerned used for day-to-day
communication about the courses.
The project had, broadly, six main phases and the Open
University (OU) and the QMW team worked in parallel
most of the time. The phases, broadly, were:
OU1. Gather information about first-year
undergraduates, design information gathering
techniques for future years based on these
results;
QMW1. Search for, or build, systems which would be
useful as calculators supporting functional
programming and formal reasoning;
OU 2. Plan and carry out “before, during and after”
assessments of students on the functional
programming and logic courses;
QMW2. Use the calculators in teaching;
OU3. Analysis of data gathered and writing-up of
results;
QMW3. Improvement of calculators based on feedback
from OU2.
It became clear during QMW1 that while there were
many, many calculators which could be used to support
the logic course (of very varied usefulness, see [3]), there
were none for the functional programming course. Hence,
we had to build our own calculator to support the
functional programming course, which used Miranda.
So, the outcome of these initial phases was the
decision to use Tarski’s World [4], developed by the
Center for the Study of Language and Information,
Stanford University, to support the logic course and our
own MiraCalc [5] which supported the functional
programming course.
In the following academic year the courses “Functional
Programming One” (FP1) and “Introduction to Logic”
(ItL) used these two calculators to support their teaching
via laboratory work and demonstrations during lectures.
The students were surveyed at the start of these courses
using questionnaires and interviews which were designed
on the basis of the results from OU1.
These initial surveys gave us a good idea of the sorts of
students we were teaching and allowed the OU team to
choose representative students according to several criteria
that we had hypothesized might make a difference to their
achievement on the courses. These criteria included gender,
level of achievement in maths, amount of programming
experience and whether they used a computer at home.
These representative students were interviewed in some
depth during and after the courses so that we could get a
very clear picture of how they had perceived the courses.
We also gathered data from tests and exams to give
some quantitative information on their achievements on
the courses. Further, we logged the amount of time that
each student used MiraCalc to see whether there was any
significant difference between students who did and did not
use that calculator.
We will see the results of this data gathering in section
five below.
4: The Calculators
Below we will work through parts of two simple
examples to show how each calculator performs, but first
we make some general remarks. The calculators had to
have two main attributes: robustness and ease-of-use.
They clearly had to be robust since in the environment
that they were to be used in we could not afford to have
them breaking due to mistakes by students or the
inevitable experiments, enthusiastically conducted, whose
goals would be to find out how to break them.
They had to be easy to use in order that the calculators
themselves did not become part of the problem that the
students were facing. They were intended to ease the
students’ burdens, not add to them. They had to have good
supporting documentation or help facilities and it had to
be clear at any point that the students had control.
4.1: Tarski’s World
The main aims of this system are to introduce the
syntax of first-order logic (actually an interpreted first-
order logic with equality), to make the bridge to its
semantics via the idea of an interpretation and to support
the teaching of semantic ideas like consistency,
inconsistency and entailment.
Central to all this is the graphical representation of the
link between syntax and semantics, the interpretation.
Experience shows that students, when presented with the
usual formal definition of an interpretation (a pair of
functions, one mapping terms to objects in the universe
and the other mapping predicate letters to relations of the
appropriate arity), start to give up. In Tarski’s World very
natural use is made of a picture, the situation, to give
meaning to terms and predicates.
The situation displays a picture of the world which
contains some objects of certain shapes (cubes, tetrahedra,
dodecahedra) with certain attributes (names, sizes and their
shapes) and standing in certain relations to other objects
(larger than, in front of, between).
Figure one shows, in the top, left-hand window, a
picture of a “world”, the situation, in which the sentences
in the lower left-hand window are interpreted.
The student can decide whether or not a given sentence
is a well-formed formula or a sentence and then the system
can check for the well-formedness of formulae and
sentences - see the small box in the top right-hand corner
of the sentence window in figure one. This allows exercise
of the syntactic knowledge of the student and, by suitable
examples and questions, allows the support of teaching of
these notions. Figure One Tarski’s World
Having mastered some of the syntax of the language,
the student can then go on into learning about the
semantics. Given a sentence and a situation, the student support tools we looked at (around 15 of them), Tarski’s
can try to reason for themselves about the truth-value of World was amongst the best.
the sentence. Having reached some conclusion, the system
can used to check it. For simple examples it is usually
enough to be told you are wrong. In more complex cases 4.2: MiraCalc
the student can use the “game” to investigate why their
choice was wrong. MiraCalc aims simply to provide an environment in
The game poses a series of questions about the truth- which students can investigate the syntactic structure of
values of parts of the sentence concerned, gradually their scripts, find out more about the type of expressions
“homing in” on a misunderstanding, which is made very in the script and evaluate expressions in the environment
clear by using the situation to show why the truth-value formed by a script in a variety of ways: either fully, as in
was wrongly calculated by the student. Experience shows the Miranda system, in largish chunks (so-called “skips”)
that this is, not surprisingly, viewed as a much more and one step at a time (in the manner of Bird and Wadler’s
interesting way of calculating truth-values than going explanations in [6]).
through seemingly endless exercises using truth-tables. The step-by-step evaluation option turned out to be
Typical problems involve being given a situation and a highly favoured by the students since it allowed them to
list of sentences and being asked which sentences are true see exactly how evaluation in Miranda works and clearly
in the situation and being given a list of sentences and supported the task of working out why their programs
being asked to build a situation that makes them all true, were not working as expected.
if possible. This allows the ideas of consistency and Since for a functional language there are none of the
entailment, and their opposites, to be introduced and complications of, for example, global variables and side-
developed. effects that afflict other sorts of language, the mechanism
As ever with systems which have a graphical for doing step-by-step evaluation and, more importantly,
component at their heart, it is impossible to do justice to the rather small amount of information that has to be
Tarski’s World by words and “snapshots”. The reader is given to the user to enable them to make sense of what is
urged to try it and see. Suffice to say that of all the logic happening at each step, means that the calculator is
relatively simple, one of the main attributes that we
decided on for a successful system.
 Figure Two MiraCalc on a Mac
Figure two shows, in the top window, a parsed script
and in the right-hand lower window the result of the first
step in the evaluation of numsfrom 5. The left-hand
lower window (actually a dialogue box) allows the user to
say how large a step should be taken next.
In order to give a flavour of MiraCalc we briefly
consider three examples. (More detailed examples can be
found in [5] and [7], also note that there are two versions
of MiraCalc: for a Mac and for an X terminal).
Once a script has been typed or read in and successfully
parsed, the Expand and Contract operations can be
used. They are meant to illustrate the structure of program
expressions, the grammar of definitions, the associativity
and precedence of operators, and the structure of curried
functions and functions defined using patterns. Expand is
used to expand expressions, Contract Left and
Contract Right to contract expressions. Back can
be used to go back to previous states of the cursor.
Expand works as follows. It takes the current cursor
selection in the script window and then expands this
selection to cover the smallest enclosing expression. For
example, using a box to represent the extent of
highlighted text in a window, the effect of successive
expansions can be illustrated as
times (double 2) (double 2)
times (double 2) (double 2)
times (double 2) (double 2)
times (double 2) (double 2)
Figure Three MiraCalc under X -
working on scopes
This example illustrates the fact that application
associates to the left.
Another feature of MiraCalc, which is relevant to any
programming language with control abstraction, is that it
gives support for the task of determining when an
occurrence of a name is free, bound or binding in an
environment. This is a part of the use of a programming
language that poses problems for many students, of
course, so it is important that a system like Miracalc
provides support for learning about it.
As an example of the sort of question that might be
posed consider the following: which binding occurrence of
x binds the second occurrence of x in
f x = x + 1 where x = 1
and does f 2 have the value 2 or 3? To answer this
question we can experiment with MiraCalc. Assuming
that we have entered the definition into the calculator we
select the second occurrence of x and check to see that it is
bound by selecting the B o u n d option from the
Calculate menu. A message is displayed confirming
our guess as correct and, importantly for deciding the
binding of this x, the cursor selection in the window is
moved to cover the binding occurrence of x. The window
will now look like the one in figure three.

As a final example of what can be done with MiraCalc,
figure four shows, in the top pane of the window, a script
which contains definitions of various functions for
producing and manipulating various sorts of lists of
natural numbers. One example is
numsfrom_inc x y =
x : (numsfrom_inc (x + y) y)
The function numsfrom_inc gives, with its first
argument being x and its second y, as much of the list of
natural numbers starting
[x,x + y,x + y + y,x + y + y + y...]
as we care to calculate.
In the lower pane of figure four is shown the first five
steps in the stepwise evaluation of
numsfrom_inc 5 1
This feature turned out to be a heavily used, and very
popular, one. An example of what one student said during
interview gives the flavour of their response to stepwise
evaluation:
- I mean to be honest without the calculator I think
everyone would be a hell of a lot more lost
because it’s so much easier. You can just go in and if
you don’t understand how it reduces you can go
Figure Four Stepwise evaluation
through step by step and it will tell you. I mean it’s
like having your own little teacher there saying well,
this is how it does it which is very useful [8].
5: Results
There is room here to present only a brief view of the
results of the project. In the following paragraphs we
paraphrase material from the fuller report [8].
There is no doubt that the use of the calculators helped
students and that they had a beneficial effect on their
learning of formal reasoning. Here we show why we can
conclude this and consider in which ways the tools have
helped and whom they have most helped.
For both MiraCalc and Tarski's World a selection of
assessment measures were used. Data in relation to
Tarski's World are qualitative, using student perceptions
and responses to questionnaires to gauge its effect. This
data is strongly positive in showing the use that students
have made of the tool and the benefits which they perceive
in using it. Replies from questionnaires were from almost
65% of the target population. Of these, 90% had found it
useful. Even more impressive is the 91% of those
students positively recommending that it should be used
for the following student intake.
The benefits that they perceived from using Tarski's
World were varied. It was used regularly, in that 63% of
questionnaire respondents used it for all or for most of the
lab sessions allocated for its use. Data from interviews
showed that students enjoyed using it. A major benefit
was its appealingly simple, user-friendly interface
encouraging exploration:
- When you use the ‘game’ bit you can see the steps,
you can see what you've done wrong. It's a bit like
computer games
- If you don't understand the sentence, then you can
build it up bit by bit. It's easy to try things in
different ways
- It makes it very easy to experiment. You can try
out ideas about what the sentences mean. It's great,
it would be a pain trying out different things on
paper, having to keep erasing things, crossing
things out
Another benefit was that of being able to visualise the
more complex ideas to which students were introduced:
- Now we're doing quantifiers it's really useful
- Now we're going a bit more into depth with you
know...[quantifiers], it's useful for that now. It helps
make sense of it.

- If there is something with lots of ‘nots’ in, then I use
it, to keep check.
- I got a sort of picture in my head, from Tarski's World,
'cos it's easy to picture and I work from there. It is very
useful as a tool. It is good to be able to see everything.
A most interesting finding from the data collected was
the absence of comments on the difficulties relating to
this area of the course. In a study of the previous intake in
QMW [9], a difficulty which students had noted in relation
to this introductory logic course was that of interpreting,
building and manipulating logical statements. The ease
with which students were able to do so using Tarski's
World suggests that this difficulty was being addressed by
the tool.
Data collected on the use of MiraCalc were both
qualitative and quantitative. Analysis and interpretation of
quantitative data indicates clearly that MiraCalc was of
benefit to the students using the system and that its use
had a positive effect on their learning of formal reasoning
methods. Qualitative data, collected from interviews and
returns from questionnaires, indicate that where students
had used it, they perceived the tool as useful and helping
them in a number of areas.
In addition to the qualitative data collected, quantitative
data were available to help assess the effect of using
MiraCalc upon learning outcomes, i.e. records of the
number and length of each student's MiraCalc sessions and
background information given by students at the
beginning of the first semester. For the purposes of this
study, learning outcomes were represented by the marks
which students scored at the end of the year for the course
Functional Programming 1. Since the frequency of scores
in FP1 did not satisfy normal distribution assumptions
the Mann-Whitney U -Wilcoxon Rank Sum W test was
used in the following data analyses.
For a number of the tests, notably those related to
crosstabulation procedures, data relating to FP1 scores
were ordered into the 25%, 50% and 75% quartiles, with
corresponding score values of 26, 56 and 88, e.g. the top
quartile referring to the 25% of students whose end of year
score for FP1 was greater than 88. For tests relating to
MiraCalc use, non-users were classed as those who had
used the software tool for only an hour or less. In relation
to the level of use a scale of five was used, ranging from
very low to very high.
Initial analysis showed a significant difference, in
relation to FP1 scores, between those who had used
MiraCalc in their first semester and those who had not (p
≤ 0. 001 Mann-Whitney). The mean score of those using
MiraCalc was higher than that of those students who did
not use MiraCalc. Next we looked at the level of
MiraCalc use in relation to FP1 scores. Here, there was a
difference between those with a low or very low level of
MiraCalc use and those with an average or above level of
use, but there were no significant differences between
groups as the level of MiraCalc use rose from average to
very high.
Other factors which might have had an effect on FP1
scores were also considered. Among these were
mathematical backgrounds prior to university, previous
experience of programming, previous home use of a
computer, students' motivations, expectations of the
course. These factors and FP1 scores were examined using
Spearman's rank correlation coefficient as a measure of the
relationships between them. Only mathematical
background showed a significant correlation with FP1
scores.
From previous studies [9] we were aware that a
significant factor was likely to be whether students had
studied A level maths or not. (“A level maths” here refers
to a British qualification: “A levels”, i.e. “advanced
levels”, are exams taking in the second-year of the sixth
form at secondary school, which is seventh-form in New
Zealand, which is the last possible year of a school
education and so happens for most students to be when
they are around 17-18 years old.) Further analysis of the
data confirmed the significance of A level maths. In
relation to FP1 scores, a significant difference was shown
between the two groups of students, those who had studied
A level maths and those who had not (p < .003 Mann-
Whitney).
Mean Score Mean Rank Cases
71.4 59.2 29 = 1 maths
49.4 41.5 64 = 2 no maths
While it is very encouraging that the use of MiraCalc
shows a positive effect in relation the end of year scores,
it is even more interesting to look at where the effect is
most evident. We next considered different groupings of
students, using levels of programming expertise and
maths backgrounds in order to address this point.
Perhaps surprisingly, there appeared to be no
significant relationship between the benefits of using
MiraCalc and levels of programming expertise in relation
to FP1 scores.
Looking at the relationship between the use of
MiraCalc, maths background and FP1 however was very
enlightening. Here, in relation to FP1 we looked at two
groups, those students with a mathematical background,
i.e. had studied maths at A levels, and those without a
maths background. Within each of these groups, we then
looked at whether or not they had used MiraCalc (figure
five).
 Mean Score (FP1) Mean Rank
55.11 10.0
78.750 17.2
(two-tailed) p < .03
Figure five Comparing advanced maths students
The results showed an interesting contrast. There is a
significant difference between those who had used the
software tool and those who hadn't, in relation to FP1 end
of year scores, in the group of students who had studied A
level maths. This difference is however is far more
pronounced within the group of students who had not
studied A level maths (figure six).
Indications are then that the use of the software tool
MiraCalc has had a positive effect upon the learning
outcomes and that this effect is most strongly seen in the
results of those students who had not studied A level
maths. Given that one of the principal motivations for
developing and introducing these software tools had been
to address difficulties which 'non' mathematical students
experience, this is a most encouraging result.
A final observation, based purely on anecdotal evidence
from teaching staff, was that students treat a ‘theoretical’
course like Introduction to Logic more seriously when it
is supported with labs just as the programming courses
were.
What appears to be happening is that the Department is
influencing the students’ perceptions about the relevance
of logic. Without the labs, which is how such courses are
often taught, the students see the courses as more
peripheral to Computer Science and hence less important,
so they treat the course less seriously. Typically this
manifests itself in students leaving the logic coursework
until last when there is competition from other courses
and being less willing to devote time to independent study
of the course. (It also has to be said that some teaching
staff are guilty of reinforcing this problem by what they
say about such courses and how they schedule
consideration of them during tutorials.)
In the students’ eyes, the fact that the Department
Mean Score (FP1) Mean Rank
21.143 17.07
57.36 36.82
(two-tailed) p < .001
Figure six Comparing non-advanced maths students
Cases Student group
9 Maths not used MiraCalc
20 Maths used MiraCalc
29 total
committed resources to supporting labs for the logic
course is significant and their attitude towards the course
improved. According to some measure of relevance and
importance, the students seemed more likely to view the
logic course as equal to the programming courses.
6: Further work
Since this project finished, various of its members
have been considering how to continue work in this area,
building on the achievements outlined above. Firstly, it
should be noted that use of the calculators is continuing at
QMW, and they are also now being used at Waikato,
where the developments mentioned below are taking place.
In one main development, work is now underway to
improve MiraCalc. The Mac version (as shown in figure
one above) does not allow use of all of Miranda, though
enough of the language is supported to allow a first course
to use it successfully. This is because of the relatively
short amount of time we had to put into developing the
calculator between the start of the project and the use of
MiraCalc in the functional programming course.
During the final part of the project, after teaching had
finished, we were able to extend MiraCalc to include all of
the language in the form of a Unix-based version which
has an X interface. We decided to follow this route so that
labs that were X workstation-based could also use
MiraCalc, and Mac-based labs could use the full system
via MacX.
Even though MiraCalc implements all of Miranda it is
not truly lazy in that although it uses normal-order
evaluation it does not use sharing. We intend, in future
work, to consider how to include the idea of sharing in the
Cases Student group
14 No maths not used MiraCalc
50 No maths used MiraCalc
64 total
 calculator, though this will pose some problems for the
step-by-step feature since it will require that we show how
a graph-reduction evaluation mechanism works, which is
likely to be much harder to show in a way which does not
confuse students too much.
Using many of the lessons learnt on the project
reported here, another calculator is being developed to
support working on program derivation within
constructive type-theory [10]. This is a single language
(syntax, semantics and proof theory) within which formal
specifications (very expressive types) can be written and
functional programs can be derived (according to the proof
rules) so the programs necessarily provably meet their
specifications. This work is aimed at graduate-level
students and the calculator as so far constructed
(PICTCalc) has been used by several classes of M.Sc.
students enrolled on the course “Constructive Mathematics
and Programming” at QMW.
7: Acknowledgments
As ever with educational research, we have to thank the
students, who suffered all of the questionnaires, interviews
and novel software associated with this work with
excellent humour, and our colleagues at QMW on the
teaching staff. Without the help and goodwill of both
these groups the work reported here would not have been
possible.
References
[1] Fung, P., O’Shea, T., Goldson, D., Reeves, S. and
Bornat, R. (1992) Computer Science Students'
Perceptions of Learning Formal Reasoning
Methods in International Journal of Mathematical
Education in Science and Technology, pp 749-760.
[2] Fung, P., O’Shea, T., Goldson, D., Reeves, S. and
Bornat, R. (1994) Understanding why computer
View publication stats
science students find formal reasoning frightening.
Appearing in December '94 issue. Journal of
Computer Assisted Learning.
[3] Goldson, D., Reeves, S., Bornat R. (1993) A
Review of several systems for the Support of
Logics in The Computer Journal, vol. 36, no. 4,
pp. 373-386.
[4] Barwise, J. and Etchemendy, J. (1991) T h e
Language of First-Order Logic, CSLI, Stanford
University.
[5] Goldson, D. (1994) A Symbolic Calculator for
Non-Strict Functional Programs in The Computer
Journal, vol. 37, no. 3, pp. 177-187.
[6] Bird, R. and Wadler. P (1988) Introduction to
Functional Programming, Prentice-Hall.
[7] Goldson, D., Hopkins, M. and Reeves, S. (1994)
MiraCalc: The Miranda Calculator, The Unix
Version. Working paper 94/5, Department of
Computer Science, University of Waikato, New
Zealand.
[8] Fung, P. and O’Shea, T. (1994) Using Software
Tools to Learn Formal Reasoning: A first
assessment CITE Technical Report no. 197 Open
University, Milton Keynes, UK.
[9] Fung, P. and O’Shea, T. (1993) Learning to reason
formally about programs: an observational study of
computer science students. CITE Technical Report
no. 168. Open University, Milton Keynes, UK.
[10] Reeves, S. (1994) Computer support for students'
work in a formal system: MacPICT, to appear in
International Journal of Mathematical Education in
Science and Technology.