education
sciences
Article
Evaluation and Use of a Student-Centered Syllabus for the
Software Process Subject in a Postgraduate Course: A
Quasi-Experiment
José Augusto de Sena Quaresma *
Citation: de Sena Quaresma, J.A.;
Oliveira, S.R.B. Evaluation and Use
of a Student-Centered Syllabus for
the Software Process Subject in a
Postgraduate Course: A
Quasi-Experiment. Educ. Sci. 2022,
12, 851. https://doi.org/10.3390/
educsci12120851
Academic Editor: Mike Joy
Received: 13 October 2022
Accepted: 21 November 2022
Published: 23 November 2022
Publisher’s Note: MDPI stays neutral
with regard to jurisdictional claims in
published maps and institutional affil-
iations.
Copyright: © 2022 by the authors.
Licensee MDPI, Basel, Switzerland.
This article is an open access article
distributed under the terms and
conditions of the Creative Commons
Attribution (CC BY) license (https://
creativecommons.org/licenses/by/
4.0/).
Educ. Sci. 2022, 12, 851. https://doi.org/10.3390/educsci12120851
and Sandro Ronaldo Bezerra Oliveira
Graduate Program in Computer Science (PPGCC), Federal University of Pará (UFPA), Belém 66075-110, PA, Brazil
* Correspondence: augustoquaresma@ufpa.br
Abstract: Background: The software-development process is considered a knowledge area in soft-
ware engineering that allows the performance of activities for the inception of a functional software.
In a globalized software context, where companies and organizations have their data and information
controlled by applications, it can be seen that the form of construction, that is, the path for which
the software is built, has relevance for users. Problem Analysis: Within this reality, it is possible
to observe that over the years, scientific research has identified competence deficits for profession-
als trained in software engineering and more specifically in the software-development processes.
Purpose: This work aims to develop and apply a syllabus for the software-development process,
with student-centered learning strategies, as it allows for centrality of the teaching and learning
process for the student. Methods: The strategies were selected through a literature review, which
informed which ones were adopted to intervene in the traditional teaching process in software.
Other methodological procedures adopted to identification of the necessary requirements for the
development of the syllabus were (i) a survey with the identification and perception of professionals
and professors on the research theme, (ii) equivalence mapping of the main documents related to the
theme of research at an educational level, and (iii) the identification of how to develop a syllabus.
After gathering the requirements, the syllabus was designed and described in each of its parts based
on the specialized literature. The evaluation of the syllabus took place in two stages: (i) an expert
panel, where experts in the software-development process and student-centered learning were se-
lected, who contributed points to be adjusted for the use of the syllabus, and (ii) the experiment,
which was the use of the syllabus in comparison with the traditional approach (use of theoretical
classes, exercise list, and written test), through the control and experimental groups. Results: Data
generated by using the syllabus in the experiment, as well as data from the control group, were
analyzed using the two-tailed Student-t technique. The results achieved in the work demonstrate
that there was a considerable gain in learning compared to the control group. Another result point
of the syllabus was that through the project strategies, students had contact with a progression of
knowledge related to hypothetical cases that simulated reality, which students reported was very
good for their knowledge. Conclusions: The use of student-centered strategies can bring benefits in
the learning of software-development processes, as it allows a practical application of real scenarios
observed in software-development companies.
Keywords: student-centered learning; syllabus; active methodologies; software-development process
1. Introduction
Software development has become increasingly necessary in a globalized world.
Companies and organizations need control over their information about products and
processes, since software can quickly and consistently present information about the
company’s production or the best ways to save money in the production process.
https://www.mdpi.com/journal/education
Educ. Sci. 2022, 12, 851 2 of 21

The process is a set of interrelated or interactive activities that transform inputs into
outputs (products) [1]. When we refer to the software-development process, the same
concept can be adopted, with the specification, as is possible to see in SWEBOK (Software
Engineering Body of Knowledge) [2], that it is identified that the product is the software
working with or without documentation.
The software process, or software-development process, is an area of software engi-
neering. As seen above, the software process is a set of interrelated or interactive activities
that transform inputs into outputs (products) [1], whose main output is working software.
The teaching of the software process takes place within the subject of software engi-
neering in undergraduate courses in computing, as can be seen through the international
curriculum of the ACM/IEEE (Association for Computing Machinery/Institute of Electrical
and Electronic Engineers) for computing [3], as well as in the Brazilian reality proposed by
the document of reference of the Brazilian Computer Society (SBC) [4].
Professionals trained in computing have difficulty adapting to the competences re-
quired by the market, either because they only know the theory and not how to apply
it in practice, or because they have competences and still do not have the expertise to
solve a real problem, such as is stated in [5]. In [6], an analysis of the perception of
professionals identified that this deficiency is still occurring in professionals in the software-
development process.
The software process has been identified as a necessary knowledge area for software
development today, in addition to the fact that professionals who have graduated or who
are already in the job market have difficulties in carrying out the activities necessary for the
smooth running of software companies.
In general, the teaching of the software process is adopted through classic lectures with
an evaluation application—that is, the teacher is at the center of the teaching and learning
process, as can be seen in [7]. However, within the literature it is possible to identify efforts
to achieve student-centered learning through the formal literature review proposed in [8].
One of the strategies adopted to improve the teaching process is student-centered
learning, which was dubbed by psychologist Carl Rogers and has as significance the central-
ity of the student to the teaching and learning process and the teacher as a facilitator of this
process [9]. Student-centered learning brings the following advantages: facilitating student
learning, approximation of the teacher with the students, and improved interpersonal
relationships. Furthermore, it can be visualized through the use of project-based learning,
flipped classroom, gamification, and serious games, which are teaching strategies that
consider student-centered learning [9]. Each of these teaching strategies will be presented
in more detail regarding their use and conceptualization in a later section of this article.
In the studies [10,11], intervention in the teaching and learning process in software
engineering is reported, with student-centered learning. As in studies [7,12,13], proposals
for teaching specific areas of the software process are discussed. Therefore, the research
trend for the teaching process is visualized, specifically for the software process.
In this sense, the guiding question of this research is: How can the software process be
taught with student-centered learning and verify whether the teaching is more effective
than the traditional process adopted? As a way of answering this research question, the
following objective was defined: Develop and apply a syllabus for a software-process
subject, with strategies adopting student-centered learning.
The proposed methodology to achieve the intended result followed three phases with
nine steps, which will be detailed more specifically in the corresponding section. As an
introduction, the quasi-experiment is an evaluation methodology that uses the syllabus
in a real environment, by which a control group is compared with an experimental one
and the gain or loss is verified in relation to the two groups. As a way of defining the
experimental methods, we identified the use of the approach with a certain objective of the
study, selecting the variables to be observed and the form of control to be adopted. This
information is corroborated in [14].
Educ. Sci. 2022, 12, 851 3 of 21

The context of the development and evaluation of the research is a federal university
in the north of Brazil within the graduate program in computer science, which has a line
of research and performance in software engineering. In this context, every semester
the subject of Technology in Software Processes is offered on an elective basis. For the
graduate program in question, both master’s and doctoral students take the same subjects.
In addition, it should be noted that a number of vacancies are available for the public
outside the university, which is called an audit student. The audit student category can
apply for a master’s or doctoral degree in the graduate program to which they apply in
the future.
The class with which the experiment was conducted had 20 students, of which 10
were in the control group and 10 were in the experimental group. Of these students, the age
group was between 20 and 40 years old, with people who already worked professionally
with computing. Classes took place in the morning shift. A total of 20% of the students
were doctoral students, 70% were masters, and 10% were listening students, and 10% were
female and 90% male.
With the execution of the experiment, there was a learning gain with the proposed
approach—that is, the syllabus in relation to the traditional approach—based on the two-
tailed Student-t analysis with the Shapiro–Wilk reliability test, which will be explained
in more detail in the data analysis. In addition, through the perception and reports of
the students and the use of strategies with student-centered learning and situations close
to reality, it was identified that there was knowledge progress through the provision of
contents related to the strategies.
The limitations and implications of this research do not allow the results to be general-
ized, so the context in the research is accurately presented and future work is necessary for
this research.
In addition to this introductory section, this article is structured as follows: In Section 2
some research related works are discussed; Section 3 defines the methodological path
adopted to reach the defined objective and answer the research question; Section 4 presents
methods defined to verify and evaluate whether the use of the syllabus was effective
for the teaching and learning process; in Section 5 the data collected by the experiments
are treated and discussed regarding whether there was a learning gain for the software-
process students; in Section 6, based on the analyses collected from the experiment data, the
implications of these results for the proposal worked in this article are discussed; Section 7
discusses information regarding threats to the validity of the evaluation through a quasi-
experiment for the syllabus; and, finally, Section 8 presents the conclusions reached in the
development of the quasi-experiment for a syllabus in software-process subject.

2. Related Works
In the academic literature it is possible to see several works on the teaching and
learning process in software-process subjects through the work elaborated by [8]. The
authors developed a literature review and identified works that report on student-centered
learning for the software process. The works related to this research are proposed for the
teaching of the software process, and include the following: Refs. [11–13].
Ref. [12] reported on the development of a game with artificial intelligence for teaching
software-process improvement. The game was evaluated through its interface, which took
into account usability and learning. In the work, the authors stated that it is just a proposal,
in addition to focusing only on the teaching of one of the contents visualized for the
software process, which is improvement. The difference in the proposal developed by
this research is that it is not being evaluated through use and verified by a control and
experimental group.
The study by [13] discusses the construction of a board game to promote understand-
ing of the definition of a software process based on ISO 29.110. With game elements, the
authors carried out an experiment that did not have a control and experimental group to
perform the necessary comparisons. The difference for the proposal developed by this
Educ. Sci. 2022, 12, 851 4 of 21

research is the evaluation format of the proposal, in addition to the fact that the game de-
veloped only focuses on the concept of the software process—that is, it lacks the necessary
content to achieve the competences required by the curricula.
Ref. [11] reported on the use of ludic resources for teaching the software process
within software engineering. One of the adopted resources reported by the authors was
group dynamics. In this sense, the authors reported, through their perception, the gains
considered in the students’ learning with the use of these resources, which are different
than the traditional approaches of the subject. The differences in this proposal are (i) the
evaluation format, as the study does not work with statistical analysis; (ii) the subject in
which it is applied—in the study, it is an undergraduate subject; and (iii) the number of
strategies adopted to bring the student to the center of the learning process.
Therefore, with the related works and the literature review, a tendency can be identified
in researching ways of teaching the software process. However, there is still no study to
propose a syllabus for content related to software-process subjects.
In this sense, the research proposed by this work presents a set of strategies focused
on the student through a syllabus in the software process, which is verified and validated
through an experiment with a control and experimental group in the context of a subject of
a graduate program.

3. Research Methodology
The evaluation of the software-process syllabus was conceived by adopting the steps
Educ. Sci. 2022, 12,described
x FOR PEERinREVIEW
Figure
1. The research can be classified as mixed methods—that is, qualitative
and quantitative [15].

Figure
Figure 1. Research 1. Research methodology.
methodology.

3.1. Literature
The methodological Review
path presented by Figure 1 identifies several paths, with the
possibility of parallelism, return to the flow
The development of the(loop by questioning),
research started withorthe sequentially following
literature review activity—t
the research. The stages of the flow were developed sequentially, as it was one
formal research was carried out for teaching the software process, with student-cenauthor’s doc-
toral research and the secondWith
approaches. wasthemonitoring,
conclusion reviewing, and guiding
of the review, there werein the development
several inputs for the sy
of activities—that is, the execution of the stages was the responsibility
on the software process, such as approaches to teaching the software of one person. In
process, diffic
the first part of faced
the figure we have Steps 1 and 2 of the research, which could be developed
in the software process, and verification of studies carried out for teaching th
in parallel. Step 3, with
ware its perception of some characters involved in the teaching and
process.
learning process of The the software
product of process, would need
the development inputs
of the collected
literature by the
review can literature
be seen In [8], wit
review, resulting from Step 1, such as the mapping of assets in process
liometric data, information on approaches, and strategies adopted teaching, resulting
for teaching the
from Step 2. Stepsware process. In addition, it presented problems when teaching theStep
3–6 were performed sequentially. There was a questioning after software pr
6, with the verification
Through of thethe need of
results forthe
adjustments to carryitout
literature review, the use to
is possible of arrive,
the syllabus.
through an an
If so, the activity
andto systematization
be developed was the results,
of the implementation of these
at a teaching catalogadjustments through
of the software process wit
the return to Steps 4 and 5, and if not, the flow proceeded
dent-centered learning. Such a result can be seen in [16].to Steps 8 and 9. At this initial
Student-centered learning has its origins in the public theory by Carl Rogers [9
proposal emphasizes the potential of the student through positive and negative aspe
their own study process. It is worth noting that in this learning modality the tea
process is bidirectional—that is, in addition to the teacher teaching, he can learn wi
students [10].
Educ. Sci. 2022, 12, 851 5 of 21

moment, only the possibilities of the flow were identified, but in the following subsections
is a description of the research step.

3.1. Literature Review

The development of the research started with the literature review activity—that is,
formal research was carried out for teaching the software process, with student-centered
approaches. With the conclusion of the review, there were several inputs for the syllabus
on the software process, such as approaches to teaching the software process, difficulties
faced in the software process, and verification of studies carried out for teaching the
software process.
The product of the development of the literature review can be seen In [8], with
bibliometric data, information on approaches, and strategies adopted for teaching the
software process. In addition, it presented problems when teaching the software process.
Through the results of the literature review, it is possible to arrive, through an analysis
and systematization of the results, at a teaching catalog of the software process with
student-centered learning. Such a result can be seen in [16].
Student-centered learning has its origins in the public theory by Carl Rogers [9]. The
proposal emphasizes the potential of the student through positive and negative aspects
of their own study process. It is worth noting that in this learning modality the teaching
process is bidirectional—that is, in addition to the teacher teaching, he can learn with his
students [10].
Therefore, for this research, we refined and adopted such teaching strategies in the
software process: flipped classroom, project-based Learning, Gamification, Concept Map,
and Dynamics.
The flipped classroom is the inversion of understanding of the agents of the teach-
ing process inside and outside the classroom [17]. For [17], this strategy proposes and
encourages debates, group work, and explanations about the subject’s concepts in the
classroom—that is, allowing students to become active in the teaching and learning process.
The students’ extracurricular activities are research on topics covered by the contents of the
subject and the teacher as a planner to facilitate the possibility of students explaining and
debating subjects.
Project-based learning (PBL) is considered a student-centered learning methodol-
ogy, as it allows students to build a solution project based on real problems through the
formation of groups and the resolution of investigative tasks [18]. This methodology is
intended to stimulate the critical thinking of the students through the collection of data and
team discussion of the best way to analyze and present the information as a result of the
developed project.
The use of game elements in contexts other than games is the definition of gamifi-
cation [19]. This strategy is configured as student-centered because it allows, through
the elements of games inserted in the classroom or in the developed works, for stimu-
lation of competition, motivation, and collaboration of students in their teaching and
learning process.
The concept map is the visual representation of how a person abstracts a concept or
content via an illustrative graphic [20]. This strategy allows the student to systematize
as well as identify the existing connections between the different subjects covered in
the classroom.
For [21], dynamics are moments of playful education in the classroom. One of the
forms presented by the authors is collective history, where a group of students is co-
responsible for describing a story about the content covered in the classroom in a narra-
tive way.

3.2. Mapping of Assets on Software-Process Teaching

The second step of the research was to map the assets of teaching the software process,
wherein the main documents on teaching the software process were analyzed. they included
Educ. Sci. 2022, 12, 851 6 of 21

the 2017 reference document for computing from the Brazilian Computer Society (RF-SBC),
the curriculum for ACM/IEEE Computing (CS-Curricula ACM/IEEE) of 2013, and the
Software Engineering Body of Knowledge (SWEBOK) version 3. Mapping and generating
correspondence between the assets allowed us to define the main contents and competence
necessary to generate the syllabus for the software process. The results of the mapping—that
is, the contents and competences worked on in this syllabus—are available in [22].

3.3. Perception of Teachers and Professionals in the Software Process

The third step of the research was to consult the perception of teachers and profes-
sionals in the software process. At that moment, a survey was built and several teachers
and professionals were consulted about the main pedagogical components of the software
process, namely, approaches, strategies, resources, and contents. The result of the research
development allowed the main pedagogical components adopted by the teachers in the
teaching of the software process to be visualized, as well as the most relevant subjects for
the professionals of the industry. A discussion of the planning, dissemination, and analysis
of the teacher and professional survey can be seen in [6].

3.4. Definition of the Construction Format for the Syllabus

The fourth step of the research was the definition of the format for building a syllabus.
At this step, basic research was conducted to identify how teaching models and teaching
plans are developed. Some studies were found, such as [23–25]. With this research, the
following format was defined in topics: Identification of Competences, Plan Contents,
Identification of Contents in the Literature, Description of Teaching Strategies, Lesson
Planning, and Evaluation Strategies for Teaching Units.

3.5. Construction of the Software-Process Syllabus

The fifth step of the research was the construction of a software-process syllabus
based on three methodological procedures: (i) literature review, (ii) mapping of assets on
software-process teaching, and (iii) a survey on the perception of professionals and teachers
about the software-process area. The construction of the syllabus, as well as its evaluation
by a panel of experts, can be consulted in [26]. Some succinct information from the syllabus
is as follows:
• Competences: (i) Apply techniques, tools, and practices to manage the process pro-
duction, acquisition, and evolution of a software; (ii) apply software-development
processes; (iii) understand software-product and -process quality standards and mod-
els; and (iv) apply process-quality concepts to the definition of a software process.
• Contents: (1) software work products and roles, (2) software-process concept, (3) software-
process models, (4) software-process representation, (5) product-quality models and
standards (national and international), (6) process-quality models and standards (national
and international).
• Teaching Units: I—Introduction to the software process, II—Software work products
and roles, III—Software process and product models and standards.
Each of the teaching units was detailed in terms of (i) prerequisites, which are the
subjects and/or contents that can facilitate learning if they are previously taken by the
students, which must provide the theoretical basis necessary to follow the planned contents
for each highlighted unit; (ii) guiding questions, which are the questions asked of the
students during the beginning of each unit that aim to start the discussion of the theme
from the exposition of a problem to the students; (iii) programmatic content, which is the
contents that will be taught in the units so that the student can achieve the competences;
(iv) teaching strategy, which is the planned way of teaching the syllabus topics for each
unit; (v) expected results, what the student should be able to learn and accomplish after
learning the unit, i.e., the learning results; and (vi) learning level—each of the expected
results was detailed with a certain level of expected cognitive ability, using a terminology
based on Bloom’s Revised Taxonomy [27].
Educ. Sci. 2022, 12, 851 7 of 21

The contents adopted In the planned teaching units were extracted from the bibliogra-
phy and literature available and commonly used for the teaching of software engineering.
They include books, reports, manuals, standards, and models.
From this, the teaching strategies adopted for the software-process subject were de-
scribed in a general way. It is worth noting that the approach adopted for development is
the cognitivist approach, where the knowledge of the parties involved in the teaching and
learning process is a continuous construction. The continuous construction of knowledge
identifies the student as a functional being who, when relating to the organization and
integration of content with their ideas, can build their learning.
Finally was (i) the planning and distribution of classes in teaching units in relation
to the adopted strategy and how it communicates with the content covered, and (ii) the
definition of the evaluation of the teaching and learning process, which took place continu-
ously within the subject through established criteria for the students to develop activities
within the teaching strategies, where for each strategy the students were evaluated by
their development.

3.6. Evaluation of the Syllabus by a Panel of Experts

The sixth step in the research was the evaluation of the syllabus through a panel of
experts, with researchers who are experts in the software process and student-centered
teaching strategies. The developed evaluation brought necessary contributions for the
improvement of the study plan before being used in a practical experience [26].

3.7. Implementation of the Requested Adjustments

The seventh step of the research was the implementation of the adjustments requested
by the experts. The implemented adjustments followed the suggestion present in the
systematization of the data carried out by the experts’ evaluation [26].

3.8. Using the Syllabus in a Quasi-Experiment

The eighth step of research was the use of the syllabus in a postgraduate course
through a quasi-experiment. The experiment was developed and planned by taking into
account a control and experimental group. In addition, for each of the content units
developed by the syllabus, a research question was stipulated with a hypothesis to be
investigated through statistical analysis.

3.9. Study-Data Analysis

The ninth step of the research was the analysis of the study data to identify whether
there was a learning gain for the experimental group—that is, the group that adopted the
new proposed strategy.

4. Syllabus Evaluation Format for the Software Process

This section describes how the syllabus was evaluated through a quasi-experiment.
The syllabus was used in a postgraduate course. It was subdivided into experiment
planning and data analysis.

4.1. Experiment Planning

During the planning phase of the experiment, the following steps were followed,
both for the control and experimental groups: characterization of the subject, scenario,
characters, schedule, and data-collection and evaluation methods.
Characterization of the subject: The subject chosen for the development and follow-up
of the software-process syllabus was the Technology in Software Processes subject offered
in the postgraduate course in Computer Science at the Federal University of Pará, in Brazil.
This subject is considered optional in this course and has a workload of 60 h, of which
30 are theoretical hours and 30 are practical hours. For both the control group and the
experimental group, it took place in person in the first semester of 2022.
Educ. Sci. 2022, 12, 851 8 of 21

Scenario: The choice was made to use the syllabus for the software process, a subject
in the postgraduate course in Computer Science. Therefore, the main scenario for the
development of the teaching and learning process was the classroom. For the control group,
there was the virtual learning environment (called SIGAA) and the face-to-face classroom,
where explanations, debates, and resolution of exercise lists took place. For the experimental
group, there was a face-to-face classroom, where explanations and the development of
student-centered strategies took place. A group in WhatsApp and the Google Classroom
tool were used as a virtual classroom, where students had direct communication with the
teacher to clarify doubts and topics related to the activities of the subject.
At this point, it is necessary to identify and describe the virtual learning environments
adopted in the experiment. They included SIGAA and Google Classroom. SIGAA is
the integrated academic-management system developed by the Technology Center of the
Federal University of Rio Grande do Norte, a public university in the northeast of Brazil,
which was adapted to the reality in which the experiment was developed, in this case a
federal university in the north of Brazil. For the purposes of understanding and use for
the experiment, SIGAA was used in student and teacher mode and the features adopted
were (i) in teacher mode, the availability of material adopted in the classroom and launch
of concepts, and (ii) in student mode, downloading and viewing files by students.
Regarding the use of Google Classroom, the functionalities used were (i) by the teacher,
the creation of a classroom, structure of ordering, and visualization of the files worked
on in the semester and the creation of exercises to attach evidence of the development of
strategies, and (ii) by the student, viewing and downloading of the content posted by the
teacher, as well as viewing the feedback given in the activities. In addition, WhatsApp was
used as a tool to complement the virtual learning environment to facilitate the teacher’s
communication with the students. In this case the features adopted for this experiment were
the creation of a group for the subject, which had discussion conferences and a conversation
channel with the teacher to resolve doubts regarding the content and teaching strategy
adopted in the classroom.
The characters involved in the experimental group were (i) a group of 10 postgraduate
students—whether masters, doctoral, or audit students, they are considered the interested
party in the learning process; (ii) one professor who was responsible for the teaching
process—that is, who passed on the contents and planned the activities to be developed
in the classroom; and (iii) one support student who helped the teacher in the teaching
process through examples in the classroom and in the correction of some related activities
for the students.
The characters involved in the control group were (i) a group of 10 students from the
postgraduate course—whether masters, doctoral, or audit students, they were considered
the interested party in the learning process, and (ii) one professor who was responsible for
the teaching process, that is, who passed on the contents and planned the activities to be
developed in the classroom.
It is worth noting that in this specific semester the number of students enrolled in the
course was 20 students, as it is an elective course in the graduate program. Thus, an equal
sample was chosen—that is, 10 students in the control group and 10 in the experimental
group. Another point to highlight is that increasing the number of students should not
interfere with the execution of the experiment, since, as seen in the description of student-
centered approaches, there is no restriction on the number of students. Therefore, we intend
to develop a new experiment with a larger number of students to comparatively evaluate
the results obtained.
The content related to the subject of the software-development process was subdivided
into three teaching units. Namely, the initial concepts, work products and roles, and
quality models and standards were presented. In Teaching Unit I the basic concepts of the
subject were taught, including an introduction to the conceptualization of the software
process, as well as fundamental and support-process activities, the composition of a process
structure, technologies, and process modeling. In Teaching Unit II the two contents that
Educ. Sci. 2022, 12, 851 9 of 21

were addressed were work products, with their identification and development, and roles,
with their identification and description in the process. Finally, in Teaching Unit III models
and quality standards were presented, which were worked on to identify and visualize
the ways in which they are addressed in their implementation and evaluation in software-
development companies.
The development of classroom classes for both the control and experimental groups
followed the schedule presented in Table 1.

4.2. Method of Data Analysis

Based on the planning of the quasi-experiment, a comparison was made between
the traditional teaching structure and the proposed approach, that is, a software-process
syllabus. By “traditional approach,” the following is understood: a lecture, with an eval-
uative activity through lists of exercises and tests. For the syllabus, we made use of
the following student-centered teaching strategies: conceptual/mind map, gamification,
flipped classroom, project-based learning, extended classroom, dynamics, and feedback.
The comparison is shown in Figure 2.
As a way of validating the proposal, research questions and hypotheses were estab-
lished that made it possible to analyze the attendance to the degree of learning of the
teaching and learning process in the experimental group versus the control group, based on
Bloom’s revised taxonomy [28]. In this context, the scores achieved in each of the teaching
units were adopted. The planning data for evaluation are shown in Table 2, which, in
addition to the research questions and hypotheses, has the analysis formulas with the
data-collection instrument.
One of the points presented in Table 2 is the weight considered for each of the activities
in the formulas. This score was assigned from the most difficult strategy to be developed
to the easiest. Therefore, for this experiment, the weight assigned to the project was 50%
of the grade. The flipped classroom, with the presentation of the contents studied by the
students, was assigned 30% of the grade to the first unit, and 40% to the other units. The
concept map was equivalent to 10% of the students’ grade and was developed after some
work content in the classroom. Finally, the dynamics, which also amounted to 10% and
were only present in Unit I. As a way of exemplifying the difficulty and weight assigned to
each of the strategies, the command related to the strategy is described below. It is worth
noting that each of the strategies had context, requirement, and evaluation criteria.
Concept map: (i) context—the concepts studied in our room so far, initial concepts, and
fundamental and support activities for the software process; (ii) requirement—development
of a concept map on initial concepts and fundamental and support activities for the software
process; and (iii) evaluation criteria—the student related the initial concepts to each other
in the concept map, as well as the fundamental and support activities.
Dynamics, collective history: (i) context—based on studies on process models we built
a joint history on how companies can use process models for the development of the same
product; (ii) requirement—in the description of the process one or more process models
was used as a basis, the basic or fundamental activities were presented, and at least two
good practices were identified per agile methodology studied; (iii) evaluation criteria—the
student described using at least one of the software-process models and presented all the
basic or fundamental activities, and in the description the student identified and described
how the good practices chosen for the process were used (in the activities or tasks). At least
two practices/characteristics per agile methodology were studied.
Flipped classroom: (i) context—based on the concept of software-process activities, the
importance of activities for companies and their context in the process were presented; (ii)
requirements—the student presented the concept related to the activity, with two examples
with a set of tasks for the activity; (iii) evaluation criteria—the student presented the concept
of related activities to the team, as well as two examples per related activity, with a set
of tasks.
Educ. Sci. 2022, 12, 851 10 of 21

Table 1. Planning for the control and experimental group class days.

Days Experimental Group Control Group

Presentation of the professor, the subject, their way of conducting Presentation of the professor, the subject, their
the teaching plan used, and the students. In addition, all the way of conducting the teaching plan used, and
Day 1 (inaugural class) teaching methodologies used in the classroom were discussed. the students.
Availability of the virtual learning environment (Google Classroom Availability of the virtual learning
and WhatsApp). environment (SIGAA).
Theoretical and practical classes: on the contents worked on in
Teaching Unit I, initial concepts of the software process.
Concept/mind map: After each presentation/explanation of
content, a concept/mind map was requested as a way of
consolidating the learning.
Gamification: use of game elements to stimulate the teaching and
learning process.
Flipped classroom: Some concepts of the syllabus were requested
for students to present, where the concept, its importance, and its
practical or hypothetical applicability were requested.
Project-based learning: Three contexts of software-development
companies were created. In this first unit the students were
Lectures: about the contents covered in
encouraged to describe the process structure adopted by the
Teaching Unit I.
company, as well as the representation of this process.
Days 1 to 13 Test: evaluation activity with multiple-choice
Extended classroom: The Google Scholar environment became a
and discursive questions about the content
fundamental tool for delivering activities and planning the subject,
taught in Teaching Unit I
as students could follow the times and deadlines for the delivery
of activities.
Dynamics: In the first teaching unit, a dynamic was established
with the class, and after the presentation of concepts about agile
methods, the development of a collective story was requested,
which was a narrative text that would need to have essential
practices for each of the agile methods studied.
Feedback: Two forms of feedback were established for the subject:
(1) the delivery of activities, scored according to the established
evaluation criteria, which took place up to a maximum of one week
after delivery, and (2) collecting information from students about
the teaching resources and strategies used in the classroom.
Theoretical and practical classes: on the contents worked in
Teaching Unit II, work products, and roles in the software process.
Concept/mind map: Same as for Teaching Unit 1.
Gamification: Same as for Teaching Unit 1. Lectures: about the contents covered in
Flipped classroom: Same as for Teaching Unit 1. Teaching Unit II.
Days 14 to 19 Project-based learning: Same as for Teaching Unit 1. In this second Test: evaluation activity with multiple-choice
teaching unit the students were encouraged to describe and and discursive questions about the content
evidence models of software work products, as well as to describe taught in Teaching Unit II.
the roles.
Extended classroom: Same as for Teaching Unit 1.
Feedback: Same as for Teaching Unit 1.
Theoretical and practical classes: on the contents covered in
Teaching Unit III, models, and standards of the software process.
Concept/mind map: Same as for Teaching Unit 1.
Gamification: Same as for Teaching Unit 1. Lectures: about the contents covered in
Flipped classroom: Same as for Teaching Unit 1. Teaching Unit III.
Days 20 to 30 Project-based learning: Same as for Teaching Unit 1. In this third Test: evaluation activity with multiple-choice
teaching unit the students were encouraged to adopt one of the and discursive questions about the content
constant processes in each of the models worked on in taught in Teaching Unit III.
the classroom.
Extended classroom: Same as for Teaching Unit 1.
Feedback: Same as for Teaching Unit 1.
OR PEER REVIEW 11 of 22
Educ. Sci. 2022, 12, 851 11 of 21

Figure 2. ComparisonFigure
between the traditional
2. Comparison teaching
between structure
the traditional versus
teaching the proposed
structure approach.
versus the proposed approach.

Project-based learning: (i) context—based on the specific context for each of the
As a way of validating
companies, the
theproposal,
consulting research questions
teams in the and hypotheses
software process developed a were
projectestab-
on software
lished that made it artifacts;
possible(ii)torequirements—each
analyze the attendance to needed
of the teams the degree
to meetof the
learning of thefor the
requirements
teaching and learning processofinthe
completion the experimental
project, which weregroup versus the
the elaboration of atcontrol group,
least four modelsbased
of software
artifacts in text-document
on Bloom’s revised taxonomy format, the
[28]. In this context, the elaboration of at least
scores achieved four of
in each models of software
the teach-
artifacts in audio and/or visual-evidence format, the completion of at least three models
ing units were adopted. The planning data for evaluation are shown in Table 2, which, in
in the software-process cycle (both for the model and for the application, filling them
addition to the research
out, and questions
using one and hypotheses,
for each fundamental has the analysis
or basic activity offormulas with
the software the and
process);
data-collection instrument.
(iii) evaluation criteria—elaboration of models in text-document format with all the sections
and an explanation of how to fill it out, preparation of the models in audio and/or visual
Table 2. Detailing the format, and detailing how team members should proceed in the generation of this artifact,
study objectives.
correctly filling them out according to the model’s indication in the document format, and
Experimental Research correctly filling them
Question out according
1 (ERQ1): Whattoisthe
themodel’s indication
learning in audio and/or
effectiveness visual format.
of Teach-
As can be seen, the proposed model for the description of each of the activities for the
ing Unit I when the proposed approach using student-centered approaches to teaching
teaching strategies allowed, through the three items (context, requirements, and acceptance
the software processcriteria),
is adopted over to
the student the traditional
acquire approach?
a broader understanding of the activity.
Hypothesis 1 (H01): In Teaching Unit I, there will be no difference between the scores
Table 2. Detailing the study objectives.
obtained by the experimental and control groups at the Apply level of Bloom’s revised
taxonomy. Experimental Research Question 1 (ERQ1): What is the learning effectiveness of Teaching Unit I when the
proposed approach using student-centered approaches to teaching the software process is adopted over the
Variables: traditional approach?
MC1—Concept/MindHypothesis
Map 1 1 (H01): In Teaching Unit I, there will be no difference between the scores obtained by the
experimental and control groups at the Apply level of Bloom’s revised taxonomy.
MC2—Concept/Mind Map 2
MC3—Concept/Mind Map 3
AI1—Presentation 1 (Flipped Classroom)
AI2—Presentation 2 (Flipped Classroom)
AI3—Presentation 3 (Flipped Classroom)
Educ. Sci. 2022, 12, 851 12 of 21

Table 2. Cont.

Variables:
MC1—Concept/Mind Map 1
MC2—Concept/Mind Map 2
MC3—Concept/Mind Map 3
AI1—Presentation 1 (Flipped Classroom)
AI2—Presentation 2 (Flipped Classroom)
AI3—Presentation 3 (Flipped Classroom)
AI4—Presentation 4 (Flipped Classroom)
PR1—Project 1 (Project-Based Learning)
PR2—Project 2 (Project-Based Learning)
DI—Dynamics (Collective History)
P1—Unit I Test
Point Scale: For all the variables adopted in the first research question, the score attributed by the evidence of
execution of the strategy was assigned a score from 0 to 10 per student.
Formulation: Ma > Mb, where:
a = experimental group
b = control group
The experimental group scores:
(( MC1+MC2
3
+MC3 + AI1+AI2+AI3+AI4 × 3+DI+ PR1+PR2 × 5
) ( 4 ) ( 2 ) )
Nai = 10 , where i is a student from Group A.

Control group scores:

Nbi = P1, where i is a student from Group B.

Average scores of students in Group A:

Σin=1 × Nai
Mai = n where n is the number of students in Group A.

Average student scores in Group B:

Σin=1 × Nbi
Mbi = n where n is the number of students in Group B.
Instruments: Concept/mind map, presentations (flipped classroom), project (project-based learning),
dynamics (collective story), and test.
Experimental Research Question 2 (ERQ2): What is the learning effectiveness of Teaching Unit II when the
proposed approach using student-centered approaches to teaching the software process is adopted over the
traditional approach?
Hypothesis 2 (H02): In Teaching Unit II, there will be no difference between the scores obtained by the
experimental and control groups at the Apply level of Bloom’s revised taxonomy.
Variables:
MC4—Conceptual/Mind Map 4
MC5—Conceptual/Mind Map 5
AI5—Presentation 6 (Flipped Classroom)
AI6—Presentation 7 (Flipped Classroom)
PR3—Project 3 (Project-Based Learning)
PR4—Project 4 (Project-Based Learning)
P2—Unit II Test
Point Scale: For all the variables adopted in the second research question, the score attributed by the evidence
of execution of the strategy was assigned a score from 0 to 10 per student.
Formulation: Ma > Mb, where:
a = experimental group
b = control group
The experimental group scores:
(( MC4+2 MC5 )+( AI5+2 AI6 ) × 4+( PR3+2 PR4 ) × 5)
Nai = 10 , where i is a student from Group A.

Control group scores:

Nbi = P2, where i is a student from Group B.

Average scores of students in Group A:

Σin=1 × Nai
Mai = n where n is the number of students in Group A.

Average student scores in Group B:

Σin=1 × Nbi
Mbi = n where n is the number of students in Group B.
Instruments: Concept/mind map, presentations (flipped classroom), project (project-based learning), and test.
Experimental Research Question 3 (ERQ3): What is the learning effectiveness of Teaching Unit III when the
proposed approach using student-centered approaches to teaching the software process is adopted over the
traditional approach?
Educ. Sci. 2022, 12, 851 13 of 21

Table 2. Cont.

Hypothesis 3 (H03): In Teaching Unit III, there will be no difference between the scores obtained by the
experimental and gontrol groups at the Apply level of Bloom’s revised taxonomy.
Variables:
MC6—Concept/Mind Map 6
MC7—Concept/Mind Map 7
MC8—Concept/Mind Map 8
AI7—Presentation 8 (Flipped Classroom)
AI8—Presentation 9 (Flipped Classroom)
AI9—Presentation 10 (Flipped Classroom)
AI10—Presentation 11 (Flipped Classroom)
PR5—Project 5 (Project-Based Learning)
PR6—Project 6 (Project-Based learning)
P3—Unit III Test
Point Scale: For all the variables adopted in the third research question, the score attributed by the evidence
of execution of the strategy was assigned a score from 0 to 10 per student.
Formulation: Ma > Mb, where:
a = experimental group
b = control group
The experimental group scores:
(( MC6+MC7
3
+MC8 + AI7+AI8+AI9+AI10 × 4+ PR5+PR6 × 5
) ( 4 ) ( 2 ) )
Nai = 10 , where i is a student from Group A.

Control group scores:

Nbi = P3, where i is a student from Group B.

Average scores of students in Group A:

Σin=1 × Nai
Mai = n where n is the number of students in Group A.

Average student scores in Group B:

Σin=1 × Nbi
Mbi = n where n is the number of students in Group B.
Instruments: Concept/mind map, presentations (flipped classroom), project (project-based learning), and test.

5. Data Analysis
In this section, the data obtained through the execution of the quasi-experiment are
analyzed. Therefore, based on the data collected, the research questions ERQ1, ERQ2, and
ERQ3 are answered.
As a way of validating or refuting each of the hypotheses established for the research,
statistical testing with reliability validation by the Shapiro–Wilk test was defined as a
strategy. This form of test has been adopted in the construction of educational research for
computer science and allows the normality and confidence of the selected sample to be
identified, as can be seen in [28].

5.1. Analysis of Experimental Research Question 1 (ERQ1)

In ERQ1, “What is the learning effectiveness of Teaching Unit I when the proposed
approach using student-centered approaches to teaching the software process is adopted
over the traditional approach?”, evidence was sought to refute H01, “In Teaching Unit I,
there will be no difference between the scores obtained by the experimental and control
groups at the Apply level of Bloom’s revised taxonomy,” by comparing the experimental
and control groups in Teaching Unit I. Data normality, variance, and the objective of
evaluating the difference between two populations with the treatment condition and two
samples (treatments) were taken into account. Therefore, the two-tailed Student-t test was
chosen for independent samples.
The experimental group scored 8.65 ± 0.76 in the evaluation, whereas the control
group scored 6.4 ± 2.09. Thus, there was a real difference between the groups, where the
experimental score was ∆ = 2.25 higher than the control, indicating a possible increase in
the learning of this group.
Educ. Sci. 2022, 12, 851 14 of 21

To perform the Student-t test, the data were submitted to tests of normality assump-
tions, which was verified using the Shapiro–Wilk test, with a result of p = 0.091. To verify
the variance, the Equal Variance Test (Shapiro–Wilk) was used, with a result of p = 0.33.
From the two-tailed Student-t test (α = 0.05), significant differences were found be-
tween the analyzed groups (Experimental—Teaching Unit I versus Control—Teaching
Unit I); thus, the test for the two independent samples showed that the use of active
methodologies was effective in the experimental group, as can be seen in the results of the
two-tailed Student-t test with p-value = 0.008111 < 0.05. Thus, the significance level derived
from the test provided statistical evidence to reject H01. Table 3 summarizes the results
obtained for ERQ1.

Table 3. Comparison of learning effectiveness between participating groups (Student-t) for Unit I.

Experimental Group Control Group

Variables
Evaluation Evaluation
Sample size 10 10
Minimum 7.37 3
Maximum 9.54 9
Sum of Points 86.55 64
Median 8.705 6.75
First Quartile 8.06 4.75
Third Quartile 9.39 8
Average 8.655 6.4
Standard Deviation 0.760 2.092

5.2. Analysis of Experimental Research Question 2 (ERQ2)

In ERQ2, “What is the learning effectiveness of Teaching Unit II when the proposed
approach using student-centered approaches to teaching the software process is adopted in
relation to the traditional approach?,” evidence was sought to refute H02, “In Teaching Unit
II, there will be no difference between the scores obtained by the experimental and control
groups at the Apply level of Bloom’s revised taxonomy”, through the comparison between
the experimental and control groups in Unit II. Data normality, variance, and the objective
of evaluating the difference between two populations with the treatment condition and
two samples (treatments) were taken into account. Therefore, the two-tailed Student-t test
was chosen for independent samples.
The experimental group scored 8.67 ± 0.56 in the evaluation, whereas the control
group scored 6.3 ± 2.32. Thus, there was a real difference between the groups, where the
experimental score was ∆ = 2.37 higher than the control, indicating a possible increase in
the learning of this group.
To perform the Student-t test, the data were submitted to tests of normality assump-
tions, which were verified using the Shapiro–Wilk test, with a result of p = 0.934. To verify
the variance, the Equal Variance Test (Shapiro–Wilk) was used, with a result of p = 0.493.
From the two-tailed Student-t test (α = 0.05), significant differences were found be-
tween the analyzed groups (Experimental—Teaching Unit II versus Control—Teaching
Unit II); thus, the test for the two independent samples showed that the use of active
methodologies in the experimental group was effective, as can be seen in the result of the
two-tailed Student-t test with p-value = 0.01047 < 0.05. Thus, the significance level derived
from the test provided statistical evidence to reject H02. Table 4 summarizes the results
obtained for ERQ2.
Educ. Sci. 2022, 12, 851 15 of 21

Table 4. Comparison of learning effectiveness between participating groups (Student-t) for Unit II.

Experimental Group Control Group

Variables
Evaluation Evaluation
Sample size 10 10
Minimum 8.01 2
Maximum 9.78 9
Sum of Points 86.73 63
Median 8.55 6
First Quartile 8.215 5.125
Third Quartile 9 8.375
Average 8.673 6.3
Standard Deviation 0.563 2.323

5.3. Analysis of Experimental Research Question 3 (ERQ3)

In ERQ3, “What is the learning effectiveness of Teaching Unit III when the proposed
approach using student-centered approaches to teaching the software process is adopted in
relation to the traditional approach?”, evidence was sought to refute H03, “In Teaching Unit
III, there will be no difference between the scores obtained by the experimental and control
groups at the Apply level of Bloom’s revised taxonomy”, through the comparison between
the experimental and control groups in Unit III. Data normality, variance, and the objective
of evaluating the difference between two populations with the treatment condition and
two samples (treatments) were taken into account. Therefore, the two-tailed Student-t test
was chosen for independent samples.
The experimental group scored 9.51 ± 0.13 in the evaluation, whereas the control
group scored 6.63 ± 1.98. Thus, there was a real difference between the groups, where the
experimental score was ∆ = 2.88 higher than the control, indicating a possible increase in
the learning of this group.
To perform the Student-t test, the data were submitted to tests of normality assump-
tions, which were verified using the Shapiro–Wilk test, with a result of p = 0.949. To verify
the variance, the Equal Variance Test (Shapiro–Wilk) was used, with a result of p = 0.659.
From the two-tailed Student-t test (α = 0.05), significant differences were found be-
tween the analyzed groups (Experimental—Teaching Unit III versus Control—Teaching
Unit III); thus, the test for the two independent samples showed that the use of active
methodologies in the experimental group was effective, as can be seen in the result of the
two-tailed Student-t test with p-value = 0.00129 < 0.05. Thus, the significance level derived
from the test provided statistical evidence to reject H03. Table 5 summarizes the results
obtained for ERQ3.

Table 5. Comparison of learning effectiveness between participating groups (Student-t) for Unit III.

Experimental Group Control Group

Variables
Evaluation Evaluation
Sample size 10 10
Minimum 9.27 2.17
Maximum 9.74 9
Sum of Points 95.19 66.33
Median 9.52 6.58
Educ. Sci. 2022, 12, 851 16 of 21

Table 5. Cont.

Experimental Group Control Group

Variables
Evaluation Evaluation
First Quartile 9.437 6.085
Third Quartile 9.6 8
Average 9.519 6.633
Standard Deviation 0.13 1.986

6. Discussion of Results
The results obtained through the statistical analysis allowed for the rejection of hy-
potheses H01, H02, and H03, which were formulated for the research. That is, it was
identified that the approach proposed through the syllabus had superior effectiveness to
the traditional approach because the average scores attained by the experimental group
during the evaluations were significantly higher than those of the control group.
The results achieved by the experimental group in relation to the control group can be
attributed to the use of student-centered learning, as the main focus of the teaching and
learning process was on the student and allowed for their engagement in the development
of practical experiences in contact with the contents of the work. In the literature, several
authors worked with the student-centered learning perspective and reported gains of
knowledge and/or competences in the teaching of software engineering [11], with related
contents such as the teaching of statistical-process control [23] and even the teaching of the
software process [12,13].
It is worth noting that in this study it was not verified whether the students had
previous knowledge of the software process, nor was the performance of the students veri-
fied in other subjects of the postgraduate course in computing. The subject of Technology
in Software Processes was offered as an option—that is, students were not required or
instructed to enroll, but enrolled in order to obtain knowledge about software processes in
the postgraduate course.
A point of analysis and discussion of the results is the feedback of the students collected
during the classes, who reported that the use of teaching strategies with student-centered
learning fostered motivation and stimulus to carry out the activities, as well as the search
and understanding knowledge related to software process. It is worth noting that both
groups considered the contents worked in the classroom relevant and sufficient for learning
about the software process.
The control group received a traditional approach—that is, theoretical classes with
tests at the end of each of the units. Therefore, their interaction with the content was
through the professor passing on what he knew. In this case, the students did not form
discussion groups on the subjects, and they did not interact with the content, developing
practical activities related to the acquisition of knowledge. This method presented during
the tests only allowed students to remember the content passed on in class, which may
have caused the control group to have lower scores than the experimental group.
During the development of the activities of the experimental group—that is, the
use of the syllabus for the software process [26]—it was possible to collect information
from students and perceive the excessive use of practices involving the need to deliver
assignments to the professors—that is, a weakness in the syllabus. For example, for each
piece of content covered in the classroom, students needed to create a concept map. In
addition, they needed to either develop a project or perform a presentation in a flipped
classroom. For these cases, it was suggested to reduce the number of concept maps
created, especially when the development of another student-centered strategy was already
suggested.
Another point of the experimental group was that the part of the class that was more
participatory in the classroom sometimes did not analyze with due attention the evaluation
Educ. Sci. 2022, 12, 851 17 of 21

criteria stipulated for the development of activities. In this case, some participants’ scores
were reduced, so, as a palliative measure, feedback was given in one of the classes. In this
feedback class, the concept of evaluation criteria was re-introduced, which the students
had already visualized in the first class of the subject, but with a practical example applied
to a concept of the subject.
A positive point was the use of dynamic collective history, as it allowed students to
experience a challenging narrative text in a postgraduate course in computing. In general,
students worked on essay texts about content. However, the students reported that it was
enriching to develop a narrative text with the concepts covered in the classroom. In a next
version of the syllabus, the adoption of at least one more dynamic collective history in the
classroom could be considered.
Another positive outcome was in project-based learning, as students were facing
hypothetical companies in different contexts in the software development market, and
needed to provide consultancy in software process. In this reality, the students in the
feedback classes gave a positive evaluation, as they were able to see that in the development
of tasks in the advancement of content on the software process they achieved growth in
learning about the software process.
Another point of discussion, comparing the control and experimental groups, is the
SWOT matrix (Strengths, Weaknesses, Opportunities, and Threats)—that is, identifying
strengths, weaknesses, opportunities, and threats regarding the use of the proposed ap-
proach in relation to the traditional one. For this study, only strengths and weaknesses
were presented, as they are internal to the experiment in question.
As strengths, we can list the following: (i) the use of strategies with student-centered
learning—that is, the student is the protagonist of their learning and it allows them to focus
on the content covered in real or hypothetical situations, but that may appear in everyday
life; (ii) the presentation of the students’ ranking by grade, which allowed students to
compete to reach first place, as well as allowing for comparisons; and (iii) fast and constant
feedback, where students, after turning in their activities, received feedback on their
performance within a day or two so they could progress in their knowledge and exert
themselves or relax in the next activities.
As weaknesses, we can list the following: (i) the amount of work needed to maintain
strategies with student-centered learning and constant feedback, since corrections by
judgment are necessary via the acceptance criteria and the amount in relation to only the
final exam of the units; (ii) the need for the virtual learning environment—that is, students
and teachers need to have access to the internet; and (iii) from the ranking, it is possible to
compare the students’ grades at the same time, which, as explained in the strengths, allows
for better engagement.
Regarding the issue of validation of the handling of the experiment itself and the
pillars that it was built on, the following section discusses this in more detail.

7. Threats to Validity
According to [29], for the experiment to be valid, it is necessary to have a level of
confidence in its investigation process in all related parties. It should be noted that scientific
research needs to be developed with caution, especially with regard to the use of inferences
to generalize to the area. Therefore, in order to guarantee the analyses carried out in this
study, threats to validity were identified, related to internal, external, construction, and
conclusion. In the following subsections the threats are presented, as well as the activities
developed to mitigate them. Thus, it is recommended to use and evaluate the results
exposed by this work within the limits exposed by the threats described.

7.1. Internal Validity

According to [30], the internal validity intends to define whether the result reached by
the study conceives a truth for the observed population. Therefore, with this result it was
Educ. Sci. 2022, 12, 851 18 of 21

possible to define whether the data presented in the research were authentic in relation to
the adopted sample.
The experimental and control groups had students enrolled in the subject of Tech-
nology in Software Processes, in alternate semesters. As the curricular component is an
optional subject in the curriculum and the students were fully aware of how the subject
would be conducted through the teaching and syllabus, the voluntary participation of the
members was considered. In addition, it is worth noting that none of the participants was
instructed to enroll in the subject. With this, it can be said that the researchers did not
influence the formation of the groups. In addition, this decision allowed the confounding
factor and the possible threat of statistical regression to be reduced. In addition, based on
this information, the control and experimental groups had statistical similarity.
Regarding the threat to internal validity related to maturation, the study did not
limit the participants’ search for knowledge to external materials, since the teaching and
learning process is dynamic and constant. Therefore, this threat is possible. As a way of
reducing this threat, the materials used in both groups addressed the same content for
the software-process subject. It should be noted that the professors were always available
through official channels to answer questions during the experiment.
As a way of mitigating the internal threat by instrumentation, an expert in the ad-
dressed content who was not the minister carried out the evaluations of the activities. It
is worth noting that a researcher with statistical knowledge, who was not involved in the
construction and implementation of the subject, had the function of analyzing the data.
For a possible threat on the level of knowledge of the professors participating in the
experiment, the instrumentation used for each group addressed the same contents. In
addition to the evaluations, they underwent analysis by experts in the teaching of the
software process.

7.2. External Validity

According to [30], after the constitution of the internal validity, the external validity
of the study worked, in this case, to identify whether the study is applicable to a broader
population—that is, it was intended to infer whether the results are achievable for a
population similar to the sampling characteristics. Thus, this study sought to adopt as a
context postgraduate students in computing who had not yet had contact with the subject
of the software process.
The experiment was carried out with a sample of 10 students in each group (experi-
mental and control). Therefore, the sample size is considered small and generalizations
should be viewed with caution due to their limitations. In addition, the study was not
tested in other populations, taking into account factors such as different academic levels of
the participants. It is not possible to guarantee that the skills and competences acquired
will be adopted by students in practical labor-market experiences.

7.3. Construction Validity

Regarding construction validity, the research of [31] was adopted as a basis, which
identified this validity as something beyond just an execution test, and was a study to
identify, measure, and analyze the bases on which the instrument of study was built.
For this purpose, research questions were developed for each of the teaching units
covered in the classroom, which were analyzed to identify the efficiency and effectiveness
of the use of teaching strategies with student-centered learning of the experimental group
in comparison with the approach used in the control group. Therefore, it stands out as one
of the most important construction validities. Another point to be taken into account is
that the distribution of activities and content taught to the experimental group took into
account Bloom’s revised taxonomy [27].
It is worth noting that the learning results achieved by students in the experimental
group do not guarantee the use of skills and abilities in different scenarios. Therefore, care
Educ. Sci. 2022, 12, 851 19 of 21

must be taken when generalizing and asserting about the data and results provided in the
research.

7.4. Validity of the Conclusion

According to the study by [31], this validity deals with interferences with regard to
the result and statistical analyses used and established correctly for the research.
A point of attention is that the data were collected from a small population of par-
ticipants. Therefore, for the research questions, several statistical tests were carried out
and the data variance for that specific population was taken into account within the set of
contents established for the unit. Therefore, as a way of mitigating this threat, the most
robust statistical tests were adopted in order to circumvent the low statistical power of the
distribution of the data obtained.

8. Conclusions
This research presents the results obtained through the development of a syllabus in
the software-development process with its actual execution via an experiment in which,
using an experimental group and control group, the proposed approach was compared
with the traditional approach.
As seen, the syllabus was designed using student-centered teaching strategies, which
included the flipped classroom, project-based learning, gamification, concept map, and
dynamics. The use of each one of them during the teaching and learning process allowed
for, according to the available statistical results, a learning gain in relation to the traditional
one. Furthermore, with the correlation to Bloom’s taxonomy, it was possible to identify the
levels of learning achieved by the students.
Using the proposed approach allows the student to be at the center of their teaching
and learning process, as this situation is reflected by the use of related and visualized
teaching strategies. For example, the use of projects and growth in difficulties and scenarios
for development allowed students to observe that the initial base of development must
be solid. Another relevant point is the flipped classroom, which allowed students to
study and bring theoretical and practical knowledge to the classroom, since, in addition
to studying theory, it was necessary to present at least one real or hypothetical case of the
worked concept. Regarding the use of the concept map, it allowed the first consolidation of
the concept—that is, the student, after each discussion/explanation of the concept in the
classroom, needed to assemble a map that related the discipline and described the concept.
The main contribution of the investigation is providing a in software-process syllabus,
and through an experiment it was possible to identify its effectiveness for a graduate class.
The use of the syllabus has some limitations, which are as follows: (i) Students need to
have a base—that is, prior knowledge in software engineering, to adopt the syllabus; (ii) the
activities require group-work skills, since most of the students’ scores come from this work
format; (iii) after using the syllabus, even though Bloom’s revised taxonomy is adopted in
its definition, adaptations are necessary for emergency teaching and for distance education;
(iv) the use of the syllabus is not yet accessible, but this situation can be resolved with the
analysis and indication of a committee specialized in this matter; (v) the syllabus provides
for a room of extended classes—that is, students and teachers need access to the internet to
use it; and (vi) the use of the syllabus does not allow for a new chance to develop activities
as a way for students to improve their knowledge even more. Therefore, such a situation
could be thought of in a new method of execution or made available via the syllabus.
As for future works continuing this research, we have (i) to carry out another experi-
ment in a new graduate class with a larger number of students to evaluate the effectiveness
of the learning and the effort made by the professor in this situation, and (ii) to verify the
possibility of adopting other student-centered learning strategies to verify which ones are
more effective in teaching the software-development process.
Educ. Sci. 2022, 12, 851 20 of 21

Author Contributions: Conceptualization, J.A.d.S.Q.; methodology, J.A.d.S.Q. and S.R.B.O.; software,

J.A.d.S.Q. and S.R.B.O.; validation, J.A.d.S.Q. and S.R.B.O.; formal analysis, J.A.d.S.Q. and S.R.B.O.;
investigation, J.A.d.S.Q.; resources, S.R.B.O.; data curation, J.A.d.S.Q.; writing—original draft prepa-
ration, J.A.d.S.Q. and S.R.B.O.; writing—review and editing, J.A.d.S.Q. and S.R.B.O.; visualization,
J.A.d.S.Q. and S.R.B.O.; supervision, S.R.B.O.; project administration, S.R.B.O.; funding acquisition,
S.R.B.O. All authors have read and agreed to the published version of the manuscript.
Funding: This research received no external funding.
Institutional Review Board Statement: Ethical review and approval were waived for this study
due to, according the Brazilian Resolution No. 510 of 7 April 2016, the research used the publicly
accessible information, and database, whose information is aggregated, without the possibility of
individual identification.
Informed Consent Statement: Informed consent was obtained from all subjects involved in the study.
Data Availability Statement: The data are available upon request.
Conflicts of Interest: The authors declare no conflict of interested.

References
1. ISO/IEC 12207; Systems and Software Engineering—Software Life Cycle Processes. ISO: Geneve, Switzerland, 2017.
2. Fairley, R.E.D.; Bourque, P.; Keppler, J. The impact of SWEBOK Version 3 on software engineering education and training. In
Proceedings of the 2014 IEEE 27th Conference on Software Engineering Education and Training (CSEE&T), Klagenfurt, Austria,
23–25 April 2014; pp. 192–200.
3. ACM/IEEE. Computer Science Curricula 2013; ACM and IEEE Computer Society, Incorporated: New York, NY, USA, 2013.
4. Sociedade Brasileira de Computação. Referenciais de Formação para os Cursos de Graduação em Computação. 2017. Avail-
able online: https://www.sbc.org.br/documentos-da-sbc/send/127-educacao/1155-referenciais-de-formacao-para-cursos-de-
graduacao-em-computacao-outubro-2017 (accessed on 11 May 2018).
5. Garousi, V.; Giray, G.; Tüzün, E.; Catal, C.; Felderer, M. Closing the gap between software engineering education and industrial
needs. arXiv 2018, arXiv:1812.01954. [CrossRef]
6. Quaresma, J.A.S.; Oliveira, S.R.B. A Study on the perception of the teaching-learning of software process in the academia and
industry: A survey application. In Proceedings of the 18th CONTECSI, São Paulo, Brazil, 13–15 October 2021.
7. Soares, E.M.; Oliveira, S.R.B.; Elgrably, I.S.; Monteiro, R.H.B. Application of a Gamified Approach to Learning in the Treatment of
Problems in Software Process Improvement: Analysis and Discussion of Results. Int. J. Emerg. Technol. Learn. 2022, 17, 242–296.
[CrossRef]
8. Quaresma, J.A.S.; Oliveira, S.R.B. Student-centered approaches and industry training practices for Software Process and Quality:
An ah-hoc literature review. In Proceedings of the 17th CONTECSI, São Paulo, Brazil, 28–30 October 2020.
9. Pinheiro, M.N.; Batista, E.C. O aluno no centro da aprendizagem: Uma discussão a partir de Carl Rogers. Rev. Psicol. Saberes 2018,
7, 70–85.
10. Souza, M.; Moreira, R.; Figueiredo, E. Playing the Project: Incorporating Gamification into Project-based Approaches for Software
Engineering Education. In Proceedings of the Anais do XXVII Workshop Sobre Educação em Computação, Belém, Brazil, 15–18 July 2019;
SBC: Porto Alegre, Brazil, 2019; pp. 71–80.
11. Castro, R.; Souza, G. O Uso de Recursos Lúdicos Para o Ensino de Processos em Engenharia de Software. In Proceedings of the
Anais do XXIV Workshop sobre Educação em Computação, Português, Brazil, 30 June 2020; SBC: Porto Alegre, Brazil, 2020; pp. 270–279.
12. Maxim, B.R.; Kaur, R.; Apzynski, C.; Edwards, D.; Evans, E. An agile software engineering process improvement game. In
Proceedings of the 2016 IEEE Frontiers in Education Conference (FIE), Erie, PA, USA, 12–15 October 2016; pp. 1–4.
13. Moura, V.; Santos, G. ProcSoft: A Board Game to Teach Software Processes Based on ISO/IEC 29110 Standard. In Proceedings of
the 17th Brazilian Symposium on Software Quality, Curitiba, Brazil, 17–19 October 2018; pp. 363–372.
14. OliveiraJr, E.; Furtado, V.; Vignando, H.; Luz, C.; Cordeiro, A.; Steinmacher, I.; Zorzo, A. Towards Improving Experimentation in
Software Engineering. In Brazilian Symposium on Software Engineering; SBC: Porto Alegre, Brazil, 2021; pp. 335–340.
15. Severino, A.J. Metodologia do Trabalho Científico; Cortez Editora: São Paulo, Brazil, 2017.
16. Quaresma, J.A.S.; Oliveira, S.R.B. Teaching and learning strategies for software process subject. In Proceedings of the 2021 IEEE
Frontiers in Education Conference (FIE), Lincoln, NE, USA, 13–16 October 2021.
17. Akçayır, G.; Akçayır, M. The flipped classroom: A review of its advantages and challenges. Comput. Educ. 2018, 126, 334–345.
[CrossRef]
18. Bender, W.N. Project-Based Learning: Differentiated Education for the 21st Century; Corwin: Newbury Park, CA, USA, 2012.
19. Kalogiannakis, M.; Papadakis, S.; Zourmpakis, A.-I. Gamification in Science Education. A Systematic Review of the Literature.
Educ. Sci. 2021, 11, 22. [CrossRef]
20. Maximo-Pereira, M.; Souza, P.V.S.; Lourenço, A.B. Mapas Conceituais e a Elaboração de Conhecimento Científico na História da
Ciência: Algumas aproximações teóricas. Ciência Educação 2021, 27. [CrossRef]
Educ. Sci. 2022, 12, 851 21 of 21

21. Schuster, T.; Lopes, T.R.C. Jogo de contar histórias: O uso de técnicas de criação de narrativas colaborativas em sala de aula.
Renote 2012, 10. [CrossRef]
22. Quaresma, J.A.S.; Oliveira, S.R.B. A mapping of the assets included in the reference standards of RF-SBC, CS-Curricula ACM/IEE
2013 and SWEBOK regarding the knowledge area about Software Process. In Proceedings of the 17th CONTECSI, São Paulo,
Brazil, 28–30 October 2020.
23. Furtado, J.C.C. Uma Abordagem para o Ensino do Controle Estatístico de Processos em Cursos Superiores de Computação.
orientador, Sandro Ronaldo Bezerra Oliveira. Ph.D. Thesis, Universidade Federal do Pará, Instituto de Ciências Exatas e Naturais,
Programa de Pós-Graduação em Ciência da Computação, Belém, Brazil, 2020.
24. Elgrably, I.S.; Oliveira, S.R. A Quasi-Experimental Evaluation of Teaching Software Testing in Software Quality Assurance Subject
during a Post-Graduate Computer Science Course. Int. J. Emerg. Technol. Learn. 2022, 17, 57–86. [CrossRef]
25. Garcia, F.W.S.; Oliveira, S.R.B.; Carvalho, E.C. Application of A Teaching Plan for Algorithm Subjects Using Active Methodologies:
An Experimental Report. Int. J. Emerg. Technol. Learn. 2022, 17, 175–207. [CrossRef]
26. Quaresma, J.A.S.; Oliveira, S.R.B. A Syllabus Proposal for Teaching of Software Development Process in Undergraduate Courses
in Computer Science. In Proceedings of the XXXVI Brazilian Symposium on Software Engineering, Virtual, 5–7 October 2022.
27. Pereira, M.W.; Pontello, L.S. Análise de Objetivos Educacionais Utilizando a Taxonomia de Bloom Revisada Aplicada a um PPC
do Curso de ADS. In Proceedings of the Anais do XXIX Workshop Sobre Educação em Computação, Belém, Brazil, 20 July 2021; SBC:
Porto Alegre, Brazil, 2021; pp. 248–257.
28. Pimentel, M.; Filippe, D.; Santoro, F.M. Metodologia de Pesquisa Científica em Informática na Educação: Concepção de Pesquisa.
2018. Available online: https://metodologia.ceie-br.org/livro-1/ (accessed on 20 November 2022).
29. Lima, V.C.M.; Neto, A.G.S.S.; Emer, M.C.F.P. Investigação experimental e práticas ágeis: Ameaças à validade de experimentos
envolvendo a prática ágil Programação em par. Revista Eletrônica Sistemas Informação 2014, 13. [CrossRef]
30. Patino, C.M.; Ferreira, J.C. Validade interna e externa: Você pode aplicar resultados de pesquisa para seus pacientes? Jornal
Brasileiro Pneumologia 2018, 44, 183. [CrossRef] [PubMed]
31. Fattore, I.D.M.; Moraes, A.B.D.; Crestani, A.H.; Souza, A.M.; Souza, A.P.R.D. Validação de conteúdo e de construto de sinais
enunciativos de aquisição da linguagem no segundo ano de vida. In CoDAS; Sociedade Brasileira de Fonoaudiologia: São Paulo,
Brazil, 2021; Volume 34.