Quality in Software Digital Ecosystems

The Users Perceptions

Giancarlo Stefanuto1, Angela Maria Alves1, Maiko Spiess2 and Paula

Drummond de Castro2
1 Centro de Tecnologia da Informação Renato Archer
Rodovia Dom Pedro I (SP - 65) Km 143,6 , Campinas, Brasil
angela.alves@cti.gov.br
WWW home page: http://www.cti.gov.br/,
2 University of Campinas, Department of Science and Technology Policy
Rua Pandiá Calógeras, 51, 13083-870, Campinas, Brazil
{spiessbnu,paulafdc}@ige.unicamp.br,
WWW home page: http://www.ige.unicamp.br/

Abstract. Brazilian Public Software (BPS) is an innovative experience in

public administration. It combines features of the free software production
model with the concept of public goods and is delivered by a portal that links
different people and interests. This paper is about the perception of quality by
the participants of the digital ecosystem BPS - Brazilian Public Software. The
concept of digital ecosystems employed here is ecosystems which digital
environment is populated by digital species (software components, applications,
online services, etc.). These ecosystems can be devoted to digital content
production, business, academic research, etc. (Kannan et al 2010).

1 Introduction
This paper is about the perception of quality by the participants of the digital
ecosystem BPS - Brazilian Public Software. The concept of digital ecosystems
employed here is ecosystems which digital environment is populated by digital
species (software components, applications, online services, etc.). These ecosystems
can be devoted to digital content production, business, academic research, etc.
(Kannan et al 2010). In this case is the former one.
From an institutional standpoint, the BPS is a government initiative, which arose in
2006, coordinated by the Brazilian Ministry of Planning, Budget and Management,
which seeks to establish a new impetus to production, distribution and use of
software, especially by public authorities (Freitas and Meffe, 2008). This initiative is
based on the concept of software as a public good, which ties the licensing of free
software to the concept of public good (Alves et al, 2009a).
The BPS has a virtual environment (http://www.softwarepublico.gov.br/spb/), where
tools are provided - for example, forum, blog, wiki, track – for collaborative
development of solutions available on the portal. Currently there are 44 solutions
available on the website and over 100,000 registered users interacting in a dynamic
and nonlinear way which characterizes the initiative as an ecosystem in constant
evolution.
When software is available on the BPS, it stimulates the formation of a community
around it. Through the development of BPS, thematic communities were established,
called interest groups. Currently there are two of them: one focusing on
municipalities’ issues (4CMBr) and other aiming quality issues (5CQualiBr). The
latter, 5CQualiBr, is the object of this paper. It emerged based on five core values
(Trust, Cooperation, Community, Knowledge and Sharing) and its purpose is to be an
environment where people discuss and improve the quality of solutions provided by
BPS through collaborative processes and emphasizing the participation of key-actors
of the ecosystem. The conception of quality adopted in this interest group is a set of
best practices for the software development and the management of its community.
Thus, the 5CQualiBr has undertaken many actions addressed to understand the how
quality is accomplished in BPS environment, and specifically in topic of the
development of the ecosystem, interoperability, product, process, service and test.
This paper presents and discusses the results of a survey conducted by the interest
group 5CQualiBr and concerned with communities leaders to whom were questioned
about their use of 5CQualiBr environment; their quality practices and the community
way of operate. The research is based on the premise that the opinions and
participation of actors are crucial to the development and maintenance of the
ecosystem and can be used to improve the environment itself.
This paper is divided into three parts: a) presentation of the maturity model of the
BPS, which will be reported its development, maturity levels, learning cycles and an
overview of the ecosystem quality, b) the result of field research, which will be
presented the results and some communities leaders statements and conclusions
arising from interviews, c) conclusions, which are discussed in the survey results and
recommendations for advances in the 5CQualiBr ecosystem.

2 Maturity model of the BPS

Was observed in BPS, that the directing actions happening on specific issues related
to stages of development and maturation of the BPS. From the resolution of certain
bottlenecks or defining new goals, the focus of attention and development activities
has changed, not eliminated, however, the attention of previous themes.
The growth process of the ecosystem, so emergent (decentralized organizational
pulverization of contributions, limits flexible participation) leads to incremental
processes of change, driven by values and general concepts (public goods, etc...), but
not a scene precise arrival. The ecosystem is self-regulating. This leads to a process of
continuous learning and improvement, which occurs in cycles. These cycles occur in
a sequence, but they are not disconnected and discontinuous. The first cycle continues
to occur in the second cycle, but at a lower intensity. And the third cycle occurs in the
first and second cycles, but is not the focus of most learning communities of BPS. The
Figure 2 expresses these considerations.
They are not separated but are inserted in circles of causation, they are present in
each community, working group as well as in the ecosystem as a whole, and are
inspired on the use of Peter Senge (Senge, 1990) theoretical contributions to learning
organizations. So, considering the BPS experience, there are 4 cycles of
apprenticeship: I) learning with the structural elements of BPS; II) learning with the
components of BPS; III) learning with the relationships of the ecosystem and IV)
learning with the patterns of behavior of BPS.

Fig. 2 – BPS learning cycles

It is important to point out that all these cycles begin from the conceptual, legal and
organizational framework of public software. This framework has created and
continues to maintain the environment that enables the interactions in the ambiance of
BPS. It also acts like an active frontier that, at the same time, promotes interactions
with the all stakeholders, absorbing their contributions but maintaining the main
objectives of BPS ecosystem. The end of the fourth cycle generates feedbacks to
redesign the basic structure of BPS ecosystem and promotes a new beginning of
apprenticeship cycles, however in a new level, like a spiral moving. So, each set of
cycles contributes to the evolution of the framework of BPS, improves its strategic
view, and incorporates new variables, and so on. It is natural to look at these cycles as
a process towards increasing maturity that indicate a base to establish a mature model.

Fig. 4 – BPS maturity levels

The formulation of the maturity levels of BPS occurred from the perception of the
learning cycles in the BPS. At 4 cycles identified was assigned a level of maturity and
added them to another level with regard to the formation stage of the ecosystem. We
then formulated a model with five levels of maturity. The Figure 2 illustrates briefly
each of these levels.

3 Field research results

Field research was conceived addressing the following issues: 1) disclosure of

5CQualiBr; 2) use of software quality processes; 3) communities' technical and
management difficulties; 4) service providers’ regulation; 5) software tests; 6)
interoperability; 7) user-friendliness and content of 5CQualiBr portal; 8) community
leader's role and 9) alignment of the community to BPS ecosystem.
At the time of the survey, there were 43 BPS active communities. Leaders from all
these communities were contacted, and effectively 30 of them participated of the
interviews carried out between November 2010 and February 2011, through telephone
and Skype. At a second moment, the interviews were systematized to allow their
tabulation and presentation, as showed below.
a) disclosure of 5CQualiBr: the percentage of leaders' negative answers about the
use of the portal ("No, I don't know the 5CQualiBr portal" and "No, I don't know it
but I intend to") represents 53% of respondents (16 leaders). When added to the
percentage of "Yes, I know it but I don't use it" (30%), it is possible to note an
effectively very low use rate of the portal and its artifacts.
b) use of software quality processes: when asked about the use of software quality
guidelines, manuals and processes in the daily routine of their community, 50% of the
respondents said they did not use these mechanisms. 23% do not use software quality
processes, but intend to do so. Among the positive responses (27%) the following
methodologies were mentioned: JUnit, Sonar, MDA, Agile Quality. The use of
internal processes from companies and organizations related to the BPS software were
mentioned as well. Negative responses indicate the widespread use of “informal” and
“non-structured” quality techniques: “For the next year, our goal is to improve our
software development quality practices. Now, we don't apply any established software
engineering procedure; it's all about daily practices, used in an informal way.”
(respondent “a”)
c) communities' technical and management difficulties: in relation to major
technical difficulties faced by the leaders, were common references about the
ecosystem's infrastructure problems, such as the portal's instability and unfriendly
interface, its low accessibility, little storage space, software's versions management
difficulties, and the existence of redundant tools (eg, forum and trac). Also mentioned
were issues related to the portal's administration, such as the centralization of content
publishing, and its access policy. With respect to community management, recurrent
difficulties are the lack of end users' knowledge about free/open source software, and
the low degree of voluntary participation. According to one interviewee, "The BPS
and 5CQualiBr portals are not intuitive enough for the end user. Both portals don't
have clear informations about their purpose, mission, etc. For which person is the
5CQualiBr designed for? (...) In general, you should indicate to the user what it does,
where it goes, what are its advantages ... " (respondent “b”)
Some of the interviewees related a demand for formal management models for their
communities – such as the creation of management comittees – to enable a process of
responsibility sharing among the community members, and to guarantee its
sustainability. Overall, these testimonies point towards a growing institutionalization
of some communities which, in turn, can be related to the ecosystem's maturity.
d) service providers regulation: Only one leader said that his community has rules
for registration of service providers. In general, the registration of providers through
the Public Market portal is seen as being inefficient – it currently doesn't allow any
kind of background check on the provider, or any feedback about their current
activities. On that matter, one of the respondents said: “We do have service providers,
but there isn’t any selection criterion... it wasn't me the one who selected them. I think
the information about it [the Public Market] should be as trustworthy as possible. “
(respondent “c”).
Several other respondents indicated a latent need for qualification and certification
mechanisms for service providers candidates.
e) software tests: 60% of respondents use a methodology for software testing. The
use of informal and unstructured tests, as well of private methodologies were
highlighted by respondents (8 and 5 mentions, respectively). The respondents
highlighted the use of the following methodologies: functional testing, unit testing,
JUnit, pair programming and MPS.BR.
f) interoperability: Approximately half of respondents (53%) said interoperability is
a frequently addressed topic in their communities. The interviews highlight the use of
established interoperability standards (for example, the federal government standard
called e-Ping, and the W3C recommendations) and the frequent interoperability with
another business solutions, either available in the software market or in the BPS
ecosystem itself.
g) user-friendliness and content of 5CQualiBr portal:  regarding the presentation
of the 5CQualiBr portal, respondents said that the portal structure is "rigid and
outdated", often attributing this to the rigid governance of the BPS website, to which
5CQualiBr is subjected. The answers show a preference for "new and dynamic"
technologies, which could improve the usability and communication with users
(advanced search engines, tag clouds, "ranking" of users / developers, and multimedia
material were among the most cited web-based technologies). Besides the demand for
technical contents on various aspects of quality (including examples, and cases about
software testing), the interviewees highlighted the need of material for service
providers qualification, and on the management of communities.
According to the respondents, these contents should be presented in a “fast and clean”
structure, and in the form of short articles and/or papers with abstracts, alongside with
a website area with a brief description of the 5CQualiBr interest group, its objectives,
mission, and structure.
h) community leader's role: The leaders' role is widely regarded as fundamental, and
their managerial qualities are perceived as being equally or more important than the
technical qualities. Two major obstacles were mentioned by the leaders: their lack of
management training and the excessive concentration of the decision powers in a few
members of the communities. In order to overcome these difficulties, once again
respondents indicated the necessity of developing alternative management models
(such as management committees), mostly to decentralize decisions and
responsibilities. Besides that, they also indicated a demand for courses and training
devised specifically for the leaders.
i) alignment of the community to BPS ecosystem: 67% of respondents rate the BPS
portal as an unprecedented, socially relevant, and important initiative. In most cases,
the community shows concern for the future and sustainability of the BPS ecosystem.
Regarding stable and mature software solutions, especially the ones closely related to
private companies, or to some very specific type of technical knowledge/activity, the
community leaders tend to perceive the BPS only as a repository and publicity
mechanism, and not as a collaborative production environment.
Also, it should be mentioned that the BPS informational infrastructure is considered
by its users as limited and old-fashioned. Besides that, its governance mechanisms are
perceived as “strict” and too centered on the government actors (even simple content
uploads, such as news and polls, must be authorized by the Ministry's staff). As a
consequence of this somewhat rigid access and management policies, the interests’
groups websites (in this case, the 5CQualiBr) are considered as strict as the BPS
portal, hence the leaders' weak interest and participation in these discussion and
content production arenas.
Overall, the survey revealed several potential ways to enhance the ecosystem's
maturity. In other words, the leaders’ behavior, and some community practices,
indicates the crescent demand for formalization of certain BPS activities. This, in
turn, indicates a potential for stabilization and improvement of the ecosystem.
However, problems of infrastructure (such as the portal's accessibility and
management, its tools and limited repositories), technical difficulties (lack of
knowledge about quality processes) and social obstacles (heterogeneous views about
collaborative production) are still difficulties to be overcome in order to achieve more
maturity for the BPS ecosystem.

7 Conclusions

The survey found a lack of information in two important aspects: (i) software quality
processes and, (ii) the use of artifacts provided by 5CQualiBr. These findings can be
regarded as an obstacle to the advance of the maturity level of the ecosystem.
The analysis of critical variables and indicators related to each level of maturity
suggest that, according to the scale referred in this research, BPS as a digital
ecosystem is at Level II in transition to Level III. In other words, BPS already has a
demarcated legal and institutional framework (which is one of the innovations of the
BPS), a minimal infrastructure, but the partnerships and supports to the BPS are still
in a consolidation phase. To a suitable development of BPS ecosystem, the perception
of quality by the leaders and other key-actors become a strategic and fundamental
factor for the evolution of the BPS.
We must also consider that the model of collaborative production of free software,
deals with aspects of quality within a distinct perspective of proprietary models. The
existence of a supportive community, which may have significant size, and their
continuous interaction, ensure continuous improvement of solutions. But this is still
true for a limited number of solutions. In communities BPS this process happens, but
the introduction of guidelines, manuals, automated tools for quality, adapted to this
dynamic, tend to generate a significant impact on the insertion of the BPS solutions in
various spheres of government.

References

Alves, A. M.; Stefanuto, G. N.; Castro, Paula F. D.; Varani, Sueli A. (2009a).
Software Público Brasileiro: Muito além do Compartilhamento de Software.
Revista InfoBrasil, Fortaleza, v. 2 (7), 19-21, Jun/Ago, 2009a. Available in:
http://www.cti.gov.br/pdf/2009/revista_software_livre.pdf .
Briscoe, Gerard & De Wilde, Philippe. Digital Ecosystems:Self- Organisation of
Evolving Agent Populations. In: International Conference on Management of
Emergent Digital EcoSystems, 2009, Lion, França. Proceedings of the
International Conference on Management of Emergent Digital EcoSystems. Nova
Iorque: ACM, 2009.
Dini, P.; Darking, M.; Rathbone, N.; Vidal M.; P. Hernandez; P. Ferronato The digital
ecosystems research vision: 2010 and beyond. European Commisssion Bruxelles
Position Paper, 2005. Available at: http://www.digital-
ecosystems.org/events/2005.05/de_position_paper_vf.pdf
Freitas, C. S. and Meffe, C. (2008). FLOSS in an Open World: best practices from
Brazil. In: Archambault, J.P.; Aigrain, P.; Laisné,J. P. Cazenave, A.. (Org.). 2020
FLOSS Roadmap. 01 ed. Paris: Creative Commons, v. 01, p. 69-73.
Kannan, Rajkumar; Balasundaram, S.R. Andres, Frederick (2010). The role of
mulsemedia in digital content ecosystem design.MEDES’10 October, 26-29,
Bangkok, Thailand, 2010.
Senge, P. M.. The Fifth Discipline: The Art & Practice of Learning Organization.
New York, NY: Doubleday, 1990.