III.
International Conference on Electrical, Computer and Energy Technologies (ICECET 2023)
Methodology for measuring and improving inter-
organizational interoperability
Mamour FALL Mamadou Samba Camara
Faculté des Sciences et Techniques Ecole Superieure Polytechnique
Cheikh Anta DIOP University Cheikh Anta DIOP University
Dakar Senegal Dakar Senegal
mamourfall@gmail.com mamadou.samba.camara@esp.sn
Abstract— Interoperability is a constantly evolving field,
particularly with the development of connected and intelligent
devices. Numerous methods have been developed to assess
interoperability, based on maturity models. However, with the
evolution of technology, the development of the Internet,
connectivity is no longer a real problem, and isolated systems
are becoming increasingly rare. Improving interoperability is
therefore no longer oriented towards interconnection, but
rather towards optimizing business processes. Activities
generated by interoperability are a priori considered to be of no
value compared with business activities and contribute to
putting data into the right format for transmission or use. This
study proposes a methodology for assessing interoperability
based on data exchanged between systems. The use of the
proposed Data Circuit based on generation and integration
processes makes it possible to identify points of improvement of
the business process to have data produced in a format that
requires the least processing before its use, especially in the
context of the execution of a process instance.
Keywords—interoperability’s improvement, system
engineering, assessment, business process, non-value-added
activities.
I. INTRODUCTION
Research on interoperability has led to the development of
technical solutions that now allow applications to
communicate with each other. However, this communication
only supports the transmission of data that is often already
standardized via data channels that require processing by
sending and receiving systems. Interoperability is therefore
limited to software level. Technical solutions partially solve
interoperability problems because they ignore the overall
analysis of the business process and its requirements.
Determining the right requirements for the system is very
critical in the system development life, hence, there is a need
to carefully consider this phase to build a high quality and
reliable system [1].
In the literature, most interoperability assessment methods
are based on maturity models that provide an assessment of
individual systems and do not consider the overall business
process that requires the collaboration of multiple systems. In
addition, these maturity models do not integrate technological
developments with the arrival of Web 2.0 which facilitates
data sharing as well as collaborative work and teleworking.
979-8-3503-2781-6/23/$31.00 ©2023 IEEE
16-17 November 2023, Cape Town-South Africa
Mouhamed DIOP
Ecole Superieure Polytechnique
Cheikh Anta DIOP University
Dakar Senegal
mouhamed.diop@esp.sn
Web applications are getting closer and closer to traditional
desktop applications with processing often transferred to
browsers. However, these treatments can be expensive.
The objective of this study is to propose a methodology for
assessing interoperability based on the processes of generation
and integration of the data exchanged between systems in
order to identify areas for improvement of the business
process to obtain data to be exchanged generated in a given
format. This approach has the advantage of taking into
account the complete functional coverage of the business
process and reduces the processing carried out on the data
received by a system before a business process instance runs.
Section 1 will provide a state-of-the-art interoperability
assessment. Since the proposed methodology is based on an
analysis of the systems involved in interoperability, Section 2
will address interoperability analysis from a system
engineering perspective to characterize requirements, while
Section 3 will discuss the proposed methodology. Finally, a
use case and discussion will close the study.
II. STATE OF THE ART
Interoperability measurement aims at defining metrics to
qualify the degree of interoperability. The implementation of
metrics is related to two principles: (1) the identification of the
parameters relating to interoperability, (2) the characterization
of these parameters by metrics [2]. Interoperability problems
usually arise when two systems are not interoperable. The
solution to this problem, mostly related to technical aspects,
consists mainly in upgrading systems or in creating a gateway
allowing the systems to interact. However, other types of
problems may exist outside of this assumption such as
optimization, reliability, or process failures, among others.
A. Measures
The fact that interoperability can be improved means that
metrics for measuring the degree of interoperability exist. An
assessment determines the As-is state of a system or system of
systems and provides a roadmap toward the To-be state [3].
Assessments are done using qualitative and/or quantitative
measurements.
1) Qualitative measure: The qualitative measures allow
to assess interoperability in relation to the approach adopted
and to define technical performance objectives. The
Qualitative measures are mainly subjective. In most cases,
this kind of measure uses a rating scale composed of
linguistic variables (e.g. “Good”, “Optimized” and
“Adaptive”) for qualifying a system [4]. It is mostly used by
the maturity models, which are approaches designed to assess
the quality (i.e. competency, capability, level of
sophistication) of a selected domain based on a more or less
comprehensive set of criteria. Among the most popular
maturity’s model, one can mention:
a) LISI which proposes five levels of maturity:
Isolated, Connected, Distributed, Domain and Enterprise. To
establish these levels of maturity, it defines four areas of
interest named PAID, which stands for Procedures,
Applications, Infrastructure and Data. From 1998 to 2006,
LISI was the template for numerous maturity model and
maturity model-like (leveling) interoperability measurement
approaches designed to measure both information and non-
information system interoperability [4]. The aim of LISI's
measurement is to capture the essence of the potential
interactions available between systems, as recorded by the
implementation choices made by developers. Three types of
measure are proposed: "generic", "expected" and "specific".
The generic type relates to the comparison of one system
against the capability model, while the other two (expected
and specific) relate to the comparison of two or more systems
against each other.
b) Organizational Interoperability Model-OIM: The
Organizational Interoperability Model [5, 6] was developed
to assess interoperability at the level of human or
organizational activity. Originally proposed for use in the
assessment of military organizations, the authors argued that
such a model could be applied to different contexts (i.e., for
general use) [6, 7]. It was specifically designed as an
extension of the LISI 2.10 maturity model, which is
technology-intensive and focuses on the compatibility of
systems and technologies. It aims to assess the compatibility
of at least two entities relative to organizational and
conceptual barriers, with an emphasis on business concerns
[5]. The OIM defined five levels of interoperability
(Independent, Cooperative or Ad hoc, Collaborative,
Integrated or Combined, Unified) and four interoperability
attributes (Readiness, Understanding, Command Style and
Ethos).
c) Levels of Conceptual Interoperability Maturity
Model-LCIM : LCIM is a maturity model that assesses the
compatibility of two entities by targeting conceptual barriers
in the field of data interoperability [8]. It is a framework
describing a spectrum from what is conceptually modeled to
how it is technically implemented. Data (entities, own
concepts), the way it is used (process concepts, methods) and
constraints are described by artifacts following the ideas of
systems architecture and modeling as known in the field of
systems engineering [9].
d) Enterprise Interoperability Maturity Model- EIMM :
The ATHENA project elaborated, for manufacturing
enterprises, the EIMM to address interoperability issues at all
levels of the company (ATHENA, 2005) [6]. The six areas of
concern covered by the assessment are: - Business strategy
and processes - Organization and skills - Products and
services - Systems and technologies - Legal environment,
security, and trust - Business modeling. The five maturity
levels of the Enterprise Interoperability Maturity Model are:
Realized, Modeled, Integrated, Interoperable and Optimized
[11].
2) Quantitative measures: Quantitative models are
approaches that make it possible to numerically measure the
characteristics of interoperability. Quantitative measures are
concerned with performance. They define numeric values to
characterize the interoperability [4]. In general, the rating
scale is from 0 to 100%.
B. Limits
Many approaches to interoperability assessment are
identified in the literature based on the definition of maturity
models that describe, for a specific domain of interest, several
levels of sophistication at which activities in that domain can
be conducted [10]. They are focused on maturity measures and
addressed potentiality, compatibility and/or performance
assessments. They are mostly oriented architecture, system
engineering and/or data with different approaches and metrics.
LISI, is focused on technical interoperability while LCIM is
on the data to be exchanged and the documentation of the
available interface. OIM describe the ability of organizations
to interact and EIMM, defined in an a priori interoperability
context, allows companies to assess their interoperability
potential, to know the likelihood that they need to support
effective integration and to detect precisely the weaknesses
that cause interoperability problems.
Technical solutions for achieving interoperability between
systems are based on the development of APIs (SOAP,
WSDL, ebXML among others) [12,13] and on standardization
initiatives concerning the format of exchanged data ((IEC
62264-1) [14],
Even if they allow communication between systems,
technical solutions do not intervene as part of an overall
analysis of the business process and ignore certain aspects of
interoperability. It is mainly about the approach, the concerns,
and the barriers. To fully achieve interoperability, it is
therefore necessary to consider all activities (business and
non-value added) of the business process as presented in [16]
by Camara et al. This approach will allow to identify
functional requirements. With communication between
systems established, the improvement in interoperability will
be in efforts made by systems to provide usable data to third
parties with minimal processing.
III. CONCEPTS
A. System engineering Process
A process is a system of activities that uses resources to
transform inputs into outputs. It describes the "what to do" in
a predefined order. It transforms data by creating added value.
Systems engineering is an iterative process of top-down
synthesis, development, and operation of a real-world system
that satisfies, in a near optimal manner, the full range of
requirements for the system.
Systems Engineering Methods are Stakeholder Analysis,
Interface Specification, Design Tradeoffs, Configuration
Management, Systematic Verification and Validation,
Requirements Engineering etc. They ensure correct external
interfaces, interfaces among subsystems and software. The
conventional view of requirements definition is that this phase
of system’s development begins with an informal description
of "what" the system is expected to do.
Systems engineering makes it possible to:
• represent of services that each system must offer to
achieve the interoperability corresponding to the
Functional Requirements (FR) and
• determine how these services are rendered or
expected (Non-Functional Requirements - NFR).
B. Interoperability efforts
The systems’ interoperability must be organized to
minimize communication between components to ensure that
performance requirements are met and avoid process failures.
Each interconnection node is a critical point insofar as it must
be considered the data exchanged, the I/O between systems,
the number of target systems and especially the process
failures. The effort made by each system to make the data
usable, before, and/or after the exchange, is decisive to assess
the interoperability.
Two types of effort should be considered:
• A one-time effort: This is an organizational and
applicative upgrade to directly have data generated in
the appropriate format in output so that it is directly
usable by other systems, or as input by the requesting
systems. In this case, systems must be upgraded to
record data exchanged in the correct format at the time
of creation or generation.
• a continuous effort: this is an application-level data
transformation that must be carried out by the system
providing or receiving the data for each process
instance.
C. NVA activities
The problems of interoperability can be of two kinds:
• Multiple and complementary systems are not
interoperable.
• The data received are not in the correct format.
Interoperability’s need, at the origin of the project, is
classified as NFR. Similarly, the activities generated by
interoperability are mostly NVA activities. They are
defined as the components of business processes that
represent efforts between partners to achieve
interoperability in information exchange [16]. However,
NVA activities can generate both FR (without adequate
data formatting, an instance of business process is bound
to fail) and NFR (process failure, optimization...). As a
result, interoperability problems can concern the activities
of a process supported by interacting systems or by an
individual system.
IV. PROPOSITION
In this study, we assume that interoperability is effective
between systems. Inter-organizational interoperability is
materialized by the existence of functional links between
several entities within a business process. The collaboration
system is analyzed from a requirement engineering
perspective with the objective of identifying the
interconnection nodes, evaluating the interoperability’s
efforts, and optimizing the data exchange and processing. The
system engineering process enables analysis to develop,
reorganize, and speculate the functionality of a system for
specific purposes. These specific goals always depend on the
user. The methodology is based on optimizing interoperability
efforts through two processes: the Data Generation Process
(DGP) for the sender and the Data Integration Process (DIP)
for the receiver. The advantage of this approach is that it
provides a conceptual analysis of the interoperability scheme
and allows for an assessment of the compliance of the "As-is"
situation.
Analyzing an activity independently of the involved actors
and the way it is considered at the application level does not
allow us to accurately measure its impact on the collaboration
process. The proposed methodology will be based on:
• identification of NVA activities,
• identification of involved actors and
• processing of exchanged data.
NVA activities representing interoperability efforts are
identified through the analysis of:
• the process of generating data to be exchanged from a
business process,
• the transformation process of the received data before
its use in a business process.
Fig. 1. Data Generation Process (DGP)
Figure 1, which shows the DGP process, also highlights the
areas of improvement. Criticisms of the latter are shown in red
and orange. From the data collected at the interconnection
node, it is possible to reverse-engineer the creation, generation
and/or transformation process.
Handling NVA activities requires the identification of the
actors involved in the process. The Data Integration Process
(DIP) is different depending on the case selected. Considering
the first case, for example, integrating data from another
medium involves sorting, entering, and validating data before
it is used by a third-party system in a business process
instance. The DIP shown in Figure 2 proposes a representation
of the options for the data processing circuit in an
interoperability project. The optimal circuit is the receipt of
data by the receiving system in a ready-to-use format by an
automated process directly from the sending system. The
transformation process depends on the level of compliance of
the data with what is expected as input for a business activity.
Fig. 2. Data Integration Process (DIP)
Integration must also consider the situation where all
exchanges are automated, and those where they are partially
or not at all.
For the second situation, four subcases have been identified:
● An actor uses an isolated system that requires
manual integration of the received data.
● An identified actor in the process who does not have
features or software interfaces that allow him to
perform what is expected of him.
● An actor identified in the process who sees the
features relevant to his work available to another
actor than him.
● The functionalities are implemented but the user
does not have access to the system due to network
constraints (e.g., geographical position).
In the case where all exchanges are automated, NVA
activities are only identified in the data integration process.
Otherwise, additional NVA activities may appear. These
include, for example the integration of an actor using an
isolated system and the development of new functionalities for
an actor.
However, we will have to consider before opting for
formatting by the sending system the number of systems
requesting a specific information and the data format required
by each system. In an integrated and unified interoperability’s
approaches, a "neutral" format can be established. Data that
compose the information is presented in a format that allows
each "receiving" system to adapt it to its needs. In this case,
there are two treatments to be foreseen before using the data,
one carried out by the "sender" and another by the “receiver”.
They are the interoperability efforts.
V. APPLICATION
The case illustrated in Figure 5 represents the course’s
planning process at “Ecole supérieure Polytechnique de
Dakar” which has an application called "VT System". In the
current mode of operation, there are several entities that can
be considered as organizational units using autonomous
systems. Each training course has a pedagogical manager who
ensures the scheduling of courses and who requests the
allocation of a room from an actor named focal point in charge
of managing the occupation of the rooms. The starting point
of the planning is the entry of schedules on an Excel file by
the pedagogical manager who transmits it to the focal point by
email or on an electronic medium.
The information contained in the file is entered into the VT
system by the focal point, which at the same time assigns the
classrooms. The output information is sent by the focal point
to the class supervisor and teacher. It may also happen that a
pedagogical manager assigns rooms instead of the focal point.
The class supervisor receives the printed form of the enriched
schedule, shares it with the students and fills in the textbook
that will be signed by the teacher once the course is done. As
the person in charge of training is not connected to the VT
system, the focal point enters the schedule transmitted in
Excel format in the application.
From the analysis of the first interconnection node, the
paths shown in bold draw the data processing circuit in the
“As-is” situation. It appears that the circuit is not optimized
with the non-existence of a data channel between the two
systems. Furthermore, the usage diagram in Figure 7 shows
that there is a confusion of actions between the pedagogical
manager and the focal point. The pedagogical manager issues
a programming request to the focal point, which registers it in
the system before processing it.
The case presented at Figure 3 corresponds to a situation
where all exchanges are not automated:
• the class supervisor uses an isolated system,
• the focal point prints the timetable and transmits it to
the class supervisor,
• the class supervisor notifies the new course’s
scheduling to students,
• teachers and class supervisor do not have access to VT
System.
From the combined analysis of Figures 4 and 5, it comes
out that in addition to improvements in the data processing
circuit, the system engineering process allows for an inventory
of functionality by user to determine activities performed in a
redundant manner as shown in Figure 5. Each time an actor
intervenes, there are NVA activities that are identified around
business’ ones. Most of NVA activities aim at putting the data
in the right format.
Fig. 3. Sequence diagram of the planning and execution of the courses
Fig. 4. Data circuit between the Pedagogical Manager and the focal Point
The execution of a business activity is often surrounded by
NVA activities. For example, the filling of the textbook by the
class supervisor is preceded by its retrieval at the focal point
level at the beginning of the day. The activity diagram
presented in Figure 5 is obtained after processing the NVA
activities carried out by the different actors. The objective of
the solution is to process the NVA activities in such a way that
each time a piece of information is available, another actor
cannot manually process it to perform a business activity.
The transfer of certain activities to the software, such as
notifications, the filling in of the textbook by the class
supervisor or its signature by the teacher, can be accompanied
by checks and reminders to ensure that all actors have received
the information intended for them or have correctly performed
the activities expected of them. In the same way, the follow-
up of the courses will be done directly in the system with the
integration of the class supervisors and the teachers. The
interest of this approach is to eliminate manual rework that can
lead to inconsistencies and errors in the execution of a
business process instance. It also limits the responsibility for
the completeness of the information to the actors involved in
the treatment process.
CONCLUSION
Functional interoperability does not imply interoperability
at software level. Similarly, technical solutions that promote
collaborative working, such as Web 2.0, web services and
APIs, only create communication gateways based on pre-
established formats. Each system is responsible for sending
data in the required format, and for receiving data using an
ETL or ELT tool before their use. These operations are
interoperability efforts. The proposed data circuit in this study
aims to improve interoperability through data-based
evaluation, from production to transfer to a third-party system.
Data presentation thus becomes an interoperability concern.
This approach breaks with the logic of evaluating systems
separately and subjecting data to repetitive processing at each
interoperation, producing them, if possible, in a ready-to-use
format.
In perspective, a global analysis of the business process
according to the indications of the data circuit will enable a
precise distinction to be made between business activities and
Fig. 5. Activity diagram for scheduling and monitoring courses
REFERENCES
[1] D. Chen, G. Doumeingts, et F. Vernadat, “Architectures for enterprise
integration and interoperability: Past, present and future”, Computers
in Industry, vol. 59, no 7, p. 647-659, sept. 2008, doi:
10.1016/j.compind.2007.12.016.
[2] N. DACLIN, D. CHEN, et B. VALLESPIR Enterprise interoperability
measurement-Basic concepts. EMOI-INTEROP, 2006, vol. 6.
[3] M. S. Camara, R. Dupas, Y. Ducq, et B. Mané, “Interoperability
Improvement in Inter-Enterprises Collaboration: A Software
Engineering Approach”, in Enterprise Interoperability VI, K. Mertins,
F. Bénaben, R. Poler, et J.-P. Bourrières, Éd., Cham: Springer
International Publishing, 2014, p. 201-211. doi: 10.1007/978-3-319-
04948-9_17.
[4] G. da Silva Serapião Leal, W. Guédria, et H. Panetto, “Interoperability
assessment: A systematic literature review”, Computers in Industry,
vol. 106, p. 111-132, avr. 2019, doi: 10.1016/j.compind.2019.01.002.
[5] CLARK, Thea et JONES, Richard. “Organisational interoperability
maturity model for C2.” In: Proceedings of the 1999 Command and
Control Research and Technology Symposium. 1999.
[6] S. FEWELL et T. CLARK “Organisational interoperability:
evaluation and further development of the OIM model.” Defence
Science and Technology Organisation Edinburgh (Australia), 2003
[7] G. da Silva Serapião Leal, W. Guédria, et H. Panetto, “Interoperability
assessment: A systematic literature review”, Computers in Industry,
vol. 106, p. 111-132, avr. 2019, doi: 10.1016/j.compind.2019.01.002.
[8] T. Ford, J. Colombi, S. Graham, et D. Jacques, “Survey on
Interoperability Measurement”, p. 67, juin 2007.
[9] WANG, Wenguang, TOLK, Andreas, et WANG, Weiping. “The levels
of conceptual interoperability model: applying systems engineering
principles to M&S.” arXiv preprint arXiv:0908.0191, 2009.
[10] W. Guédria, Y. Naudet, et D. Chen, “Interoperability Maturity Models
– Survey and Comparison – " in On the Move to Meaningful Internet
Systems: OTM 2008 Workshops, R. Meersman, Z. Tari, et P. Herrero,
NVA activities. The processing carried out on these NVA
activities will make it possible, among other things, to limit
the number of participants to those performing business
activities, and to translate programmable tasks into
functionalities managed directly at software level.
Éd., in Lecture Notes in Computer Science, vol. 5333. Berlin,
Heidelberg: Springer Berlin Heidelberg, 2008, p. 273-282. doi:
10.1007/978-3-540-88875-8_48.
[11] C. Campos, R. Chalmeta, R. Grangel, et R. Poler, “Maturity Model for
Interoperability Potential Measurement”, Information Systems
Management, vol. 30, no 3, p. 218-234, juill. 2013, doi:
10.1080/10580530.2013.794630.
[12] G. Paschina, A. Clematis, G. Zereik, et D. D’Agostino,
“Implementation of an interoperable interface to exchange B2B
messages between heterogeneous computer platforms”, 2014.
[13] H. R. Motahari Nezhad, B. Benatallah, F. Casati, et F. Toumani, “Web
Services Interoperability Specifications”, Computer, vol. 39, no 5, p.
24-32, mai 2006, doi: 10.1109/MC.2006.181.
[14] IEC, IEC. 62264-1: Enterprise-Control System Integration–Part 1:
Models and Terminology. IEC: Geneva, Switzerland, 2013.
[15] ISO 19440:2020 Enterprise modelling and architecture — Constructs
for enterprise modelling
[16] M. S. Camara, Y. Ducq, et R. Dupas, “A methodology for the
evaluation of interoperability improvements in inter-enterprises
collaboration based on causal performance measurement models”,
International Journal of Computer Integrated Manufacturing, vol. 27,
no 2, p. 103-119, févr. 2014, doi: 10.1080/0951192X.2013.800235.