Congratulations to all :)

Če kdo rabi predstavitev še naredit, je tle moj primer:

https://docs.google.com/presentation/d/1W-lrL7mmtY0KSrlRTxR0j4ypV6VbZJmXhJWfU8sT
BLg/edit?usp=sharing
ON THE EXAM WE CAN HAVE 1 CHEAT SHEET

DYNAMO​ IN ​BIGTABLE WON’T BE ON EXAM

INTRODUCTION
1) What is a distributed database system? 
A distributed database (DDB) is a collection of multiple, logically interrelated databases 
distributed over a computer network. 

2) What are the promises of parallel DBMS? 

Transparent management of distributed, fragmented, and replicated data Improved 
reliability/availability through distributed transactions Improved performance Easier and 
more economical system expansion 

1
3) What is the meaning of the transparency in distributed database systems. 
Transparency is the separation of the higher level semantics of a system from the lower 
level implementation issues. 
Lower level implementation: 
The data is distributed in a network - location 
The data is replicated  
The data is fragmented 
 
High level:  
The DB is used like a normal DB the user is not concerned about the implementation 
level 

4) Describe the most important issues (problems) that are handled by a distributed 
database system. 
How to distribute the database - replication ? fragmentation? 
How to optimize query? the problem is NP-hard 
Concurrency control- many users many systems, access synchronization, consistency & 
isolation of transactions 
Reliability- how to make the system resilient to failures - Atomicity and durability of 
transactions 

5) Present the classical architecture of distributed database systems. 

 
User interface + view management 
Control: semantic integrity + authorization checking 
Compilation: query decomposition and optimization + access plan management 
in: relational calculus  out: relational algebra 
Execution: access plan execution control + algebra operation execution 
in: relational algebra  out: retrieval/update 
Data Access: buffer management + access methods 
Consisteny: concurrency control + logging 
DATABASE  

6) Describe the client/server architecture of a database system. 

2
  
+ 
 
 

7) Present the database server and the distributed database server architectures. 

3
Introduction to no-SQL database systems

1) What were the main motives for the design of the novel database systems? 
to decrease complexity 
Horizontal scalability and running on comodity hardware -> hight throughput 
Decrease cost of setting up a database cluster 
Cloud computing 
 

2) What are the main design considerations in modern database management systems? 
Main memory -> ram DB 
Multi thread and resource control 
high availability -> shared nothing  
Remove bottleneck of redo-log and undo-log 

3) List the main directions used for the design of H-Base (Stonebraker) database 
system. 
 

4) Present a few critical aspects of the NoSQL database systems. 

● NoSQL is nothing new
● NoSQL as a hype - evolves to some point and fades away

5) Present briefly the main types of the NoSQL database systems. 

   

4
DISTRIBUTED DATABASE DESIGN 
Database distribution design
 
1) Describe the concepts of a fragment and the fragmentation of the relational 
distributed database. 
 
Fragment: we fragment subsets of relations  
-> because transactions access only different portions of relations rarely the whole 
relation 
-> increase the level of concurrency 
-> views defined over multiple fragments suffer performance degradation 
-> hard to perform semantic data control, like integrity checking, because relations are 
split and attributes are decomposed on different fragments on different sites 
 
 
2) Explain the concept of a degree of fragmentation. 
 
The degree of fragmentation goes from one extreme, that is, not to fragment at all 
(relations), to the other extreme, to fragment to the level of individual tuples (in the case 
of horizontal fragmentation) or to the level of individual attributes (in the case of vertical 
fragmentation) 
-> We need to find a suitable level of fragmentation that is a compromise between the 
two. 
-> How to find it? we have to understand the application needs!!! 
 
3) Describe the allocation alternatives for the given fragmentation of the database. 
 
•Non-replicated ➡ partitioned : each fragment resides at only one site  
•Replicated ➡ fully replicated : each fragment at each site  
➡ partially replicated : each fragment at some of the sites 
RULE OF THUMB - read query / update query << 1  
replication is advantageous 
otherwise replication may cause problems 
 
 
 
 
 
 

5
4) What information requirements does the process of the distributed database design 
have? 
 
MANY FACTORS CONTRIBUTE TO THE OPTIMAL DESIGN!!! 
->Database information - logical organization 
➡ Application information  
➡ Communication network information 
➡ Computer system information 
 
5) Describe the problem of the primary horizontal fragmentation. 
 
HORIZONTAL FRAGMENTATION: 
It refers to the division of a relation into subsets (fragments) of tuples (rows). Each
fragment is stored at a different node, and each fragment has unique rows. However,
the unique rows all have the same attributes (columns).
The data of a relation (table) is split between nodes on a criteria defined like a value.
example: data about product -> split between nodes by price
price <200 on node A price >= 200 on node B
 
Primary horizontal fragmentation ​of a relation is performed using predicates that are 
defined on that relation 
 
PROBLEM: 
Completeness of a predicate -> a predicate is complete if an only if there is an equal 
probability of access by every application to any tuple or fragment defined according to 
the predicate -> we get a balanced load!! 
A predicate is minimal iff it causes a fragment f to be further fragmented into, say, fi 
and fj), there should be at least one application that accesses fi and fj differently. 
  
MINTERM: combination of more predicates 
 
 
6) Present the algorithm COM_MIN for finding a complete and minimal set of simple 
predicates. 
 
This algorithm, called COM MIN, an iterative algorithm that would generate a complete 
and minimal set of predicates Pr0 given a set of simple predicates Pr.  
 
Given: a relation R and a set of simple predicates Pr  
Output: a complete and minimal set of simple predicates Pr' for Pr 
 
7) Describe the algorithm PHORIZONTAL for computing the set of minterm predicates 
from the set of simple predicates. 
 

6
Makes use of COM_MIN to perform fragmentation. 
Input: a relation R and a set of simple predicates Pr  
Output: a set of minterm predicates M according to which relation R is to be fragmented   
 
Pr' COM_MIN (R,Pr)   
determine the set M of minterm predicates   
determine the set I of implications among pi Pr 
eliminate the contradictory minterms from M  
 
8) Explain derived horizontal fragmentation. 
 
Relations: PAY & EMPLOYEE 
 
Owner: Pay 
Member: Employee 
-> pay 1 - N Employee 
 
-> We can split Employee by pay!! 
 
 
A derived horizontal fragmentation is defined on a member relation (pay) of a link 
according to a selection operation specified on its owner (Employee).​ It is important to 
remember two points. First, the link between the owner and the member relations is 
defined as an equi-join. Second, an equi-join can be implemented by means of semijoins. 
This second point is especially important for our purposes, since we want to partition a 
member relation according to the fragmentation of its owner, but we also want the 
resulting fragment to be defined only on the attributes of the member relation. 
 
 
 
9) What are the information requirements for the vertical fragmentation? 
 
VERTICAL FRAGMENTATION: 
It refers to the division of a relation into attribute (column) subsets. Each subset
(fragment) is stored at a different node, and each fragment has unique
columns—with the exception of the key column, which is common to all fragments.

INFORMATION REQUIREMENTS:
The major information required for vertical fragmentation is related to applications

Vertical partitioning places in one fragment those attributes usually accessed

together. This measure is the​ affinity​ of attributes, which indicates how closely

7
related the attributes are. Unfortunately, it is not realistic to expect the designer or
the users to be able to easily specify these values

 
10) Describe the role of the clustering algorithm employed for vertical fragmentation. 
 
It is designed specifically to determine groups of similar items as opposed to, say, a 
linear ordering of the items (i.e., it clusters the attributes with larger affinity values 
together, and the ones with smaller values together) 
 
HOW TO MEASURE AFFINITY??? 
 
affinity (Ai , Aj ) = SUM( (number of query access) all queries that access Ai and Aj) 
 
 
ACCESS FREQUENCY MATRIX: 
rows : query  
Columns: nodes (sites) 
cell: number of execution of that query Qi on site Sj 
 
AFFINITY MATRIX: 
rows: Atributes Ai 
Cols: Atributes Aj 
cell: number of execution of a query that accessed Ai and Aj at the same time 
 
CLUSTERING ALGORITHM: 
Take the attribute affinity matrix AA and reorganize the attribute orders to form clusters 
where the attributes in each cluster demonstrate high affinity to one another 
 
 
11) Give a general description of the clustering algorithm Bond-Energy. 
 
BEA finds an ordering of entities (in our case attributes) such that the global affinity 
measure is maximized 
 
Out clustered affinity matrix -> atributes with high affinity are near 
 
12) Explain the cluster forming algorithm for vertical fragmentation. 
 
in: Clustered affinity matrix 
result: 

8
How can you divide a set of clustered attributes {A1 , A2 , …, An} into two (or more) sets 
{A1 , A2 , …, Ai} and {Ai , …, An} such that there are no (or minimal) applications that 
access both (or more than one) of the sets. 
 
13) Describe the information requirements for the database allocation. 
Database Information  
➡ selectivity of fragments  
➡ size of a fragment  
•Application Information  
➡ number of read accesses of a query to a fragment  
➡ number of update accesses of query to a fragment  
➡ A matrix indicating which queries updates which fragments 
➡ A similar matrix for retrievals  
➡ originating site of each query  
•Site Information  
➡ unit cost of storing data at a site  
➡ unit cost of processing at a site  
•Network Information  
➡ communication cost/frame between two sites  
➡ frame size 
 
14) Describe the allocation model for assigning fragments to servers. 
 
We are searching for the minimum total cost:response time constraint  
storage constraint processing constraint 
 
Response time: execution time of a query 
 
Query processing cost: acces cost + integrity enforcement cost + concurrency control  
acces cost = SUM_allSites(SUM_allFragments(updates + read access)) 
 
Storage constraint: storage requirements of a fragment on the site -> for all fragments 
 
Processing constraints : processing load on a site 
 
•Attempts to reduce the solution space 
➡ assume all candidate partitionings known; select the “best” partitioning  
➡ ignore replication at first  
➡ sliding window on fragments 
 
 
 
 
   

9
DISTRIBUTED QUERY
PROCESSING
Introduction

1) Describe the problem of distributed query optimization. 

 
Transform high level query (relational calculus) to low level query (relational algebra) 
The transformation must achieve both correctness and efficiency 
 
CORRECTNESS it produces the same result of the high level (easy) 
EFFICIENCY Since each equivale nt execution strategy can lead to very different 
consumptions of computer resources, the main difficulty is to select the execution 
strategy that minimizes resource consumption 
 

2) Present the main components and characteristics of distributed query processors. 

 
Query language that is used ➡ SQL: “intergalactic dataspeak”  
•Query execution methodology ➡ The steps that one goes through in executing 
high-level (declarative) user queries. 
•Query optimization ➡ How do we determine the “best” execution plan? 
 
WHAT WE WANT TO OPTIMIZE? 
Minimize cost functions: I/O cost + CPU cost + Communication cost 
-> everithing counts differently in different environments WAN / LAN 
 

3) What are the main approaches to distributed query optimization? 

 
Exhaustive search (Test all the possibilities)   
➡ Cost-based  
➡ Optimal  
➡ Combinatorial complexity in the number of relations 
 
Heuristics (IMPORTANT) 
➡ Not optimal  
➡ Regroup common sub-expressions  
➡ Perform selection, projection first  
➡ Replace a join by a series of semijoins  

10
➡ Reorder operations to reduce intermediate relation size  
➡ Optimize individual operations 
 
OPTIMIZATION WHEN? 
Static -> recieve query -> optimize -> execute 
 
Dynamic -> recieve query -> optimize in runtime 
 
 

3) Present the layers of distributed query processing. 

 

Query decomposition and localization

1) Describe the process of the distributed query decomposition. 

11
Query decomposition is the first phase of query processing that transforms a relational
calculus query into a relational algebra query.
Both input and output queries refer to global relations, without knowledge of the distribution
of data. -> Therefore, query decomposition is the same for centralized and distributed
systems.
Process of the distributed query decomposition:
Normalization:​ Lexical and syntactic analysis + Transform query in conjunctive normal form
(p1 V p2 V p3) ∧ ( ….) ∧ ( …. )
Analysis:​ Reject type incorrect or semantic incorrect query
Simplification:​ eliminate redundancy -> redundant predicates
Restructuring​: Rewrites the relational calculus in relational algebra
Problem: more than one transformation can exist ! Some are faster than others!!

2) Present the query graph and the join graph together with their function. 
 

 
A query is semantically incorrect if its query graph is not connected! It shows the whole 
query. -> in picture a query graph is connected -> The query is sematically correct! 
 
 
The join graph is particularly useful in the query optimization phase. 
 

3) Describe query simplification. 

 
eliminate redundancy via those rules:

12
 

4) Describe the representation of queries using the relational algebra operations. 

 
Relational algebra:  

 
Projection = SELECT in SQL
Selection = WHERE in SQL

5) Describe the use of the relational algebra equivalence rules. 

 
● JOIN (ali cartesian) is commutative R ​⨝​ S ⇔ S ⨝
​ ​ R  
​ ​ S) ​⨝​ T ⇔ R ⨝
● JOIN (ali cartesian) is associative (R ⨝ ​ ​ (S ​⨝​ T) 
● Group subsequent projections: σ1 p1(σ2 p2 (R)) = σ p1∧p2 (R) 

Commutativity of binary operations R × S ​⇔​ S × R isto R ⋈S ​⇔​ S ⋈R

Associativity of binary operations (R ⋈S) ⋈T ​⇔​ R ⋈ (S ⋈T)

13
Group subsequent projections: σ1 p1(σ2 p2 (R)) = σ p1∧p2 R 

Commuting selection with projection:  

Commuting selection with binary operator:  
 
Commuting projection with binary operator:  
 
6) Present the main functions of the query localization process. 
 
The localization layer must take into account the distribution of data! 
It translates an algebraic query on global relations (DB model) on an algebraic query 
expressed on physical fragments. 
Localization uses information stored in the fragment schema 

7) Describe the approaches to the parallel execution of distributed queries on the 

fragmented relations. 
 
We can parallelize inner joins to nodes, to make the processing faster. (Potrebno je malo 
filozofirati kako naj bi porazdelili algebricni query in več vozlišč) 

8) Overview reductions of queries for the horizontal and vertical fragmentations. 

HORIZONTAL: split data by a value

We take into account the range of the attribute and on which node are these values!
We query only the node which has these values.

Push JOIN down and UNION up (in the execution tree)

DERIVED HORIZONTAL: data split by a value from another relation

Same as Horizontal normal

VERTICAL: split data by columns

Take into account only attributes that you need (only needed columns) if a relation is split on
two nodes and you need data from the relation but only columns on one node ! Query only
one node!

Query optimization

1) Describe the global query optimization in distributed database systems. 

14
The idea is to find the best global schedule - minimize a cost function = I/O cost + CPU cost
+ communication cost
Query first goes into search space generation which provides an QEP (Query Execution
Plan), then it provides search strategy, test the search strategy with the cost model
Iteratively repeat to find the best QEP.

2) Explain the concept of the search space in query optimization. 

The search space is defined by all the alternative combinations of a query, that produces the
same result. For N relations it is O(N!)
We can restrict the search space by heuristics like:
push selection and projection down and JOIN up
Restrict the search space by the shape of the join tree:

3) Explain the concept of the strategy in distributed query optimization. 

It describes, how to move in the search space. It can be ​deterministic ​(start from base
relations and build plans by adding one relation per step) or ​randomized ​(search for
optimalities around a particular starting point)

4) Describe the cost function in distributed query optimization. 

cost function = I/O cost + CPU cost + communication cost

I/O - disk
communication cost: Message Passing Protocol
Depends also on the network - LAN , WAN or INTERNET

5) Present the main ideas of the dynamic distributed query optimization. 

PUSH down all selection and proiection to sites

Decompose complex SQL query (multy variable) into many smaller irreducible parts (mono
variable) Q -> q1,q2,q3,q4….

Chose the part qi that produces the smallest result and build the linear tree.
Determine the processing site

15
 Determine fragments to move
Repeate

execution plan:
q3 -> q2 -> q4 ->q1 equivalent to Q

6) Present the main ideas of the static distributed query optimization. 

Dynamically generate all the best permutations of the query. Exhaustive search
For every candidate query evaluate the cost functio.

First find the best access method to each individual relation by using selection.
Second for each relation R​ find the best JOIN ordering​ -> based also on indexes

-> Choose the best execution plan that minimizes the cost function

For joins take into account NESTED LOOP (inner relation indexed or small) and MERGE
JOIN (good if relations are sorted)

7) Explain the concept of the join ordering as used in distributed query optimization. 
 
Where should we make the JOIN??  
The Data is distributed / fragmented ! -> At which node should we JOIN? 
 
By data size!!  
if size(node1) < size(node2) move the data from node1 to node2 and join in node2!! 
HOW? JOIN: L ⋈​ ​R L=outer R=inner  (depends on JOIN 
implementation loop, merge and indexes) 
1. SHIP WHOLE: outer to the inner site -> outer rows can be joined as they arrive (if 
merge join) 
2. SHIP WHOLE: inner to the outer site -> inner rows can’t be joined as they arrive, so 
we need to store them in an TMP relation. 
3. FETCH-AS-NEEDED: for each row in outer FETCH a row in inner site and join as it 
arrives. (good if inner is small and indexed. works as semijoin) 
4. MOVE BOTH to a THIRD SITE: firts inner aand save it in TMP then outer and 
compute JOIN as the outer arrive 
 
 
 
 
WHAT IS SEMIJOIN? 
Returns all the records from the left side that have matches from the right. The
result table contains columns from the left side only 

16
 
 
 
 

DISTRIBUTED TRANSACTION
MANAGEMENT

Transactions

1) What is a transaction? Describe its principles, behavior, and possible states. 

Transaction is a collection of actions that make consistent transformations of system states
while preserving system consistency. Transactions follow ACID principles.
ACID stands for:
Atomicity​ (all or nothing)
Consistency​ (no violation of integrity constraints) after the execution of a transaction the DB
is in consistent state (various levels)
Isolation​ (concurrent changes invisible => serializable) if transactions are executed
concurrently, the result must be the same as if they were executed in a certain order
(serially)
An incomplete transaction, cannot reveal its results to other transactions before its commit
(various levels)
Durability​ (commited updates persist) Once a transaction commits, the system must
guarantee that the results will never be lost

A transaction can be in begin, execution or end state.

2) Explain the ACID properties of transactions. 

Atomicity:
- Either all or none of the transaction operations are performed.
- Atomicity requires

3) Present the well-known types of transactions (classified by applications, timing, 

organization, and structure). 

Application:​ distributed vs non-distributed

Timing: ​on-line??
Organization of read and write actions:​ Two-step, restricted , action
Structure:​flat(simple),nested transactions,workflows(a collection of tasks)

17
4) Give a bird's-eye view of the transaction processing. What components and software 
modules are involved? 
**?????**??*??
Ne bo uprašal!

5) Present an architecture of a distributed transaction manager. 

TM:​ Transaction manager- manages the transaction states also defines the distributed
transaction execution model and has replica control
SC:​Scheduler - distributed concurrency control over READ and WRITE
RM:​Recovery manager in local

Concurrency control

1) Present the concepts of a schedule, history, serial and serializable history, and global 
serializability. 

The problem of synchronizing concurrent transactions such that the consistency of the
database is maintained while, at the same time, maximum degree of concurrency is
achieved.
Problems:​ Lost updates- The effects of some transactions are not reflected on the database
Inconsistent retrievals- reads inconsistent data :C

18
Schedule or history:​An order in which the operations of a set of transactions are executed
a partial order over the operations of a set of transactions

Serial history:​ All the actions of a transaction occur consecutively

Leaves the DB in a consistent state after execution of a serial history

Serializable history:​ transactions run concurrently, but the result is equivalent to serial
history (zaporedno izvajanje)

Global serializability:​ the global history, to be serializable must

● each local history should be serializable
● two conflicting operations should be in the same relative order in all of the
local histories where they appear together

2) Explain the naive and two-phase locking algorithms. 

 
NAIVE: 
TM (transaction manager ) passes to the LM(lock manager ) single commands like 
READ and WRITE 
if it is possible to lock -> lock resource, execute read and write on x  
when execution ends release the lock. 
-> otherwise delay execution 
 
Good for concurrency BUT 
problems: resource are locked only for commands not whole transactions! 
loss of atomicity and isolation!! 

TWO PHASE LOCKING: 2PL 2PL 2PL 2PL 2PL 2PL

rule: No transaction should request a lock after it releases one of its locks.
-> when you want to release a lock you must be ready to release all LOCKs

2PL execute transactions in two phases: GROWING phase + SHRINKING phase

GROW - obtain locks
SHRINK - releasing locks
LOCK POINT: when the transaction has gained all of its locks (end of grow start of shrink)

ANY HISTORY GENERATED BY 2PL IS ​SERIALIZABLE​ (lahko izvedena vzporedno)

STRICT 2PL: release all locks at the same time ! saves some problems about aborting and
easier to implement.

3) Present centralized two-phase locking protocol in distributed database systems. 

 

19
ONLY ONE CENTRALIZED SCHEDULER!!! -> one central LM (lock manager)  

TM (transaction manager) at each site is sending lock request to central LM and gets back
lock granted to TM and then TM allow to execute operation on the site!!

4) Present distributed two-phase locking protocol in distributed database systems. 

 
DISTRIBUTED SCHEDULER: every node has TM and LM 
 
Node that receive transaction: 
coordinating TM sends Lock request to LM on ALL nodes ! 
 
All other nodes: 
LM doesn't send lock granted, but it handles lock request for data at its site and when it 
can gain the lock on data it executes the operation! 
When the operation ends the LM send to coordinating TM end of operations 
 
When 2PL in SHRINK (releasing all the locks): 
coordinating TM send release lock to appropriate (or all) LM  
 
 
 
IN CASE OF REPLICATION: 
READ: read lock only on one copy 
WRITE: write lock on all copies 

Deadlocks

1) Define the deadlock in a distributed database environment. 

 
Deadlock is a state of a database system having two or more transactions, when each
transaction is waiting for a data item that is being locked by some other transaction. A
deadlock can be indicated by a cycle in the wait-for-graph. This is a directed graph in
which the vertices denote transactions and the edges denote waits for data items.
Source:
https://www.tutorialspoint.com/distributed_dbms/distributed_dbms_deadlock_handling.ht
m

2) Present the wait-for graph and its use in distributed database systems. 
Wait-for graph is a directed graph used for deadlock detection in database systems. If 
there is no cycle in a directed graph there is no deadlock present. If there is one however, 

20
we are stuck in a circular wait. In that case, transaction manager aborts some of the 
transactions to free up resources. 

Source:
http://www.mathcs.emory.edu/~cheung/Courses/554/Syllabus/8-recv+serial/deadlock-waitfor
.html

3) List and abstract the main approaches to deadlock handling. 

Deadlock prevention ->​ ​does not allow any transaction to acquire locks that will lead to
deadlocks. The convention is that when more than one transactions request for locking
the same data item, only one of them is granted the lock.

knjiga:
Deadlock cannot occur in any case! 
The TM permits the execution of a transaction only if all the recources needed by the 
transaction are availbable.
Resource must be reserved, but not necessary allocated apriori.

Evaluation:
● reduced concurrency due to preallocation
● load overhead to calculate if an allocation is safe
● difficulty to determine
● no need for transaction rollback

Deadlock avoidance ->​ ​handles deadlocks before they occur. It analyzes the
transactions and the locks to determine whether or not waiting leads to a deadlock.

Deadlock detection and removal ->​ r​ uns a deadlock detection algorithm periodically and
removes deadlock in case there is one. It does not check for deadlock when a
transaction places a request for a lock. When a transaction requests a lock, the lock
manager checks whether it is available. If it is available, the transaction is allowed to lock
the data item; otherwise the transaction is allowed to wait.

Source:

21
https://www.tutorialspoint.com/distributed_dbms/distributed_dbms_deadlock_handling.htm

4) Describe the deadlock prevention approach to deadlock handling. 

The deadlock prevention approach does not allow any transaction to acquire locks that will
lead to deadlocks. The convention is that when more than one transactions request for
locking the same data item, only one of them is granted the lock.
One of the most popular deadlock prevention methods is pre-acquisition of all the locks. In
this method, a transaction acquires all the locks before starting to execute and retains the
locks for the entire duration of transaction. If another transaction needs any of the already
acquired locks, it has to wait until all the locks it needs are available. Using this approach,
the system is prevented from being deadlocked since none of the waiting transactions are
holding any lock.

5) Describe the deadlock avoidance approaches to deadlock handling. 

The deadlock avoidance approach handles deadlocks before they occur. It analyzes the
transactions and the locks to determine whether or not waiting leads to a deadlock.

The method can be briefly stated as follows. Transactions start executing and request data
items that they need to lock. The lock manager checks whether the lock is available. If it is
available, the lock manager allocates the data item and the transaction acquires the lock.
However, if the item is locked by some other transaction in incompatible mode, the lock
manager runs an algorithm to test whether keeping the transaction in waiting state will
cause a deadlock or not. Accordingly, the algorithm decides whether the transaction can
wait or one of the transactions should be aborted.

There are two algorithms for this purpose, namely wait-die and wound-wait. Let us assume
that there are two transactions, T1 and T2, where T1 tries to lock a data item which is
already locked by T2. The algorithms are as follows −
● Wait-Die − If T1 is older than T2, T1 is allowed to wait. Otherwise, if T1 is younger than
T2, T1 is aborted and later restarted.
● Wound-Wait − If T1 is older than T2, T2 is aborted and later restarted. Otherwise, if T1 is
younger than T2, T1 is allowed to wait.

Source:
https://www.tutorialspoint.com/distributed_dbms/distributed_dbms_deadlock_handling.htm

6) Present the deadlock detection and recovery methods (centralized, distributed, and 
hierarchical). 
The deadlock detection and removal approach runs a deadlock detection algorithm
periodically and removes deadlock in case there is one. It does not check for deadlock
when a transaction places a request for a lock. When a transaction requests a lock, the
lock manager checks whether it is available. If it is available, the transaction is allowed
to lock the data item; otherwise the transaction is allowed to wait.

22
Since there are no precautions while granting lock requests, some of the transactions
may be deadlocked. To detect deadlocks, the lock manager periodically checks if the
wait-forgraph has cycles. If the system is deadlocked, the lock manager chooses a
victim transaction from each cycle. The victim is aborted and rolled back; and then
restarted later. Some of the methods used for victim selection are −
● Choose the youngest transaction.
● Choose the transaction with fewest data items.
● Choose the transaction that has performed least number of updates.
● Choose the transaction having least restart overhead.
● Choose the transaction which is common to two or more cycles.

This approach is primarily suited for systems having transactions low and where fast
response to lock requests is needed.
Source:
https://www.tutorialspoint.com/distributed_dbms/distributed_dbms_deadlock_handling.h
tm
 
Centralized deadlock detection: 
One central site is the deadlock detector 
Every site sends periodically WFG to the central site 
The central site merge all WFG to a global WFG to determine cycles. 
 
Distributed deadlock detection: 
Each site has a local WFG which is passed to other sites. 
If a cycle occur in a local WFG the deadlock can be handled locally  
If a cycle occur in a merged WFG it indicates a potential global deadlock -> pass info to 
other sites in the cycle and solve the deadlock 
 
Hierarchical: 
Similar to distributed, but deadlock are not solved globally but hierarchically 
 
 
   

23
DATA REPLICATION
Replication

1) Present the taxonomy of the replication issues. 

The main problem is maintaining consistency. How do we keep the values of physical copies
of a logical data item synchronized?
Another problem is in replication transparency. Because apps and users need to know who
the master is.
Source:
http://osebje.famnit.upr.si/~savnik/predmeti/IPPB/Replication.pdf
page 4

2) Why do we replicate? 
We replicate to increase system availability. By doing that we avoid single points of failure.
With replication we increase performance and scalability (in numbers and geographic area)
Source:
http://osebje.famnit.upr.si/~savnik/predmeti/IPPB/Replication.pdf
page 2

3) Explain the concepts of mutual, strong, and weak consistencies. 

Mutual consistency -> How do we keep the values of physical copies of a logical data item
synchronized? Replicas converging to the same value! Mutual consistency can be either
strong or weak.
Strong consistency -> all copies are updated within the context of the update transaction.
When the update transaction completes all copies have the same value.
Weak consistency -> Copies are not identical when update transaction completes, but they
eventually converge to the same value.
Source:
http://osebje.famnit.upr.si/~savnik/predmeti/IPPB/Replication.pdf
page 5

4) Present the definition of 1SR transactional consistency? 

1SR (One-copy serializability) is the effect of transactions performed by clients on replicated
objects should be the same as if they had been performed one at-a-time on a single set of
objects.
Source:
http://osebje.famnit.upr.si/~savnik/predmeti/IPPB/Replication.pdf
page 6

Knjiga:

24
1SR states that the effect of transactions on replicated data items should be the same as if
they had been performed one at a time on a single set of data items. (celotna zgodovina
update na replication je isti, kot da se bi zgodil zaporedno )

5) Present the main approaches used in update management strategies. 

They can differ depending on when the updates are propagated
Eager ​-> Changes are propagated within the scope of the transaction making the changes
when the transaction commits, all the copies have the same value
Lazy​ -> First executes the updating transaction on one copy. After the transaction commits,
the changes are propagated to all other copies.
the transactions commit as soon as one replica is updated

Or depending on where the updates can take place

Centralized -> Only one copy which can be updated (master), the other ones (slaves) are
updated reflecting the changes to the master.
Distributed -> Changes can be initiated at any of the copies. That is any of the sites which
owns a copy can update the value of the data item.
Source:
http://osebje.famnit.upr.si/~savnik/predmeti/IPPB/Replication.pdf
page 9

6) Describe eager centralized protocols (limited and full transparency) for replication. 
 
Limited Transparency:​ -> user application knows who is MASTER
Applications submit update transactions directly to the master.
Master then reads locally and returns the data to the user. Writing gets executed locally on
master and then replicated to slaves.

Read to slaves is ossible, but the slave must obtain read lock on MASTER, to confirm that
the data is consistent!!

Full Transparency:​ User application don’t know who is the MASTER

Slaves work as a coordinator:
when slave recive a read -> set read lock on master master sends back granted
then allow read
when slave recive a write -> send write to master site which set a write lock
perform write
informs coordinator lock granted and COORDINATOR propagate write

differences:
Master doesn’t deal with reads but it just confirm locks
Master doesn’t propagate write ! -> this is the job of the coordinator that takes the request

Source:

25
http://osebje.famnit.upr.si/~savnik/predmeti/IPPB/Replication.pdf
page 17

7) Present eager distributed protocols for replication. 

Updates originate at any copy. Each site uses 2 phase locking to prevent deadlocks.
Read operations are done locally while write operations are performed at all sites (using
distributed locking protocol). Issue with it is that concurrent writes initiated at different master
sites are executed in the same order at each slave site.

HIGH comunication overhead

Source:
http://osebje.famnit.upr.si/~savnik/predmeti/IPPB/Replication.pdf
page 25

8) Describe lazy centralized protocols for replication. 

Updates are first applied to a master replica and then propagated to the slaves.
Propagation does not take place within the update transaction but after the transaction
commits separate refresh transaction sent to slaves.
Slave site performs read operations on its local copy.

Source:
http://osebje.famnit.upr.si/~savnik/predmeti/IPPB/Replication.pdf
page 27

9) Present lazy distributed protocols for replication. 

Reads, writes, commits/aborts are done locally and returned to the user.
We receive messages from other sites about install changes (writes) and conflicts.
Reconcialiation is necessary (združiti write skupaj določiti kdo ima prednost) by heuristics
depending on the implemented application (timestamps, priority of a node...)
Source:
http://osebje.famnit.upr.si/~savnik/predmeti/IPPB/Replication.pdf
page 36
 
   

26
PARALLEL DATABASE SYSTEMS
Parallel database systems

1) Explain the I/O bottleneck problem in database systems and its solution. 
An I/O bottleneck is a problem where a system does not have fast enough
input/output performance. I/O bottlenecks can be caused by various things and
require various solutions.
Solution is to increase I/O bandwidth, by data partitioning or in other words​ parallel
data access​.

Source:
http://osebje.famnit.upr.si/~savnik/predmeti/IPPB/Parallel-DBMS.pdf
page 2

2) Describe the data server architecture. What are the benefits and drawbacks? 

Separation of concerns:
Application server + Data server

Benefits:
- Integrated data control by the server (black box)
- increased performance by dedicated systems
- can better exploit parallelism
- fits well in distributed environments
Drawbacks:

27
 - Communication overhead between application and data server
- high cost of mainframe servers

Source:
http://osebje.famnit.upr.si/~savnik/predmeti/IPPB/Parallel-DBMS.pdf
page 6

3) What is data-based parallelism? 

Data parallelism is parallelization across multiple ​processors​ in ​parallel computing
environments. It focuses on distributing the data across different nodes, which
operate on the data in parallel. There are two types of data parallelism:

Source:
http://osebje.famnit.upr.si/~savnik/predmeti/IPPB/Parallel-DBMS.pdf
page 9

4) Present the definition and objectives of the parallel database system. 

 
Parallel DBMS is a DBMS implemented on a tightly coupled multiprocessor.
Def knjiga:​ A parallel DBMS is a DBMS implemented on a parallel computer.

Its objectives are to achieve the following:

- Performance:​Much better cost/performance ratio than mainframe solution
- High performance through parallelism
- Aviability:​High availability and reliability by exploiting data replication
- Extensibility​ easily increasing performance or disk size

Source:
http://osebje.famnit.upr.si/~savnik/predmeti/IPPB/Parallel-DBMS.pdf
page 11

28
5) Explain the linear speed-up and linear scale-up. 
Linear speed-up:
Linear increase in performance for a constant DB size and proportional increase of the
system components (processor, memory, disk)
Constant DB -> performance increase if we add more components

Linear scale-up:
Sustained performance for a linear increase of database size and proportional increase of
the system components.
DB increase -> performance stays the same if we add more components

Source:
http://osebje.famnit.upr.si/~savnik/predmeti/IPPB/Parallel-DBMS.pdf
page 11

6) Describe the functional architecture of parallel database systems. 

29
Session manager: ​transaction monitor providing support for client interactions. Perform the
connection and disconnection of client and maintain a user session for executing
transactions.

Transaction manager - Request manager:​ recieves client transactions verifing semantic

and running execution of transactions then returning error code or result to client.

Data manager:​ low level functions to run compiled queries in parallel.

Source:
http://osebje.famnit.upr.si/~savnik/predmeti/IPPB/Parallel-DBMS.pdf
page 15

7) Which parallel architectures are used as frameworks for the implementation of a 

parallel database system? 
Multicore architecture alternatives:
- Shared memory
- Shared disk
- Shared nothing
Hybrid architecture:
- NUMA (Non-Uniform Memory Architecture)
- Clusters

Source:
http://osebje.famnit.upr.si/~savnik/predmeti/IPPB/Parallel-DBMS.pdf
page 16

8) Explain the shared memory architecture of a parallel DBMS. 

30
This is what has come to be known as an SMP (symmetric multiprocessor). Memory
and peripherals are shared among many processors. The advantages are easy
programming and load balancing. The problem is low fault isolation because of the
shared memory.

Many processors connected on a bus, that shares the same memory.

Inter query parallelism always possible

Intra query parallelism requires some parallelization
Load balancing easy

problems:
high cost
conflict acces the shared memory if we have many processors
Limited extensibility of processors

Source:
https://www.sciencedirect.com/topics/computer-science/parallel-database

9) Describe the shared disk architecture of a parallel DBMS. 

Each processor has its own memory, but it shares the disk subsystem with other
processors. In addition, there is a communication network. This design allows for an
easy migration from a single node configuration, it offers fairly good fault isolation,
and it goes well with electronic disks for performance improvement.

Lower cost , high extensibility

Problem: global cache consistency !!!! -> need to use 2PL and distributed locking

Source:
https://www.sciencedirect.com/topics/computer-science/parallel-database

10) Describe the shared-nothing architecture of a parallel DBMS. 

Each processor has its own memory and its own peripherals;
Each node can be viewed as a local site (with its own OS and DB system)
communication is through messages only. This design has great fault isolation and
scales well. The problem is the communication bottleneck.

Low cost + high extensibility + high availability

Source:
https://www.sciencedirect.com/topics/computer-science/parallel-database

31
11) Present the NUMA (CC-NUMA) architecture of a parallel DBMS. 
Main memory is distributed so that every processor has its own memory but all the
processors can access all other processors memories. Since different processors
can access the same data, cache consistency protocols are needed.

NUMA = single address space on distributed physical memory

Any processor can access all other processors memory

NEED cache consistency.

Done in hardware very efficient CC-NUMA = Cache Coherent

Source:
http://osebje.famnit.upr.si/~savnik/predmeti/IPPB/Parallel-DBMS.pdf
page 27

12) Describe the cluster architecture of a parallel DBMS. 

A computer cluster is a set of loosely or tightly connected ​computers​ that work
together so that, in many respects, they can be viewed as a single system.​ ​Computer
clusters have each ​node​ set to perform the same task, controlled and scheduled by
software.​ With simple PCs we can form a powerful SMP. Combines a good load balancing
of shared memory with extensibility of shared nothing.

Source:
http://osebje.famnit.upr.si/~savnik/predmeti/IPPB/Parallel-DBMS.pdf
page 28

13) Describe the data partitioning methods and the forms of parallel query execution for 
each method. 
 
Data partition: how to split data to nodes 
Each relation is divided in n partitions(subrelations), where n is a function of relation size 
and access frequency 
 
 
Round-robin: 
Maps i-th element to node i mod n.  
Simple but only exact match queries are possible. 
 
 

32
B-tree index: 
Supports range queries but large index 
Data is split in intervals 
 
Hash function: 
Split data by hash function 
Supports only exact-match queries but small index. 

Source:
http://osebje.famnit.upr.si/~savnik/predmeti/IPPB/Parallel-DBMS.pdf
page 31

14) Explain the inter-query, intra-query, inter-operation, and intra-operation parallelisms. 

Inter-query parallelism: 
It is a form of parallelism where many different Queries or Transactions are executed in
parallel with one another on many processors.

Intra-query parallelism: 
It is the form of parallelism where Single Query is executed in parallel on many
processors.

Inter-operation parallelism: 
the process of speeding up a query through parallelizing the execution of individual
operations. The operations which can be parallelized are Sort, Join, Projection,
Selection and so on.

Intra-operation parallelism: 
One operator is decomposed in a set of indipendent sub-operators that can be
executed in parallel
the process of speeding up a query through parallelizing various operations which
are part of the query. For example, a query which involves join of 4 tables can be
executed in parallel in two processors in such a way that each processor shall join
two relations locally and the result1 and result2 can be joined further to produce the
final result.

Source:
http://www.exploredatabase.com/2014/03/interquery-and-intraquery-parallelism.html
 
15) Describe three main algorithms for parallel evaluation of the join operator. 
- Parallel nested loop join​: no special assumption
R join S:
each fragment of R is sent and replicated at each node containing a fragment of S.
Each node containing a fragment of S recieves the whole relation R and joins Whole R with
local S.

33
Then the union of all joins is computed

Whole operation:

- Parallel associative join​: one relation is declustered on join attribute and equi-join
- Parallel hash join​: same as parallel associative join but with indexes on R and S

Source:
http://osebje.famnit.upr.si/~savnik/predmeti/IPPB/Parallel-DBMS.pdf
Page 43

16) Present the parallel associative join algorithm. 

R Equijoin S -> on predicate A from R and B from S

relation S is partitioned according to the hash function h applied to join attribute B

-> all tuples of S have the same value for h(B) are placed on the same node.

No knowledge about R partitioning.

Algorithm:
hash(R.A) and send tuples to the correct node
(receive only usefull data ) Perform the join locally at each S node
Compute the union of the results

34
In the first phase, relation R is sent associatively to the S-nodes based on the hash function
h applied to attribute A. In the second phase, each Sj-node receives in parallel from the
different R-nodes the relevant subset of R (i.e., Rj ) and joins it locally with the fragments Sj.

Source:
http://osebje.famnit.upr.si/~savnik/predmeti/IPPB/Parallel-DBMS.pdf
Page 49

17) Describe the search space, the search strategy, and the cost model of a distributed 
query optimizer. 
Search space: 
Is the space of a problem domain where search problems are conducted. Execution 
plans are abstracted by means of operator trees. 
 
Search strategy: 
A search strategy is an organised structure of key terms used to search a database. The
search strategy combines the key concepts of your search question in order to retrieve
accurate results. It functions as a ​heuristic based ​optimization.

Cost model of a distributed query optimizer: 

To generate execution plans, the optimizer uses a cost model that is based on estimates of
the cost of various operations that occur during query execution. The optimizer has a set of
compiled-in default “cost constants” available to it to make decisions regarding execution
plans. Basically, cost model is responsible for estimating the cost of a given execution plan. 

 
Source: 
https://en.wikipedia.org/wiki/Search_algorithm  
https://library.leeds.ac.uk/info/1404/literature_searching/14/literature_searching_explai
ned/4  
https://dev.mysql.com/doc/refman/5.7/en/cost-model.html  
   

35
NOSQL DBMS: CONCEPTS,
TECHNIQUES AND PATTERNS
1) Present the main characteristics of the BASE consistency model? 

Source:
http://osebje.famnit.upr.si/~savnik/predmeti/IPPB/NoSQL-Concepts.pdf
page 4

2) Explain the CAP theorem for NoSQL DBMS? 

- Consistency​: System is in a consistent state after the execution of an operation;
After an update of some writer all readers see his updates in some shared data
source
- Availability:​ A system is designed and implemented in a way that allows it to
continue operation if e.g. nodes in a cluster crash or some hardware or software
parts are down due to upgrades
- Partition tolerance:​ The ability of the system to continue operation in the presence
of network partitions (the data is replicated, to make the system resistent to failures
-> replication produces inconsistency or difficulty to maintain it)

CAP theorem -> you can have only two of them in a shared-data system

Source:
http://osebje.famnit.upr.si/~savnik/predmeti/IPPB/NoSQL-Concepts.pdf
page 5

3) Explain the ordering of events in distributed systems with Lamport timestamps. 

 

36
Lamport has definied a local logical ordering where the order is defined by a logical 
relation A(x) -> B(x+1) = A happened before B   
 
All nodes have a local clock (lamport timestamp) at the beginning is 0 
A node increments its counter for each event: execution of an operation, message sent, 
message recieved 
 
When a node sends a message, it includes its incremented counter value with the 
message 
 
on the recieving of a message its counter is updated to the max(local clock,recieved 
clock) +1 
 
Ordering of events is determined by transitivity: A->B and B->C then A->C 
 
we get only​ a partial ordering​: not every two events can be comparable 
If two events are not comparable, then this events are ​concurrent​ they happened at the 
same time and are (neodvisni) 
 
But you can’t determine if two events are concurrent or it’s an error. 

4) Explain ordering events in a distributed system with vector clocks. 

 
Every node has a local clock ( an incrementing number ) which is incremented at the 
recieving of a message or at the execution of an operation or at the sending of a 
message. 
 
VECTOR clock: a vector containing all the clocks of nodes V[0] node 0 to V[n] 
 
Each node is maintaining a vector clock by reading messages from other nodes 
 
At each sending message the node is also sending its vector clock! 
 
The node recieves the message and the vector clock and it updates each element in the 
vector clock by taking the maximum and incrementing it’s own counter 
-> each node can determine a partial ordering by all the vector clocks. 
 
-> can determine which operations happened before or later 
 
We get a vector for each events -> we can detect if events are concurrent and have not 
influenced each other 
 
advantages:  no synchronized clocks 
No total ordering 

37
 

5) Present the main idea of optimistic concurrency control. 

OCC assumes that multiple transactions can frequently complete without interfering with each
other.
While running, transactions use data resources without acquiring locks on those resources.
Before committing, each transaction verifies that no other transaction has modified the data it has
read. If the check reveals conflicting modifications, the committing transaction rolls back and can
be restarted

6) Describe state propagation algorithms based on gossip. 

 
Its a metod to propagate a cluster status very fast. 
 
Each node has it’s own vector clock and a status. 
 
If one wants to spread data it selects one node and it sends the data via gossip with it’s 
vector clock. 
 
The node recieves the data and in the next iteration every node that has this data selects 
another node to spread the data 
 
The data spreads in the cluster in O(lg n) 

7) Describe the use of memory caches (e.g., Memcached system) for distributed NoSQL 
systems. 
 
MEMCACHED: 
light-weight key/value interface where all key-valuetuples are stored and served from 
DRAM 
 
Interface -ADD/SET add a key/value object from the disk to the RAM 
- GET(key) retrieve the value 
- DELETE delete a key/value object 
 
uses HASH table to index key/value pairs  
key/values are stored in pages subdivided into chunks 
 
-> replicate most frequent request in the main memory 
 

38
8) Explain the database cluster architecture of distributed database systems. 

Major difference with a parallel DBMS: “black-box” DBMS at each node

Fast network interconnect; Low disk and network latency essential
● Clustering uses multiple servers to work with a shared database system
Can have a shared-disk or shared-nothing (preferred) architecture
Parallel data management must be implemented via middleware (layer between DBMS and
Network)

High availability

9) Explain the sharding approach to data partitioning in distributed database systems. 

Partition the data in such way that the data typically requested and updated together resides
on the same node
-> horizonta partitioning of the whole data store not only tables!
each shard may have the same schema but different data

Sharding logic directs the application to the appropriate shard!!

(Physical organizzation of data)

Load and storage volume is evenly distributed among servers

Easy to scale - just add nodes and repartition

off the shelf hardware
reduce data contention and improve performance by balancing workload
Shards can be located physically close to the user

WHY sharding??

Lookup sharding:
we have a logic that maps virtual shards to physical partitions

Range sharding:
Grouping data by a value in the same node ! -> like group by month
good for range query
problems one month produces more data than others

good for data location close to users!

Hash sharding:
Partition data by a hash function
good for load balancing
bad for range query

39
10) Describe the consistent hashing approach to partitioning in a distributed database 
system.

Consistenc hashing:

we have n nodes and a hash function that maps h(x) in {0,1,....,n,....}

we want to cache web pages with URL x at the cache server h(x) mod n

Save values in a server as a bucket!

The bucket is defined on the left of a server!
Where to save x? h(x)=x2 then move to the right and find s0 -> save the value on s0

How to search?
h(x)=x0 go to position x0 and move to right find s2 and search for the value in s2.

What if we want to add a server? h(s)=value and move only objects on the left of value to the
new server

How to split data more evenly between nodes? -> make everything virtual! -> virtual copies
of server in the hash range that maps data to the physical server
Split each node in more virtual values well separated in hash range

40