21cs1401-Unit 5 - Dbms

21CS1401 – Database Management Systems Unit –V
UNIT V IMPLEMENTATION TECHNIQUES AND NON-RELATIONAL MODEL

Data on External Storage-RAID – File Organizations – Indexing and Hashing -Trees – B+ tree and B tree
Index Files.Hashing: Static Hashing – Dynamic Hashing – Query Processing and Query Optimization –
Introduction to NoSQL & MongoDB: Advantages, Architecture, Data Models, MongoDB Data types and
CRUD Operations.
5.1 Data On External Storage
A database system provides an ultimate view of the stored data. However, data in the form of bits, bytes get
stored in different storage devices.
Types of Data Storage
For storing the data, there are different types of storage options available. These storage types differ from one
another as per the speed and accessibility. There are the following types of storage devices used for storing
the data:
○ Primary Storage
○ Secondary Storage
○ Tertiary Storage
Primary Storage
It is the primary area that offers quick access to the stored data. We also know the primary storage as volatile
storage. It is because this type of memory does not permanently store the data. As soon as the system leads to
a power cut or a crash, the data also get lost. Main memory and cache are the types of primary storage.
○ Main Memory: It is the one that is responsible for operating the data that is available by the storage
medium. The main memory handles each instruction of a computer machine. This type of memory
can store gigabytes of data on a system but is small enough to carry the entire database. At last, the
main memory loses the whole content if the system shuts down because of power failure or other
reasons.
1
1. Cache: It is one of the costly storage media. On the other hand, it is the fastest one. A cache is a tiny
storage media which is maintained by the computer hardware usually. While designing the
algorithms and query processors for the data structures, the designers keep concern on the cache
effects.
Secondary Storage
Secondary storage is also called as Online storage. It is the storage area that allows the user to save and store
data permanently. This type of memory does not lose the data due to any power failure or system crash.
That's why we also call it non-volatile storage.
There are some commonly described secondary storage media which are available in almost every type of
computer system:
○ Flash Memory: A flash memory stores data in USB (Universal Serial Bus) keys which are further
plugged into the USB slots of a computer system. These USB keys help transfer data to a computer
system, but it varies in size limits. Unlike the main memory, it is possible to get back the stored
data which may be lost due to a power cut or other reasons. This type of memory storage is most
commonly used in the server systems for caching the frequently used data. This leads the systems
towards high performance and is capable of storing large amounts of databases than the main
memory.
○ Magnetic Disk Storage: This type of storage media is also known as online storage media. A
magnetic disk is used for storing the data for a long time. It is capable of storing an entire database. It
is the responsibility of the computer system to make availability of the data from a disk to the main
memory for further accessing. Also, if the system performs any operation over the data, the modified
data should be written back to the disk. The tremendous capability of a magnetic disk is that it does
not affect the data due to a system crash or failure, but a disk failure can easily ruin as well as
destroy the stored data.
Tertiary Storage
2
It is the storage type that is external from the computer system. It has the slowest speed. But it is
capable of storing a large amount of data. It is also known as Offline storage. Tertiary storage is
generally used for data backup. There are following tertiary storage devices available:
○ Optical Storage: An optical storage can store megabytes or gigabytes of data. A Compact Disk
(CD) can store 700 megabytes of data with a playtime of around 80 minutes. On the other hand, a
Digital Video Disk or a DVD can store 4.7 or 8.5 gigabytes of data on each side of the disk.
○ Tape Storage: It is the cheapest storage medium than disks. Generally, tapes are used for archiving
or backing up the data. It provides slow access to data as it accesses data sequentially from the start.
Thus, tape storage is also known as sequential-access storage. Disk storage is known as direct-access
storage as we can directly access the data from any location on disk.
5.2 RAID- Redundant Arrays Of Independent or Inexpensive Disks

RAID provides disk organization techniques that manage a large numbers of disks, providing a view
of a single disk, which aims
● high reliability by storing data redundantly, so that data can be recovered even if a disk fails and
● high capacity and high speed by using multiple disks in parallel.
Improvement of reliability via Redundancy:
Let us first consider reliability. The chance that some disk out of a set of N disks will fail is much
higher than the chance that a specific single disk will fail. Suppose that the mean time to failure of a disk is
100,000 hours, or slightly over 11 years. Then, the mean time to failure (MTTF) of some disk in an
array of 100 disks will be 100,000 / 100 = 1000 hours, or around 42 days, which is not long at all.
If only one copy of the data is stored, then each disk failure will result in loss of a significant amount
of data. Such a high rate of data loss is unacceptable. The solution to the problem of reliability is to
introduce redundancy; that is, to store extra information that can be used in the event of failure of a disk to
rebuild the lost information.
The simplest (but most expensive) approach to introducing redundancy is to duplicate every disk.
This technique is called mirroring (or, sometimes shadowing). A logical disk then consists of two
physical disks, and every write is carried out on both disks. If one of the disks fails, the data can be read
from the other. Data will be lost only if the second disk fails before the first failed disk is repaired. The mean
time to failure (where failure is the loss of data) of a mirrored disk depends on the mean time to failure of the
individual disks, as well as on the mean time to repair, which is the time it takes (on an average) to replace a
3
failed disk and to restore the data on it. Power failures, and natural disasters such as earthquakes, fires may
result in damage to both disks at the same time.
Improvement in Performance via Parallelism
Consider the benefit of paral1el access to multiple disks. With disk mirroring, the rate at which read
requests can be handled is doubled, since read requests can be sent to either disk. The transfer rate of each
read is the same as in a single-disk system but the number of reads per unit time has doubled.
With multiple disks, the transfer rate can be improved as well by striping data across multiple disks.
In its simplest form, data striping consists of splitting the bits of each byte across multiple disks. Such
striping is called bit-level striping. For example, if there is an array of eight disks, then bit i of each byte is
written to disk i. The array of eight disks can be treated as a single disk with sectors that are eight times the
normal size, and, more important, that has eight times the transfer rate.
In such an organization, every disk participates in every access (read or write), so number of accesses
that can be processed per second is about the same as on a single disk, but each access can read eight times as
many data in the same time as on a single disk.
Block-level striping stripes blocks across multiple disks. It treats the array of disks as a single large
disk, and it gives blocks logical numbers. It is assumed that the block numbers start from 0.
With an array of n disks, block-level striping assigns logical block i to the (i/n) th physical block
of the disk (i mod n) + 1. For example, with 8 disks, logical block 0 is stored in physical, block 0 of disk 1,
while logical block 11 is stored in physical block 1 of disk 4.
While reading a large file, block-level striping fetches n blocks at a time in parallel from n disks,
giving a high data transfer rate for large reads. When a single block is read the data transfer rate is the same
as on one disk, but the remaining n - 1 disks is free to perform other actions.
RAID Levels
Mirroring provides high reliability, but it is expensive. Striping provides high data transfer
rates, but does not improve reliability. Various alternative schemes aim to provide redundancy at lower cost
by combining disk striping with "parity" bits. These schemes have different cost-performance trade-offs. The
schemes are classified into RAID levels. RAID level 0 refers to disk arrays with striping at the level of
blocks, but without any redundancy. The below Figure: 5.3(a) shows an array of size 4.
4
Figure: 5.3 – (a) Raid Level 0 and (b) Raid Level 1

RAID level l refers to disk mirroring with block striping. The above Figure: 5.3(b) shows a mirrored
organization that holds four disks worth of data.
RAID level 2 known as memory-style error-correcting-code (ECC) organization, employs parity bits.
Memory systems have long used parity bits for error detection and correction.
Usually each byte in a memory system may have a parity bit associated with it that records whether
the number of bits in the byte that are set to 1 is even (parity = 0) or odd (parity = 1). If one of the bits in the
byte gets damaged (either a 1 becomes a 0, or a 0 becomes a 1), the parity of the byte changes and thus will
not match the stored parity. Similarly, if the stored parity bit gets damaged, it will not match the computed
parity. Thus, all 1-bit errors will be detected by the memory system. Error-correcting schemes store 2 or
more extra bits, and can reconstruct the data if a single bit gets damaged.
The idea of error-correcting codes can be used directly in disk arrays by striping bytes across disks.
For example, the first bit of each byte could be stored in disk 1, the second bit in disk 2, and so on until the
eighth bit is stored in disk 8, and the error-correction bits are stored in further disks.
The below Figure: 5.4(c) shows the level 2 scheme. The disks labeled P store the error correction
bits. If one of the disks fails, the remaining bits of the byte and the associated error-correction bits can be
read from other disks, and can be used to reconstruct the damaged data. The RAID level 2 requires only
three disks overhead for four disks of data, unlike RAID level 1, which required four disks overhead.
5
Figure: 5.4 – (c) Raid Level 2 and (d) Raid Level 3
RAID level 3, bit-interleaved parity organization, improves on level 2 by exploiting the fact that disk
controllers, can detect whether a sector has been read correctly. So, single parity bit can be used for error
correction, as well as for detection. The idea is as follows. If one of the sectors gets damaged, the disk
controller knows exactly which sector it is, and, for, each bit in the sector, the system can figure out whether
it is a 1 or a 0 by computing the parity of the corresponding bits from sectors in the other disks. If the
parity of the remaining bits is equal to the stored parity, the missing bit is 0. Otherwise, it is 1. The Figure:
5.4(d) shows the RAID level 3.
RAID level 3 is as good as level 2, but is less expensive in the number of extra disks (it has only a
one-disk overhead), so level 2 is not used in practice.
RAID level 3 has two benefits over level 1. It needs only one parity disk for several regular disks,
whereas Level l needs one mirror disk for every disk, and thus reduces the storage overhead.
RAID level 4, block-interleaved parity organization, uses block level striping, like RAID 0, and in
addition keeps a parity block on a separate disk for corresponding blocks from N other disks. This scheme
is shown pictorially in the Figure: 5.5(e). If one of the disks fails, the parity block can be used with the
corresponding blocks from the other disks to restore .the blocks of the failed disk.
A block read accesses only one disk, allowing other requests to be processed by the other disks. Thus,
the data-transfer rate for each access is slower, but multiple read accesses can proceed in parallel, leading to a
higher overall I/O rate. The transfer rates for large reads is high, since all the disks can be read in parallel;
large writes also have high transfer rates, since the data and parity can be written in parallel.
Small independent writes, on the other hand, cannot be performed in parallel. A write of a block has
to access the disk on which the block is stored, as well as the parity disk, since the parity block has to be
updated. Moreover, both the old value of the parity block and the old value of the block being written have to
be read for the new parity to be computed. Thus, a single write requires four disk accesses: two to read the
two old blocks, and two to write the two blocks.
6
Figure: 5.5(e) - Raid Level 4

RAID level 5, block-interleaved distributed parity, improves on level 4 by partitioning data and parity
among all N + 1 disks, instead of storing data in N disks and parity in one disk. In level 5, all disks can
participate in satisfying read requests, unlike RAID level 4, where the parity disk cannot participate, so level
5 increases the total number of requests that can be met in a given amount of time. For each set of N logical
blocks, one of the disks stores the parity, and the other N disks store the blocks. The Figure: 5.6(f) shows the
setup. The P's are distributed across all the disks. The following table indicates how the first 20 blocks,
numbered 0 to 19, and their parity blocks are laid out. The pattern shown gets repeated on further blocks.
Figure: 5.6(f) - Raid Level 5
Figure: 5.7 – Distribution of Parity Block PK

RAID level 6, the P + Q redundancy scheme is much like RAID level 5. It stores extra redundant
information to guard against multiple disk fail Instead of using parity; level 6 uses error-correcting codes
such as the Reed Solomon codes. The scheme is shown in the Figure: 5.8(g) bits of redundant data are stored
for every 4 bits of data-unlike 1 parity in level 5 and the system can tolerate two disk failures.
7
Figure: 5.8(g) - Raid Level 6

Choice of RAID level
● The factors to be taken into account when choosing a RAID level are
● Monetary cost of extra disk storage requirements
● Performance requirements in terms of number of I/O operations
● Performance when a disk has failed
● Performance during rebuild (that is, while the data in a failed disk is being rebuilt on a new disk)
5.3 File Organization:

A file is organized as a sequence of records. These records are mapped onto disk blocks. The record
organization methods are of the following
Fixed Length Records:
Consider a file of account records for the bank database. Each record of the file is defined as :
type deposit = record
account-number: char(10);
branch-name: char(22);
balance:real;
end
If it is assumed that each character occupies 1 byte and that a real occupies 8 bytes, then account
record is 40 bytes long. A simple approach is to use the first 40 bytes for the first record, the next 40 bytes for
the second record and so on. The Figure: 5.9 show how
fixed length records are stored in the file.
Figure: 5.9 – Fixed Length Records

However, there are two problems with this simple approach:
8
⮚ It is difficult to delete a record from this structure. The space occupied by the record to be
deleted must be filled with some other record of the file, or we must have a way of marking
deleted records so that they can be ignored.
⮚ If the block size happens to be a greater than 40 bytes, some records will cross block
boundaries. That is part of the record will be stored in one block and part in another. It would
thus require two block accesses to read or write such a record.
The deletion can be performed in several ways:
The first approach is when a record is deleted all the records after it should be moved up to the
deleted position. Instead of this approach, it might be easier simply to move the final record of the file into
the space occupied by the deleted record. Another approach is to reuse the space of the deleted record
by inserting new record in that place. This approach avoids the movement of records. Since it is hard find the
available space, it is desirable to use some additional structure. At the beginning of the file, a certain number
of bytes are allocated as a file header. The header will contain a variety of information about the file. In
addition to all the information it maintains the address of the first record whose contents are deleted. And this
first record is used to store the address of the second available record and so on. This stored address is called
as pointers, since they point to the location of a record. The deleted records thus form a linked list, which is
often referred to as a free list. The Figure: 5.12 Shows the file with the free list after records 1, 4, and 6 have
been deleted.
Figure: 5.12 - File of Figure: 5.9, with free list after deletion of records 1,4 and 6.
On insertion of a new record, we can use the record pointed by the header. The header pointer is
change to point to the next available record after insertion. If no space is available the insertion is done at the
end of the file.
Variable Length Records:
Variable-length records arise in database systems in several ways:
9
o Storage of multiple record types in a file.

o Record types that allow variable lengths for one or more fields.
o Record types that allow repeating fields (used in some older data models).
Consider a different representation of the account information, in which one variable length record is
used for each branch name and for all account information in that branch. The format of the record is:
type account-list = record
branch-name: char (22);
account-info: array [1 .. ∞] of
record;
account-number: char(l0);
balance: real;
end
end
The account-info is defined as an array with an arbitrary number of elements. That is, the type
definition does not limit the number of elements in the array, although any actual record will have a specific
number of elements in its array. There is no limit on how large a record can be (up to, of course, the size of
the disk storage).
Byte String Representation:
A simple method for implementing variable-length records is to attach a special end-of-.record (⊥)
symbol to the end of each record. Then each record is stored as a string of consecutive bytes.
The Figure: 5.13 show, such an organization to represent the account file as variable-length records.
Figure: 5.13 - Byte-String Representation of Variable-Length Records

An alternative version of the byte-string representation stores the record length at the beginning of
each record, instead of using end-of-record symbols.
10
The byte-string representation as described has some disadvantages:

● It is not easy to reuse space occupied formerly by a deleted record.
● There is no space, in general, for records to grow longer. If a variable-length record becomes longer,
it must be moved-movement is costly if pointers to the record are stored elsewhere in the database
(e.g., in indices, or in other records), since the pointers must be located and updated.
Thus, the basic byte-string representation described here not usually used for implementing variable-length
records. However, a modified form of the byte-string representation, called the slotted-page structure, is
commonly used for organizing records within a single block. The slotted-page structure appears in the
Figure: 5.14
Figure: 5.14 - Slotted Page Structure
There is a header at the beginning of each block, containing the following information:
● The number of record entries in the header.
● The end of free space in the block
● An array whose entries contain the location and size of each record
The actual records are allocated contiguously in the block, starting from the end of the block. The free
space in the block is contiguous, between the final entry in the header array, and the first record . If a
record is inserted, space is allocated for it at the end of free space, and an entry containing its size and
location is added to the header.
If a record is deleted, the space that it occupies is freed, and its entry is set to deleted (its size is set to
-1, for example). Further, the records in the block before the deleted record are moved, so that the free space
created by the deletion gets occupied and all free space is again between the final entry in the header array
and the first record. The end-of-free-space pointer in the header is appropriately updated as well. Records can
be grown or shrunk by similar techniques, as long as there is space in the block.
11
Fixed-length Representation
Another way to implement variable-length records efficiently in a file system is to use one or more
fixed-length records to represent one variable-length record.
There are two ways of doing this:
● Reserved space: If there is a maximum record length that is never exceeded, then fixed-length
records of that length is used. Unused space (for records shorter than the maximum space) is filled
with a special null, or end-of-record, symbol.
● List representation: variable-length records can be represented by lists of fixed length records,
chained together by pointers.
The Figure: 5.15 shows, how the file of account would be represented if, maximum of three accounts
per branch are allowed.
Figure: 5.15 - Fixed-Length Representation

A record in this file is of the account-list type, but with the array containing exactly three elements.
Those branches with fewer than three accounts (for example, Round Hill) have records with null fields. The
symbol (⊥) is used to represent this situation in Figure. The reserved-space method is useful when most of
the records have a length close to the maximum. Otherwise, a significant amount of space may be wasted.
In the bank example, some branches may have many more accounts than others. This situation leads
to consider the linked list method. To represent the file by the linked list method, a pointer field should be
added. The resulting structure appears in the Figure: 5.16.
12
Figure: 5.16 - Pointer Method

A disadvantage to the structure of the above Figure: 5.16 is that space is wasted in all records except
the first in a chain. The first record needs to have the branch-name value, but subsequent records do not. But,
it is needed to include a field for branch-name in all records. This wasted space is significant.
To deal with this problem, two kinds of blocks are allowed in a file:
● Anchor block, which contains the first record of a chain
● Overflow block, which contains records other than those that are the first record of a chain
Thus, all records within a block have the same length, even though not all records in the file have the
same length. The Figure: 5.17 show this file structure.
Figure: 5.17 - Pointer Method-Using Anchor block Overflow block
Organization of Records in Files:

An instance of a relation is a set of records. Given a set of records, the next question is how to
organize them in a file. Several of the possible ways of organizing records in files are:
● Heap file organization (heap files): In this simplest and most basic type of organization, records are
placed in the file in the order in which they are inserted, so new records are inserted at the end of the
file. Such an organization is called a heap or pile file. Sequential file organization (sorted files):
Records are stored in sequential order according to the value of a "search key" of each record.
● Hashing file organization: A hash function is computed on some attribute of each record. The result
of the hash function specifies in which block of the file the record should be placed.
● Clustering file organization: Generally, a separate file is used to store the records of each relation.
However, in a clustering file organization, records of several different relations are stored in the same
file; further, related records of the different relations are stored on the same block, so that one I/O
13
operation fetches related records from all the relations. For example, records of the two relations can
be considered to be related if they would match in a join of the two relations.
Heap file organization (heap files or unordered files):
In this simplest and most basic type of organization, records are placed in the file in the order in
which they are inserted, so new records are inserted at the end of the file. Such an organization is called a
heap or pile file. This organization is often used with additional access paths, such as the secondary indexes.
It is also used to collect and store records for future use.
Inserting a new record is efficient. The last disk block of the file is copied into a buffer. The new
record is added and then the block is then rewritten back to the disk. The address of the last file block is
kept in the file header. However, searching for a record using any search condition involves a linear
search through the file by block, which is an expensive procedure. If only one record satisfies the search
condition, then, on the average, a program will read into memory and search half the file blocks before it
finds the record. For a file b blocks requires searching (b/2) blocks on average. If no records satisfy the
search condition, the program must read and search all b blocks in the file.
To delete a record, a program must first find its block, copy the block into the buffer, and finally
rewrite the block back to the disk. This leaves unused space in the disk block. Deleting, a large number of
records in this way results in wasted storage space. Another technique used for record deletion is to have
an extra byte or bit, called a deletion marker, stored with each record. A record is deleted by setting the
deletion marker to a certain value. A different value of the marker indicates a valid (not deleted) record.
Search programs consider only valid records in a block when conducting their search. Both of these
deletion techniques require periodic reorganization of the file to reclaim the unused space of deleted
records. During reorganization, the file blocks are accessed consecutively, and records are packed by
removing deleted records. After such reorganization, the blocks are filled to capacity once more. Another
possibility is to use the space when inserting records although this requires extra bookkeeping to keep
track of empty locations.
Sequential File Organization (sorted files or ordered files):
A sequential file organization is designed for efficient processing of records in sorted order based on
some search key. A search key is any attribute or set of attributes. It need not be the primary key, or even a
super key. To permit fast retrieval of records in search-key order, the records are chained together by
pointers. The pointer in each record points to the next record in search-key order. Furthermore, to minimize
14
the number of block accesses in sequential file processing, the records are stored physically in search-key
order, or as close to search-key order as possible.
Figure: 15.18 show a sequential file of account records taken from the banking example. In that
example, the records are stored in search-key order, using branch-name as the search key.
Figure: 5.18 - Sequential File for account Records

The sequential file organization allows records to be read in sorted order; that can be useful for
display purposes, as well as for certain query-processing algorithms. It is difficult, however, to maintain
physical sequential order as records are inserted and deleted, since it is costly to move many records as a
result of a single insertion or deletion or deletion. Deletion can be managed by using pointer chains. For
insertion, the following rules are applied:
● Locate the record in the file that comes before the record to be inserted in search-key order.
● If there is a free record (that is, space left after a deletion) within the same block as this record, insert
the new record there. Otherwise, insert the new record in an overflow block. In either case, adjust the
pointers so as to chain together the records in search-key order.
15
The Figure: 5.19 show the file of account, after the insertion of the record (North Town, A-888, 800).
The structure in the figure allows fast insertion of new records, but forces sequential file-processing
applications to process records in an order that does not match the physical order of the records. If relatively
few records need to be stored in overflow blocks, this approach works well. Eventually, however, the
correspondence between search-key order and physical order may be totally lost, in which case sequential
processing will become much less efficient. At this point, the file should be reorganized so that it is once
again physically in sequential order. Such reorganizations are costly; and must be done during times when
the system load is low. The frequency with which reorganizations are needed depends on the frequency of
insertion of new records. In the extreme case in which insertions rarely occur, it is possible always to keep
the file in physically sorted order.
Figure: 5.19 - Sequential File Organization after an

Insertion Clustering File Organization:
Many relational-database systems store each relation in a separate file, so that they can take full
advantage of the file system that the operating system provides. This simple approach to relational-database
implementation becomes less satisfactory as the size of the database increases. There are performance
advantages to be gained from careful assignment of records to blocks, and from careful organization of the
blocks themselves. The database system stores all relations in this one file, and manages the file itself. To see
the advantage of storing many relations in one file, consider the following SQL query for the bank database:
select account-number, customer-name, customer-street, customer-city from depositor,
customer where depositor. customer_name = customer.customer_name
This query computes a join of the depositor and customer relations. Thus, for each tuple of depositor,
the system must locate the customer tuples with the same value for customer-name. Regardless of how these
records are located, however, they need to be transferred from disk into main memory. In the worst case,
16
each record will reside on a different block, forcing us to do one block read for each record required by the
query. As an example,
Figure: 5.20 - Depositor Relation
Figure: 5.21 - Customer Relation

The Figure: 5.22 show a file structure designed for efficient execution of queries involving
depositor customer.
Figure: 5.22 - Multiple clustering file structure

The depositor tuples for each customer-name are stored near the customer tuple for the corresponding
customer name. This structure mixes together tuples of two relations, but allows for efficient processing of
the join. When a tuple of the customer relation is read, the entire block containing that tuple is copied from
disk into main memory. Since the corresponding depositor tuples are stored on the disk near the customer
tuple, it is also copied. If a customer has so many accounts that the depositor records do not fit in one block,
the remaining records appear on nearby blocks.
17
A clustering file organization is a file organization, such as that illustrated in the Figure: 5.22 that
stores related records of two or more relations in each block. Such a file organization allows us to read
records that would satisfy the join condition by using one block read.
The use of clustering has enhanced processing of a particular join (depositor customer), but it
results in slowing processing of other types of query. For example,
Select * from customer
requires more block accesses than it did in the scheme under which each relation is stored in a separate file.
Instead of several customer records appearing in one block each record is located in a distinct block.
Indeed, simply finding all the customer records is not possible without some additional structure. To locate
all tuples of customer relation in the structure of the Figure: 5.22, it is needed to chain together all records
of that relation using pointers, as in the Figure: 5.23. The usage of clustering depends on the types of query
that the database designer believes to be most frequent. Careful use of clustering can produce significant
performance gains in query
processing.
Figure: 5.23 – Multiple Clustering File Structure with Pointer Chains

5.4 Indexing
Database system indices play the same role as book indices or card catalogs in the libraries. For
example, to retrieve an account record given the account number, the database system would look up an
index to find on which disk block the corresponding record resides, and then fetch the disk block, to get the
account record.
There are two basic kinds of indices:
● Ordered indices: Based on a sorted ordering of the values.
● Hash indices: Based on a uniform distribution of values across a range of buckets. The bucket to
which a value is assigned is determined by a function called a hash function.
18
Several techniques exist for both ordered indexing and hashing. No one technique is the best. Rather,
each technique is best suited to particular database applications.
Ordered Indices
To gain fast random access to records in a file, an index structure is used. Each index structure is
associated with a particular search key. Just like the index of a book or a library catalog an ordered index
stores the values of the search keys in sorted order, and associates with each search key the records that
contain it.
Ordered indices can be categorized as primary index and secondary index.
Primary Index
In this index, it is assumed that all files are ordered sequentially on some search key. Such files, with
a primary index on the search key, are called index-sequential files. They represent one of the oldest index
schemes used in database systems. They are designed for applications that require both sequential processing
of the entire file and random access to individual records.
The Figure: 5.24 show a sequential file of account records taken from the banking example. In the
example figure, the records are
stored in search- key order, with
branch-name used as the search
key.
Figure: 5.24 – Sequential file for account records.

Dense and Sparse Indices
An index record, or index entry, consists of a search-key value, and pointers to one or more records
with that value as their search-key value. The pointer to a record consists of the identifier of a disk block and
an offset within the disk block to identify the record within the block.
There are two types of ordered indices that can be used:
Dense index: an index record appears for every search-key value in the file. In a dense primary index, the
index record contains the search-key value and a pointer to the first data record with that search-key value.
19
The rest of the records with the same search key-value would be stored sequentially after the first record,
since, because the index is a primary one, records are sorted on the same search key.
Dense index implementations may store a list of pointers to all records with the same search-key
value; doing so is not essential for primary indices. The Figure: 5.25 show the dense index for the account
file.
Figure: 5.25 - Dense Index.

Sparse index: An index record appears for only some of the search-key values. To locate a record we
find the index entry with the largest search-key value that is less than or equal to the search key value for
which we are looking. We start at the record pointed to by that index entry, and follow the pointers in the file
until we find the desired record. The Figure: 5.26 show the sparse index for the account file.
Figure: 5.26 - Sparse Index.

Suppose that we are looking up records for the Perryridge branch. Using the dense index we follow
the pointer directly to the first Perryridge record. We process this record and follow the pointer in that record
to locate the next record in search-key (branch-name) order. We continue processing records until we
encounter a record for a branch other than Perryridge. If we are using the sparse index, we do not find an
index entry for "Perryridge". Since, the last entry (in alphabetic order) before "Perryridge" is "Mianus" we
20
follow that pointer. We then read the account file in sequential order until we find the first Perryridge record,
and begin processing at that point.
Thus, it is generally faster to locate a record in a dense index; rather than a sparse index. However,
sparse indices have advantages over dense indices in that they require less space and they impose less
maintenance overhead for insertions and deletions.
Multi level indices
Even if the sparse index is used, the index itself may become too large for efficient processing. It is
not unreasonable, in practice, to have a file with 100,000 records, with 10 records stored in each block. If we
have one index record per block, the index has 10,000 records. Index records are smaller than data records,
so let us assume that 100 index records fit on a block. Thus, our index occupies 100 blocks. Such large
indices are stored as sequential files on disk.
If an index is sufficiently small to be kept in main memory, the search time to find an entry is low.
However, if the index is so large that it must be kept on disk, a search for an entry requires several disk block
reads. Binary search can be used on the index file to locate an entry, but the search still has a large cost. If
overflow blocks have been used, binary search will not be possible. In that case, a sequential search is
typically used, and that requires b block reads, which will take even longer. Thus, the process of searching a
large index may be costly.
To deal with this problem, we treat the index just as we would treat any other sequential file, and
construct a sparse index on the primary index, as in the Figure: 5.27
To locate a record, we first use binary search on the outer index to find the record for the largest
search-key value less than or equal to the one that we desire. The pointer points to a block of the inner index.
We scan this block until we find the record that has the largest search-key value less than or equal to the one
that we desire. The pointer in this record points to the block of the file that contains the record for which we
are looking
.
21
Figure: 5.27 – Two-level Sparse Index

Using the two levels of indexing, we have read only one index block, rather than the seven we read
with binary search, if we assume that the outer index is already in main memory. If our file is extremely
large, even the outer index may grow too large to fit in main memory. In such a case, we can create yet
another level of index. Indices with two or more levels are called multilevel indices. Searching for records
with a multilevel index requires significantly fewer I/O operations than does searching for records by binary
search.
A typical dictionary is an example of a multilevel index in the non database world. The header of
each page lists the first word alphabetically on that page. Such a book index is a multilevel index: The words
at the top of each page of the book index form a sparse index on the contents of the dictionary pages.
Secondary Indices:
Secondary indices must be dense, with an index entry for every search-key value, and, a pointer to
every record in the file. A primary index may be sparse, storing only some of the search-key values, since it
is always possible to find records with intermediate, search-key values by a sequential access to a part of the
file. If a secondary index stores only some of the search-key values, records with intermediate search-key
values may be anywhere in the file and, in general, we cannot find them without searching the entire file.
A secondary index on a candidate key looks just like a dense primary index, except that the records
pointed to by successive values in the index are not stored sequentially. In general, however, secondary
indices may have a different structure from primary indices. If the search key of a primary index is not a
candidate key, it suffices if the index points to the first record with a particular value for the search key, since
the other records can be fetched by a sequential scan of the file.
In contrast, if the search key of a secondary index is not a candidate key, it is not enough to point to
just the first record with each search-key value. The remaining records with the same search-key value could
be anywhere in the file, since records are ordered by the search key of the primary index, rather than by the
search key of the secondary index. Therefore, a secondary index must contain pointers to all the records.
We can use an extra level of indirection to implement secondary indices on search keys that are not
candidate keys. The pointers in such a secondary index do not point directly to the file. Instead, each points
to a bucket that contains pointers to the file.
22
The Figure: 5.28 shows the structure of a secondary index that uses an extra level of indirection on the
account file, on the search key balance.
Figure: 5.28 - Secondary index on account file on noncandidate key balance.

A sequential scan in primary index order is efficient because records in the file are stored physically
in the same order as the index order. However, we cannot (except in rare special cases) store a file physically
ordered both by the search key of the primary index, and the search key of a secondary index. Because
secondary-key order and physical-key order differ, if we attempt to scan the file sequentially in secondary-
key order, the reading of each record is likely to require the reading of a new block from disk, which is very
slow. The procedure described earlier for deletion and insertion can also be applied to secondary indices.
Hashing:
For a huge database structure, it can be almost next to impossible to search all the index values through all its
level and then reach the destination data block to retrieve the desired data. Hashing is an effective technique
to calculate the direct location of a data record on the disk without using index structure.
Hashing uses hash functions with search keys as parameters to generate the address of a data record.
23
Hash Organization
Bucket − A hash file stores data in bucket format. Bucket is considered a unit of storage.
A bucket typically stores one complete disk block, which in turn can store one or more
records.
Hash Function − A hash function, h, is a mapping function that maps all the set of search-keys K to
the address where actual records are placed. It is a function from search keys to bucket addresses.
Static Hashing
In static hashing, when a search-key value is provided, the hash function always computes the same address.
For example, if mod-4 hash function is used, then it shall generate only 5 values. The output address shall
always be same for that function. The number of buckets provided remains unchanged at all times.
Operation
24
Insertion − When a record is required to be entered using static hash, the hash function h computes the bucket
address for search key K, where the record will be stored.
Bucket address = h(K)
Search − When a record needs to be retrieved, the same hash function can be used to retrieve the address of
the bucket where the data is stored.
Delete − This is simply a search followed by a deletion operation.
Bucket Overflow
The condition of bucket-overflow is known as collision. This is a fatal state for any static hash function. In
this case, overflow chaining can be used.
1. Overflow Chaining − When buckets are full, a new bucket is allocated for the same
hash result and is linked after the previous one. This mechanism is called Closed
Hashing.
2. Linear Probing − When a hash function generates an address at which data is already
stored, the next free bucket is allocated to it. This mechanism is called Open Hashing.
25
Dynamic Hashing
The problem with static hashing is that it does not expand or shrink dynamically as the size of the database
grows or shrinks. Dynamic hashing provides a mechanism in which data buckets are added and removed
dynamically and on-demand. Dynamic hashing is also known as extended hashing.
Hash function, in dynamic hashing, is made to produce a large number of values and only a few are used
initially.
26
Organization
The prefix of an entire hash value is taken as a hash index. Only a portion of the hash value is used for
computing bucket addresses. Every hash index has a depth value to signify how many bits are used for
computing a hash function. These bits can address 2n buckets. When all these bits are consumed − that is,
when all the buckets are full − then the depth value is increased linearly and twice the buckets are allocated.
Operation
Querying − Look at the depth value of the hash index and use those bits to compute the
bucket address.
Update − Perform a query as above and update the data.
Deletion − Perform a query to locate the desired data and delete the same.
Insertion − Compute the address of the bucket
27
If the bucket is already full.

Add more buckets.
Add additional bits to the hash
value. Re-compute the hash
function.
Else
Add data to the bucket,
If all the buckets are full, perform the remedies of static hashing.
Hashing is not favorable when the data is organized in some ordering and the queries require a range of data.
When data is discrete and random, hash performs the best.Hashing algorithms have high complexity than
indexing. All hash operations are done in constant time.
5.5 B+ tree Index Files:
B+-tree indices are an alternative to indexed-sequential files.

● Disadvantage of indexed-sequential files
o performance degrades as file grows, since many overflow blocks get created.
o Periodic reorganization of entire file is required.
● Advantage of B+-tree index files:
o Automatically reorganizes itself with small, local, changes, in the face of insertions and deletions.
o Reorganization of entire file is not required to maintain performance.
● (Minor) disadvantage of B+-trees:
o Extra insertion and deletion overhead, space overhead.
● Advantages of B+-trees outweigh disadvantages
o B+-trees are used extensively
A B+-tree is a rooted tree satisfying the following properties:
● All paths from root to leaf are of the same length ,Each node that is not a root or a leaf has between
⎡ n/2⎤ and n children.
28
● A leaf node has between ⎡ (n–1)/2⎤ and n–1 values
29
Special cases:
⮚ If the root is not a leaf, it has at least 2 children.
⮚ If the root is a leaf (that is, there are no other nodes in the tree), it can have between 0 and (n–1)
values.
B+-Tree Node Structure
● Typical node
Figure: 5.38 - B+-Tree Node Structure

o Ki are the search-key values
o Pi are pointers to children (for non-leaf nodes) or pointers to records or buckets of records (for
leaf nodes).
● The search-keys in a node are ordered K1 < K2 < K3 < . . . < Kn–1
Leaf Nodes in B+-Trees
Properties of a leaf node:
● For i = 1, 2, . . ., n–1, pointer Pi either points to a file record with search-key value Ki, or to a bucket of
pointers to file records, each record having search-key value Ki. Only need bucket structure if search-key
does not form a primary key.
● If Li, Lj are leaf nodes and i < j, Li’s search-key values are less than Lj’s search-key values ,Pn points to
next leaf node in search-key order
Figure: 5.39 – A Leaf Node for account B+-Tree index(n=3).
Non-Leaf Nodes in B+-Trees

● Non leaf nodes form a multi-level sparse index on the leaf nodes. For a non-leaf node with m
pointers:
o All the search-keys in the subtree to which P1 points are less than K1
o For 2 ≤ i ≤ n – 1, all the search-keys in the subtree to which Pi points have values greater than
or equal to Ki–1 and less than Ki
30
o All the search-keys in the

subtree to which Pn points have
values greater than or equal to Kn–1
Figure: 5.40 - Non leaf node

Example of a B+-tree
Figure: 5.41 - B+-tree for account file (n = 3)
Figure: 5.42 - B+-tree for account file (n = 5)

● Leaf nodes must have between 2 and 4
values (⎡ (n–1)/2⎤ and n –1, with n = 5).
● Non-leaf nodes other than root must have between 3 and 5 children (⎡ (n/2⎤ and n with n =5).
● Root must have at least 2 children.
Queries on B+ Trees:
Find all records with a search-key value of k.
1. Start with the root node
▪ Examine the node for the smallest search-key value > k.
▪ If such a value exists, assume it is Kj. Then follow Pi to the child node
▪ Otherwise k ≥ Km–1, where there are m pointers in the node. Then follow Pm to the
child node.
2. If the node reached by following the pointer above is not a leaf node, repeat the
above procedure on the node, and follow the corresponding pointer.
31
3. Eventually reach a leaf node. If for some i, key Ki = k follow pointer Pi to the desired record
or bucket. Else no record with search-key value k exists.
Updates on B+-Trees: Insertion

▪ Find the leaf node in which the search-key value would appear
▪ If the search-key value is already there in the leaf node, record is added to file and if necessary
a pointer is inserted into the bucket.
▪ If the search-key value is not there, then add the record to the main file and create a bucket
if necessary. Then:
⮚ If there is room in the leaf node, insert (key-value, pointer) pair in the leaf node
⮚ Otherwise, split the node
Splitting a node:
⮚ take the n(search-key value, pointer) pairs (including the one being inserted) in sorted order.
Place the first ⎡ n/2 ⎤ in the original node, and the rest in a new node.
⮚ let the new node be p, and let k be the least key value in p. Insert (k,p) in the parent of the
node being split. If the parent is full, split it and propagate the split further up.
The splitting of nodes proceeds upwards till a node that is not full is found. In the worst case the root
node may be split increasing the height of the tree by 1.
Figure 5.43 Result of splitting node containing Brighton and Downtown on inserting Clearview
Insertion Example:
32
Figure 5.44: B+-Tree before and after insertion of “Clearview”
Updates on B+-Trees: Deletion

1. Find the record to be deleted, and remove it from the main file and from the bucket (if present)
2. Remove (search-key value, pointer) from the leaf node if there is no bucket or if the bucket
has become empty
3. If the node has too few entries due to the removal, and the entries in the node and a sibling fit into
a single node, then
⮚ Insert all the search-key values in the two nodes into a single node (the one on the left), and
delete the other node.
⮚ Delete the pair (Ki–1, Pi), where Pi is the pointer to the deleted node, from its
parent, recursively using the above procedure.
4. Otherwise, if the node has too few entries due to the removal, and the entries in the node and a
sibling fit into a single node, then
⮚ Redistribute the pointers between the node and a sibling such that both have more than the
minimum number of entries.
⮚ Update the corresponding search-key value in the parent of the node.
The node deletions may cascade upwards till a node which has ⎡ n/2 ⎤ or more pointers is found. If the
root node has only one pointer after deletion, it is deleted and the sole child becomes the root.
33
Figure 5.55 before and after deleting “Downtown”

5.6 B-Tree Index Files:
● Similar to B+-tree, but B-tree allows search-key values to appear only once; eliminates redundant storage
of search keys.
● Search keys in nonleaf nodes appear nowhere else in the B-tree; an additional pointer field for each
search key in a nonleaf node must be included.
● Generalized B-tree leaf node(a) and Non leaf node(b) – pointers Bi is the bucket or file record pointers.
Figure: 5.6.1 - (a) B-tree leaf node ,(b) Non leaf node
B-Tree Index File Example
34
Figure: 5.6.2
– B-tree
● Advantages of B-Tree indices:
o May use less tree nodes than a corresponding B+-Tree.
o Sometimes possible to find search-key value before reaching leaf node.
● Disadvantages of B-Tree indices:
o Only small fraction of all search-key values are found early
o Non-leaf nodes are larger, so fan-out is reduced. Thus, B-Trees typically have greater depth than
corresponding B+-Tree
o Insertion and deletion more complicated than in B+-Trees
o Implementation is harder than B+-Trees.
● Typically, advantages of B-Trees do not out weigh disadvantages.
5.7 Query Processing & Optimization:

Query processing refers to the range of activities involved in extracting data from a database. The activities
include translation of quires in high-level database languages into expressions that can be used at the physical
level of the file system, a variety of query optimizing transformations, and actual evaluation of queries.
Steps in query Processing:
The steps involved in the query processing are

1. Parsing and translation
2. Optimization
3. Evaluation
These steps are illustrated in the Figure: 5.7
35
Figure: 5.7 – Steps in Query Processing.

Parsing and translation
⮚ This step translates the query into its internal form. The translation process is similar to the work
performed by the parser of a compiler. This is then translated into relational algebra expression.
⮚ Parser checks the syntax, and also verifies relations involved.
Evaluation:
⮚ Given the query there are generally a variety of methods for computing the answer.
⮚ The query-execution engine takes a query-evaluation plan, executes that plan, and returns the
answers to the query.
⮚ A relational algebra expression may have many equivalent expressions
o Example, balance2500(balance(account)) is equivalent to
balance(balance2500(account))
⮚ Each relational algebra operation can be evaluated using one of several different algorithms
o Correspondingly, a relational-algebra expression can be evaluated in many ways.
⮚ Annotated expression that specifies the detailed evaluation strategy of an expression is called an
Evaluation-plan.
o Example: can use an index on balance to find accounts with balance < 2500,
o or can perform complete relation scan and discard accounts with balance  2500
Query Optimization:
⮚ This chooses amongst all equivalent evaluation plans, the one with lowest cost.
o Cost is estimated using statistical information from the database catalog
▪ Example: number of tuples in each relation, size of tuples, etc.
Measures of Query Cost:
The cost of query evaluation can be measured in terms of a number of different resources, including
disk accesses, CPU time to execute a query, and, in a distributed or parallel database system, the cost of
communication. The response time for a query evaluation plan (that is, the clock time required to execute the
plan), assuming no other activity is going on the computer, would account for all these costs, and could be
used as a good measure of the cost of the plan.
36
In large database systems, however, disk accesses (which we measure as the number of transfers of
blocks from disk) are usually the most important cost, since disk accesses are slow compared to in-memory
operations. Moreover, CPU speeds have been improving much faster than have disk speeds. Thus, it is likely
that the time spent in disk activity will continue to dominate the total time to execute a query. Finally,
estimating the CPU time is relatively hard compared to estimating the disk-access cost. Therefore, most
people consider the disk-access cost a reasonable measure of the cost of a query evaluation plan.
We use the number of block transfers from disk as a measure of the actual cost. To simplify our
computation of disk-access cost, we assume that all transfers of blocks have the same cost. This assumption
ignores the variance arising from rotational latency (waiting for the desired data to spin under the read-write
head) and seek time (the time that it takes to move the head over the desired track or cylinder). To, get more
precise numbers, we need to distinguish between sequential I/O, where the blocks read are contiguous on
disk, and random I/O, where the blocks are noncontiguous, and an extra seek cost must be paid for each disk
I/O operation. We also need to distinguish between reads and writes of blocks, since it takes more time to
write a block to disk than to read a block from disk. A more accurate measure would therefore estimate
⮚ The number of seek operations performed
⮚ The number of blocks read
⮚ The number of blocks written
and then add up these numbers after multiplying them by the average seek time, average transfer time for
reading a block, and average transfer time for writing a block, respectively. Real-life query optimizers also
take CPU costs into account when computing the cost of an operation
The cost estimates given above ignore the cost of writing the final result of an operation back to disk.
These are taken into account separately where required. The costs of all the algorithms that we consider
depend on the size of the buffer in main memory. In the best case, all data can be read into the buffers, and
the disk does not need to be accessed again. In the worst case, we assume that the buffer can hold only a few
blocks of data-approximately one block per relation. When presenting cost estimates, we generally assume
the worst case.
Overview of Query Evaluation
❖ DBMS keeps descriptive data in system catalogs.
❖ SQL queries are translated into an extended form of relational algebra:
▪ Query Plan Reasoning:
• Tree of operators
37
• with choice of one among several algorithms for each operator

Query Plan Evaluation
▪ Query Plan Execution:
• Each operator typically implemented using a `pull’ interface
• when an operator is `pulled’ for next output tuples, it `pulls’ on its inputs and
computes them.
Query Plan Optimization :

▪ Ideally: Want to find best plan.
Practically: Avoid worst
plans!
Two main issues in query optimization:
▪ For a given query, what plans are considered?
• Algorithm to search plan space for cheapest (estimated) plan.
▪ How is the cost of a plan estimated?
• Cost models based on I/O estimates
System Catalogs
❖ For each index:
▪ structure (e.g., B+ tree) and search key fields
❖ For each relation:
▪ name, file name, file structure (e.g., Heap file)
▪ attribute name and type, for each attribute
▪ index name, for each index
▪ integrity constraints
❖ For each view:
▪ view name and definition
❖ Plus statistics, authorization, buffer pool size, etc.
❖ Need : Information about relations and indexes.
Catalogs typically contain at least:
▪ # tuples (NTuples)
# pages (NPages) for each relation.
38
▪ # distinct key values (NKeys) and NPages for each index.
39
▪ Index height, low/high key values (Low/High) for each tree index.
❖ Catalogs updated periodically.
▪ Updating whenever many data changes occurred;
▪ Lots of approximation anyway, so slight inconsistency ok.
How Catalogs are stored
• Attr_Cat(attr_name, rel_name, type, position)
• System Catalog is itself a collection of tables.
• Catalog tables describe all tables in
database, including catalog tables
themselves.
5.8 INTRODUCTION TO NOSQL:
NoSQL is a type of database management system (DBMS) that is designed to handle and store large
volumes of unstructured and semi-structured data. Unlike traditional relational databases that use
tables with pre-defined schemas to store data, NoSQL databases use flexible data models that can
adapt to changes in data structures and are capable of scaling horizontally to handle growing
amounts of data.
40
The term NoSQL originally referred to “non-SQL” or “non-relational” databases, but the term has
since evolved to mean “not only SQL,” as NoSQL databases have expanded to include a wide range
of different database architectures and data models.
NoSQL databases are generally classified into four main categories:
1. Document databases: These databases store data as semi-structured documents, such as

JSON or XML, and can be queried using document-oriented query languages.
2. Key-value stores: These databases store data as key-value pairs, and are optimized for
simple and fast read/write operations.
3. Column-family stores: These databases store data as column families, which are sets of
columns that are treated as a single entity. They are optimized for fast and efficient
querying of large amounts of data.
4. Graph databases: These databases store data as nodes and edges, and are designed to
handle complex relationships between data.
5. NoSQL databases are often used in applications where there is a high volume of data that
needs to be processed and analyzed in real-time, such as social media analytics, e-
commerce, and gaming. They can also be used for other applications, such as content
management systems, document management, and customer relationship management.
However, NoSQL databases may not be suitable for all applications, as they may not provide the
same level of data consistency and transactional guarantees as traditional relational databases. It is
important to carefully evaluate the specific needs of an application when choosing a database
management system.
NoSQL originally referring to non SQL or non relational is a database that provides a mechanism
for storage and retrieval of data. This data is modeled in means other than the tabular relations used
in relational databases. Such databases came into existence in the late 1960s, but did not obtain the
41
NoSQL moniker until a surge of popularity in the early twenty-first century. NoSQL databases are
used in real-time web applications and big data and their use are increasing over time.
● NoSQL systems are also sometimes called Not only SQL to emphasize the fact that they
may support SQL-like query languages. A NoSQL database includes simplicity of
design, simpler horizontal scaling to clusters of machines and finer control over
availability. The data structures used by NoSQL databases are different from those used
by default in relational databases which makes some operations faster in NoSQL. The
suitability of a given NoSQL database depends on the problem it should solve.
● NoSQL databases, also known as “not only SQL” databases, are a new type of database
management system that have gained popularity in recent years. Unlike traditional
relational databases, NoSQL databases are designed to handle large amounts of
unstructured or semi-structured data, and they can accommodate dynamic changes to
the data model. This makes NoSQL databases a good fit for modern web applications,
real- time analytics, and big data processing.
● Data structures used by NoSQL databases are sometimes also viewed as more flexible
than relational database tables. Many NoSQL stores compromise consistency in favor of
availability, speed and partition tolerance. Barriers to the greater adoption of NoSQL
stores include the use of low-level query languages, lack of standardized interfaces, and
huge previous investments in existing relational databases.
● Most NoSQL stores lack true ACID(Atomicity, Consistency, Isolation, Durability)
transactions but a few databases, such as MarkLogic, Aerospike, FairCom c-treeACE,
Google Spanner (though technically a NewSQL database), Symas LMDB, and
OrientDB have made them central to their designs.
● Most NoSQL databases offer a concept of eventual consistency in which database
changes are propagated to all nodes so queries for data might not return updated data
immediately or might result in reading data that is not accurate which is a problem known
as stale
42
reads. Also some NoSQL systems may exhibit lost writes and other forms of data loss.
Some NoSQL systems provide concepts such as write-ahead logging to avoid data loss.
● One simple example of a NoSQL database is a document database. In a document
database, data is stored in documents rather than tables. Each document can contain a
different set of fields, making it easy to accommodate changing data requirements
● For example, “Take, for instance, a database that holds data regarding employees.”. In a
relational database, this information might be stored in tables, with one table for
employee information and another table for department information. In a document
database, each employee would be stored as a separate document, with all of their
information contained within the document.
● NoSQL databases are a relatively new type of database management system that have
gained popularity in recent years due to their scalability and flexibility. They are designed
to handle large amounts of unstructured or semi-structured data and can handle dynamic
changes to the data model. This makes NoSQL databases a good fit for modern web
applications, real-time analytics, and big data processing.
Key Features of NoSQL :
1. Dynamic schema: NoSQL databases do not have a fixed schema and can
accommodate changing data structures without the need for migrations or schema
alterations.
2. Horizontal scalability: NoSQL databases are designed to scale out by adding more nodes
to a database cluster, making them well-suited for handling large amounts of data and
high levels of traffic.
3. Document-based: Some NoSQL databases, such as MongoDB, use a document-
based data model, where data is stored in semi-structured format, such as JSON or
BSON.
4. Key-value-based: Other NoSQL databases, such as Redis, use a key-value data
model, where data is stored as a collection of key-value pairs.
43
5. Column-based: Some NoSQL databases, such as Cassandra, use a column-based

data model, where data is organized into columns instead of rows.
6. Distributed and high availability: NoSQL databases are often designed to be highly
available and to automatically handle node failures and data replication across
multiple nodes in a database cluster.
7. Flexibility: NoSQL databases allow developers to store and retrieve data in a flexible and
dynamic manner, with support for multiple data types and changing data structures.
8. Performance: NoSQL databases are optimized for high performance and can handle a
high volume of reads and writes, making them suitable for big data and real-time
applications.
Advantages of NoSQL: There are many advantages of working with NoSQL databases such as
MongoDB and Cassandra. The main advantages are high scalability and high availability.
1. High scalability : NoSQL databases use sharding for horizontal scaling. Partitioning of
data and placing it on multiple machines in such a way that the order of the data is
preserved is sharding. Vertical scaling means adding more resources to the existing
machine whereas horizontal scaling means adding more machines to handle the data.
Vertical scaling is not that easy to implement but horizontal scaling is easy to implement.
Examples of horizontal scaling databases are MongoDB, Cassandra, etc. NoSQL can
handle a huge amount of data because of scalability, as the data grows NoSQL scale
itself to handle that data in an efficient manner.
2. Flexibility: NoSQL databases are designed to handle unstructured or semi-structured data,
which means that they can accommodate dynamic changes to the data model. This makes
NoSQL databases a good fit for applications that need to handle changing data
requirements.
3. High availability : Auto replication feature in NoSQL databases makes it highly
available because in case of any failure data replicates itself to the previous consistent
state.
44
4. Scalability: NoSQL databases are highly scalable, which means that they can handle
large amounts of data and traffic with ease. This makes them a good fit for applications
that need to handle large amounts of data or traffic
5. Performance: NoSQL databases are designed to handle large amounts of data and traffic,
which means that they can offer improved performance compared to traditional relational
databases.
6. Cost-effectiveness: NoSQL databases are often more cost-effective than traditional
relational databases, as they are typically less complex and do not require
expensive hardware or software.
Disadvantages of NoSQL: NoSQL has the following disadvantages.
1. Lack of standardization : There are many different types of NoSQL databases, each
with its own unique strengths and weaknesses. This lack of standardization can make
it difficult to choose the right database for a specific application
2. Lack of ACID compliance : NoSQL databases are not fully ACID-compliant, which
means that they do not guarantee the consistency, integrity, and durability of data.
This can be a drawback for applications that require strong data consistency
guarantees.
3. Narrow focus : NoSQL databases have a very narrow focus as it is mainly designed for
storage but it provides very little functionality. Relational databases are a better choice in
the field of Transaction Management than NoSQL.
4. Open-source : NoSQL is open-source database. There is no reliable standard for NoSQL
yet. In other words, two database systems are likely to be unequal.
5. Lack of support for complex queries : NoSQL databases are not designed to handle
complex queries, which means that they are not a good fit for applications that require
complex data analysis or reporting.
45
6. Lack of maturity : NoSQL databases are relatively new and lack the maturity of
traditional relational databases. This can make them less reliable and less secure
than traditional databases.
7. Management challenge : The purpose of big data tools is to make the management of a
large amount of data as simple as possible. But it is not so easy. Data management in
NoSQL is much more complex than in a relational database. NoSQL, in particular, has a
reputation for being challenging to install and even more hectic to manage on a daily
basis.
8. GUI is not available : GUI mode tools to access the database are not flexibly available
in the market.
9. Backup : Backup is a great weak point for some NoSQL databases like
MongoDB. MongoDB has no approach for the backup of data in a consistent
manner.
10. Large document size : Some database systems like MongoDB and CouchDB store data
in JSON format. This means that documents are quite large (BigData, network
bandwidth, speed), and having descriptive key names actually hurts since they increase
the document size.
Types of NoSQL database: Types of NoSQL databases and the name of the databases system that
falls in that category are:
1. Graph Databases: Examples – Amazon Neptune, Neo4j

2. Key value store: Examples – Memcached, Redis, Coherence
3. Tabular: Examples – Hbase, Big Table, Accumulo
4. Document-based: Examples – MongoDB, CouchDB, Cloudant
When should NoSQL be used:
1. When a huge amount of data needs to be stored and retrieved.
46
2. The relationship between the data you store is not that important
47
3. The data changes over time and is not structured.

4. Support of Constraints and Joins is not required at the database level
5. The data is growing continuously and you need to scale the database regularly to
handle the data.
In conclusion, NoSQL databases offer several benefits over traditional relational databases, such as
scalability, flexibility, and cost-effectiveness. However, they also have several drawbacks, such as a
lack of standardization, lack of ACID compliance, and lack of support for complex queries. When
choosing a database for a specific application, it is important to weigh the benefits and drawbacks
carefully to determine the best fit.
5.9 MONGODB
Advantages of MongoDB over RDBMS

● Schema less − MongoDB is a document database in which one collection holds different
documents. Number of fields, content and size of the document can differ from one
document to another.
● Structure of a single object is clear.
● No complex joins.
● Deep query-ability. MongoDB supports dynamic queries on documents using a document-
based query language that's nearly as powerful as SQL.
Tuning.
● Ease of scale-out − MongoDB is easy to scale.
● Conversion/mapping of application objects to database objects not needed.
● Uses internal memory for storing the (windowed) working set, enabling faster access of data.
Why Use MongoDB?

● Document Oriented Storage − Data is stored in the form of JSON style
documents.
● Index on any attribute
● Replication and high availability
● Auto-Sharding
48
● Rich queries
● Fast in-place updates
● Professional support by MongoDB
Data Model Design

MongoDB provides two types of data models: — Embedded data model and Normalized data model. Based
on the requirement, you can use either of the models while preparing your document.
Embedded Data Model

In this model, you can have (embed) all the related data in a single document, it is also known as de-
normalized data model.
For example, assume we are getting the details of employees in three different documents namely,
Personal_details, Contact and, Address, you can embed all the three documents in a single one as shown
below −
{
_id: ,
Emp_ID: "10025AE336"
Personal_details:{
First_Name: "Radhika",
Last_Name: "Sharma",
Date_Of_Birth: "1995-09-26"
},
Contact: {
e-mail: "radhika_sharma.123@gmail.com",
phone: "9848022338"
},
Address: {
city: "Hyderabad",
Area: "Madapur",
State: "Telangana"
}
}
Normalized Data Model

In this model, you can refer the sub documents in the original document, using references. For example, you
can re-write the above document in the normalized model as:
Employee:
49
{
_id: <ObjectId101>,
Emp_ID: "10025AE336"
}
Personal_details:
{
_id: <ObjectId102>,
empDocID: " ObjectId101",
First_Name: "Radhika",
Last_Name: "Sharma",
Date_Of_Birth: "1995-09-26"
}
Contact:
{
_id: <ObjectId103>,
e-mail: "radhika_sharma.123@gmail.com",
phone: "9848022338"
}
Address:
{
_id: <ObjectId104>,
city: "Hyderabad",
Area: "Madapur",
State: "Telangana"
}
Considerations while designing Schema in MongoDB

● Design your schema according to user requirements.
● Combine objects into one document if you will use them together. Otherwise separate
them (but make sure there should not be need of joins).
● Duplicate the data (but limited) because disk space is cheap as compare to compute time.
● Do joins while write, not on read.
● Optimize your schema for most frequent use cases.
● Do complex aggregation in the schema.
Example
50
Suppose a client needs a database design for his blog/website and see the differences between RDBMS and
MongoDB schema design. Website has the following requirements.
Every post has the unique title, description and url.

Every post can have one or more tags.
Every post has the name of its publisher and total number of likes.
Every post has comments given by users along with their name, message, data-time and likes.
On each post, there can be zero or more comments.
In RDBMS schema, design for above requirements will have minimum three tables.
While in MongoDB schema, design will have one collection post and the following structure −
{
_id: POST_ID
title: TITLE_OF_POST,
description: POST_DESCRIPTION,
by: POST_BY,
url: URL_OF_POST,
tags: [TAG1, TAG2, TAG3],
likes: TOTAL_LIKES,
comments: [
{
user:'COMMENT_BY',
message: TEXT,
dateCreated: DATE_TIME,
like: LIKES
},
{
user:'COMMENT_BY',
message: TEXT,
dateCreated: DATE_TIME,
51
like: LIKES
}
]
}
So while showing the data, in RDBMS you need to join three tables and in MongoDB, data will be shown
from one collection only.
5.9.1 DATA TYPES IN MONGODB:
MongoDB supports many datatypes. Some of them are −
String − This is the most commonly used datatype to store the data. String in MongoDB must
be UTF-8 valid.
Integer − This type is used to store a numerical value. Integer can be 32 bit or 64 bit depending
upon your server.
Boolean − This type is used to store a boolean (true/ false) value.
Double − This type is used to store floating point values.
Min/ Max keys − This type is used to compare a value against the lowest and highest BSON
elements.
Arrays − This type is used to store arrays or list or multiple values into one key.
Timestamp − ctimestamp. This can be handy for recording when a document has been
modified or added.
Object − This datatype is used for embedded documents.
Null − This type is used to store a Null value.
Symbol − This datatype is used identically to a string; however, it's generally reserved for
languages that use a specific symbol type.
Date − This datatype is used to store the current date or time in UNIX time format. You can
specify your own date time by creating object of Date and passing day, month, year into it.
Object ID − This datatype is used to store the document’s ID.
Binary data − This datatype is used to store binary data.
Code − This datatype is used to store JavaScript code into the document.
Regular expression − This datatype is used to store regular expression.
5.10 CRUD OPERATIONS:
52
As we know that we can use MongoDB for various things like building an application (including web and
mobile), or analysis of data, or an administrator of a MongoDB database, in all these cases we need to
interact with the MongoDB server to perform certain operations like entering new data into the application,
updating data into the application, deleting data from the application, and reading the data of the application.
MongoDB provides a set of some basic but most essential operations that will help you to easily interact with
the MongoDB server and these operations are known as CRUD operations.
Create Operations –
The create or insert operations are used to insert or add new documents in the collection. If a collection does
not exist, then it will create a new collection in the database. You can perform, create operations using the
following methods provided by the MongoDB:
Method Description
db.collection.insertOne() It is used to insert a single document in the collection.
db.collection.insertMany() It is used to insert multiple documents in the collection.
53
db.createCollection() It is used to create an empty collection.
Example 1: In this example, we are inserting details of a single student in the form of document in the student
collection using db.collection.insertOne() method.
Example 2: In this example, we are inserting details of the multiple students in the form of documents in the
54
student collection using db.collection.insertMany() method.
Read Operations –
The Read operations are used to retrieve documents from the collection, or in other words, read operations
are used to query a collection for a document. You can perform read operation using the following
method provided by the MongoDB:
55
Method Description
db.collection.find() It is used to retrieve documents from the collection.
56
Example : In this example, we are retrieving the details of students from the student collection using
db.collection.find() method.
Update Operations –
The update operations are used to update or modify the existing document in the collection. You can perform
update operations using the following methods provided by the MongoDB:
57
Method Description
It is used to update a single document in the collection

db.collection.updateOne()
that satisfy the given criteria.
It is used to update multiple documents in the collection

db.collection.updateMany()
It is used to replace single document in the collection

db.collection.replaceOne()
Example 1: In this example, we are updating the age of Sumit in the student collection using
db.collection.updateOne() method.
58
Example 2: In this example, we are updating the year of course in all the documents in the student collection
59
using db.collection.updateMany() method.
60
Delete Operations –
The delete operation are used to delete or remove the documents from a collection. You can perform delete
operations using the following methods provided by the MongoDB:
Method Description
It is used to delete a single document from the collection

db.collection.deleteOne()
It is used to delete multiple documents from the collection that

db.collection.deleteMany()
satisfy the given criteria.
Example 1: In this example, we are deleting a document from the student collection using
db.collection.deleteOne() method.
61
59
Example 2: In this example, we are deleting all the documents from the student collection using
db.collection.deleteMany() method.
60

21cs1401-Unit 5 - Dbms

Uploaded by

Copyright:

Available Formats

You might also like

21cs1401-Unit 5 - Dbms

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

21cs1401-Unit 5 - Dbms

Uploaded by

Copyright:

Available Formats

21CS1401 – Database Management Systems Unit –V

UNIT V IMPLEMENTATION TECHNIQUES AND NON-RELATIONAL MODEL

5.1 Data On External Storage

Types of Data Storage

5.2 RAID- Redundant Arrays Of Independent or Inexpensive Disks

Figure: 5.3 – (a) Raid Level 0 and (b) Raid Level 1

Figure: 5.4 – (c) Raid Level 2 and (d) Raid Level 3

Figure: 5.5(e) - Raid Level 4

Figure: 5.6(f) - Raid Level 5

Figure: 5.7 – Distribution of Parity Block PK

Figure: 5.8(g) - Raid Level 6

5.3 File Organization:

Figure: 5.9 – Fixed Length Records

o Storage of multiple record types in a file.

Figure: 5.13 - Byte-String Representation of Variable-Length Records

The byte-string representation as described has some disadvantages:

Figure: 5.14 - Slotted Page Structure

Figure: 5.15 - Fixed-Length Representation

Figure: 5.16 - Pointer Method

Figure: 5.17 - Pointer Method-Using Anchor block Overflow block

Organization of Records in Files:

Figure: 5.18 - Sequential File for account Records

Figure: 5.19 - Sequential File Organization after an

Figure: 5.20 - Depositor Relation

Figure: 5.21 - Customer Relation

Figure: 5.22 - Multiple clustering file structure

Figure: 5.23 – Multiple Clustering File Structure with Pointer Chains

Figure: 5.24 – Sequential file for account records.

Figure: 5.25 - Dense Index.

Figure: 5.26 - Sparse Index.

Figure: 5.27 – Two-level Sparse Index

Figure: 5.28 - Secondary index on account file on noncandidate key balance.

Delete − This is simply a search followed by a deletion operation.

If the bucket is already full.

5.5 B+ tree Index Files:

B+-tree indices are an alternative to indexed-sequential files.

● A leaf node has between ⎡ (n–1)/2⎤ and n–1 values

Figure: 5.38 - B+-Tree Node Structure

Figure: 5.39 – A Leaf Node for account B+-Tree index(n=3).

Non-Leaf Nodes in B+-Trees

o All the search-keys in the

Figure: 5.40 - Non leaf node

Figure: 5.41 - B+-tree for account file (n = 3)

Figure: 5.42 - B+-tree for account file (n = 5)

Updates on B+-Trees: Insertion

Figure 5.44: B+-Tree before and after insertion of “Clearview”

Updates on B+-Trees: Deletion

Figure 5.55 before and after deleting “Downtown”

B-Tree Index File Example

5.7 Query Processing & Optimization:

Steps in query Processing:

The steps involved in the query processing are

These steps are illustrated in the Figure: 5.7

Figure: 5.7 – Steps in Query Processing.

• with choice of one among several algorithms for each operator

Query Plan Optimization :

▪ # distinct key values (NKeys) and NPages for each index.

5.8 INTRODUCTION TO NOSQL: