Professional Documents
Culture Documents
Pinotcoursera 151103183418 Lva1 App6892
Pinotcoursera 151103183418 Lva1 App6892
Pinotcoursera 151103183418 Lva1 App6892
Kishore Gopalakrishna
Columnar
SQL-like Real-time
storage and
interface data load
indexing
Mutation
Broker Helix
Pinot
Real
time
Historical Architecture
Tuesday, August 18, 15
Star tree Index
Data aware
pre-computation
Brokers
Helix
S1 S2 S3
S3 S1 S2
Servers
Tuesday, August 18, 15
Pinot Query Execution: Distributed
1.Query
Brokers
Helix
S1 S2 S3
S3 S1 S2
Servers
Tuesday, August 18, 15
Pinot Query Execution: Distributed
1.Query
Helix
S1 S2 S3
S3 S1 S2
Servers
Tuesday, August 18, 15
Pinot Query Execution: Distributed
1.Query
S3 S1 S2
Servers
Tuesday, August 18, 15
Pinot Query Execution: Distributed
1.Query
Servers
Tuesday, August 18, 15
Pinot Query Execution: Distributed
1.Query
Servers
Tuesday, August 18, 15
Pinot Query Execution: Distributed
1.Query 6. Return Response
Servers
Tuesday, August 18, 15
Pinot Query Execution: Single Node Architecture
PLANNER
EXECUTION ENGINE
INVERTED BITMAP
INDEX INDEX
COLUMN FORMAT
SELECT
Aggregation
campaignId, Group by campaignId,Click tuple
sum(clicks) Operator
Servers
Brokers and Servers register themselves in Helix
All servers start with no use case specific configuration
XLNT
Tag XLNT
Controller Brokers
TableName XLNT_T1
Servers 3 Helix
Brokers 1
Create Table
XLNT XLNT XLNT
command
Servers
S1
Controller Brokers
S2
Helix
Upload Segment S1 S2 S3
S3 TableName XLNT_T1
Copies 2 S3 S1 S2
Servers
Always On all columns, Configurable on per Allows trade off between scanning v/s
Inverted Index inverted index + scanning. More data can be
Fixed column basis
fit in given memory size
Organizing data provides speed/better
Data organization N/A Sorts data compression and removes the need for
inverted index
Smart pre-
N/A star-tree Allows trade off between latency and space
materialization
Query Execution Split into Planning Smart choices can be made at runtime
Fixed Plan
Layer and execution based on metadata/query.
30
Tuesday, August 18, 15