SQL

1. How would you outline the process of crafting a relational database

schema for a complex business domain? Could you touch upon key aspects
such as normalization, denormalization, and optimal indexing strategies?

Solution :- Crafting a relational database schema for a complex business domain

involves several key steps. Firstly, understanding the business requirements and
relationships between entities is essential. Then, normalization techniques are applied
to minimize redundancy and dependency issues. Denormalization may be
considered for performance optimization, although it should be approached
cautiously to prevent data inconsistencies. Finally, implementing optimal indexing
strategies, such as primary keys, foreign keys, and composite indexes, can enhance
query performance.

2. Can you explain the steps involved in creating an ETL process for moving data
between databases? I'm curious about handling data validation, errors, and
incremental loading.

Solution :- Creating an ETL process involves several steps:

1. Extract: Retrieve data from the source database using extraction methods such
as SQL queries or API calls.
2. Transform: Apply transformations to the extracted data to ensure it meets the
requirements of the target database. This may include cleaning, filtering,
aggregating, or joining data sets.
3. Load: Insert the transformed data into the target database using loading
methods such as bulk insert or individual record insertion.

When it comes to handling data validation, errors, and incremental loading:

● Data Validation: Perform validation checks during the transformation phase to

ensure data accuracy and integrity. This may involve checking for missing values,
data types, constraints, or business rules.
● Error Handling: Implement error handling mechanisms to manage exceptions
that may occur during the ETL process. This includes logging errors, retrying
failed operations, and notifying stakeholders of critical issues.
● Incremental Loading: Incremental loading involves only transferring new or
modified data since the last ETL run, reducing processing time and resource
usage. Implement techniques such as timestamp-based or change data capture
(CDC) methods to identify and extract incremental changes efficiently.

3. You are given a database schema representing an e-commerce platform

with the following tables:
a. `customers`:
i. `customer_id` (Primary Key)
ii. `name`
 iii. `email`

b. `orders`:
i. `order_id` (Primary Key)
ii. `customer_id` (Foreign Key referencing `customers.customer_id`)
iii. `order_date`
iv. `total_amount`

c. `order_items`:
i. `order_item_id` (Primary Key)
ii. `order_id` (Foreign Key referencing `orders.order_id`)
iii. `product_id`
iv. `quantity`
v. `unit_price`

Write an SQL query to identify the top 5 customers who have spent the most
money on the platform. The query should calculate the total amount spent by
each customer across all their orders and include their names and email addresses
in the result. Consider scenarios where customers may have multiple orders and
optimize the query for performance.

Solution :-
SELECT c.name, c.email, SUM(o.total_amount) AS total_spent
FROM customers c
JOIN orders o ON c.customer_id = o.customer_id
GROUP BY c.customer_id
ORDER BY total_spent DESC
LIMIT 5;

This query joins the customers and orders tables on the customer_id field, calculates
the total amount spent by each customer using the SUM() function, groups the results by
customer_id, orders them in descending order based on total spent, and limits the
output to the top 5 customers.

4. Problem: Analyzing Flight Data

You are given three tables:

a. `flights` - Contains information about flights:

i. `flight_id` (unique identifier for each flight)
ii. `departure_airport_code`
iii. `arrival_airport_code`
iv. `departure_time`
v. `arrival_time`
vi. `duration` (in minutes)
vii. `airline_code`
b. `airports` - Contains information about airports:
 i. `airport_code` (unique identifier for each airport)
ii. `airport_name`
iii. `city`
iv. `country`
c. `airlines` - Contains information about airlines:
i. `airline_code` (unique identifier for each airline)
ii. `airline_name`
iii. `Headquarters`

Write a SQL query to find the following:

1. The top 5 busiest airports (by total number of flights departing or arriving) along
with their respective cities and countries.
2. The airline that has the highest average flight duration.
3. The airport pair (departure and arrival) with the highest number of flights
between them.

Solution :-
1. Top 5 busiest airports :

SELECT a.airport_name, a.city, a.country,

COUNT(*) AS total_flights
FROM flights f
JOIN airports a ON f.departure_airport_code = a.airport_code
OR f.arrival_airport_code = a.airport_code
GROUP BY a.airport_code
ORDER BY total_flights DESC
LIMIT 5;

2. Airline with the highest average flight duration :

SELECT airline_name,
AVG(duration) AS avg_flight_duration
FROM flights f
JOIN airlines al ON f.airline_code = al.airline_code
GROUP BY al.airline_code
ORDER BY avg_flight_duration DESC
LIMIT 1;

3. Airport pair with the highest number of flights between them:

SELECT departure.airport_name AS departure_airport,

departure.city AS departure_city,
departure.country AS departure_country,
arrival.airport_name AS arrival_airport,
 arrival.city AS arrival_city,
arrival.country AS arrival_country,
COUNT(*) AS total_flights
FROM flights f
JOIN airports departure ON f.departure_airport_code =
departure.airport_code
JOIN airports arrival ON f.arrival_airport_code = arrival.airport_code
GROUP BY f.departure_airport_code, f.arrival_airport_code
ORDER BY total_flights DESC
LIMIT 1;

5. Problem Statement : Sales Analysis

You're given a database schema with the following tables:

a. `Orders`: Contains information about orders placed, including `order_id`,

`customer_id`, `order_date`, and `total_amount`.
b. `Order_Items`: Contains details about items ordered in each order, including
`order_id`, `product_id`, `quantity`, and `unit_price`.
c. `Customers`: Contains information about customers, including `customer_id`,
`name`, `email`, and `city`.

You're tasked with writing a SQL query to find the top 5 customers who have spent the
most money (total_amount) on orders, along with their total spending and the
number of orders they've placed. Additionally, include the city of each top-spending
customer.

Your query should return the following columns:

● `customer_id`
● `name`
● `total_spending`
● `order_count`
● `city`

Solution :-
SELECT

c.customer_id,

c.name,

SUM(o.total_amount) AS total_spending,

COUNT(o.order_id) AS order_count,
 C.city

FROM

Customers c

JOIN

Orders o ON c.customer_id = o.customer_id

GROUP BY

c.customer_id, c.name, c.city

ORDER BY

total_spending DESC, order_count DESC

LIMIT 5;

Explanation :-

We first connect the Customers table with the Orders table using the customer_id to
gather details about orders placed by each customer. Next, we group the data by
customer_id, name, and city, then calculate the total spending for each customer by
adding up the total_amount from all their orders. Additionally, we count the number of
orders placed by each customer. Finally, we arrange the results by total_spending and
order_count in descending order and limit the output to the top 5 customers. This
approach ensures we identify the top-spending customers while effectively handling
ties in spending.

6. How would you optimize a query that is running slowly due to a large
dataset? Discuss techniques such as indexing, query restructuring, and
materialized views.

Solution :-
a. Indexing:
i. Identify frequently used columns in queries and create indexes on those
columns to speed up data retrieval.
ii. Be cautious not to over-index, as it can impact data modification
operations.
b. Query Restructuring:
i. Simplify complex queries by breaking them into smaller, more
manageable parts.
ii. Use JOIN conditions effectively to filter data early in the query execution
process.
iii. Specify only necessary columns to reduce unnecessary data retrieval.
c. Materialized Views:

i.Create precomputed views of frequently accessed or resource-intensive

queries.
ii. Refresh materialized views periodically to maintain data accuracy and
relevancy.
d. Partitioning:
 i. Divide large tables into smaller partitions based on specific criteria to
improve query performance.
ii. Utilize partition pruning to skip irrelevant partitions during query
execution.
e. Statistics Maintenance:
i. Keep table and index statistics up to date to assist the query optimizer in
making efficient query execution plans.
ii. Regularly update statistics, especially after significant data changes.
f. Query Optimization Tools:
i. Use database-specific optimization tools to analyze query execution plans
and identify performance bottlenecks.
ii. Experiment with different optimization techniques to find the most
efficient solution.

7. Describe the risks of SQL injection attacks and techniques for preventing
them in SQL queries. Provide examples of vulnerable code and how it could be
secured.

Solution :-

Risks of SQL Injection Attacks:

1. Data Leakage: Attackers can extract sensitive data from the database, such as
usernames and passwords.
2. Data Manipulation: Attackers can modify or delete data in the database, leading
to data corruption or loss.
3. Unauthorized Access: Attackers can gain unauthorized access to restricted areas
of the application or escalate their privileges.

Techniques for Preventing SQL Injection:

1. Parameterized Queries (Prepared Statements):

● Separates SQL code from user input, preventing malicious code injection.
● Uses placeholders for user input, treating them as data, not executable
code.
2. Input Validation:
● Validates and sanitizes user input to ensure it meets expected formats and
doesn't contain malicious characters.
● Whitelist validation only allows known-safe inputs.
3. Least Privilege Principle:
● Limits database user privileges to what's necessary for the application,
preventing unauthorized actions.

4. Escaping Special Characters:

● Neutralizes special characters in user input to prevent them from altering
the meaning of SQL queries.
 ● Done using functions like mysqli_real_escape_string() or by using
parameterized queries.
5. Web Application Firewall (WAF):
● Filters and blocks incoming requests with suspicious SQL injection
patterns.
● Provides an extra layer of defense against SQL injection attacks.

8. Compare and contrast subqueries and JOINs in SQL. Provide examples of

scenarios where each would be preferred, considering performance and
readability.

Solution :-

Subqueries:
● Definition: Nested queries within another query.
● Usage: Filter or manipulate data based on results of another query.
● Performance: Can impact performance, especially if inner query is complex or
returns large result set.
● Readability: Makes queries more concise and readable, useful for expressing
complex logic.

JOINs:

● Definition: Combine rows from multiple tables based on related column.

● Usage: Retrieve data from multiple tables based on common column or
relationship.
● Performance: Generally offers better performance, especially with large datasets.
● Readability: Clear and explicit in indicating how tables are related, useful for
joining multiple tables efficiently.

Scenarios:

● Subqueries: Preferred for filtering based on results of another query or applying

conditional logic.
● JOINs: Preferred for retrieving data from multiple tables efficiently based on
common column or relationship.

9. What are star and snowflake schemas? When would you use each schema
design in a data warehousing environment? Provide examples to illustrate your
answer.

Solution :-
Star Schema:
● Definition: Central fact table surrounded by denormalized dimension tables.
● Usage: Simple, efficient for querying, suitable for clear relationships between fact
and dimensions.
● Example: In retail, sales fact table surrounded by product, time, and store
dimensions.
 Snowflake Schema:

● Definition: Similar to star schema, but with normalized dimension tables.

● Usage: More normalized, reduces redundancy, suitable for hierarchical
relationships between dimensions.
● Example: Product dimension further broken down into product category,
subcategory, and brand tables.

Comparison:

a. Star Schema: Simple, denormalized, suitable for better query performance and
clearer relationships.
b. Snowflake Schema: More normalized, reduces redundancy, suitable for complex
hierarchical relationships.

10. How would you integrate SQL Server with big data technologies such as
Hadoop or Apache Spark? Discuss challenges and best practices for data
integration in a hybrid environment.

Solution :-

Challenges:
1. Data Formats and Protocols:
● Different formats and protocols between SQL Server and big data
technologies can hinder seamless data exchange.
2. Data Consistency and Integrity:
● Maintaining consistency across heterogeneous systems is crucial for
ensuring data accuracy and reliability.
3. Performance and Scalability:
● Achieving optimal performance and scalability when processing
large datasets distributed across multiple systems.
4. Security and Access Control:
● Ensuring data security and access control across hybrid
environments with diverse authentication mechanisms.
5. Metadata Management:
● Managing metadata and data lineage across systems for data
governance and compliance.

Best Practices:

1. Use Compatible Formats and Protocols:

● Choose data formats and protocols supported by both SQL Server
and big data platforms for smooth integration.
2. Leverage Connectors and APIs:
● Utilize connectors and APIs provided by both systems for seamless
data transfer and processing.
3. Implement Data Pipelines:
● Design data pipelines to orchestrate data movement and processing
between systems efficiently.
4. Ensure Consistency and Integrity:
● Implement mechanisms like change data capture (CDC) and
transactional consistency to maintain data integrity.
5. Monitor and Tune Performance:
● Monitor data transfer and processing performance and optimize
workflows for better efficiency.
6. Implement Security Measures:
● Secure data in transit and at rest with encryption, authentication,
and access control mechanisms.
7. Establish Data Governance Practices:
● Manage metadata, lineage, and data quality for consistent and
compliant data management.