Query Optimisation Tips & Tricks

Some uncommon yet important SQL query optimization tips, along with examples comparing
the regular approach with the optimized approach, each illustrated with a real-world example:

1. Avoid using SELECT *:

- Regular Approach:
```sql
SELECT * FROM employees WHERE department = 'HR';
```
- Optimized Approach:
```sql
SELECT employee_id, first_name, last_name FROM employees WHERE department = 'HR';
```
Explanation: In the regular approach, we are selecting all columns using `SELECT *`, which
may retrieve unnecessary data, leading to increased I/O and slower query execution. By
explicitly listing only the required columns, as shown in the optimized approach, we reduce the
data volume and improve query performance.

2. Minimize Subqueries:
- Regular Approach:
```sql
SELECT product_name, (SELECT AVG(price) FROM sales WHERE product_id =
products.id) AS avg_price
FROM products;
```
- Optimized Approach:
```sql
SELECT p.product_name, AVG(s.price) AS avg_price
FROM products p
JOIN sales s ON p.id = s.product_id
GROUP BY p.product_name;
```
Explanation: In the regular approach, we use a subquery to calculate the average price for
each product. This can be slow and inefficient, especially for large datasets. The optimized
approach uses a JOIN and GROUP BY to achieve the same result, resulting in better
performance.
3. Use EXISTS or JOIN instead of IN:
- Regular Approach:
```sql
SELECT order_id, order_date
FROM orders
WHERE customer_id IN (SELECT id FROM customers WHERE country = 'USA');
```
- Optimized Approach:
```sql
SELECT o.order_id, o.order_date
FROM orders o
WHERE EXISTS (SELECT 1 FROM customers c WHERE c.id = o.customer_id AND
c.country = 'USA');
```
Explanation: The regular approach uses the IN operator with a subquery, which can be
inefficient when the subquery returns a large result set. The optimized approach uses EXISTS
to check for the existence of a matching record, which is often faster.

4. Use UNION ALL instead of UNION:

- Regular Approach:
```sql
SELECT employee_id, first_name FROM employees
UNION
SELECT customer_id, full_name FROM customers;
```
- Optimized Approach:
```sql
SELECT employee_id, first_name FROM employees
UNION ALL
SELECT customer_id, full_name FROM customers;
```
Explanation: The regular approach uses UNION, which removes duplicate rows from the
result set. If you know that the queries' results are distinct, use UNION ALL, which avoids the
overhead of removing duplicates and improves performance.

5. Be cautious with ORDER BY in subqueries:

- Regular Approach:
```sql
SELECT product_name
 FROM products
WHERE product_id IN (SELECT product_id FROM sales ORDER BY sale_date DESC
LIMIT 10);
```
- Optimized Approach:
```sql
SELECT p.product_name
FROM products p
JOIN (
SELECT product_id
FROM sales
ORDER BY sale_date DESC
LIMIT 10
) s ON p.product_id = s.product_id;
```
Explanation: The regular approach uses ORDER BY in the subquery, which can have a
performance impact, especially when retrieving a large number of rows. The optimized
approach uses a JOIN and moves the ORDER BY to the subquery, reducing the number of
sorted rows and improving query speed.

6. Use appropriate indexing:

- Regular Approach:
```sql
SELECT product_name
FROM products
WHERE product_type = 'Electronics';
```
- Optimized Approach:
```sql
ALTER TABLE products ADD INDEX idx_product_type (product_type);
```
```sql
SELECT product_name
FROM products
WHERE product_type = 'Electronics';
```
Explanation: The regular approach doesn't utilize any index, resulting in a full table scan. The
optimized approach adds an index on the `product_type` column, which allows for faster data
retrieval when filtering based on this column.

Remember, optimization techniques may vary based on the database system and specific use
cases. Always analyze the query execution plans, profile query performance, and test different
optimization approaches to determine the most effective strategy for your environment.
JOINS
Some uncommon yet important JOINs tips in SQL queries to optimize their performance:

1. Choose the Appropriate JOIN Type:

- Regular Approach:
```sql
SELECT orders.order_id, customers.customer_name
FROM orders, customers
WHERE orders.customer_id = customers.customer_id;
```
- Optimized Approach:
```sql
SELECT orders.order_id, customers.customer_name
FROM orders
JOIN customers ON orders.customer_id = customers.customer_id;
```
Explanation: The regular approach uses an implicit join (comma-separated tables in the
FROM clause), which can be less readable and is prone to Cartesian products if not specified
correctly. The optimized approach uses an explicit INNER JOIN, making the query more
readable and helping the database engine optimize the join execution.

2. Use LEFT JOIN Instead of Subqueries:

- Regular Approach:
```sql
SELECT customers.customer_name, (SELECT SUM(order_total) FROM orders WHERE
customer_id = customers.customer_id) AS total_spent
FROM customers;
```
- Optimized Approach:
```sql
SELECT c.customer_name, COALESCE(SUM(o.order_total), 0) AS total_spent
FROM customers c
LEFT JOIN orders o ON c.customer_id = o.customer_id
GROUP BY c.customer_name;
```
 Explanation: The regular approach uses a correlated subquery to calculate the total amount
spent by each customer, which can be inefficient. The optimized approach uses a LEFT JOIN
and GROUP BY to achieve the same result, providing better performance.

3. Be Mindful of JOIN Order:

- Regular Approach:
```sql
SELECT p.product_name, c.category_name
FROM products p
JOIN categories c ON p.category_id = c.category_id
JOIN suppliers s ON p.supplier_id = s.supplier_id;
```
- Optimized Approach:
```sql
SELECT p.product_name, c.category_name
FROM products p
JOIN suppliers s ON p.supplier_id = s.supplier_id
JOIN categories c ON p.category_id = c.category_id;
```
Explanation: The order of JOINs can impact performance. In the regular approach, the JOIN
with `suppliers` is done before the JOIN with `categories`, even though the final result requires
data from both tables. In the optimized approach, we reorder the JOINs based on the query's
logical needs, potentially reducing the number of rows processed during the JOIN operations.

4. Utilize Appropriate Indexes for JOIN Columns:

- Regular Approach:
```sql
SELECT product_name, category_name
FROM products
JOIN categories ON products.category_id = categories.category_id
WHERE categories.category_name = 'Electronics';
```
- Optimized Approach:
```sql
ALTER TABLE categories ADD INDEX idx_category_name (category_name);
```
```sql
SELECT product_name, category_name
FROM products
JOIN categories ON products.category_id = categories.category_id
 WHERE categories.category_name = 'Electronics';
```
Explanation: The regular approach doesn't utilize any index for the
`categories.category_name` column, which can lead to a slower query. The optimized approach
adds an index on the `category_name` column, improving the join performance when filtering
based on this column.

5. Be Mindful of Multiple JOINs:

- Regular Approach:
```sql
SELECT o.order_id, c.customer_name, p.product_name
FROM orders o
JOIN customers c ON o.customer_id = c.customer_id
JOIN order_items oi ON o.order_id = oi.order_id
JOIN products p ON oi.product_id = p.product_id;
```
- Optimized Approach:
```sql
SELECT o.order_id, c.customer_name, p.product_name
FROM orders o
JOIN customers c ON o.customer_id = c.customer_id
JOIN (
SELECT oi.order_id, p.product_name
FROM order_items oi
JOIN products p ON oi.product_id = p.product_id
) p ON o.order_id = p.order_id;
```
Explanation: In the regular approach, multiple JOINs can lead to a complex execution plan.
The optimized approach uses a subquery to combine the `order_items` and `products` tables
before the final JOIN, potentially simplifying and improving the performance.

Remember to analyze query execution plans, profile query performance, and test different join
strategies to find the most efficient approach for your specific use case and database system.
Properly setting up JOINs is crucial for query optimization and can significantly impact the
overall performance of your SQL queries.
GROUP BY & Aggregations
Following are some tips for setting up GROUP BY clauses in SQL queries to optimize their
performance:

1. Use GROUP BY with Aggregates:

- Regular Approach:
```sql
SELECT department, employee_name
FROM employees
GROUP BY department;
```
- Optimized Approach:
```sql
SELECT department, COUNT(*) AS employee_count
FROM employees
GROUP BY department;
```
Explanation: In the regular approach, the GROUP BY clause is used without any aggregates,
which can lead to non-deterministic results and potential errors. The optimized approach uses
the COUNT() aggregate function to count the number of employees in each department.

2. Minimize the Number of Grouped Columns:

- Regular Approach:
```sql
SELECT department, employee_name, hire_date
FROM employees
GROUP BY department, employee_name, hire_date;
```
- Optimized Approach:
```sql
SELECT department, employee_name
FROM employees
GROUP BY department, employee_name;
```
Explanation: In the regular approach, all columns are included in the GROUP BY clause,
which might lead to a large number of groups and increased query execution time. The
optimized approach reduces the number of grouped columns to only the necessary ones.
3. Use HAVING Wisely:
- Regular Approach:
```sql
SELECT department, COUNT(*) AS employee_count
FROM employees
GROUP BY department
HAVING COUNT(*) > 5;
```
- Optimized Approach:
```sql
SELECT department, COUNT(*) AS employee_count
FROM employees
GROUP BY department
HAVING employee_count > 5;
```
Explanation: In the regular approach, the HAVING clause uses an aggregate function
(COUNT(*)) in the condition, which can be less readable. The optimized approach uses the
alias `employee_count` directly in the HAVING clause, making the query easier to understand.

4. Be Mindful of Ordering in GROUP BY:

- Regular Approach:
```sql
SELECT department, employee_name, MAX(salary) AS max_salary
FROM employees
GROUP BY department, employee_name
ORDER BY max_salary DESC;
```
- Optimized Approach:
```sql
SELECT department, employee_name, MAX(salary) AS max_salary
FROM employees
GROUP BY department, employee_name
ORDER BY department, max_salary DESC;
```
Explanation: In the regular approach, the ORDER BY clause includes `max_salary` without
specifying `department`, which can lead to inconsistent sorting within departments. The
optimized approach includes `department` in the ORDER BY clause to ensure proper sorting
within each department.

5. Utilize Indexes for GROUP BY Columns:

- Regular Approach:
 ```sql
SELECT department, COUNT(*) AS employee_count
FROM employees
GROUP BY department;
```
- Optimized Approach:
```sql
ALTER TABLE employees ADD INDEX idx_department (department);
```
```sql
SELECT department, COUNT(*) AS employee_count
FROM employees
GROUP BY department;
```
Explanation: The regular approach doesn't utilize any index for the `department` column,
which can lead to a slower query, especially with a large number of rows. The optimized
approach adds an index on the `department` column, improving the GROUP BY performance.

Remember to analyze query execution plans, profile query performance, and test different
GROUP BY strategies to find the most efficient approach for your specific use case and
database system. Properly setting up GROUP BY clauses is essential for query optimization
and can significantly impact the overall performance of your SQL queries.

WINDOW FUNCTIONS
Window functions and GROUP BY serve different purposes, but they can often achieve similar
results. Here are some tips on using window functions in comparison with GROUP BY and other
use cases:

1. Aggregation with Window Functions vs. GROUP BY:

- GROUP BY Approach:
```sql
SELECT department, SUM(salary) AS total_salary
FROM employees
GROUP BY department;
```
- Window Function Approach:
 ```sql
SELECT department, salary, SUM(salary) OVER (PARTITION BY department) AS
total_salary
FROM employees;
```
Explanation: Both approaches provide the total salary for each department. The GROUP BY
approach aggregates the data and returns a single row per department. The Window Function
approach uses the SUM() window function to calculate the total salary for each department
without collapsing the result into a single row per group.

2. Ranking Rows with Window Functions:

- Regular SELECT Approach:
```sql
SELECT employee_id, first_name, salary
FROM employees
WHERE department = 'HR'
ORDER BY salary DESC
LIMIT 5;
```
- Window Function Approach:
```sql
SELECT employee_id, first_name, salary
FROM (
SELECT employee_id, first_name, salary,
ROW_NUMBER() OVER (ORDER BY salary DESC) AS rank
FROM employees
WHERE department = 'HR'
) ranked_employees
WHERE rank <= 5;
```
Explanation: The Window Function approach utilizes the ROW_NUMBER() window function
to rank employees based on their salary within the 'HR' department. It then filters the result to
retrieve the top 5 highest salaries.

3. Moving Averages and Aggregations with Window Functions:

- Regular SELECT Approach (Moving Average):
```sql
SELECT date, sale_amount,
(SELECT AVG(sale_amount) FROM sales s2
 WHERE s2.date BETWEEN DATE_SUB(s1.date, INTERVAL 2 DAY) AND s1.date) AS
moving_average
FROM sales s1;
```
- Window Function Approach (Moving Average):
```sql
SELECT date, sale_amount,
AVG(sale_amount) OVER (ORDER BY date ROWS BETWEEN 2 PRECEDING AND
CURRENT ROW) AS moving_average
FROM sales;
```
Explanation: The Window Function approach calculates the moving average using the AVG()
window function with a window frame defined by the current row and the two preceding rows
based on the order of the date column. This avoids the need for a correlated subquery.

4. ROWS vs. RANGE Clause in Window Functions:

- ROWS Clause Approach:
```sql
SELECT date, sale_amount,
SUM(sale_amount) OVER (ORDER BY date ROWS BETWEEN 2 PRECEDING AND
CURRENT ROW) AS total_sales
FROM sales;
```
- RANGE Clause Approach:
```sql
SELECT date, sale_amount,
SUM(sale_amount) OVER (ORDER BY date RANGE BETWEEN INTERVAL 2 DAY
PRECEDING AND CURRENT ROW) AS total_sales
FROM sales;
```
Explanation: In this case, the ROWS and RANGE approaches yield similar results. However,
when dealing with date-based data, the RANGE clause can produce different results as it
considers the distance between values, while the ROWS clause looks at the number of rows.

When using window functions, consider the specific use case, the window frame, and the
desired result set. Window functions offer powerful capabilities to perform complex calculations
and aggregations without collapsing the data, making them suitable for various analytical tasks.
However, for simple aggregations and grouping, GROUP BY remains a suitable choice. Choose
the appropriate approach based on the query's complexity, performance requirements, and the
specific analysis needed for your data.