What is Query Folding?

Query folding refers to the process of translating and pushing data transformation steps back to the
source database server. This means that instead of retrieving all the data to your local machine and
performing transformations there, Power Query attempts to perform as many transformations as
possible within the data source itself.

Benefits of Query Folding

Efficiency: Performing operations on the server side leverages the power and performance of the
database server.

Reduced Data Transfer: Limits the amount of data that needs to be transferred over the network.

Optimized Processing: Takes advantage of indexes and optimized SQL queries, resulting in faster data
processing.

When Does Query Folding Occur?

Query folding occurs when Power Query can convert a transformation step into a query that the data
source can understand and execute. This is possible when using data sources that support query
languages, like SQL databases (e.g., SQL Server, Oracle, MySQL).

Steps that Typically Support Query Folding

Filtering rows (Table.SelectRows)

Removing columns (Table.RemoveColumns)

Grouping (Table.Group)

Sorting (Table.Sort)

Joins (Table.Join)

Aggregations (e.g., Table.Group with aggregations)

Checking Query Folding

To see if query folding is occurring, you can check the "View Native Query" option in Power Query:

In Power BI Desktop:

Open Power Query Editor.

Right-click on a step in the Applied Steps pane.

If "View Native Query" is enabled, query folding is happening for that step.

In Excel Power Query:

The steps are similar to Power BI Desktop.

Common Reasons Query Folding Might Not Occur

Using functions that the source doesn’t support.

Performing operations in an order that breaks folding.

Transformations that require complex logic not easily translatable to the source's query language.

Best Practices for Maximizing Query Folding

Order of Operations: Perform steps that support query folding early in your transformation process.

Simplify Transformations: Avoid overly complex or custom transformations that the data source cannot
natively process.

Leverage Source Capabilities: Use database functions and capabilities as much as possible.

Example of Query Folding

Let’s consider a scenario where you have a SQL Server table and you perform the following steps in
Power Query:

Connect to the SQL Server table.

Filter the rows to select records where OrderDate > '2023-01-01'.

Remove columns that are not needed.

Sort the data by OrderDate.

Power Query will attempt to fold these steps into a single SQL query that might look something like this:

sql

Copy code

SELECT OrderDate, CustomerID, OrderAmount

FROM Sales

WHERE OrderDate > '2023-01-01'

ORDER BY OrderDate
Troubleshooting Query Folding Issues

If query folding is not occurring as expected:

Review Each Step: Check each transformation step to identify which step breaks the folding.

Check Source Compatibility: Ensure the data source supports the transformations you are applying.

Adjust Transformations: Modify the order or method of transformations to enable folding.

Conclusion

Understanding and leveraging query folding in Power Query can significantly enhance the performance
and efficiency of your data transformations. By ensuring that as many steps as possible are folded back
to the data source, you can take full advantage of the processing power of your database server and
minimize data movement.