How to use query in excel

26 Sep ’22 by Antonio Nakić-Alfirević

SQL Query function in Excel

If you’re reading this article you probably know that Google Sheets has a QUERY function that allows you to run SQL-like queries against data in the sheet. This function lets you do all sorts of gymnastics with the data in your sheet, be it filtering, aggregating, or pivoting data.

Being a fully-fledged desktop app, Excel tends to be more feature-rich than Google Sheets. This is especially true in the data analytics department where Excel shines with advanced Excel functions as well as Power Query functionality.

However, Excel doesn’t natively have a QUERY function that you can use in cells on the sheet.

In this blog post, I’m going to show you how to add a QUERY function to Excel and give a few examples of how to use it.

First look

Let’s start by taking a look at the function in action.

The function is pretty straightforward. It accepts the SQL query as the first parameter and returns a table with the results of the query.

The results automatically spill to the necessary amount of space. This spilling behavior relies on the dynamic array functionality that’s available in Excel 365 (but isn’t in earlier versions of Microsoft Excel).

Works with Excel tables

In Google Sheets, the QUERY function references data by address (e.g. “A1:B10”) while columns are referenced by letters (e.g. A, B, C…).

This works but has some drawbacks:

• It makes the query sensitive to the location of the data. If the data is moved or if columns are reordered, the query will break.
• It makes the query difficult to read since it uses range addresses and column letters instead of table and column names (e.g. Employees, DateOfBirth…)
• Adding or removing rows can break the query. For example, if the provided range is “A1:H10” the query will only take into account the first 10 rows. If additional rows are added, the query will not take them into account. You can get around this by omitting the end row number (e.g. “A1:H”), but this means that there must be no other content below the data range.

Excel, on the other hand, allows explicitly defining tables (aka ListObjects) that delineate the areas that hold data. Each Excel table has a name, as do its columns. This makes Excel tables very similar to database tables and makes them easier to work with from SQL.

Full SQL syntax support (SQLite)

Under the hood, the Windy.Query function is powered by SQLite – a small but powerful embedded database engine.

When called, the function passes the query to the built-in SQLite engine which has an adapter that lets it use Excel tables as its data source.

This means that the entire SQLite syntax is available for use in queries. In comparison, in Google Sheets, the query syntax is rather limited. It only supports a single table (no joins) and a very small set of built-in functions.

Examples of use

Since the engine under the hood is SQLite, queries can use all operations available in SQLite, including table joins, temp tables, column table expressions, window functions etc… Let’s go over some examples of how to use these in Excel.

Joining tables

Here’s an example of a simple one-to-many join:

The usual way of doing a simple operation such as this one in Excel would be to use xlookup or PowerQuery, but SQL is now another option. And if we needed anything more complex than a simple join, SQL would quickly shine as the most powerful and convenient option of the three.

Merging table rows (union)

Another way we might want to combine two (or more) tables is to combine their rows. We can do this with a SQL UNION operator.

The tables might have some rows in common. If we want to keep only one instance of such rows we would use the regular UNION operator. If we want to keep both versions of rows that are in common, we would use the UNION ALL operator.

Finding differences between two tables

In the previous example, we had two tables that had some rows in common and some rows not. Let’s assume, for example, that the first table contains last year’s list of employees and the second table is the new list of employees.

If we wanted to find out the differences between the two tables, we could easily do that with a bit of SQL.

All of the rows that are in the first table but not in the second one we will mark as “deleted”. All of the rows that are in the second table but not in the first one we will mark as “added”. Here’s what that SQL query looks like:

select
	id, name, 'deleted'
from
	employees e where not exists (select * from Employees_New en where e.id == en.Id)
union
select
	id, name, 'added'
from
	employees_new en where not exists (select * from Employees e where e.id == en.Id)

And here’s what the result looks like:

Ranking rows

Another useful thing we might want to do is rank rows based on some criteria. For example, suppose we have a table with a list of cities. For each city we have its population and the country it belongs to.

Our task is to find the top 3 cities in each country based on population. Here’s how we might do that in SQL.

-- we use this CTE so we can reference the calculated 'rank_pop' column in the where clause
with cte as (
	select
		city,
		country,
		population,
		-- using the RANK() window function
		RANK() OVER (PARTITION BY country ORDER BY population) as rank_pop
	from
		cities c)
select
	*
from
	cte
where
	-- filtering by the 'rank_pop' column from the CTE
	rank_pop <= 3
order by country, rank_pop

This query is a bit more complex than the previous ones. It uses a common table expression and a window function (the rank function), and showcases the ability to write complex SQL in queries.

Queries can also make use of dozens of built-in SQLite functions. Various specialized extended functions such as RegexReplace, GPSDist (GPS distance between two points) and LevDist (fuzzy text matching) are also available.

Updating tables

OK, this next example is a bit of a hack, but a useful one… The query you supply doesn’t need to be a SELECT query. You can do UPDATE/INSERT/DELETE statements as well, and these will modify the data in the target Excel tables.

This can be a handy way to clean and transform data in your tables in place, without having to export/import the data to an external database (e.g. SQL Server, MySql, Postgres…).

This works because the SQLite engine isn’t copying the data. Rather, it’s using an adapter that lets it access live data in the Excel table.

How does the function see Excel tables?

At first glance, it might seem strange that the query can access your workbook tables. After all, we did not pass them in as parameters, and functions normally only work with parameters that are passed to them.

However, the Windy.Query function is aware of the workbook it’s being called from and it can read data from the workbook’s tables without the need for passing them in as parameters. This makes the function much easier to call especially when working with multiple tables.

Column Headers

Results returned by the Windy.Query function can optionally include headers. This is controlled by the second parameter of the function.

The texts in the column headers are determined by the SQL query itself. You can easily rename result columns by aliasing them in the select list.

Automatically refresh results

By default, the SQL query runs as a one-off operation when you enter the formula but does not refresh if the source tables change. However, if you want the query to refresh whenever one of the source tables changes, you can easily do so by setting the autoRefresh argument to true.

Note that the auto-refresh functionality relies on Excel’s RTD (Real-Time Data) server. The RTD server usually throttles updates so functions don’t overwhelm Excel with frequent updates. The default throttle interval is 2s meaning that the function will not update more than once every 2s. To improve responsiveness, you can lower this value to something like 20ms. The simplest way to do this is through the “Configure” dialog in the QueryStorm runtime’s ribbon.

Passing parameters

When needed, SQL queries can use values from cells as parameters. To use a cell as a parameter in a query, start by giving the cell a name (named range).

Once the cell has a name, you can reference it in the query using the @paramName or $paramName syntax.

If automatic refresh is turned on, results will automatically refresh whenever one of the parameter cells changes its value.

Performance

This is all well and good for small tables, you might think, but how does it handle large data sets? Well, it handles them quite well. The function can read source tables of 100k rows and 10 columns within a few milliseconds and can return this amount of data in a second or two. In addition to this, all columns are automatically indexed so searches and joins are extremely performant as well.

This makes the function perform very well, both from the data throughput standpoint as well as from the computational one.

OK, so is this better than the Google Sheets version of the QUERY function?

Yes, dah. Did you read the previous chapters? 😛

Installing the Windy.Query function

So how do you install this function into your Excel? It’s a simple 2-step process.

Step 1 is to install the QueryStorm Runtime add-in (if you don’t already have it). This is a free, 4MB add-in for Excel that lets you install and use various extensions for Excel. It’s basically an app store for Excel.

Step 2 is to click the “Extensions” button in the “QueryStorm” tab in the Excel ribbon, find the Windy.Query package in the “Online” tab, and install it.

What happens if I share the workbook with a user who doesn’t have the function?

Nothing bad. If the other user doesn’t have the function installed, they will see the last results of the query that were returned on your machine. They just won’t be able to refresh the results.

Advanced SQL query editor

Writing SQL queries in the formula bar can get a bit unwieldy. To make queries easier to write, it’s better to use a proper editor, preferably one that offers syntax highlighting and code completion for SQL and knows about the tables in your workbook.

For this purpose, I recommend using the QueryStorm IDE. This is an advanced IDE that lets you use SQL in Excel. You can write the query in the QueryStorm code editor and then paste the query into the Windy.Query function when you’re happy with it (if needed).

The IDE does more than just allow using SQL in Excel. You can use it to create and share functions and addins for Excel. In fact the QueryStorm IDE was used to create the Windy.Query function itself.

The IDE has a free community version for individuals and small companies, while users in larger companies can make use of the free trial license. For paid licenses, check out the pricing page.

You can read more in this blog post that’s dedicated to the QueryStorm SQL IDE.

Video demonstration

For a video demonstration of the Windy.Query function, take a look the following video:

Written by Puneet for Excel 2010, Excel 2013, Excel 2016, Excel 2019

If you are one of those people who work with data a lot, you can be anyone (Accountant, HR, Data Analyst, etc.), power query can be your power tool.

Let me come straight to the point, Power Query is one of the advanced Excel skills that you need to learn and in this tutorial, you will be exploring power query in detail and will be learning to transform data with it.

Let’s get started.

What is Excel Power Query

Power Query is an Excel add-in that you can use for ETL. That means, you can extract data from different sources, transform it, and then load it to the worksheet. You can say POWER QUERY is a data cleansing machine as it has all the options to transform the data. It is real-time and records all the steps that you perform.

Why Should You Use Power Query (Benefits)?

If you have this question in your mind, here’s my answer for you:

• Different Data Sources: You can load data into a power query editor from different data sources, like, CSV, TXT, JSON, etc.
• Transform Data Easily: Normally you use formulas and pivot tables for data transformations but with POWER QUERY you can do a lot of things just with clicks.
• It’s Real-Time: Write a query once and you can refresh it every time there is a change in data, and it will transform the new data which you have updated.

Let me share an example:

Imagine you have 100 Excel files that have data from 100 cities and now your boss wants you to create a report with all the data from those 100 files. OKAY, if you decide to open each file manually and copy and paste data from those files and you need at least one hour for this.

But with the power query, you can do it in minutes. Feeling excited? Good.

Further in this tutorial, you will learn how to use Power Query with a lot of examples, but first, you need to understand its concept.

The Concept of Power Query

To learn power query, you need to understand its concept that works in 3 steps:

1. Get Data

Power query allows you to get data from different sources like web, CSV, text files, multiple workbooks from a folder, and a lot of other sources where we can store data.

2. Transform Data

After getting data in the power query you have a whole bunch of options that you can use to transform it and clean it. It creates queries for all the steps you perform (in a sequence one step after another).

3. Load Data

From the power query editor, you can load the transformed data to the worksheet, or you can directly create a pivot table or a pivot chart or create a data connection only.

Where is Power Query (How to Install it)?

Below you can see how to install access to the power query in the different versions of Microsoft Excel.

Excel 2007

If you are using Excel 2007, I’m sorry PQ is not available for this version so you need to upgrade to the latest version of Excel (Excel for Office 365, Excel 2019, Excel 2016, Excel 2013, Excel 2010).

Excel 2010 and Excel 2013

For 2010 and 2013, you need to install an add-in separately which you can download from this link and once you install it, you’ll get a new tab in the Excel ribbon, like below:

• First, download the add-in from here (Microsoft’s Official Website).
• Once you have downloaded the file, open it and follow the instructions.
• After that, you’ll automatically get the “Power Query” tab on your Excel ribbon.

If somehow that “POWER QUERY” tab doesn’t appear, there is no need to worry about it. You can add it using the COM Add-ins option.

• Go to File Tab ➜ Options ➜ Add-ins.

• In “Add-In” options, select “COM Add-ins” and click GO.
• After that, tick mark “Microsoft Power Query for Excel”.

• In the end, click OK.

Excel 2016, 2019, Office 365

If you are using Excel 2016, Excel 2019, or you have OFFICE 365 subscription, it’s already there on the Data tab, as a group named “GET & TRANSFORM” (I like this name, do you?).

Excel Mac

If you are using Excel in Mac I’m afraid that there is no power query add-in for it and you can only refresh an existing query but you can’t create a new one and or even edit a query (LINK).

Power Query Editor

Power Query has its own editor where you can get the data, perform all the steps to create queries, and then load it to the worksheet. To open the power query editor, you need to go to the Data Tab and in the Get & Transform ➜ Get Data ➜ Launch Power Query Editor.

Below is the first look at the editor which you will get when you open it.

Now, let’s explore each section in detail:

1. Ribbon

Let’s look at all the available tabs:

• File: From the file tab, you can load the data, discard the editor, and open the query settings.
• Home: In the HOME Tab, you have options to manage the loaded data, like, delete and move columns and rows.
• Transform: This tab has all the options which you need to transform and clean the data, like merge columns, transpose, etc.
• Add Column: Here you have the option to add new columns to the data you have in the power editor.
• View: From this tab, you can make changes to the view for the power query editor and data loaded.

2. Applied Steps

On the right side of the editor, you have a query setting pane which includes the name of the query and all the applied steps in a sequence.

When you right-click on a step you have a list of options that you can perform, like, rename, delete, edit, move up or down, etc. and when you click on a step, the editor will take you to the transformation done on that step.

Look at the below where you have the total five steps applied and when I click on the 4th step it takes me to step four’s transformation where the columns name hasn’t changed.

3. Queries

The queries pane on the left side lists all the queries you have in the workbook right now. It’s basically one place where you can manage all the queries.

When you right-click on a query name you can see all the options that you can use (copy, delete, duplicate, etc.)

You can also create a new query by simply right click on the blank space on the queries pane and then select the option for the data source.

4. Formula Bar

As I said, whenever you apply a step in the editor it generates M code for that step, and you can see that code in the formula bar. You can simply click on the formula bar to edit the code.

Once you learn to use M code you can also create step by writing the code and simply clicking on the “FX” button to enter a custom step.

5. Data Preview

The data preview area looks like an Excel worksheet but there’s a little different than a normal worksheet where you can edit a cell or data directly. When you load data into the editor (we will do it in a while) it shows all the columns with the headers with the columns name and then rows with data.

At the top of each column, you can see the data type of the data in the column. When you load data into the editor the power query applies the right data type (almost every time) to each column automatically.

You can click on the top left button on the column header to change the data type applied to the column. It has a list of all the data types from where you can.

And on the left side of the column header there you have the filter button which you can use to filter values from the column. Note: When you filter values from a column, the power query takes it as one step and lists it in the applied steps.

If you right-click on the header of the column you can see that there is a menu that includes a list of the options which you can use to transform the data and use any of the options and PQ stores it as a step in the applied steps.

Data Sources for Power Query

The best part of the power query is you have the option to get data from multiple sources and transform that data and then load it into the worksheet. When you click on the Get Data in the GET & TRANSFORM you can see the complete list of data sources that you can get data load into the editor.

Now let’s look at some of the data sources:

• From Table/Range: With this option, you can load data into the power query editor directly from the active worksheet.
• From Workbook: From a different workbook that you have on your computer. You just need to locate that file using an open dialog box and it will get data from that file automatically.
• From Text/CSV: Get data from a text file or a comma-separated file and then you can load it into the worksheet.
• From Folder: It takes all the files from the folder and load data from them into the power query editor. (See this: Combine Excel Files from a Folder).
• From Web: With this option, you get data from a web address, imagine you have a File that is stored on the web or you have a web page from where you need to get the data.

How to Load Data into Power Query Editor

Now let’s learn to load data into the power query editor. Here you have a list of student names and their scores (LINK).

You will be loading data directly from the worksheet, so you need to open the file first and then follow the below steps:

• First, apply an Excel table to the data (Even if you don’t do it Excel will do it for you before loading data into PQ editor).
• Now, select a cell from the table and click on the “From Table/Range” (Data Tab Get & Transform).

• Once you click on the button, Excel confirms the range of data to apply an Excel table to it.

• At this point, you have the data into the power query editor, and it looks something like below.

• Here you can see:
  • In the Formula bar, PQ has generated the M code for the table you have just loaded into the editor.
  • On the left side of the editor, you have the queries pane where you have the list of the queries.
  • On the right side, in the query settings, you have the section called “Applied Steps” where you have all the steps listed. Note: You must be thinking that you haven’t performed any “Changed Type” but there’s a step called “Changed Type” is there. Let me tell you the SMARTNESS of POWER QUERY when you load data into the editor it checks and applies the correct data types for all the columns automatically.

Power Query Examples (Tips and Tricks)

You can learn to perform some of the basic tasks which you normally do with functional formulas in Excel, but with power query, you can do it with a few clicks:

How to Load Data Back to the Worksheet

Once you transform your data, you can load it to the worksheet and use it for further analysis. On the home tab there is a button called “Close and Load” when you click on it you get a drop-down which has options further:

• Close and Load
• Close and Load To

• Once you click on the button, it will show the following options:

• Select how you want to view this data in your worksheet.
  • Table
  • Pivot Table Report:
  • Pivot Chart
  • Only Create Connection
  • Where do you want to put the Data?
    • Existing Worksheet
    • New Worksheet.
  • Add this data to the Data Model.
• Just select the table option and new worksheet and don’t tick mark the data model and click OK.

• The moment you click OK, it adds a new worksheet with the data.

More Examples to Learn

Auto Refresh a Query

From all the examples that I have mentioned here, this one is the most important. When you create a query, you can make it auto-refresh (you can set a timer).

And here are the steps:

• On the Data tab, click on “Queries & Connections” and you’ll get the Queries and Connection pane on the right side of the window.

• Now, right-click on the query and tick mark “Refresh every” and enter the minutes.

How to use a Formula and a Function in Power Query

Just like you can use functions and formulas in Excel worksheet, the power query has its own list of functions that you can use. The basics of function and formulas in power query are the same as Excel’s worksheet functions.

In PQ, you need to add a new custom column to add a function or a formula.

Let’s take an example: In the below data (already in the PQ editor) you have the first name and last name (DOWNLOAD LINK).

Imagine you need to merge both names and create a column for the full name. In this case, you can enter a simple formula to concatenate names from both columns.

• First, go to the Add Column tab and click on the “Custom Column”.

• Now in the custom column dialog box, enter the name of the new column “Full Name” or anything you want to name the new column.

• The custom column formula is the place where you need to enter the formula. So enter the below formula in it:

[First Name]&" "&[Last Name]

• When you enter a formula in the “custom column formula”, PQ verify the formula that you have entered and shows a message “No syntax error have been detected” and if there’s an error it will show an error message based on the type of the error.
• Once you enter the formula and that formula doesn’t have any errors in it, simply press OK.
• Now you have a new column at the end of the data which has values from two columns (first name and the last name).

How to use a Function in Power Query

In the same way, you can also use a function while adding a custom column and Power Query has a huge list of functions that you can use.

Let’s understand how to use a function with an easy and simple example. I’m continuing the above example where we have added a new column by combining the first name and last name.

But now, you need to convert that full name text which you have in that column into the upper case. The function which you can use is “Text.Upper”. As the name suggests, it converts a text to an upper-case text.

• First, go to the add column tab and click on the custom column.

• Now in the custom column dialog box, enter the column name and below formula in the custom column formula box:

Text.Upper([Full Name])

• And when you click OK it creates a new column with all the names in the uppercase.
• The next thing is to delete the old column and rename the new column. So right-click on the first column and select remove.

• In the end, rename the new column his “Full Name”.

There are a total of 700 functions that you can use in power query while adding a new column and here is the complete list provided by Microsoft for these functions, do check them out.

How to Edit a Query in PQ

If you want to make some changes in the query which is already in your workbook you can simply edit it and then make those changes. On the Data tab, there’s a button named Queries and Connections.

When you click on this button, it opens a pane on the right side that lists all the queries that you have in the current workbook.

You can right-click on the query name and select edit and you will get it in the power query editor to edit.

When you edit a query, you can see that all the steps which you have performed earlier are listed in the “Applied Steps” that you can also edit or you can perform new steps.

And once you are done with your changes you can simply click on the “Close & Load” button.

Export and Import Connections

If you have a connection which you have used for a query and now you want to share that connection with someone else, you can export that connection as an odc file.

On the query table, there’s a button called “Export Connection” and when you click on it, it allows you to save that query’s connection in your system.

And if you want to import a connection that is shared by someone else, you can simply go to the Data tab and in the Get & Transform click on the existing connections.

And then click on the “Browse for More” button from where you can locate the connection file which has been shared with you and import it to your workbook.

Power Query Language (M Code)

As I mentioned earlier that for every step you performed in power query it generates a code (at the backend) which is called M Code. On the Home tab, there is a button called “Advanced Editor” which you can use to see the code.

And when you click on the advanced editor it will show you the code editor and that code looks something like below:

M is a case sensitive language and like all the other languages it uses variables and expressions. The basic structure of code looks like below where the code starts with the LET expression.

In this code, we have two variables and the values defined to them. In the end, to get the value, IN expression has been used. Now when you click OK it will return the value assigned to the variable “Variablename” in the result.

Check out this resource to learn more about Power Query Language.

In the End

You can’t afford to avoid the POWER QUERY. If you think like this, a lot of things which we do with Excel functions or VBA codes can be automated using it, and I’m sure this tutorial inspires you to use it more and more.

But now you need to tell me one thing. Which thing do you like most about the POWER QUERY?

You must check out these tutorials

• Create a Pivot Table from Multiple Files