Skip to main content

How to combine multiple files with Power Query (with no VBA and just 10 mouse clicks!)

The need to combine information from multiple files is one that most users of Excel will have come across at some point in the use of Excel. I've personally spent far too many hours aggregating data from multiple files, that are identical in structure, so that I can analyse larger datasets and provide insights into products and processes.

For anyone who has also done this and not yet discovered Power Query you'll probably be amazed how simple the process has become. I realise there might be some who will say "just use VBA, its easy once you learn how to code..." and they would be right. The method using Power Query provides a zero code solution that is an evolution of the Excel interface that many will already be familiar with.

In this example, I've created a sample file and created a number of duplicates of the file which I've saved in a folder. The folder contains only these files and i'd recommend you do the same if you're looking to try out this process. It is possible to use an existing folder with other files within it but this method is simpler.


Once you have something similar the process you need to carry out is as follows:
  • Open PowerBI or Excel, both of which contain Power Query. I'll be using PowerBI for this demo.
  • Click on the button labelled 'Get Data', then 'more' and then when the following window appears, select 'Folder' and then click 'Connect'
  • Then in the next window, enter the folder name which contains the files you'd like to combine. If you would prefer to navigate to the folder, use the 'browse' button, find the folder and click 'ok' and then click 'ok in the window shown below
  • The following window will display the files identified in the folder and some options to load the files. In this example, we'll set the files to 'combine & load', this method will automatically combine the files and then load them to the data model.

  • The next screen allows you to select the information you wish to combine using one of the files as a sample. If your data is a table, select the table from the option on the left. If your data is a sheet, select the sheet and then click 'Ok'.
  • Once the above sequence has completed, the combined data will be added to the data model.in the example I've created, it is listed in the fields section of Power BI as shown here

  • Now that the files are combines, one method to confirm the data has been loaded is to create a table showing the number of records for each file, this can be done using the 'table' visualisation and using the 'Source.name' value which is the filename as shown below
  • An additional way to check the data is present and correct is to view it using the 'data' view located in the left hand section of the window, like this



And that's it!





Comments

Popular posts from this blog

Extracting data from Word (.docx) files into Power Query

Word and Excel don’t usually get along too well so it's no surprise that Power Query isn't directly compatible with its estranged cousin Word either. If you are presented with the need to import data from Word into Power Query you'll be please to hear it is possible however it requires a couple of manual steps to make it work.  The manual steps could fairly easily be completed by a batch file which would automate the process further. Here is the Excel data pasted 'as values' in a Word file which i'll use for the first example Here is the Excel data pasted with 'keep source formatting' which i'll reference a couple of times in the article. Although the steps I've covered below aren't complex, this whole process has some unknowns around it so you may find the result in your instance varies from mine. The Word file I've used contains the contents of a range of excel cells that I deliberately pasted as values into Word to create a test file f

How to automate the import of all files in a Google Drive folder to PowerBI, now updated, please read the first paragraphs!! (PowerQuery)

Update!!! This method no longer works although there is a new Google Sheets connector for PowerQuery that is currently in beta. If you're using PowerBI, you'll need to enable the preview features to enable it. https://docs.microsoft.com/en-us/power-query/connectors/googlesheets In the interest if demonstrating how it 'did' work, the original post is provided below. If you attempt to replicate this, you'll quickly realise the website doesn't behave as it used to. (here is the original article)  Using PowerQuery to access multiple files within the same folder on a local or network drive is a game-changing feature that will almost certainly save many people hours of effort. This functionality is great if your data exists in a place that is easily accessible but what do you do if your data is somewhere less accessible like Sharepoint, OneDrive or even Google Docs? I have previously connected to data on SharePoint and found it fairly straight forward which raises the