Sort Your Pivot Table by Using Slicer

Sometimes different users want pivot table sorted in different ways. If column has full names of people, it is possible to sort column either by using first names or by using last names. An additional requirement is the ability to change the sort order using a slicer.

In the example bellow, we can see that our pivot table can be sorted by "Points" (1), but two other possible sorting orders are by "Last Name" (2) or by "First Name" (3). We will create solution only for this three specific sorting methods. Points will be sorted from highest to lowest, and first and last names will be sorted alphabetically.

Helper Tables

First we have to create table in Power Pivot that will hold values for our slicer (1). We will call this table "SlicerValues".
Second, we have to create table in Power Pivot, that will have columns with sort order for every row (1). We need two such columns, one for last names and one for first names. This table will have name "SortOrdersTable".

General Idea

We know that it is possible to sort some column, in pivot table, by using some other column as sort order. First we have to right click on column which we want to sort, and then we go to Sort > More Sort Options (1). New dialog will open. In this dialog we will select "Descending (A to Z) by:" and then we will choose which column will define our sort order. In the image bellow, we decided that "Sum of Points" column (2) in DESC order is used for sorting. Now, "Full Name" column (3) is not sorted alphabetically, but according to "Sum of Points" column.

We can use mechanism, explained above, to control column "Full Name" sorting order. Idea is to create a measure, with a name "SortMeasure", that will be added to our pivot table (1). This measure values will change according to selection in the slicer (3,4,5). Our column "Full Name" will be set to sort by order defined by "SortMeasure" (2).

Now, when the user selects "First Names" (3), "Last Names" (4) or "Points" (5) in the slicer, column "SortMeasure" values will change (1) and that will sort our column "Full Name" (2). We can see in (3) that column "Full Name" is sorted by first name, in (4) by last name, and in (5) by points. At the end, column "SortMeasure" will be hidden from the user by hiding the whole spreadsheet column.

Creative process

First we have to make relation between "Full Name" columns in "SortOrdersTable" and "Data" table (1). "Data" table is the name of a table which is the base for our pivot (with "Full Name" and "Points" columns). Relation has to be "1:N" in direction "Data" to "SortOrdersTable" (2). This means that in relation dialog "Data" table has to be at the bottom. "Data" table can filter "SortOrdersTable".

Next step is to create our measure. This measure has to be reactive to slicer selection. ALLSELECTED function will return values selected in slicer as a column. With green part of code we will check whether specific individual values in slicer are part of that column. If they are, that means that those terms are selected and we will return TRUE(), otherwise we will return FALSE(). At the end, we will use SWITCH function to decide what values the measure should return. SWITCH function will return values for the first variable ( FirstNamesSelected, LastNamesSelected, PointSelected ) that returns TRUE(). For example, if FirstNamesSelected is FALSE(), and LastNamesSelected is TRUE(), values from "Last Name order" column will be returned. Those are the values that will be used to sort "Full Name" column.

SortMeasure:=
VAR FirstNamesSelected = IF( "First Names" IN ALLSELECTED( SlicerValues[SortBy] ); TRUE(); FALSE() )
VAR LastNamesSelected = IF( "Last Names" IN ALLSELECTED( SlicerValues[SortBy] ); TRUE(); FALSE() )
VAR PointsSelected = IF( "Points" IN ALLSELECTED( SlicerValues[SortBy] ); TRUE(); FALSE() )
VAR Result = SWITCH( TRUE(); FirstNamesSelected; SUM( SortOrdersTable[First Name order] ) 
                           ; LastNamesSelected; SUM( SortOrdersTable[Last Name order] )
                           ; PointsSelected; 10000 - SUM( Data[Points] ))
RETURN Result

All columns with possible result ( "First Name order", "Last Name order", "Points" ) are just wrapped in SUM function, but "Points" column has one more detail. In our measure we will transform all "Points" values into "10000 – Points". This is because, we want ascending sorting for "First Name order" and "Last Name order", but descending sorting for "Points". Because pivot table can only be in ascending or descending order at once, we need this transformation so that ascending ordering of our measure will always correctly sort our pivot table. Values of "Points" and "10000 – Points" are inversely correlated, and this solves our problem.

Descending sorting of "Points" is the same as ascending sorting of a transformation "10000 – Points".

Result

Bellow we can see our results and we can observe a few details. Columns "C" and "G" are hidden (1). This is how we will hide our "SortMeasure" column from the user. In green pivot table (2) we can see that while "SortMeasure" column is in ascending order, important column "Sum of Points" is actually in descending order, as per requirement.

In blue and orange pivot tables, there are two values selected in each slicer (3). It is not possible to force single selection in slicer in Excel without resorting to VBA. To solve this problem, we made our measure so that the first selected value is the one determining sorting order. That is why both blue and orange pivot tables are sorted by "First Names", that is the first value selected in their slicers.

Sample table can be downloaded from here:

Analysis Toolpak and Descriptive Statistics

Analysis ToolPak is an Excel add-in that provides many statistical tools. This add-in comes in two flavors, and each of them has to be enabled in "Add-ins" dialog (1) in order to use it. They both have the same functionality but "Analysis ToolPak" has its functionality available only through graphical interface. "Analysis ToolPak – VBA" is accessible both through graphical interface and VBA code.

After we enable "Analysis ToolPak" (1), a button will appear in Data > Analysis (2). By clicking on that button, we will open dialog with many statistical tools (3). In order to use any of these tools we just have to select it and click on OK. New dialog will appear which provides options for that specific statistical tool.

Descriptive Statistics

Descriptive statistics is based on a brief descriptive coefficients that quantitatively describe or summarize features of a data set. Data set is set of results of measurements made on a population or a sample. This coefficients comprise measures of central tendency like mean, median, mode, and measures of variability like Standard error, Variance, Range, Kurtosis.

We will select "Descriptive statistics" tool in Analysis ToolPak. "Descriptive statistics" dialog asks us to enter what range will be used as a source of data (1). This range, in our example, has two columns and we will get results for each of them. Program doesn't know whether data in selected range is organized in Columns or Rows. We will choose Columns (2) in our case, to solve that dilemma. Our data has labels in the first row, so we have to check option (3) in order to have first row excluded from calculations.

Final output can be directed to a range in the same sheet or some other sheet in the same workbook (A). This range is defined by it's top left cell. Option (B) means that new sheet will be created with provided name and result will be pasted in "A1" cell in that new sheet. Selection of option (C) will create new workbook with the name "Book1", and the result will be pasted in sheet "Sheet1" in cell "A1" in that workbook.

Last options (4,5,6,7) are for selecting what results should be calculated. Option (4) "Summary statistics" should always be selected because without it most of important indicators (mean, variance…) will not be part of a final result. Option (5) needs confidence level for calculating confidence interval for the mean. Without it checked, we will not get this result. Options (6,7) will give us the same results as SMALL and LARGE functions in Excel.

Descriptive Statistics Results

This is what the results of descriptive statistics would look like. We have results both for Col1 and Col2. For each column there is a list of different indicators which describe our data set. We will only analyze results for the first column.

To the right, we can see all the results for first column, with the same results calculated by using Excel formulas. We can see that all the results are the same but there is one potential difference. If there are multiple values in data set that could be considered as Mode ( e.g. 1,1,2,5,5 ), Analysis ToolPak would return the last one ( 5,5 ), but the MODE function would return the first one ( 1,1 ).

Formulas used in Analysis ToolPak descriptive statistics

Bellow are listed formulas used for calculation of some descriptive indicators.

Standard DeviationVariance
Standard ErrorConfidence Interval
KurtosisSkewness

Let's calculate Confidence Interval by using its formula. Because our sample is smaller than 30 observations, we will use t distribution. For 95% confidence level, with 3 degrees of freedom, t value would be 3.1824. That means that t * s = 3.1824 * 0,75 = 2.3868. This is the same result as Excel CONFIDENCE.T function is returning. Now our confidence interval is ( 2,75 – 2,3868; 2,75 + 2,3868 ), which is ( 0,3632; 5,1368 ).

Kurtosis and Skewness are used to describe how much our distribution fits into normal distribution. If Kurtosis and Skewness are zero, or close to zero, then we have normal distribution.

Analysis ToolPak Descriptive Statistics through VBA

Descr Subprocedure

Different functionalities of Analysis ToolPak can be called through VBA subprocedures. Bellow we can see how to provide arguments and call procedure with the name "Descr" for descriptive statistics. All arguments except the first one are optional. All arguments are of a type Variant.

Sub subDescrFunction()
Dim inputRng As Range: Set inputRng = Range("Sheet1!A3:B7")
Dim outputRng As Range: Set outputRng = Range("Sheet1!K3")
Application.Run "Descr", inputRng, outputRng, "C", True, True, 1, 1, 95
End Sub
inprng This is input range, with our data set.
outrngThis is output range. Output range is declared with a reference to a top left cell of the output range. If we type some word here, Analysis Toolpak will export result into new worksheet with such name. If this argument is False, blank or omitted, new workbook will be created.
groupedIs our data set organized in rows or columns? Depending on that, we type "R" or "C". "C" is default.
labelsHere we should type "True" if our data set has header. If False or omitted, then we claim that there are no labels.
summaryType "True" to get all major descriptive statistics. If this argument is False, blank or omitted, those values will not be calculated.
ds_largeWhat k-th largest value to get? If this argument is omitted, this value will not be calculated.
ds_smallWhat k-th smallest value to get? If this argument is omitted, this value will not be calculated.
confidWhat is confidence level for calculating confidence interval for the mean? If this argument is omitted, there will be no result.

VBA project for Analysis ToolPak (ATPVBAEN.XLAM) is protected with the password "Wildebeest!!". There you can find declarations of all VBA procedures.

DescrQ Subprocedure

There are two variants of subprocedures for descriptive statistics. First has name Descr, and the second one has name DescrQ. "Descr" would create result immediately by using arguments provided in the code. "DescrQ" would first open GUI dialog filled with values provided in the code (1). We can then accept those values, or we can change them, and then we can click on OK button. Only then, final results will be created (2). "DescrQ" give us opportunity to customize values provided in the code.

All the code for "Descr" and "DescrQ" is the same. Only difference is the name of subprocedure.

Application.Run "DescrQ", inputRng, outputRng, "C", False, True, 1, 1, 95
"DescrQ" version has one bug. Ranges in the dialog will have starting "grave accent" missing.
[New Microsoft Excel Worksheet.xlsm]Sheet1'!$D$9:$D$12 'initial formatting
'[New Microsoft Excel Worksheet.xlsm]Sheet1'!$D$9:$D$12 'corrected starting sign

Sample file can be downloaded here:

Power Query and protected sheet in Excel

We made a report. We'll give that report to some colleague. He will refresh that report periodically and he will then send it to end users. In the image bellow we can see that our colleague has to prepare 4 Access files as data source, and to open our report (1). He will than refresh it (2). At the end of refreshing MsgBox (3) will appear. After this, our colleague can save the file and he can then send it to other colleagues (4). Idea is to make this process as much easier as possible for our colleague.

One more requirement is that spreadsheet with tables has to be protected so that users can not accidentally change or delete some numbers. When we try to make report, as described, we will find few problems to do so:
1) Tables on protected sheet will not be able to refresh.
2) Combining Power Query and VBA asks for queries to be synchronous, which will prolong queries execution.
3) If we can not use VBA, how to create MsgBox (3)?
Let's see how to solve this problems.

Tables on Protected Sheet Will Not Refresh

When we try to "Refresh All" tables that are located in protected sheet, this is the message we will get. We will have to remove our protection before refreshing our data, and then to protect the sheet again after. Idea is to avoid such step and to automate things.

We could try to solve this problem by using VBA. Usually we would lift the protection, do the changes, and then set protection back. We can use VBA code similar to this bellow:

Sub RefreshTables()
ThisWorkbook.Worksheets("Queries").Unprotect Password:="fff"
ThisWorkbook.RefreshAll
ThisWorkbook.Worksheets("Queries").Protect Password:="fff"
MsgBox "All Tables Refreshed"
End Sub

Problem is that Power Query queries are executed asynchronously. Queries will still be executing (2), VBA will not wait for them to finish, but will show final MsgBox immediately (1). This means that protection on the sheet is already placed. After we click on OK button (1), we will again receive message (3) because of that.

As explained in this post, all we need to do is to make queries synchronous. This is done by unchecking option "Enable background refresh" in query properties.
Unfortunately this would make Refreshing lasting much longer. Instead of executing all of the queries at the same time (1), queries would be executed sequentially, one by one (2). This makes refreshing lasting two times longer, on my computer.
We can try to protect our cells by making some data validation rule that will never be fulfilled. This would prevent users from overwriting numbers in our cells, but it will not stop them from deleting values in our cells. Data validation is only triggered when cell is in Edit mode and the user is trying to enter some data into cell. Edit mode is not needed for deleting cell content, we just select the cell and press Delete key.

Solution for protected sheet

Best solution is to place refreshable tables in one sheet that is unprotected and hidden (2). Numbers from that sheet can be mirrored (3), by using formulas, in another sheet which is protected (4). Cells in protected sheet always have the same, unchanged formulas (5), so they will never trigger sheet protection.

User can now click on Refresh All button in Data tab (1). Tables in unprotected sheets would refresh, and tables in protected sheet would just reflect those results.

MsgBox without VBA

We said that synchronised queries would take longer to execute, so we will avoid VBA. Question is how to create MsgBox at the end of refreshing that would inform user that refreshing is over. The only way to create message in Power Query is by using "error" statement. We have to make sure that "error" statement happens at the end of refreshing when all individual queries are finished.

We can create a query like the one bellow. This query is referencing all other queries. This means that the query bellow has to be executed the last. At the end of refreshing, this query will show user the message that is written in Error.Record. This query has to be loaded into spreadsheet in order to work correctly.

let   
  Result = if { FactSales1, FactSales2, FactSales3, FactSales4 } = { "A","A","A","A" } 
    then "A" else error Error.Record( "Finished", "Refreshing is over." )
in
    Result

We can see bellow that the final query is dependent on all other queries (1). This MsgBox query will never load any data into spreadsheet, so its footprint will be like (2). This query will show us a message that looks like (3) after all other queries are refreshed.

Sample files can be downloaded from here. File "ContosoSales1.accdb" should be copied three more times to create files "ContosoSales2.accdb", "ContosoSales3.accdb", "ContosoSales4.accdb". In "Parameters" Excel sheet, change fullpaths toward those MS Access files.

Data Types in Power Query

BuiltIn Types

Power Query has built-in record with identifier #shared. This record will give us all identifiers that exist in Power Query in current context, and their types. We will transform this record into table and we will then remove this query itself from the list of identifiers, in order to avoid cyclic reference. We will filter only identifiers that represent some of Power Query types. At the end we will sort by type name.

let
    IdentifiersAndTypes = Record.ToTable( #shared )   
    , RemoveItself = Table.SelectRows(IdentifiersAndTypes, each [Name] <> "Query1" )    
    , FilterOnlyTypes = Table.SelectRows( RemoveItself, each Value.Type( [Value] ) = Type.Type )   
    , SortTypes = Table.Sort( FilterOnlyTypes, { "Name", Order.Ascending })
in
    SortTypes
#Shared (1) returned all identifiers in Power Query, together with their types. We will delete (2) the name of our query from this list in order to avoid cyclic reference. We will filter only identifiers that refer to Power Query types (3).

Result will show us, that there are 63. built in types in Power Query. They are presented below.

AccessControlEntry.ConditionContextTypeCompression.TypeGuid.TypeMissingField.TypeRelativePosition.Type
AccessControlEntry.TypeCsvStyle.TypeIdentity.TypeNone.TypeRoundingMode.Type
AccessControlKind.TypeCurrency.TypeIdentityProvider.TypeNull.TypeSingle.Type
SapHanaDistribution.TypeDate.TypeInt16.TypeNumber.TypeTable.Type
SapHanaRangeOperator.TypeDateTime.TypeInt32.TypeODataOmitValues.TypeText.Type
SapBusinessWarehouseExecutionMode.TypeDateTimeZone.TypeInt64.TypeOccurrence.TypeTextEncoding.Type
Any.TypeDay.TypeInt8.TypeOrder.TypeTime.Type
Binary.TypeDecimal.TypeJoinAlgorithm.TypePassword.TypeTraceLevel.Type
BinaryEncoding.TypeDouble.TypeJoinKind.TypePercentage.TypeType.Type
BinaryOccurrence.TypeDuration.TypeJoinSide.TypePercentileMode.TypeUri.Type
Byte.TypeExtraValues.TypeLimitClauseKind.TypePrecision.TypeWebMethod.Type
ByteOrder.TypeFunction.TypeList.TypeQuoteStyle.Type
Character.TypeGroupKind.TypeLogical.TypeRecord.Type

Primitive types

Most of those types above are so called "ascribed types". It means that we can declare them, Power Query will remember them, but it will not enforce them. Power Query only enforce primitive types. This is list of primitive types in Power Query:

type anytype anynonnulltype binarytype datetype datetimetype datetimezone
type durationtype functiontype listtype logicaltype nulltype none
type numbertype recordtype tabletype texttype timetype type
If we try to define a function, arguments can only be declared as primitive types. We can see on the image that only primitive types any, binary, text are colored green by intelisense (1). Even the longer syntax like "Any.Type" is not acceptable. All other types (2) are incorrect and can not be used.
It is the same if we want to use "is" operator. This operator will only work with primitive types ( number is green by intelisense (1) ). Even the longer syntax "Number.Type" will not work. We have to type exactly "number".
If we select the cell with "Decimal.Type" type in our #shared table, we will see in graphic interface that this type is actually of "number" type (1). It is the same for many other types "Double.Type", "Int16.Type", "Int64.Type", "Percentage.Type" etc., "number" is their real type.

We can create a function that will return primitive type for each of our 63. built in types in Power Query. This is the function:

( TypeToName as type ) as text =>
let
      ListOfTypes = { type any, type anynonnull, type binary, type date, type datetime
        , type datetimezone, type duration, type function, type list, type logical
        , type null, type number, type record, type table, type text, type time, type type }
    , ListOfTypeNames = { "type any", "type anynonnull", "type binary", "type date"
        , "type datetime", "type datetimezone", "type duration", "type function", "type list"
        , "type logical", "type null", "type number", "type record", "type table", "type text"
        , "type time", "type type" }
    , ZipedTypes = List.Zip( { ListOfTypes, ListOfTypeNames } )
    , NameForType = List.Select( ZipedTypes, each  Type.Is( _{0}, TypeToName ) ){0}{1}
in
    NameForType
By using the function above, we will create column "Primitive types". We can see that for each built in type (1), we can get one primitive type (2). This is true for 59/63 built in types. For some built in types (3), we will get Error. In graphic interface we can see that such types are either record or function (4).

The reason for Error in image above is in the fact that table, function and record types are abstract types. There is no value that has type of table, function of record. We can create one table, record and function and compare their types with such abstract types, but the result will never be TRUE.

let
      Table = #table( { "Column" }, { { "Value" } } ) 
    , Record = [Field="Value"]
    , Function = () => let Result = "Value" in Result    
    , IsTableType = Value.Type( Table ) = type table
    , IsRecordType = Value.Type( Record ) = type record 
    , IsFunctionType = Value.Type( Function ) = type function 
    , IsTableTypeCompatible = Type.Is( Value.Type( Table ), type table )
in
    [ IsTableType=IsTableType, IsRecordType=IsRecordType
    , IsFunctionType=IsFunctionType, IsTableTypeCompatible=IsTableTypeCompatible ]
We can see that equality will never be reached (1). But if we use "Type.Is" function to check compatibility with primitive types we will get TRUE. Compatibility means that any table is compatible with the "type table". The same is true for records and functions.

When we declare type for function arguments, that argument will accept any value that is compatible with our declaration. If our argument is of type "number", such argument can receive values of all the compatible types "Double.Type", "Int16.Type", "Int64.Type", "Percentage.Type" etc.

How to check ascribed type

We will create three values of type Int64.Type, Decimal.Type and Text.Type. The question is how to read ascribed types of this values. We can do that from "metadata".

let    
      Int64Type = Value.ReplaceType( 99, Int64.Type )
    , DecimalType = Value.ReplaceType( 99.99, Decimal.Type )
    , TextType = Value.ReplaceType( "hundred", Text.Type )
    , OneTable = #table( { "values" }, { { Int64Type }, { DecimalType }, { TextType } } )
    , AddMetadata = Table.AddColumn( OneTable, "Metadata"
        , each Value.Metadata( Value.Type( [values] ) ) )
    , AddAscribedTypeName = Table.AddColumn( AddMetadata, "AscribedTypeName"
        , each [Metadata][Documentation.Name] )
in
    AddAscribedTypeName 
Value.Metadata function will return record which has Documentation.Name fild which contains ascribed type name. This is where Power Query store information about ascribed types.

Sample file can be downloaded here:

Today() function is slowing down Excel

Today() function is one of volatile functions in Excel. Volatile functions are recalculated each time any cell in the spreadsheet is changed. Some other actions can also cause recalculation, such as renaming sheets, inserting columns, deleting rows, etc. If you have many cells in Spreadsheet that are referring to cell that contains Today() function or you have many cells using Today() function, everything in your file will slow down. Constant recalculations will make working in such file really unpleasant experience.

Today() function will change its result only when the new day arrives. Because of this we can find another way to get today's date. There are actually several ways to accomplish this.

Offline Solutions

VBA Solution

In VBA, if we place "Application.Volatile" as the first line in our UDF (User Define Function), that function will become volatile. We will not do that. That way we can make VBA UDF function that is not volatile and it is replacement for Excel today() function. This function will refresh itself each time we open the file.

Function vbaToday() As Date
vbaToday = Date
End Function
User now just have to type
"=vbaToday()"
into spreadsheet and he will get
today's date inside that cell.

Power Query Solution

We will first make a query that only returns today's date.

let
Source = #table( type table [ #"pqToday"=date ]
, { { DateTime.Date(DateTime.LocalNow()) } } )
in
Source
We will load this query into
spreadsheet so that we have
today's date in cell A3.

We will then set the option so that query is refreshed every time the file is opened.

In the pane with the queries (1), we should right click on our query and choose its Properties (2). In the new dialog we want to check option "Refresh data when opening the file" (3).

Possibility of error

There is a small problem with VBA and Power Query solutions. If we open the file just before a midnight, when the midnight pass, new day will arrive, but our date will not change. For VBA solution, we would need to enter the cell with F2 and press Enter ( F9 and Calculate Now would not work on UDF function ). For Power Query solution we would need to Refresh our query.

Closing and opening the file would also give us new date, or we can just turn off our computer and go to sleep before midnight.

Online Solution

Power Automate Solution

VBA and Power Query will not work for Excel online. We can use Power Automate to change date in one cell every day in midnight. We will combine Power Query solution and Power Automate solution, so that it doesn't matter if file is in cloud or on user's computer. First, we will change our query because Power Automate needs one more column. We are going to add "Key" column to our Power Query table.

let
Source = #table( type table [ #"Key"=text, #"pqToday"=date ]
, { { "paDate", DateTime.Date(DateTime.LocalNow()) } } )
in
Source

Now we can create flow that will change date in cell B3 every day in midnight.

We click on "Create" button (1) for the new flow. Then we select "Scheduled cloud flow" (2). We will be offered dialog where we give our flow name (3), we can decide that from now, each day the flow will run (4,5). Next, we will get diagram view with "Recurrence" as the first step (6). As a next step we will add "Update a row" (7). Now that we have both our steps (8), we can fill necessary data.

In Recurrence step we just have to show advanced options (9) and decide what time zone should be used (10). In "Update a row" step we have to write where is our file and what is its name (11,12). (13) is the name of the declared table in which we are going to insert today's date. "Key" column is considered as primary key column so we have to give a value for primary key which defines row where we want changes to happen (14).


Name of declared table is "pqToday".

That row is made of fields. Those fields will appear at the bottom (15). We have only one field to fill. In this field we will write:
convertFromUtc(utcNow(),'Central Europe Standard Time','dd.MM.yyyy').
This code will take current date according to given time zone and it will format it. This date will be written to our Excel file in cell B3.

For some reason step "Update a row" wasn't able to find my Excel file when I was using XLSM extension. It seams that file has to be XLSX.


Sample file can be downloaded from here: