Explanations: Identify Reasons for Changes in Your Data
When you see a spike or sudden decrease in your data, you want to find the root causes that contributed to that behavior, to help you react accordingly and plan ahead. Manually seeking the key drivers for a change is time consuming and ineffective. You would need to drill down into different dimensions, or meet with people in your organization to find the reasons for the behavior in your data. You would also have to come up with hypotheses for the explanations you want to investigate further. Aside from the time this takes, this method is error-prone and there may be things that you have not thought about.
The Explanations feature (available in Sisense L8.2.6 and later) automatically identifies the most probable key drivers for a given change in a time series. It covers hundreds of possible contributing factors and presents the top five dimensions (or combinations of dimensions) that contributed most significantly to the change in behavior between the selected point and a previous point. You can explore these dimensions to see what effect they had on your data. This helps you get the answers you need inside Sisense and enables decision makers to easily understand connections and root causes in their KPIs.
Knowledge Graph recommendations rely on past usage (the dimensions being queried with the explained measure, sometime in the past). When there are not enough suggestions from the Knowledge Graph, Explanations now uses the whole data model as a source of possible explanations, This yields results, even when there is little usage history.
For a short demonstration of Explanations, see the video below:
How it Works
Explanations analyzes all possible factors (dimensions or combinations of dimensions) that have influenced the change in a selected point in a time series, compared to a previous point in time. Explanations then displays the most probable explaining dimensions, ranked by their contribution to the change in the selected point.
As indicated in the example, below, using textual descriptions (A) and automatically generated visualizations (B), Explanations tells you which dimension(s) drove the behavior (C), and within each dimension, how much each member contributed (B). Sometimes, Explanations identifies several possible contributing factors that could explain an anomaly in your data. Since you know your data and your business best, Explanations displays these multiple possibilities, ranked by their calculated contribution. You can then evaluate them and make decisions based on the one that makes the most sense in the context of your business. For example, in the image below, Explanations identified “Gender”, “Age Range”, “Category”, and also the combination “Age Range & Gender” as possible explanations for the 910% increase in revenue.
Sisense provides either one or two visualizations that enable you to investigate the data from different perspectives: "View by Value Distribution" and "View by % Change". For more information about these visualizations, see Using Explanations, below.
You can understand the Explanation “Category” in this example, as follows:
The business owner is attempting to understand the causes for a sudden increase in the business’s total revenue. Explanations found that in relatively high probability, the identity of the Categories that the business sold that month (cell phones, PDAs, tablets, etc.) had an effect on the revenue increase.
Explanations found that the sales in two categories - PDAs and cell phones - accounted for most of the total increase in revenue.
However, the score for the Explanation “Category” shows that it isn't the strongest Explanation (it isn't the largest contributor to the change in Total Revenue between August and September 2012.). In this case, the Gender and Age Range of the buyers seemed to have a much higher contribution to the September revenue increase.
Which Dimensions Appear as Possible Explanations?
For a dimension to appear as an explanation, it needs to have shown a connection to the examined measure, somewhere across the data model (queried with that measure in any widget in any dashboard built on the data model). If your environment includes a large number of widgets, which contain queries of the measure with a variety of relevant dimensions, then you don’t need to prepare anything in advance for Explanations to work. Explanations will rely on the past knowledge that has been accumulated across your system.
If your Sisense environment includes an especially small number of widgets (for example, in case of a new deployment), and you see that certain dimensions you expected to see don’t appear as explanations, it could be because they were never queried with the examined measure. Try including them in a widget together with the measure you wish to explain.
Looking for Explanations Outside the Dashboard
Explanations leverages Sisense’s Knowledge Graph to identify relevant dimensions and to pull them into the analysis, even if they aren't included in the dashboard. This ensures that your Viewers don't miss possible explaining dimensions, even if you didn't think to add them in advance. It also allows you to keep your dashboards succinct, fast, and focused, not needing to add unnecessary information to them.
Security and Privacy
Explanations respects your security and privacy. Data is fully encrypted and sensitive data isn't saved to the cloud. Furthermore, for security and privacy, data isn't saved in the logs.
Sisense has row-level security and Viewers will only see the members of fields that they're authorized to see. You can decide if Viewers can see Explanations based on all fields within the Data Model (this is the default), or only based on fields in the current dashboard. For more information, see Changing Explanations Settings.
Since data is sent to the cloud service, Explanations requires that your Sisense server has Internet access.
For Explanations to work, make sure that the following addresses are whitelisted on your network:
Sisense views your security very seriously and takes the following steps to limit your data exposure:
- Only the time-value series and measure are sent to the cloud, and this is done using the https protocol
- Dimension members are encrypted before they're sent to the cloud
- The data isn't persisted in the cloud (neither the original data, nor the processed data)
- Explanations algorithms are stored on the Sisense cloud service, which is hosted securely on AWS
- The data isn't identifiable to your company
- The Explanations service only uses Sisense code, and doesn't use third-party web services
- The Sisense Administrator can deactivate this feature at any time
- Enabling Explanations will make it available by default on all applicable widgets in all dashboards.
- Explanations may include all dimensions from across all dashboards in the data model. You can change this and limit Explanations to include only fields that appear in the specific dashboard. For more information, see Changing Explanations Settings.
To enable Explanations in your system:
From the Admin tab, select Feature Management.
Toggle on Explanations.
- Explanations works optimally on measures that are based on the aggregate functions SUM and COUNT ALL (DUPCOUNT). Formulas with those aggregation functions with the mathematical operation addition (+) can also be used.
- Measures based on other formulas, such as: AVERAGE, MEDIAN, COUNT, Quotients (results of a division of two numbers), aren't supported.
- If your measure is based on a custom formula, this is how you can check if your formula is supported and whether Explanations will provide meaningful results:. The aggregate function or formula used should meet the following distributive property: Suppose the data are partitioned into n groups (break bys). Apply the aggregation function or formula to each partition, resulting in n aggregate values. If the result derived by applying the aggregate function SUM to the n aggregate values is the same as that derived by applying the function or formula on the entire data set, without partitioning, the aggregate function or formula can be used in Explanations.
Working with Explanations
Generating Explanations from a Widget
There are two methods for generating Explanations for a given point:
- Directly from a point in a Time Series
- In a time-series widget, right-click the point you would like Explanations for.
- Select Explanations.
- From the Widget Menu
- In a time-series widget, click Analyze It > Explanations.
- If there are multiple metrics in the widget, select the one you would like to generate an Explanation for.
- On the graph, click the point you would like to generate an Explanation for.
The Explanations screen shows the graph that you selected. Anomalies and Breaking Points are highlighted for you, to help you find interesting points worth exploring.
- Select a point on the graph to investigate (A).
- (Optional) Change the comparison time period, if required (B). By default, Explanations compares the selected point to the previous point in time. However, you can be very flexible in the periods that you compare a data point to. For example, you can compare the data point to a different period, such as the same month last quarter, or the same month last year). The available time periods are the ones that appear in the widget.
- Review the Possible Explanations (C). The Possible Explanations are given a score, calculated based on:
- The relative contribution of certain members (values) of that dimension to the change (increase or decrease)
- The growth rates of the largest contributors (the % change in their values between the two time periods).
Keep in mind that, depending on your data, the Possible Explanation with the highest score might not always be the most suitable. You'll usually know, based on context, which explanations make the most sense in your case.
You can delete a Possible Explanation, if you like. The Sisense AI will learn from this action and provide more appropriate Possible Explanations in the future.
For each Explanation, you'll see either one or two visualizations. Move between them by clicking the yellow button at the bottom:
View by Value Distribution : This visualization helps you see the changes in the values of each member between the previous period and the selected data point. This helps you see which members accounted for most of the total change.
View by % Change : This visualization helps you see how much each member (value) has changed as a percentage, compared to its previous value.
Identifying Anomalies and Breaking Points
Most of the time, interesting points to explore are easy to see, such as a sudden increase or decrease in a given metric. However, sometimes the most significant changes aren't so obvious, especially in a noisy time series. This is why Explanations helps you by identifying anomalies and breaking points in your data.
- Anomalies : A data point that differs significantly from the rest of the points. An anomaly might indicate an unusual occurrence.
- Breaking Point : A data point in which there was a change in the behavior (distribution) of the data, such as the beginning of an increase or a decrease.
Exploring Other Dimensions
Sisense shows the dimensions that have the highest Explanation scores. However, you might like to add other dimensions to see how they affect the Explanations. By doing this, you can bring your own domain knowledge into the equation, which might enhance the Explanations that Sisense provides.
Under Possible Explanations, click Explore Other Dimensions.
In the following screen, select the dimensions to consider as contributing factors to the Explanations. You can select any dimension that has been used with this measure in the current dashboard, or across all dashboards on this data source.
Click Apply and the dimensions you selected are added to the Possible Explanations.
Changing Explanations Settings
As a Dashboard Designer, you can change the way Explanations works for Viewers by deciding which fields will be included in the Explanations results. By default, Explanations can include any dimensions from across all dashboards created over the same data model. This enables viewers to leverage Sisense’s Knowledge Graph to uncover relevant explaining factors, even if they weren't included in the dashboard in advance. You can change this and limit Explanations to include only fields that appear in the specific dashboard.
In both settings, Date Security is respected. Viewers will only see Explanations that are based on dimensions that they are authorized to see.
From your dashboard, click and select Dashboard Settings.
In the Explanations section, select the required option from the dropdown menu.Note:
By default, All fields within the data model is selected.
To disable Explanations:
From the Admin tab, select Feature Management.
Toggle off Explanations.Note:
Explanations can only be disabled for the entire license and not for individual widgets.
When you have found an Explanation worth sharing for the data point you are investigating, you can download it as a .png file.
In the Observed Explanation section, click.
Click Download Image.
An image of the Explanations screen, including the time-series and the Observed Explanations, is downloaded as a .png file to your computer.
- Explanations only works on cartesian charts, such as column, line, area, and bar charts.
- Explanations only works on time series with a single date dimension.
- Only certain measures are supported. For more information, see Supported Measures.
- Explanations doesn’t work properly when the measure is filtered by a dependent filter.
- Explanations can only run on a widget when the relevant table is limited to 115K rows (cannot run if the dataset is larger than 115K rows). This limitation may be solved by applying a widget or dashboard filter to reduce the amount of data in the widget.