Interpret a Funnels Report

Mixpanel’s Funnels allows you to examine how end-users perform events in a series. This article explains how to interpret a Funnels report. Learn about Funnels report basics in this article. Learn about how to build a Funnels report in this article

Interpret the results of your funnel by selecting different analysis charts from the drop down under Funnel steps. These are described in detail below.

Interpret1.png

Breakdown

Below the query builder you can choose to segment your funnel by a property or cohort to break down the data in the Overall table below the funnel. Click the Breakdown button and select an event property, people property, or cohort.

Screen_Shot_2019-06-13_at_4.25.35_PM.png

When segmenting a funnel by a property, the top 200 property values will be shown in the table. All other segments will be grouped into an $other category.

Screen_Shot_2019-06-18_at_11.38.12_AM.png

If you need to view all of the individual segments beyond the 200-Property limit, use the download button in the top right of the Funnels report and select Download CSV. Your export will contain all properties, including those bucketed into $other, up to 10k segments.

Build9.png

If you select any order funnels in the Conversion Criteria section, you have the option to select Top Paths under Breakdown to determine which order the steps are being performed in. 

Screen_Shot_2019-09-26_at_11.49.37_AM.png

In the Overall chart below your funnel the top paths will be displayed as a list of different path orders so that you can examine which paths have a higher or lower conversion rate or conversion time.

Screen_Shot_2019-09-26_at_11.52.56_AM.png

Statistical Significance

Statistical significance in funnels validates an increase or decrease in conversion rate for a property or cohort segment. It attempts to identify random chance with respect to overall conversion. A p-value indicating statistical significance is calculated in the overview table when you choose a property or cohort to group by.

In statistical hypothesis testing, the p-value or probability value is the probability that the variation in a segment’s conversion rate, compared to the overall conversion rate, is not driven by a random chance. This value is shown for every segment by default.

In order to clarify this statistical significance, the segmentation chart shows the confidence level of each segment. Confidence level is defined as 1 - p.

  • > 0.95 = statistically significant, indicated in green. This variation in conversion rate is likely not driven by random chance.
  • < 0.95 = not statistically significant, indicated in red. This variation in conversion rate is likely driven by random chance.

Screen_Shot_2019-03-14_at_10.47.34_AM.png

Scrolling further down the table takes you to the statistically insignificant segments. If a segment has less than 30 samples, p-value is not shown, as the sample size is too low to detect difference from overall population. This is indicated by “Insufficient samples”.

The number of samples is the same as the count of entries into the funnel. If the funnel is looking at the unique count, this will be the number of unique users who entered the funnel in that segment. If the funnel is looking at total count, this will be the total number of entries into the funnel in that segment. 

Screen_Shot_2019-03-14_at_10.47.55_AM.png

You can choose to sort by any of the columns of the overview table in descending or ascending order by clicking on the header. If you sort by statistical significance, values with confidence level of > 0.95 are shown first, and then values with confidence level < 0.95. The secondary sorting is determined by the overall conversion rate for the funnel. 

Create Cohorts from Funnels

You can create a cohort from a funnel segment in order to examine that group of users more closely. Learn more about this feature here.

Funnel Steps

This is the primary view for Funnels which shows how users are converting across your steps.

Interpret2.png

Funnel Trends

Select Funnel trends from the drop-down list to see how your users are converting over time. The trends chart shows the percentage of successful conversions from the first event in the funnel to the last event in the funnel for a day, week, or month. A user that completes the funnel within the conversion window will be counted on the day, week, or month corresponding to when they performed the first event in the funnel.

Interpret3.png

Time to Convert

Select Time to convert from the drop-down list to see how long users take to convert through your funnel.

Interpret4.png

You can choose to switch between viewing percentage or whole numbers in the chart by clicking % or # in the top right of the chart.

The dotted line indicates the median of the chart.

The chart updates if you select a specific step in the funnel above to indicate the conversion rate up to that step, rather than the full funnel.

Click on a segment of the bar chart to create a cohort or zoom in on that segment. If your users are converting within a small window of time, as seen above, it is useful to zoom into that bar in order to see a more clear breakdown of the range of time users are taking to convert.

Interpret5.png

Top Paths

If you have selected any order for your funnels steps, you have the option to select Top paths from the drop-down list to view a Sankey visualization of how your users are performing the steps in your funnel. 

Interpret6.png

Simultaneous Events

Mixpanel Funnels include a built-in two second grace period. This means any consecutive steps that have timestamps within two seconds of one another are interchangeable. This window allows for inconsistencies that arise with cross-platform tracking, batch event sending, mobile time tracking, or variations in connection speed.

What does this mean in terms of your funnel? In a funnel with step A -> B, if a user completes event B first and then performs event A within two seconds, Mixpanel counts this as a conversion from A -> B or B -> A.

Forwardfilling

Mixpanel “forwardfills” event properties in instances where properties are sent in earlier steps of a Funnel but not sent in subsequent steps of the same funnel. This means that the property that is present in early steps of a funnel is appended to the later steps of the funnel where it was previously absent.

For example, say that the user is shopping on your website and you want to track a funnel from Log In to Purchase. There are three events in this journey: Log In, Item View, and Purchase. Each event has its own properties, and the “Name” property is only sent with the “Log In” event. This property will be added to the subsequent events, where it was previously absent:

ForwardfillingDiagram.png

As you can see in the diagram above, the property of “Name” is only a property for the event “Log In”, but that property is forwardfilled to the subsequent events.

Backfilling

Mixpanel “backfills” event properties in instances where properties are sent in later steps of a Funnel but not sent in the earlier steps of the same funnel. This means that the property that is present in later steps of a funnel is appended to the earlier steps of the funnel where it was previously absent.

For example, the user shopping on your website chooses to buy a shirt that costs $5. In this instance, properties of the item that is purchased are not sent until the “Purchase” event, so the property is backfilled and added to the “Item View” and “Log In” events:

BackfillingDiagram.png

As you can see, the property of “Item Price” is only a property of the event “Purchase”, but that property backfilled to the previous two steps. Also, the properties sent with the event “Item View” are backfilled to “Log In”.

Backfilling Event Property Values

If an event property is sent with all steps of a funnel but the value of the property changes across steps, then the most recent value is applied to all steps of the Funnel.

For example, the user shopping on your website views a blue shirt and adds it to their cart. However, when they make their purchase, they have changed their mind and decided to get a red shirt instead:

BackfillingReplaceDiagram.png

As you can see, the property of “Item Color” is Blue for Event 2, but changes to Red for Event 3. As a result of backfilling, the property of “Item Color” will be appended to Red in the “Item View” event, and further backfilled to “Log In”  as well.

Deep Dive for Statisticians

Note

This section is intended for users who want to understand the mathematics involved in statistical significance in depth.

To determine whether a particular segment’s conversion rate is significantly different from the overall conversion rate, we use a hypergeometric distribution to calculate statistical significance. The hypergeometric distribution is used to model the probability of picking k items of a particular type in n attempts without replacement from a population of size N having K items of the same type.

For example, let’s say we have a sock drawer with 20 socks, 10 blue and 10 red. If we randomly picked 10 socks one at time from the drawer without putting them back between picks, and we wanted to know the probability of 9 of those socks being red and 1 of them being blue, we would use a hypergeometric distribution to calculate that.

This is applied to funnels by considering the total number of users who enter the funnel to be a finite population of size N, out of which a subset of users convert (K). We then estimate the probability of getting k conversions in a particular segment (given that there were n users who entered the funnel in that segment) if users in that particular segment were picked at random from the overall user set. The higher the probability, the higher likelihood that variations we see in conversion rate are due to random chance.

To calculate the actual p-value, we estimate the hypergeometric cumulative distribution function (CDF) for N, K, n.

Screen_Shot_2019-03-14_at_1.49.33_PM.png

In the CDF, the value of any point (X) represents the probability that a random draw would result in fewer conversions P(k < X). 1 - P(k < X) represents the probability that a random draw would result in more conversions P(k >= X).

These two probabilities are used to represent the probability that the selected segment will either outperform (P(k < X)) or underperform (P(k > X)) the overall set of users. Mixpanel takes the higher probability of the two, and calculate the p-value as 1 - max(P(outperform), P(underperform)).

Is this article helpful?

Comments

0 comments

Article is closed for comments.