Any Order Funnels vs. Closed Funnel Analysis

First off, I’m Cameron Hozouri, CEO and Co-Founder of ByteBrew, an all-in-one game analytics platform where we give developers all the tools they need to go from development to scale in one lightweight integration – completely free. After that short intro, let’s jump right into the topic.

Defining Funnels

Before breaking down the differences between funnel types, let’s start by defining a funnel. So, what’s a user funnel? Funnels are methods for understanding how users transition through stages, events, steps, etc in your project. Most common use cases for utilizing funnels when analyzing your apps are to quantify how many users reach or drop off key conversion points like purchases or level completes. Even though funnels are often highlighted within most analytic solutions, how a platform computes funnels varies based on the technological capability of the platform. 

Any Order Funnels

Like their name suggests, Any Order funnels are a form of funnel that only calculates based on the number of users or events that complete an individual step without boundary. Breaking that statement down further, steps in a Any Order funnel are queried to only search for users who have completed the series of specified steps in any order. Any user that has completed the steps listed in the query will be included when calculating. This funnel logic does not take into consideration (1) matching the consecutive order of steps completed or (2) the chronological order of steps completed when computing results. 

To illustrate see the sample funnel query for a 4 step funnel where: “Step 1 = Event A”, “Step 2 = Event B”, “Step 3 = Event C”, and “Step 4 = Event D” using any order funnel logic:

In the query diagram above, we examine 7 users that have interacted with the specified steps. Using Any Order funnel logic, all of the events performed by the users are incorporated into the funnel calculation regardless of the specified order in the query of “Step 1 = Event A”. The visual funnel output for this data would be the following chart diagram: 

Utilizing the 7 users paths, the Any Order Funnel will simply count the number of events that match the listed steps in the query as long as the user has interacted with all previous events in the query and sorts the data by the slotted step number; resulting in the Any Order Funnel Chart sample shown above. Showcasing that any event regardless of it’s predecessor or required step number will be included in results. For example, User #2 does not match Step 1 (Event A) of the query in their user path, but is still incorporated in the funnel data. Looking further, User #5 starts with the last step of the query (Event D) and ends their path with the first step of the query (Event A), but is still added to the calculation. 

Any Order funnels are fabricated funnels that completely mislead the developer analyzing their apps performance or conversion paths. Data calculated in this fashion does not represent a funnel of any kind, but instead a simple representation of the count of events performed by your users. Utilizing this funnel logic also restricts what data can be derived from the analysis. Vital KPIs for understanding your app’s funnel performance such as “time to complete”, “median time to step”, or real user “dropoff” can’t be computed since the events in the query aren’t chronologically organized. If your analytics provider describes their funnel analysis using Any Order, be aware of what their data is representing when making product altering decisions. 

Closed Funnels

Closed funnels are a type of funnel calculated using data from users that participate in events based on set step order. This funnel logic sets strict boundaries to only compute events that follow both (1) consecutive step order of events listed in the query, and (2) the timestamped chronological order of the events by users. Computing funnels using this methodology renders true user conversion data with user drop offs down to each individual user and unlocks visibility into how long converting takes from one step to the next.  

Using the exact same user data set shown from Any Order funnels, let’s examine how Closed funnels would calculate the 4 Step sample funnel where: “Step 1 = Event A”, “Step 2 = Event B”, “Step 3 = Event C”, and “Step 4 = Event D”: 

Examining the diagram above, only user paths that begin with Event A and follow subsequent step order will be incorporated into the funnel output. For example, User #4’s events will only partly be qualified as the system will rule out events that do not have all the matching predecessors in the set order. Instead the system will recognize Event A as the start of a user’s conversion path and begin looking for subsequent steps matching the query order to include – making Event A and Event B in User #4’s path accepted by Closed funnel logic. Additionally, User #3 will only have one event qualify to be included in the funnel calculation since the start of the user’s conversion path matches “Step 1 = Event A”, but the next subsequent step is not “Step 2 = Event B”. The visual funnel output for this Closed funnel data set would be the following: 

The Closed funnel chart paints a completely different picture for how a user group transitions through the funnel set in the query. Accurately excluding user events that do not abide by the queried order set by the developer, uncovers exactly how cohorts of users either reach or drop off a path to Event D.

Comparing the Any Order and Closed funnel data:

In mobile development, funnels are a benchmark used in key decision making that affects both the product in development, but also the user experience – and to that extent the success of your business. Platforms that calculate Any Order funnels display a skewed misrepresentation of what developers are looking to achieve when creating a user funnel. The delta between the results of these two methods using the exact same data set is substantial and highlights how vital the decision is for choosing a proper analytic partner. The entire purpose of an analytics platform is to provide your developers with both the tools and technology to produce the insights they depend on to make impactful business decisions in their products. 

What type of funnel does ByteBrew use?

ByteBrew uses Close funnel logic to give a precise and deep analytical breakdown of your app’s performance that delivers in seconds, so you as the developer are able to make the critical product updates necessary to build the best possible games and apps for your users.  

Sign up for ByteBrew 100% free here: https://dashboard.bytebrew.io/register 

About ByteBrew 

San Diego-based ByteBrew is an all-in-one mobile game analytics platform that provides developers with everything they need to grow their games business. Co-founded in San Diego, California, by brothers and veteran games developers Cameron Hozouri (CEO) and Kian Hozouri (COO), ByteBrew has raised $4M to-date from early-stage investment firm Konvoy, VC firm Valhalla Ventures, early-stage venture fund Node Ventures, and other angels. 

 1,185