“In today’s landscape, applications and websites, which are essential parts of businesses, need to be analyzed deeply in order to understand the impact they create. This is especially important for BI analysts. Every phone application or website has a predefined customer on-boarding journey, where all customers are given the same form of communication to drive them towards purchasing a product or service the company is selling.
Let’s take the example of a pizza delivery application: they have a predefined customer journey where, once users sign up, they are shown a coupon for a free first order delivery to encourage them to order a pizza.
To analyze whether these journeys are effective and what impact they have, we use diagnostic analytics with the help of funnels and averages. Let’s elaborate on each of them.
Funnels: Funnels are structures with broad openings that gradually narrow down as we progress through the steps. This means that at the beginning of the funnel, we have many users, but the number reduces as we move to the next step of the journey.
Averages: The numbers we obtain at the end of these funnels, and the point in the funnel where the user base is the largest, are the most important ones. These numbers are calculated using moving averages of past days, weeks, or any other period. [Preferably with a graphic – Averages]
Now, let’s dive into a case study to better understand how to create a system for actively monitoring and gaining quick insights on the customer journey, and how to stay on top of business activities.
Fashion Hub Case Study : E-commerce website for clothing and accessories
FashionHub has a simple yet powerful on-boarding journey to convert registrations into customers. Recently, the number of orders on the website has decreased, and the CEO has tasked the data analyst with identifying the reason and finding a way to bring the numbers back to their previous levels. Further investigation revealed that customer retention (repeat orders) is at the same level as before, but the issue lies with new or first-time orders. To pinpoint the issue and take appropriate action, we must analyze the customer on-boarding journey.
The current on-boarding journey works as follows:
1. Sign Up and Referral Code
2. In-app Popup – Offer for Flat 50% off on the first order.
3. Notification with the coupon code (after 1 minute of sign up) – if not already used.
4. SMS with the coupon code (after 30 minutes of sign up) – if not already used.
Questions to consider as a data Analyst
As a data analyst, you must ask basic questions holistically, and clear thinking is crucial. Some questions to consider are:
1. Have the number of sign-ups decreased in the past few days? What are the numbers?
2. Is the offer (50% off) competitive enough compared to other competitors’ offers?
3. What is the conversion rate from sign-ups to those who click on the pop-up/notification/SMS?
The first two questions will give us an idea of whether the issue lies with acquisition or tough competition. However, to answer the third question, we need to design a funnel and understand the past averages to identify where the journey is facing a bottleneck.
Upon initial analysis, we can see that the number of sign-ups has not decreased, but the number of clicks on the in-app pop-ups has decreased by almost half compared to the previous value. Other numbers for notifications and SMS are elevated, indicating an issue with our on-boarding journey. Further analysis and root cause analysis reveal that there was an issue with the in-app pop-up being blocked with the newest OS version, preventing users from viewing the offer. As a result, the conversion rates for notifications and SMS were better. However, relying solely on numbers is not enough, as the number of sign-ups can increase. Therefore, we must calculate averages.
Averages and ratios perfectly show which part of the population got filtered. In this case, the data analyst created an alerting system to notify the tech team whenever any of the funnel averages dropped below the 30-day moving average, making the system robust and directly adding value to the company.
| No Of Clicks | FY 21-22 | FY 22-23 |
| Sign Up Registrations | 288,317 | 301,584 |
| In App Popup | 71,449 | 33,963 |
| Notification | 67,898 | 84,983 |
| SMS | 1,307 | 2,577 |
Conclusion
Diagnostic analysis is not only restricted to on-boarding journeys; it can be applied to any process, such as user migration or reactivation of old users, to name a few. Here are a few points to consider when using funnels and averages:
1. Always consider all the parameters in the funnel and do not forget or skip any, as it may skew the results and lead to an incorrect diagnosis.
2. Rely on averages rather than raw values, as raw values can be misleading.
3. Averages should be resistant to volatility, which is why moving averages are preferred.
We at Vindiata believe that despite being underrated diagnostic analytics is still relevant. We have deployed several such analytics project for an array of different business or technical problems. Breaking down a complex problems requires a lot of expertise not only in data but in that domain as well which we pride ourselves with. If you feel this blog is relevant to your current business needs feel free to contact us and do check out our website for other interesting blogs.
In the next chapters, we will learn about MECE: a simple way to divide any problem into pieces for easy analysis, as well as how to code and report these warnings to the appropriate shareholders and responsible individuals.”
