A generic framework for analyzing a change in a key metric:

1. Clarifying Questions:

Before diving into the analysis, it's important to clarify some basic information about the key metric. This step involves asking questions to understand the context and scope of the metric change. By answering these questions, you can ensure that you have a clear understanding of the problem you are trying to solve.

If say, Facebook Likes is down by 10%, few of the clarifying questions which you could ask:

• Is the drop in likes consistent across all types of content (e.g. posts, photos, videos)?

• Are there any changes in user behavior or trends that could be contributing to the drop in likes?

• Are there any changes in the algorithm that could be affecting the number of likes?

2. Describe Product/Feature and State Objective/Goal:

In this step, you'll describe the product or feature that the key metric relates to. This will help you understand the context of the metric and its intended purpose. You'll also need to state the objective or goal for the product or feature, along with the key metric that you're analyzing. This will help you focus your analysis and make sure that you're measuring the right thing.

For the above example, Product/Feature Description and Objective:

The Facebook app is a social networking platform that allows users to connect with friends and family, share content such as posts, photos, and videos, and engage with other users' content through likes, comments, and shares. The objective is to understand why the average number of likes has dropped by 10% over the past year and identify potential solutions to address the issue.

3. Generate Hypotheses:

The purpose of this step is to brainstorm possible causes for the change in the key metric. Each hypothesis should be a potential explanation for the metric change. By generating multiple hypotheses, you can explore a range of possibilities and avoid jumping to conclusions too quickly.

Example hypotheses for Facebook Likes:

• "Algorithm changes hypothesis" - Changes in Facebook's algorithm have decreased the visibility of posts and made it harder for users to discover new content, resulting in fewer likes.

• "Content quality hypothesis" - The quality of content posted on Facebook has decreased, leading to lower engagement from users.

• "Feature changes hypothesis" - Changes in Facebook's features or user interface have made it more difficult for users to engage with content, resulting in fewer likes.

• "Competitor hypothesis" - The rise of competing platforms such as TikTok or Instagram has drawn users away from Facebook, resulting in fewer likes.

• "Demographic shift hypothesis" - A shift in the demographic makeup of Facebook's user base has resulted in a different mix of users who are less likely to engage with content through likes.

• "User behavior hypothesis" - Substitution for other behavious (comments, reactions), but overall visits or time spent on app not dropping.

4. Operationalize Each Hypothesis:

Once you've generated your hypotheses, you'll need to operationalize them. This means defining specific steps you'll take to test each hypothesis. Describe the analytical steps to support or refute the hypothesis and explain so someone else could reproduce your analysis. By operationalizing each hypothesis, you can design a clear plan for testing each one and gathering evidence to support your conclusions.

Ways to operationalize above hypothesis for drop in Likes:

• Algorithm changes hypothesis: Analyze the engagement metrics (e.g., likes, comments, shares) for posts before and after significant algorithm changes and compare them to the overall trend.

• Content quality hypothesis: Analyze the engagement metrics for high-quality posts (e.g., those with more likes or shares than average) and compare them to low-quality posts to identify any patterns.

• Feature changes hypothesis: Analyze the engagement metrics for posts before and after significant feature changes and compare them to the overall trend.

• Competitor hypothesis: Analyze user behavior metrics (e.g., time spent on Facebook, frequency of visits) for users who also use competing platforms to identify any correlations.

• Demographic shift hypothesis: Analyze engagement metrics for different demographic groups to identify any changes in the mix of users engaging with content.

• User behavior hypothesis: Add likes+comments+reacts and chart over time. Compare to time series for overall time spent.

5. Drive Towards Conclusion and Final Recommendation:

After analyzing each hypothesis, it's time to draw conclusions and make a final recommendation. This involves synthesizing the evidence you've gathered and making a judgment about the most likely cause of the metric change. Based on your conclusions, you can then make a recommendation about whether any action is required, and if so, what steps should be taken to improve the key metric. Finally, you'll want to prioritize actions based on their potential impact and feasibility, so that you can focus on the most important and achievable changes first.

For the Facebook example, it is recommended that Facebook focus on addressing the content quality hypothesis and the user behavior hypothesis. Improving the quality of content posted on Facebook could lead to higher engagement, while understanding and adapting to changes in user behavior could help Facebook stay relevant and engaging to users. Additionally, changing the topline metric definition from "likes" to "likes+reactions" could provide additional insights into the drop in likes.