Hey everyone! Inspired by a recent post, I wanted to share a guide for tackling open-ended interview questions in data science. Examples of such questions might include:
- A c-suite exec tells you day-over-day revenue is decreasing. What do you do?
- A PM asks you to opportunity size a new product version.
- A PM comes with mixed A/B test results and needs your help interpreting them.
Disclaimer: While I’m a senior DS, I mostly conduct coding interviews, not these types. This guide is based on my own experiences, and I welcome your feedback to improve it.
These questions usually aren’t about getting the “right” answer, but instead about showing your ability to:
- Break down complex problems into manageable steps.
- Take a systematic approach to analysis.
- Clearly communicate your reasoning.
Framework for Open-Ended Questions:
Example: Exec says revenue is dropping. What do you do?
Step 0: Outline your approach
Start by giving the interviewer a high-level overview of your process.
Example: “First, I’d understand if the issue is isolated or systemic, then break down the metric, de-aggregate, and finally suggest preventive measures.”
Step 1: Understand the big picture
Ask clarifying questions, check related metrics, and assess if the problem is widespread.
Example: “Is this gross revenue? What about active users or subscriptions? This helps gauge how broad the issue is.”
Step 2: Narrow the scope
Look at trends and other possible explanations.
Example: “Did revenue drop this time last year? Could we look at revenue per user to pinpoint whether the issue is user loss or lower spending?”
Step 3: Dig deeper
Now de-aggregate or identify new metrics.
Example: “Let’s split by revenue streams or geography. Ads vs. purchases, US vs. non-US users.”
Step 4: Prevent it from happening again
Show you’re thinking long-term.
Example: “We could add a topline metric for revenue per user and a year-over-year growth view to spot cyclical trends.”
Step 5: Advanced techniques (optional)
If time allows, showcase deeper knowledge.
Example: “We could use causal analysis with tools like DoWhy for root cause analysis in similar future problems.”
Opportunity Sizing Example:
PM asks you to size the potential of a new product version.
- Step 0: Outline your approach.
- Step 1: “Is this for all users? Have we launched a similar product before?”
- Step 2: Identify key metrics like revenue per user or engagement.
- Step 3: Compare with historical launches, use effect sizes from previous experiments.
- Step 4/5: Suggest experimenting for future launches, or apply causal modeling techniques.
Final thoughts:
There’s no perfect formula for open-ended questions. Interviewers might steer the conversation, so be flexible! Please share any resources or ideas to enhance this guide.
Looking forward to your thoughts!