Hey guys, I really love programming. I spent money on a full stack web dev boot camp to learn the basics and I’m even studying data science in uni. I have even gotten a junior developer job for the government. The thing is, they currently have me working on basic front-end issues. For the most part it’s basic JSX components they need pumped out with some CSS styling. There is logic implemented here and there but not much. It’s gotten to the point that I mostly just finish this work within the first couple hours of the day. I want to actually learn logic. I don’t find leet code interesting and I’m so uninterested in basic code projects other people recommend. Any advice??
6 Likes
Forget statistics.
Imagine you’re running the business, and your gross profit margin is at 17% rather than 20%… what’s the impact. Ask the business stakeholders when you don’t know.
Basically work out what each KPI does for the business, and what impact they have as they change. Most of the time the issue is not with the bounds of the KPI but with the KPI itself. For example a business’ NPS is already baked into how they operate, but a change in the NPS is significant. Putting a bound on NPS is a total waste of time, what you need is to throw away NPS as a KPI, and instead the KPI should be change in NPS.
When in doubt, convert everything back to dollars. How much money does the business lose if NPS drops by one point?
4 Likes
Heavily agree with this.
Statistically, 0.9 precision is a pretty good metric. For our business, 0.94 precision is so much better than 0.92 precision, that it justifies a 0.1 absolute recall drop.
All because precision impacts the business harder than recall. In statistics, no way, that would be a stupid tradeoff.
3 Likes
Does “as relatively expected” only refer to averages, meaning that your leadership cares whether this is a rapid, unusual development or not? If you simply track on a weekly or monthly basis, then probably not.
Such questions are typically asked by leadership as a gauge for “should we worry?” in my experience. Additionally, their concerns are business-related and domain-specific, such as “sales numbers shouldn’t drop more than X% MoM.” To determine what (pretty stable) safe ranges are in this situation, I would collaborate with a product manager, SME, or business lead in your region. I would also look at data from the previous one to three years, for example, to see whether any instances in which the data fell outside of the band were truly significant.
A different way to phrase that would be “given the current numbers, how much are we off from reaching our yearly goal and are we on track”—a phrase that may include a projection and a percentage—or “progress against some goal.”
2 Likes
I typically work with KPIs tied to goals/north stars. We either hit goal or we don’t, so there’s no range to speak of - but we do sometimes assign a “stretch” goal. Ultimately, a KPI should be tracked if it’s contributing to the business in some way. If it’s not tied to a strategy, why track? If there isn’t a goal already for that KPI, I’ll backstep into a goal and create one.
Anyway, depending on your business, you could compare the KPI to a 12 week average, or how that KPI was performing this time last month/quarter/year. Moving avgs could work as well. I’d keep it simple and not include standard dev since interpretable for KPIs is crucial, and not all biz folks understand std dev. All these options provide context when you don’t have a goal. Without understanding your business, it’s hard to give better advice
1 Like
What about calculating a confidence interval? Let’s say 95% CI?
I would start plotting the timeseries to check if there is any saisonality in the data. If not you can take the data points you have and calculate the CI. If not you can calculate it as well but group it by the date ranges that show similar saisonality pattern.
I had the same problem and we were also looking into FB prophet, but the statistical approach outperformed FB prophet.
Out of curiosity: Is this monitoring supposed to trigger some action once your KPI crosses one of the boundaries?