Changing Model Performance Metrics Mid-Project

Seeking Advice Forum
1

Hi everyone,

I wanna know if anyone else has faced a similar situation in their data science projects. At the beginning of a recent project, my team and I settled on a model performance metric that we believed would adequately capture the success of our efforts. However, as we started running experiments, we quickly realized that the models we were developing just couldn’t meet the acceptance criteria we had set.

After discussing it with my team, we came to the conclusion that we might need to rethink our approach and consider adjusting the performance metric. This was a bit daunting, as I didn’t want to seem like we were lowering our standards, but I also wanted to ensure that we were being realistic about what we could achieve with the data and algorithms at our disposal.

Has anyone else suggested a change in performance metrics to their stakeholders mid-project? How did you approach the conversation, and what was the outcome? Any tips on how to navigate this kind of situation :roll_eyes: :roll_eyes:

2

Yes, adjusting metrics mid-project can be necessary. Approach it by explaining the data challenges and how the new metrics better reflect the project’s goals. Show how the change will improve outcomes.

3

Changing model performance metrics mid-project can be tough but necessary. Here’s how to handle it:

  1. Assess Impact: Check how changes affect goals.
  2. Communicate: Inform your team and stakeholders.
  3. Update Models: Adjust and retrain models.
  4. Validate Results: Test performance with new metrics.
  5. Document Changes: Record reasons and methods for future reference.

Ensure changes are justified and well-documented.