I'm known at work for championing inclusive language, and I call it out regardless of how person saying it identifies. Regardless of how the folks in the group feel at the time, it continues people using it and feeling that its OK. Those people will then use it outside of that circle, and they will inevitably make someone feel unwelcome. I feel its one of those things that, although can be a pain to rewire our brains, does make a difference, and trying to move to a more inclusive place to be is a great end goal