I don't like the answer "I don't know"

I realized recently that I dislike the answer "I don't know". I understand sometimes we don't have an opinion or fully formed thought on something, but to say "I don't know, couldn't tell you," can annoy me if someone said it a few days/weeks prior, and when asked again, gives the same response.

Any other ENTJs feel this way about it? Or is it just me?