I interviewed with a Twitter search engineer and he told me that that's because their in-memory search index is not big enough hold all tweets ever posted. On top of that, their efforts to scale up the index are counteracted by the ever-increasing volume of tweets posted per day.

And the reason it's not big enough is because they don't want to invest in it. They found out search is not as monetizable as they thought it would be, and at the same time they cannot take away the functionality. The compromise is to keep what they have.