> Most will say the time to optimize is once you're fairly well along in development.

I call BS. But, if most do say that they are stunningly wrong. The time to optimize is after it is determined that some portion of code is too slow to meet a (non)functional or business requirement.