Looking at "fit" from an aero perspective, frames are quite aero, but stem spacers are not. It's better to have a taller frame with the stem slammed down, than a smaller one with stacks of spacers (or a taller "stem system" in the case of the new superbikes). The frame that fits best from an aero standpoint is the one that (when you place the saddle and bars where you need them) allows the bar setup to sit as level as possible with the toptube. Since there is no standard for bike sizing, some frames will require more spacers than others; this damage to ideal aerodynamics usually exceeds the differences in the frames themselves.