It honestly depends on where you live and your preferred driving style. Everybody dreams of owning an M3 but the honest truth is that it's not a daily driving machine. You are correct that you will achieve much better gas mileage and low end torque in the 335i.
This is not directly applicable to your case but I love driving my N52 car to work every day. It's easily fast enough to pass most of the clunkers on city streets, but not fast enough to earn me a speeding ticket every time my right foot gets heavy-- plus it's considerably more comfortable than an M car. There is tremendous value in a balanced and usable car