There is a big difference between performance and only you know if you want the extra performance of the 335d.
Having said that, the 320d seems no slouch these days, looking at the performance figures. Will also feel more nimble being lighter. Also the F30 320d will bode much better in future years for depreciation as its the current model.
Heart = 335d
Head = 320d