Size does matter for websites. The faster your webpages are downloaded, especially from mobile devices, the more likely the visitor will stay/engage/buy on your website. Fast pages will also increase the confidence of the visitor in your company. (source)
Japplis Website Optimizer is a utility that will allow you to increase the speed of your websites using different techniques.
Try your website at www.webpagetest.org and check in performance review if GZip, Cache Static and Minify JS is ok.