Comment on WordPress SEO Plugin Options by SEO Dave.
Some of the stuff you discuss above are covered in the above article and the comments. Not everything can be fixed under WordPress, some of it requires server settings.
Some pagespeed issues can’t be fixed and some shouldn’t be ‘fixed’: the fixes are worse than the ‘issues’.
Currently (February 2016) I run WordPress 4.4.2, Stallion Responsive 8.3, W3 Total Cache 0.9.4.1 (caching, CDN etc…) and EWWW Image Optimizer 2.5.9 (image optimization). Other plugins not related to pagespeed Google XML Sitemaps,
Subscribe to Comments Reloaded and a plugin for selling Stallion. So I’m running with 5 plugins, 2 related to pagespeed.
W3 Total Cache is a major part of the pagespeed optimization, also use it’s manual CDN option which is a CDN on my own server (a sub-domain holds the CDN files).
Stallion deals with Gravatar issues by caching them locally (see under the Stallion Performance options: all of the top options On except Widget Cache, have that off).
Since I use the Stallion Rocket Lazy Load Images On option my images are lazy loaded as well without using Jquery: most lazy load plugins use Jquery, so you’ve added a speed improvement (image lazy loading) by loading something (Jquery) which causes a pagespeed issue!
If you want to use AdSense, social media like buttons I’m afraid it’s at a pagespeed cost. I’ve found barely anyone shares my posts on social media and since social media has no direct SEO benefit and conversion of social media traffic is low, I’m no longer bothering with the share buttons for sites like this one.
I’m building a new site to sell children’s books (writing/illustrating children’s books with my wife: heck of a lot more fun than SEO) and social media will be a big factor for that site, so will take the pagespeed hit.