Matt Medeiros does some testing to determine if Jetpack slows down a WordPress site. The short answer according to him? Not really. But his methods and opinions in his post are worth the read.
While you are checking that out, make sure you give a listen to Matt’s interview with Helen Hou-Sandi on what it’s like leading the WordPress 4.7 release.
