Welcome to the Post Status Draft podcast, which you can find on iTunes, Google Play, Stitcher, and via RSS for your favorite podcatcher. Post Status Draft is hosted by Joe Hoyle — the CTO of Human Made — and Brian Krogsgard.
In this episode, Joe and Brian talk scaling WordPress, and what to do when you think you might’ve reached WordPress’s limits. From meta data, to users, to traffic management, they break down some of the most common scaling issues.
Topics
- Posts
- Meta
- Search
- Database
- Users
- Traffic (types of caching)
Links
- Elasticsearch
- ElasticPress
- More Like This Query
- Elasticsearch WP_Query
- Memcached
- Redis
- Rarst Fragment Cache Plugin
- Human Made Fragment Cache drop-in
Sponsor: iThemes
iThemes has a full suite of excellent products to help you level up your WordPress website. From iThemes Security, to BackupBuddy’s new live backups, to Exchange for your next membership site, iThemes has you covered. Thanks to the team at iThemes being a Post Status partner!
Was just listening to your Podcast.
I’m working on a WordCamp talk series related to people who think they’re exceeding WordPress limits somehow.
I’ve been working with WordPress since it was B2 + so far, every WordPress scaling challenge I’ve come across has a solution.
Currently I’m collecting case study data for sites which seem to exceed WordPress limits to suggest resolutions in my talk series.
Love to be in touch with you guys + have you introduce me to folks having scaling problems, to add their specific situation as a talk example..
Skype is best way to reach me.
I’m happy to contribute to your PodCast anytime. My life right now revolves around WordPress site tuning.
– David, Skype: davidfavor
Easy way to resolve looking up content by unbounded values.
Say for example all prices < $50 for an E-Commerce site.
The fix for making this search fast + scalable.
1) Install MariaDB-10.1.x version (removes MyISAM/Aria engine FTS requirement)
2) Ensure all tables are InnoDB
3) Setup FullText indexing on whatever column requires lookup
This type of lookup will run at around 30%-50% of the speed of an indexed lookup. Considerably faster than scanning millions of records + doing substring searching of every record.
@Joe: There’s another fragment caching technique that is quite powerful but doesn’t get a lot of press. You mentioned fragment caching requiring to go back to the WordPress layer to make use of the object cache, and this caching works well for fragments that are expected to live longer than the containing page, or which are updated infrequently. However, you can also do fragment caching with frontend caches as well via Edge Server Includes (ESI). Varnish supports ESI, as does Squid and CDNs like Akamai. ESI tags are like old-school Server Side Includes, except they communicate over HTTP and they respect caching headers in the HTTP requests for the fragments. Using ESI for fragment caching is particularly useful when there is one fragment of a page that updates more frequently than the rest of the page. Consider, for example, a news publisher’s homepage, where there is the top stories that need to be updated every minute but the other sections can be updated every 5 minutes. The Top Stories can be served via an ESI tag which can be defined with a URL to the endpoint that just serves that one Top Stories fragment.
See some more discussion about this at: https://github.com/pantheon-systems/wordpress-at-scale/issues/8
Well done podcast.
Thank you, it is quite insightful.
I found WordPress very scalable and this can be done easily, especially if you plan for it from the beginning of your website’s life.
With proper maintenance and infrastructure I can’t really see a limit to the scalability of WordPress. I think a lot of people fall for cheaper options when it comes to hosting and endup blaming WordPress rather than the server.
I’ve been a huge fan of the podcast for many months now, but this may well be one of my favourite episodes. Joe did an excellent job of demystifying performance and I now have an abundance of sites that need some attention!
Adam’s right on.
WordPress can easily scale to 5000-10000 requests/second with straight Apache + libapache2-mod-php.
No need for CDNs or load balancers or proxies (NGINX/Varnish) or other cruft.
Well tuned WordPress sites are blazing fast.
Get it David! 👻
Specifics would be scaling WordPress from one server box to distinct server nodes (nginx + redis, mariadb, volumes + CDN) and managing all that.
I recommend ansible for WordPress DevOps in a modern workflow
Managing many sites is a bit cumbersome.
I tend to stick with simple tech – Apache + MariaDB + PHP (libapache2-mod-php) + Ubuntu for host OS + Ubuntu in LXC containers.
Skipping all other non-essential tech – CDNs, NGINX, REDIS, LVM, Varnish, Haproxy.
My experience is all these technologies tend to slow down sites.
This is easy to see by way of using ApacheBench.
Having a simple WAMPL Stack to work with, then I use custom tools which replace much of the wp-cli technology with more robust tech. wp-cli has the nasty habit of being very fragile. For example, referencing a WP debug log which can’t be written can cause wp-cli to fail silently.
Working with 1000s of sites, a 0% failure rate for tools is required.
Ansible can be useful + I go through a long checklist setting up new sites + an even longer checklist for tuning new sites + sites I migrate out of other hosting companies.
Attempting to describe the process for Ansible or other tools tends to be complex + many times a step will consist of – setup + test + monitor logs – so Ansible isn’t really designed to handle realtime monitoring of logs after a step for a given time increment + then modifying the step, repeatedly targeting a result.
Tuning a site for Speed + Stability (speed under load) tends toward tricky.