WordPress has a table for handling metadata of posts. Quite simply, it’s a good place to store “stuff” that relates to a post, but doesn’t belong in one of the standard parts of the posts table.
Metadata can be nearly anything. And it can be displayed however a theme wants to display it. You can grab it all and list it out together (as was a common use case for WordPress’ early blog structures). Or you can control what metadata goes where, very specifically.
Using custom metadata gets us out of the_content()
Today, metadata is used by plugins and themes that are distributed, and very often by developers creating custom things on WordPress.
Here’s a sample list of metadata attached to one of my posts:
But it can get much more complicated than that. Take an eCommerce site, for instance. If a product is a custom post type, then the price, shipping details, dimensions, and many more things could all be metadata stored for that product.
Custom meta boxes and custom meta field interfaces allow us to structure these things nicely in the WordPress admin.
Then, on the front end of the website, you have control over how to grab and utilize that data. For example, the metabox shown above enables the use of that data on a page like this:
Metadata UI interfaces are fractured
Much of what I’m saying is old news to you. I get that. You know, use, and love making creative use of post metadata in WordPress. However, what I’m sure you also know, is that the landscape of tools for creating interfaces for entering custom metadata is incredibly fractured.
There are dozens (if not hundreds) of publicly available plugins, drop-in classes, and generators for creating custom metaboxes and interfaces for inputting metadata. Here’s a screenshot of an incredible breakdown of various available tools.
This screenshot is from Scott Kingsley Clark’s comparison table. If it’s overwhelming to you, that’s my point.
WordPress’ core metadata / custom fields interface is lacking. So tools have come about to make making custom meta boxes, defining fields, and interfacing with custom metadata easier.
Problems arise from non-standard meta handling
Some of the most popular tools to enhance the WordPress meta-data handling experience are Advanced Custom Fields, Pods, Custom Field Suite, and more. There are also many classes that are meant to be dropped into a custom theme or plugin.
What’s happened is that these tools have taken different approaches, and over time some of them even change the way they store the data. This means that backward compatibly issues can arise, and if not handled properly, can cause problems for developers and site owners using these tools.
Trust me, I’m not trying to pin these issues on WordPress core. Developers should be careful about the tools they use, and builders of these tools should consider the backward compatibility issues their users may run into.
A real life example happening right now
I’ve spent some time in the last day or so trying to learn exactly what problems people are having with the upcoming changes to Advanced Custom Fields (ACF). ACF has fast grown into one of the most popular tools for creating custom metaboxes and fields.
Advanced Custom Fields also has paid add-ons so the developer can monetize the work he’s put into the plugin.
The problem is, as Chris Lema did a great job of explaining, is that he has not considered backward compatibility properly.
Basically, developers use ACF to build custom interfaces for their clients. Now, ACF is changing the add-on model to be a single Pro plugin, that’s not an add-on at all, but a whole new plugin.
While he’s at it, it’s very confusing what will work and not work between ACF versions 4 and 5, leaving developers confused, and worst of all, many site owners may upgrade to version 5 of the free plugin and see features disappear or their sites break.
This is not good. And folks are unhappy. The developer of ACF should ensure that he’s created workarounds for those that upgrade from version 4 to 5, that he doesn’t break their sites.
Worst of all, this isn’t the first time this has happened for ACF. When going from version 3 to 4, there were backward compatibility issues, and it was a tough upgrade on a lot of developers and site owners. It resulted in a fork of ACF, and Custom Field Suite was born.
There’s a saying: “Fool me once, shame on you; fool me twice, shame on me.”
After the last time, I would’ve put the brakes on my level of trust that ACF would properly consider back compat moving forward. However, whether they should’ve kept using it or not, a lot of people are worried now about the future integrity of websites they built that depend on ACF.
Backward compatibility is important
Backward compatibility is important in WordPress. As a WordPress developer I talked to this morning noted, we’ve been training our clients for years that it’s safe to update their WordPress sites. And with irresponsible development practices, it puts that trust in jeopardy.
So I encourage two things:
- I hope that Elliot from ACF will re-consider the strategy he’s using, so that ACF users can have a more seamless upgrade. And I hope he’ll document exactly what actions users and developers should take (the existing posts and documentation is very confusing).
- Developers need to strongly consider the tools they’re using, and determine whether it’s worth creating a dependency on a tool like ACF to make life a little easier during development.
It’s time for WordPress core to enhance the metadata API
Many times with software, it’s tough to know exactly how people will use things. In the case of WordPress’ custom fields, I doubt the core team really considered exactly how people would be using custom fields and the post meta table 5 or 8 years from then.
But today, the post meta table is a huge part of WordPress. It’s pivotal to storing data from interfaces that allow WordPress to be used as much more than “just a blogging platform.”
And I believe it’s high time core supports a common format for creating metadata UI interfaces.
Thankfully this process is already underway.
One of the WordPress core components that’s being actively worked on is that for WordPress Options and Meta APIs. The aforementioned Scott Kingsley Clark (also the lead developer of Pods) is heading up the team, and they are utilizing regular meeting times to organize the project.
Most of the people involved in this core component have either created their own metadata framework or utilize metadata extensively in their own work. The massive spreadsheet comparison of existing tools was the first part of the process, to see how people are approaching it now, and analyze what the best route for a core solution would be.
In its current state, the new metadata API would make it easier than it currently is to create new fields and field groups to utilize a variety of core-supported field types. Furthermore, the new API would be extensible so that developers could utilize a core-supported framework to take custom fields even further.
You can check out more on their Github page for more specific information, but something like this is really promising, and could be a game-changer for creating WordPress content if and when it goes into core.
Make it easier for developers, and better for everyone
Creating custom content types was possible before WordPress 3.0. But WordPress 3.0 made it so much easier, and clearer to folks with less development experience.
The metadata API already does a great job at making it easy to grab and use metadata. But creating beautiful custom metaboxes in the admin to enhance the publishing experience in WordPress is still relatively difficult. It’s totally possible, but I think it could be easier.
I strongly believe that an enhanced API to make it as easy as making WordPress custom post types would make WordPress a better, more reliable platform, and I think that most developers would be ecstatic to utilize a core API, versus attempting to choose amongst the fragmented metadata API landscape that exists today.
Thanks Brian for shining a light on this. I could not agree more with your position. 🙂
+1 from me too 🙂
Totally agree. I’ve always thought one of these custom field plugins will end up getting rolled into core as it adds almost essential functionality to WordPress as a CMS.
ACF is great and I love what Elliot has done to put this plugin together. I wasn’t aware of the upcoming changes with the new version – will have to look in to this a bit deeper!
It would be great if the additional plugins that you mentioned were interchangeable and only represented the UI part of handling custom fields, while core took on how this data was stored. I’m surprised this hasn’t taken more focus in WordPress 4.0.
Totally second this. Core API for metaboxes sounds like the next logical step…
I hope that Elliot from ACF will listen! Like you said ACF users need a more seamless upgrade process.
I have serious issue with referring to backwards compatibility breaks as “irresponsible” and so on.
Yes, when something stops working it sucks. It’s loss aversion — it makes you feel not just bad, but out of proportion bad to the actuals problems it creates for you.
When one of my staple (tens-times-a-day-used) desktop apps removed major feature from free version I threw a fit. Then I changed to alternative. But the size of the fit I threw was way larger that practical challenge I was facing (take out credit card or move to different app).
The thing is backwards compat compromises long term life cycle of software. It’s just technical reality. You cannot extend code base endlessly with perfect backwards compatibility without major technical sacrifices. And yes — WordPress core is compromised in serious ways by it, however chooses to live with it, in line with their priorities.
But! WordPress core priorities are not our own. We are too used to off-the-shelf priorities like that handed out and treat as holy scriptures. Comes with being a young industry.
But major versions and backwards compatibility breaks happened around much longer than WP did. They serve purposes. There are code development and consumption practices to handle practical implications of them, just as there are to handle land of eternal compat in WordPress.
We benefit greatly from backwards compatibility.
We also benefit greatly from software that is not coding itself into a toxic mess.
Nailing both is software development level hardcore. WordPress thinks it’s handling it. Maybe it does. Maybe it doesn’t as well as it thinks, depending on whom you ask.
Not everyone is WordPress. Especially not one (or even two, or even five) developer shop.
We are not owed eternal backwards compatibility. We are not owed for every theme, plugin, and framework to only ever have one major version because it’s convenient for *us* and their developers can drunk themselves to sleep for all we care.
The client’s site broke because they changed major version of a plugin? That is *your* screw up (not talking to Brian personally). You made a site on the foundation of assumption other people will bend backwards to make life effortless for you. Now go look up “dependency management”, you are supposedly a professional dammit.
Don’t like that ACF released major version? Fix the things that got broken, learn from that what your processes lack. Maybe switch to competitor if you feel that’s necessary technically or emotionally.
But the pestering of developer for it is unseemly, mean, and entitled.
I agree in principal, but from a business point of view, the convenience of your customers is a costly thing to throw away.
I suspect, most of ACF’s customers are developers doing client work. Affording them a degree of laziness is, effectively, a feature. And, as you point out, being in the WordPress industry, one that is more or less assumed. Removing it therefore will upset your customers. They may, as you suggest, take their custom elsewhere – but as a business, you don’t really want that.
Of course, it’s a matter of weighing up the code-architecture pros with the business-loss cons.
Note that in no part I claim developer handled it perfectly.
But there is a line between being upset customer and being a jerk. I think in this case WP community is being a jerk and justifies that with its [unrealistic and bubbled up] backwards compatibility expectations.
Releasing major version is not “out to ruin your life” situation. It’s normal. The reason it’s not normal in WordPress is that what “normal” is skewed and people need to be much more aware of it to throw well grounded judgments around.
In short, Stephen expresses my feelings exactly.
“Of course, it’s a matter of weighing up the code-architecture pros with the business-loss cons.”
You are correct, although one of the reasons people use WordPress is for backwards compatibility, so a plugin not meeting their expectations is sure to cause a raised eyebrow or two.
Based on? I mean it’s part of the “story”. And it has objective benefits. But how important it really is to people?
And instead of wild guess let us go look at stats http://wordpress.org/about/stats/ showing how 84% of said people don’t bother to update to current stable version. I highly doubt developer concerns like backwards compatibility are genuinely important to them.
I said “one of the reasons”, not the main reason 😛
For what it’s worth, I consider backwards compatibility to be extremely important for my own projects. I am lazy and can’t be bothered testing every update and just hope like heck that some putz isn’t going to break my site. But I’m incredibly anal about what plugins I run, so I’ve never had any problems.
Hey @Rarst,
While I pretty much agree with your sentiments, I think directing them toward Brian might be a bit misplaced? I read his post as:
OTOH I think your comment might have been more appropriately posted here:
http://chrislema.com/backward-compatibility-advanced-custom-fields/
Just sayin… 🙂
Not as much at Brian as at topic. 🙂 I explicitly wrote that I am not talking about Brian personally closer to the end.
But as for his coverage of topic I do think he screwed up with ACF angle here both by praising questionable (for my taste) post by Lema as explanation and muddling the issue in general.
Plenty people (from how discussion on twitter was going today) ended up with a lot of misunderstanding about the actual issues with and stage of ACF update, not to mention that documentation of said looks great to me and was bashed unfairly in my opinion (not sure if Brian *saw* the latest documentation however, depending on timing of it and this post being written).
That’s why I suggested Lema’s post. Seemed like commenting there would be closer to the source…
…but, I do have to agree with you there, on both counts.
Thankfully, I’ve stayed out of that fray today.
BTW, seems like about 3+ years ago you said to me it was best not to comment on the ideological rantings of this community. I guess either your opinion on that has changed, or you’ve lost the self-control not to comment, eh? 😉
I know that Brian is one of those it won’t be lost at. 🙂
@Rarst
Good point!
Rarst and I have had an open channel for many years. We certainly don’t always agree, but we always listen to one another! His is one of the most valued voices I hear in the WordPress world, even if he is oft an antagonist 🙂
@Brian – Hey, just trying to help here! I think we all know each other for many years and so I thought all voices would be valued on the topic, no?
Absolutely, I appreciate your comments Mike!
For managing a single site, I completely agree.
I think many developers forget the variety of client – developer relationships that exist in the world though. This is going to be an incredible burden on a lot of folks.
Note that I should’ve clarified I think backward compatibility is important, but shouldn’t be “not allowed”. As Stephen noted, it needs to be balanced as a code architecture vs business decision. If I have to manually configure a plugin every year for every site I use it on (let’s say that number is 50, but it could be hundreds for some folks), that’s a very real cost. And I’d rather use one where the developer chooses to make back compat a priority.
Yep, I’ve learned this lesson before. And I took your exact advice below that statement, and switched to a competitor. As Mike noted, I warned folks that they should have noted it the first time.
In this case, I think (despite how pretty the docs site is), Elliot didn’t answer some pretty important questions, which are mostly workflow related, and simply things that could aid developers bringing ACF up to speed for their clients. In addition, he was requested numerous times to do so, and from my POV he could’ve handled the feedback much better.
I get someone needing to change their business model. With a “tool” plugin like ACF, it could easily be a foundational piece of a website. So in this case, back compat is even more important, and it’s not unreasonable to be concerned about what happens when you upgrade, and if you’re going to be bombarded by past and current clients. On the other hand, it was everyone’s choice that used ACF. I myself do not use it for clients, partly for these very reasons I’ve noted.
Anyway, thanks for the comment. You make some really good points. I probably could’ve left this ACF aspect out of the article, as I would’ve much preferred the conversation to be centered around metadata UI in core, vs this. But alas…
Just a comment, in alignment with @Rarst’s comments; I’m not sure someone providing a free plugin has any obligation to users whatsoever. Yes ACF has muddied those waters by having both free and paid plugins, but even so he made his choices and now the chips can fall where they may.
Now I’m not saying that people should use plugins where the developer has no obligation, only that they should not expect anything of the developer. If they are building a site that would be harmed if a plugin causes them a problem they should know in advance that that is a risk they are taking and if the risk becomes reality then they’ve got no one to blame but themselves. Or they should buy a plugin that is better supported, or build (have built) a custom one.
From the developer’s perspective, if they want to keep and grow users then they need to do a better job of maintaining backward compatibility. But it’s not a moral imperative, it’s just sound “business” where “business” here could mean maintaining a large user base or strengthening a personal brand, if that’s what motivates the developer.
But even so, companies like Oracle, Microsoft, and Apple make breaking changes all the time, so commercial support is no guarantee of panacea. The irony is that most companies just suck it up and fix the problems but the users most effected are the ones with no budget to fix problems because they no business model or one that generates very little revenue. And that’s probably why they get so mad about it, even though they really have no real right to.
There’s an old story told about a man that got kicked by a jackass; he considered the source and then went about his business.
Fully agree with the post.
I believe this is being worked on in core but they need some more help:
https://github.com/wordpress-metadata/metadata-ui-api
In my opinion a good API for metaboxes (inc repeatable data etc) would be a huge improvement for WordPress and stop the current fragmentation of methods.
Those of you having problems are probably better off using the version maintained by 10up anyway … https://github.com/10up/secured-advanced-custom-fields
I’m guessing they’ll keep it backwards compatible, but that’s just a guess on my part.
I’ve never been wiling to use ACF in my own projects. I was recently introduced to Custom Meta Boxes by Human Made though and quite like it. I find it is simpler and easier to work with than ACF, albeit the feature set isn’t as good … https://github.com/humanmade/Custom-Meta-Boxes
Estou entrando agora? Tudo é novidade mais estou de olho, no WORDPRESS?
Both this page: http://www.advancedcustomfields.com/resources/updates/upgrading-v4-v5/
and the general “What’s New in Version 5” page:
http://www.advancedcustomfields.com/resources/updates/whats-new-version-5/
cover what is changing in ACF. I can see if someone clicks “Update” blindly they might be in trouble, but I don’t understand the headache – “(the existing posts and documentation is very confusing)”. I just read it and it seems pretty straight forward.
Brian, you shot the nail!
I think that plugins like ACF are great but they aren’t the final solution to push WP towards CMS functionalities, some improvement to handle post meta data should be implemented into the core, but I would like to choose where put these data if inside wp_postmeta or in other tables/sources.
The power of WP is simplicity and extensibility so we should think how add custom (strongly typed?) properties to posts (and, why not, users, categories, etc), without breaking WP philosophy but with the power to automatically display metaboxes for these “properties”. It should be interesting also make searches and filtering posts on these properties, am I asking for the moon?
Hi @Taloweb,
Hi @mikeschinkel,
thanks for your interest, but this is not a simple feature request nor an issue posting by now, my comment was a “can we push WP to be a real CMS? How?”.
We have to discuss about it, so WP can go “a step further”, don’t you think so?
@taloweb:
Certainly. Which is why I asked for specifics. If we only speak in abstract then we’ll not move forward. BTW, I’m currently working with Scott K Clark and others on the #metadata feature-as-plugin which is why I was specifically interested in your ideas. But if you (and others) don’t post your ideas there to discuss, we’ll have no other option than to simply build what we think is important which might not cover everyone’s needs.
Said another way, if you want to have discussions and/or input, then https://github.com/wordpress-metadata/metadata-ui-api/issues is going to be your best bet.
We’ve been discussing this exact topic for years, and more recently as the WP Metadata API / UI project has moved forward, we’ve had numerous plugin authors who have built Custom Field plugins involved. We’ve considered a lot of options, patterns, and features, but we’re still open to good ideas. We won’t turn away a good idea if it makes sense and meets the requirements for implementation in WP core.
I posted a new issue on Github, I hope it could be useful, but I never used Github before, so please be patient…
https://github.com/wordpress-metadata/metadata-ui-api/issues/35
Just wondering why neither this article, nor Chris Lema’s, not S.Clark’s Gdocs sheet mention the very flexible: https://wordpress.org/plugins/custom-field-template/
Send me the column details and I’ll add it. It wasn’t intentionally left off, there are too many to cover on my own.
Hey guys.
Just wanted to quickly address the upgrade concern before I write an article about it.
As mentioned in the upgrade documentation, if an ACF4 user upgrades to ACF5 (when it is publicly released in a month or so) and is using 1 or more of the ACF4 add-ons, a clearly written message will display and allow the user to downgrade from ACF5 back to ACF4 with the click of a button.
Thanks
E
Why, for the love of god, WHY is wp_postmeta a separate table?
Why not just add an additional field in the wp_posts table? WHY WHY WHY??
This is stupid design right there. To this date there is no easy way to grab post data together with its meta. Everything just has to result in a gazhillion database requests..
This is why your WordPress, ACF and all other kinds of Advanced Custom Fucks suck.
I’d probably venture to say you’ve never dealt with scalable database design with millions upon millions of sites..
I won’t deny, tables are better when they have columns for what they need, indexes, etc. But putting everything into wp_posts would be even worse because there could be tens to hundreds or more post types using that table with god knows how many weird custom fields someone would want.
Konnie, wp_postmeta is a great place to store additional info for posts since you can’t store it in wp_posts: we don’t know before “what” info needs to be stored.
The problem about wp_postmeta is that every theme or plugin can store any information in it, using any key, that can cause conflict or a lot of garbage written (and forgotted when plugin is deleted), an improvement could be to ask to plugin developers to declare (in some easy but usable way) which values are stored in meta_key…