Shortcode syntax and API changes drafted and under review

It’s been a busy few days for shortcodes. Most, if not all, core committers and project leads would tell you that they’d rather burn the shortcodes API than anything else. But it’s one of those things that exists and now must be dealt with.

In short, parsing shortcodes is super hard, and the lack of a strict API has meant that people use shortcodes in all sorts of crazy ways. The lack of defined ways to use it makes it harder to parse them and harder to make sure that they are safe.

I understand that after the big backlash to the first draft for refactoring shortcode syntax, a team spent a very long day going back to the drawing board and coming up with further options and another roadmap. That discussion was pared down quite a bit into the second post on the matter, which further describes some of the challenges at hand and sets a time for a meeting on Wednesday, September 9th.

I think Lead Developer Andrew Nacin’s response to the feedback on the first place is worth copying in its entirety:

Thanks for your feedback, everyone. Developers, truly, honestly, really: Thank you for paying attention.I often feel that requests for comments usually fall on deaf ears, and I know I am not the only committer who feels that way. This has been one of the most substantial and productive threads on make/core I’ve seen, and it’s encouraging to see.

Here are some thoughts on the proposal:

• This was clearly labeled a “draft”. I am glad many of you noticed that this was not a decree, but a call for feedback. (And thank you again for obliging.)
• The vision of shortcodes should be something like “it should be easy and intuitive for non-technical authors to embed and enrich their posts” (I have cribbed this from @matt).
• This vision does not actually mean that shortcodes are the solution. I actually have a strong dislike for shortcodes. I think they are a terrible user experience, and that even something like Shortcake is still only a marginal improvement.
• Some changes to shortcode parsing is absolutely necessary in order to keep WordPress secure. Yes, this is about security, not just general sanity.
• The proposed syntax significantly clashes with the proposed vision. And given all of your feedback, we’re clearly going to have to go back to the drawing board. Please note that we still need to do something, but maybe we can think further outside the box.
• And for goodness sake, PLEASE STOP RELYING ON HTML IN SHORTCODE ATTRIBUTES. This also does not align with the outlined vision, either, and it’s what largely precipitates proposals like these.
• Thank you @miqrogroove for your incredibly hard work on this. Honestly, folks, many of you simply cannot see how much work Robert has poured into shortcodes over the last year, as it takes place in private security discussions. He’s been a tremendous asset to the project.

I’m looking forward to future make/core threads getting this much high quality feedback.

I don’t have a ton to add to this conversation yet. Basically, I see both sides. I agree with Mika that the curly bracket option is a non-start for users (and also brought up later as a big language issue). But something must be done. Parsing endless layers of shortcodes is tedious to say the least.

Mika Epstein has really made great comments on both points. She does a great job on the second post describing some questions folks might still have, and I look forward to more discussion on her points, and of course I look forward to the chat Wednesday.