Drupal 8: quá tuyệt vời nhưng quá phức tạp

Two weeks ago (hey, I've been busy and trying to sleep for once), after 1716 days of work by more than 3312 people the Drupal community finally released Drupal 8, the latest release of the best community-driven web software in the world. The blogosphere is already filled with congratulatory blog posts celebrating the immense accomplishment, and deservedly so.

>> Hướng dẫn Creating a Custom Filter trong Drupal 8

>> Thông tin về HTTP với Drupal::httpClient

A number of people recently have asked me how I feel about Drupal 8's release, especially around the PHP community. Overall, my answer has to be that I'm happy, but not satisfied.

Happy...

Where to start? There's so much to be happy about in Drupal 8, even if we just look at the parts I was involved in (which is not all of them, or even all of the important parts).

Drupal 8 is proof that you can, in fact, teach an old codebase new tricks. As I cover in my Eating ElePHPants keynote (slides), Drupal is the poster child for the PHP Renaissance, shifting itself in a "mere" 3-4 years (depending on how you count) from a PHP 4 procedural system to a PHP 5 mostly-OOP system. Yes, massive refactoring and revolution can be done. Drupal has proven it. You can do it too.

For me, though, the story goes back even further. I was talking about the need for separation between the body of a response message and its "envelope" as far back as 2008. The first public presentation for what later evolved into both the WSCCI and SCOTCH initiatives was at DrupalCon San Francisco 2010, with follow-ups at Copenhagen 2010 and Chicago 2011 before the Core Initiatives even kicked off.

Drupal 8.0.0 is the last 7 years of my life coming to fruition. Yeah, I'm happy.

I'm happy that we have a real framework underneath Drupal now, one that doesn't assume the whole world is complete HTML pages rendered and sent all at once.

I'm happy that Drupal has finally broken its NIH chains, and encouraged other projects to do so. When we started WSCCI, I hoped to use one or two 3rd party libraries for low-level handling, maybe. At least get some inspiration from them. I never dreamed we'd break this far out of our old mold and embrace so much code-from-someone-else.

I'm happy that Drupal has finally shifted to modern, less-proprietary development styles, such as OOP, dependency injection, real solitary unit testing, interfaces, and so on. When we started WSCCI, I insisted that we build at least our new stuff in a clean, OOP fashion. I figured that, like the Drupal 7 database layer before it, it would be another beach head to showing how much better modern dev practices are than what we'd been doing for years and, over the next 2 major releases or so, we'd finally start to embrace OOP. I never dreamed that would happen in "only" one major release, but once the dam broke there was no holding it back. The community wanted, needed that shift and was waiting only for an excuse to do so. We gave it to them. And I've learned a ton about good OOP myself in the process, including how much good OOP seems to mimic good functional code. Even Dependency Injection is something I didn't properly grok until more than a year into the Drupal 8 cycle! (I blame one of Fabien Potencier's presentations for finally helping me over that hump.)

I'm happy that we're finally starting to understand decoupled components. There's even a push to start breaking off Drupal components to allow others outside of Drupal to use them as well. That was very much the goal of splitting our namespace into \Drupal\Core and \Drupal\Component, and why I pushed for that in the first place. By 8.1 it may finally get completed, and you'll be able to composer install Drupal-provided libraries in your own non-Drupal projects.

I'm happy that we have a common extension mechanism in core, finally, in the form of plugins. I first presented on "Handlers", as I was calling them at the time, at DrupalCon Szeged in 2008. The design has shifted greatly since then, of course. If you go back to some of my old blog posts you can see me trying to figure out dependency injection along the way. In a sense, what I was originally talking about forked into the Plugins system and the Dependency Injection container (which we borrowed from Symfony, of course). I had incorrectly conflated the two originally. Where we ended up — with a common, easily-learnable, reasonably self-documenting system — is in some ways what I was hoping for and in others far more than I could have ever hoped for.

I'm happy that we finally have a concept of configuration! Drupal has always valued user-configuration as its primary feature, so it's quite ironic that until Drupal 8 it didn't have a common configuration toolchain. That configurability was enforced solely through cultural expectation, which is why the technical side was always so frustratingly weak and why anyone who's built a Drupal site knows the pain of trying to sort out deployment, shared development environments, and staging changes. Drupal 8 tackled that head-on, with a configuration system that resolves 80% of the pain of dealing with configuration as both a developer and a site builder.

I'm happy that others were able to run with and build on the work we've done. The best example here is the support for alternate page rendering strategies, including BigPipe, that finally snuck in very late in the game, even during RCs. Support for rendering blocks out-of-order, out-of-band, and caching them independency of each other was one of if not the root goal of the Butler project, which eventually became WSCCI. We were even discussing BigPipe all the way back in San Francisco in 2010. Although the mechanism used is very different, the underlying concept is still there and it is, finally, a reality for Drupal site builders the world over.

I'm happy, no, ecstatic, that we were able to finally break the logjam and bring large numbers of front-end developers into core development. That's been a dire need for Drupal for a long time, and in Drupal 8 it finally happened. You can see that in the vastly improved markup Drupal 8 produces, in the abandonment of ancient browsers (we were finally forward-looking for a change!), in the adoption of new tools and techniques on the front-end like CSS3 and SVG, in the "responsive-first" approach taken by most of core now, and even in the experimental support for new technologies like the picture element. Drupal 8 had one of the first experimental implementations of that API, and it evolved in core along with the development of the specification itself. And the picture element is now in the release or development branch of every single major browser save Safari. I'd like to think Drupal had some small part in that, even past the original impromptu meeting that spawned it (which included a number of leading Drupalers).

I'm happy that we have been paying far closer attention to usability and consistency in the UI. We finally added UX maintainers to the core team, and that can be seen in the improved workflow throughout the system. That dovetails with the improved APIs, too. By building better underlying tools to help developers build out UIs without much work, especially around entities, we've created a much more consistent and therefore learnable and usable system for all users.

I'm happy that we did end up adding more capability to core, such as the Views module. To be honest, I was initially against adding Views to core. I felt it needed to evolve independently, and given the number of other moving parts already going into Drupal 8 felt it would be a distraction. One more thing to break patches, to compete with on developer time, and to compete with on funding for developer time. I was, I'm pleased to say, quite wrong. Views was self-contained enough that it didn't end up blocking other work, and if anything acted as a canary to really prove that we thought through the new underlying systems (or in some cases didn't, and needed to fix them). It brought several new people into core development, including some of our most important Drupal 8 developers. But it was really Deane Barker who, unwittingly, convinced me that a CMS without a content assembly tool was incomplete. Even keeping Drupal core "slim", content assembly is just as important as content creation. And with Drupal 8, we have both, in core, from day 1.

I am happy that we managed to add a WYSIWYG/Rich Text Editor to core that doesn't suck. It only took until 2015 for such a thing to exist in the first place. :-)

I'm happy that all of the changes we've made have enabled Drupal to switch to a more traditional, predictable, stable release schedule. Drupal has never really had feature releases before; we've had bug fixes, and API-breaking majors, but our architecture has never really supported non-breaking improvements before, at least not on a large scale. Now it can, and we're now shifting our development culture to support it, too. That makes an investment in Drupal 8, and in helping to develop Drupal 8 itself, much more valuable than ever before.

I'm happy that the Drupal community has spearheaded one of the best new-contributor programs in all of Free and Open Source software. Mentoring is such an integral part of learning, and I don't know any project that puts more time and energy into it than Drupal. At conferences and sprints, in IRC, through online office hours and even just ad hoc instances, there are many people in Drupal whose main contribution is simply in helping others get involved. It's not possible to overstate how important that is, not just in terms of recruitment but in terms of culture. Drupal has always been, and remains, a culture that believes in helping others, in doing good for someone else, in supporting other people, in learning and in teaching. That core value is something that didn't change in Drupal 8, and may it never change.

... but not Satisfied

With so much to be happy for, why aren't I satisfied? Well, we shouldn't be. Satisfied means there's nothing left to do, nothing left to improve, nothing left to fight for. Satisfied means done. Software is never done. Certainly not Drupal. :-)

I'm not satisfied with our OOP. Certainly it's leaps and bounds better than it was in the past. Yet we still rely too heavily on "procedural code wrapped in class keywords". We still rely too heavily on Arrays of Doom in many places.

I am not satisfied with our use of inheritance. Too many parts of the system, especially around Plugins, only pay lip services to interfaces and in practice mandate extending base classes. These limitations will bite us in the future, and also undermine the self-documentation benefits of interface-driven design. (The Views and Blocks systems are worst here.) In most cases that was done in the name of making systems easier. As is so often the case, that means we've sacrificed simple in the name of easy. We likely won't be able to fix that for real until Drupal 9, although some workarounds may be possible in the interim.

I am not satisfied with how far we pushed proper dependency injection. A statement I repeated at length throughout the Drupal 8 development process was that if you're accessing the container through the global static \Drupal class from a method, you're doing it wrong. I still firmly hold to that position. Sadly, there are some parts of the system where that is the only possible way to implement an API. That means in those areas, we did it wrong. Entity API and Typed Data are far and away the worst offenders here, and I fear that is going to be a problem hanging around our necks for the entire Drupal 8 cycle.

I am not satisfied with our testability. We've made enormous strides here, and adopting PHPUnit as the basis of our testing strategy was long-overdue, but we still fell short in many areas. Every one of those places that some class statically calls the global container is an API that cannot be solitarily united tested, and, more importantly, neither can anything that builds on it. That includes the aforementioned Entity API. While we have various container-hacking mechanisms in place that can handle some of those cases, those are still nothing but cheap hacks to avoid properly writing code in the first place. Those don't count. If it's not straightforward to solitary unit test, you're doing it wrong.

I'm not satisfied that we still have render arrays. I've made no secret of my dislike for render arrays, including calling them "Drupal's biggest mistake". In Drupal 8 they actually serve a useful purpose, specifically as the common communication channel for the all-important cache information that is at the core of Drupal's performance capabilities. Their DX still royally sucks, though, as even those who built the new systems on it will freely admit. While I attempted to build the start of a new, more self-documenting toolchain for it that was removed in the name of consistency and time. (I've started rebuilding it independently of Drupal for later use.) I am very disappointed that with all the great work that went into this area we chickened out on improving our largest DX weak spot in the name of expediency. In the end, I think that cost us more time than it saved.

I am not satisfied with our Composer adoption. A great deal of work is still happening in this area, but in the end our Composer usage in 8.0 is still weaker than it should be, mostly because we dragged our feet for so long. This is an entirely self-inflicted wound, based on our collective unwillingness to bite the bullet for over 3 years. By the time we mostly did, it was too late. Hopefully this should be cleaned up by 8.1, though.

I am not satisfied with our REST support. Ironic given my role, but there you go. Core's REST support is orders of magnitude better than Drupal 7's, but we didn't go far enough. Despite our best efforts Drupal 8 not truly a REST-first system. Some of that is simple hindsight being 20/20; there's a number of things I would have recommended we do differently had I known 2 years ago what I know now, mostly around decoupling storage and serialization. However, we also did not go anywhere near far enough with hypermedia, in part due to time but also due to our at times haphazard and disorganized development process that threw a wrench into the API work needed to support it and delayed it by several months. We have a roadmap defined for improving that greatly from contrib or in 8.1, though. Volunteers welcome!

I am not satisfied with the degree to which we've moved away from hooks, or rather haven't. Many in Drupal's leadership said 2.5 years ago that hooks were doomed in the long run, and events, plugins, or tagged services were appropriate instead. We did remove a huge number of hooks in favor of cleaner, more testable alternatives but not enough. We even avoided changing some over, mainly in the entity API, again in the name of "we don't have enough time" 2 years before Drupal 8 even reached RC. These will also be drag for years to come if we don't start building their transition path now. (Tip: If you're a contrib author writing a module you want others to extend, do not define a new hook. Use an event, plugin, or tagged service depending on what you're trying to do. No more hooks.)

I'm not satisfied with our lack of layout support in core. The Butler initiative, which turned into the WSCCI initiative, which the forked the SCOTCH initiative (notice a pattern emerging?), was born not from web services but from better blocks and layouts, inspired by the Panels suite. Much work was done here under the SCOTCH initiative, building on the plumbing laid down by WSCCI and others, but once again most of it it got relegated to contrib in the name of time and avoiding scope creep years before RC happened. In the end perhaps that's for the best, as the work happening now for Page Manager/Panels/Panelizer in contrib, with the fully developed Drupal 8 toolchain, is looking very promising. Time will tell. Even then, though, despite my criticisms above the Blocks system in Drupal 8 is only an order of magnitude better than Drupal 7's.

In short, I am ecstatic by how much we have accomplished, but I know that we could have and should have done better. Many improvements were within our grasp, but we didn't get to them in many cases out of not fully committing, which ended up costing more time than it saved and leaving us with weaker systems than if we'd just let go of our fear and committed.

Still thankful

OK, I missed Thanksgiving by a week too. I have international readers, OK?

It's cliche to call Drupal 8 a team effort at this point, but it's still absolutely true. No one person "made" Drupal 8. Over 3000 did. This blog is long enough that I cannot thank everyone individually, but there are still a number of people and organizations I want to call out specifically (in absolutely no particular order).

I am thankful for Daniel Wehner and Tim Plunkett. Both were originally part of the Views team, and when Views landed in core they were left with nothing to do. So they started working on just about everything else in core. I don't know that there's a system they didn't touch on their way to becoming the undisputed #1 and #2 code contributors to Drupal 8. Daniel seemed to somehow have infinite time to help out with WSCCI issues, and Tim periodically got fed up with something and so moderned an entire subsystem out of spite. (We need to make Tim fed up with us more often...)

I am thankful for Jess Myrbo, Andrea Soper, Cathy Theys, Gabor Hojtsy, and all of the other mentorship leaders who put together Core Office Hours, organized new contributor sprints at DrupalCons year after year, and otherwise served as the on-ramp for hundreds of new contributors. Some stayed, some did not, but the presence of that welcome wagon — every day, every event, every IRC channel — is crucial to Drupal as a project and as a community.

I am thankful for Fabien Potencier, Lukas Smith, Christophe Coevoet, David Buckmann, and the rest of the Symfony and Symfony CMF teams. Symfony provided more than just code to Drupal; it provided a concrete example around which to crystalize and on which to build. We saved hundreds of hours when faced with a new issue by simply going "What Would Symfony Do?" The answer to that wasn't always ideal, but it was usually workable and saved us figuring things out from scratch ourselves so we could move forward. Fabien in particular went out of his way, repeatedly, to help Drupal developers grok what they were getting into, accept PRs to Symfony that made sense for Drupal, and otherwise be a good PHP citizen for us.

I am thankful for Klaus Purer and Lin Clark, who actually built the REST, HAL, and Serialization modules for Drupal 8's built-in REST support. My original plan for the routing system essentially stopped at making such modules possible; It was Klaus and Lin that ran with that capability and built something on top of it that site builders could leverage.

I am thankful for Jen Lampton, Morten DK, Fabian Franz, and the other leaders of the Twig informal initiative. Not just because Twig is a huge improvement for Drupal's theming layer (which it is), but because that project brought dozens of front-end developers into core development for the first time. Drupal is not just by-code nerds for-code nerds. It now has HTML nerds, CSS nerds, and design nerds, too. Morten in particular has had a tenacity (OK, stubbornness) for exterminating divs and making the output of Drupal more amenable to front-end developers that matches my own tenacity in stomping out the last vestiges of PHP 4 from the code base. :-)

I am thankful for my fellow Official Initiative Leads — Greg Dunlap, Alex Pott, Jacine Luisi, Gabor Hojtsy, John Albin Wilkins, Daniel Wehner, and Kris Vanderwater. Being an initiative lead is a tough, thankless, officially powerless job that we all made up largely as we went. Yet in the end were able to shepherd an army of cats into a remarkably similar direction.

I am thankful for Wim Leers and Fabian Franz of the "cache all the things" de facto initiative. I like to tease Wim about his first DrupalCon, Barcelona 2007 when he was my roommate and still star struck at even being in the same room as people like chx, webchick, or myself. (How I ended up on that list back in 2007 I still don't know.) Now, he's one of the lynchpins of Drupal's performance efforts. Wim and I have not always agreed on architectural direction, but he's always listened and always been open to discussion and debate. He's also been very supportive of me, personally, in the last year when the stress of core development has hit me especially hard.

I am thankful for Alex Bronstein, who is perhaps the most level-headed, conflict-defusing person I've ever met in my life, and a razor-sharp developer to boot.

I am thankful for the PHP community at large, which has welcomed Drupal with open arms as it's come out of its isolated shell and into the broader PHP community. And especially for people like Cal Evans and Eli White that have actively helped to spread the Proudly Invented Elsewhere, "Off the Island" message far and wide.

I am thankful for Acquia, who far and away has put more cold hard cash behind Drupal developer time than any other company. So much of Drupal 8 would simply never have existed without their continual financial backing.

I am thankful for the many other Drupal companies that have funded significant developer time on Drupal 8, including (but by no means limited to) Chapter Three, Previous Next, Blink Reaction (now FFW), NodeOne (now Wunderkraut), BlackMesh, and Commerce Guys, just to name the ones I can think of off hand. (I'm sure I'm missing some key ones.)

I am incredibly thankful for my employer, Palantir.net, who's kept me fed through the last several years. Palantir was for a time supporting not one but two core initiative leads (myself and John Albin Wilkins), in addition to direct financial support to the Drupal Association in various forms. While extremely little of the code I wrote for Drupal 8 was on Palantir time, if I reviewed your patch or ran a meeting or took a phone call it's a safe bet that Palantir was paying for it (even if technically I was supposed to be doing client work). For the past two years I've been effectively a Drupal developer evangelist, spreading both Drupal and Palantir knowledge across the PHP world (and I mean world quite literally, from Bulgaria to Bogota to Wellington) as part of my role at Palantir. Palantir is a company that firmly believes in the collaborative atmosphere that has taken over PHP in the last few years, and has been very supportive of my role in helping to push and celebrate that evolution.

And of course I am thankful for our project lead, Dries Buytaert. I've disagreed with Dries publicly on numerous occasions, both directly and implicitly, and rarely shy from calling him out when I think he's wrong. Nonetheless, Dries has managed to build a truly amazing community around Drupal. Not many development communities could survive, much less become even stronger from, a dramatic upheaval like Drupal has gone through in recent years and no doubt will continue to go through. He's patient and gracious to a fault, and still surprisingly down to earth and unassuming. It was Dries that sat down on the floor next to me at the DrupalCon Sunnyvale after-party in 2007 to give me the go-ahead and encouragement for what became the Drupal 7 database layer, the GoPHP5 project, and all the work that came after. He also took the gamble on Symfony that I was pushing for at the start of 2012, and it has paid off in spades (and diamonds, clubs, and especially hearts).

There's always Drupal 9 Drupal 8.1

In the end, Drupal 8 is an amazing piece of work by an amazing group of people, and I am proud for the role I was able to play in this transformative, ground-breaking process. Hopefully it continues to serve as an example to other projects of just what a healthy community can do if it puts its mind to it.

That said, make no mistake. We're not done. Not even close. There is plenty of work still to do to make Drupal the system it has the potential to become, and to help ensure that we never fall behind again the way we had before Drupal 8. Some of that work will need to wait for Drupal 9, but much of it can be done today. It just takes the commitment to keep improving, the refusal to settle for "good enough", and the willingness to challenge yourself and others to make tomorrow's work ever better than today's, even if it's uncomfortable to admit how far there is left to go.

That's how we got this far, and that's what will take us the rest of the way and beyond.

We should all be happy of our accomplishments. But never, ever be satisfied.