The State of Drupal 7 Contrib

While this article focuses primarily on the state of Drupal “contrib” (modules and themes which are not part of the “core” Drupal download), it also takes a look at the greater “State of Drupal” in terms of sites known to be running on some version of Drupal, comparisons of the rate of uptake after Drupal 6 and Drupal 7 release, and a small case study involving attempting to perform a “major upgrade” from Drupal 6 to Drupal 7 on a site using a significant number of contributed modules.

The recent history of Drupal core usage

As a starting point, I think it is helpful to look at the recent history of Drupal core usage and compare the uptake of Drupal 6, after its release, with the uptake of Drupal 7. On June 22nd, 2008, when Drupal 6.0 was released, there were already significantly more sites using Drupal 6 than Drupal 5 (almost 32,000 on Drupal 6 vs almost 17,000 on Drupal 5). Both core versions of Drupal steadily gained users for a time, with Drupal 5 reaching a peak of about 24,000 sites about 7 months later, but by that time Drupal 6 was running on more than 100,000 sites. By late July 2009 (a similar point to now in terms of months after the major version release), Drupal 5 usage had dropped to about 20,000 sites and Drupal 6 was running on more than 160,000 sites; more than eight times as many installations. Since then, Drupal 5 usage has tapered to about 7,000 sites; a bit more than 1% of total Drupal usage (please note: it’s likely that many of the existing Drupal 5 sites do not report usage back to Drupal.org).

Now let’s look at the usage of Drupal 6 and Drupal 7 since the time of Drupal 7’s release. Drupal 6 peaked with about 355,000 sites, shortly after Drupal 7’s release in January 2011. At that time Drupal 7 was running on about 24,000 sites, a fraction of Drupal 6 usage at that time. Since then, while sites running on Drupal 7 have steadily increased to their present values, about 280,000 sites, Drupal 6 has hovered around the same value, drifting between about 320,000 and 350,000 sites, but not yet significantly dropping. Almost 13 months after Drupal 7’s official release, we still have more sites running Drupal 6 than Drupal 7 (and I suspect that a significant percentage of the Drupal 7 sites are in development rather than production). But what does this really say? Let’s look a bit closer at the numbers and trends:

Note: I banged this graph out in Excel since the Google chart of Drupal usage, normally displayed on project pages, seems to fail as “too large to process” for “core” usage statistics.

Drupal usage has grown by leaps and bounds since Drupal 6’s release. In June 2008, there were fewer than 50,000 sites using Drupal 5 and 6 combined. Now, a bit more than three-and-a-half years later, there are more than 615,000 sites running on some version of Drupal — more than a 12-fold increase in that time period! A year ago, this figure was less than 400,000, so Drupal 7 sites make up a large proportion of the more-than-200,000 Drupal sites added since then. The growth was steeper after Drupal 6’s release, but we still did not have 200,000 sites, total, by July 2009. In any case, it’s safe to say that for most use cases, we have the modules necessary to build a good site based on Drupal 7, so if you are hesitant to use it, don’t be. There are many great advantages to Drupal 7 and with the continual improvement of the contributed modules, we should probably build new sites on Drupal 6, only if modules critical to the use case are lacking for Drupal 7 (or if the “new” site is another site in an existing Drupal 6 “multi-sites” installation). Even if a “critical” module exists for Drupal 6 and not yet for Drupal 7, it may still be worth building the site on Drupal 7 if you have the coding experience to port the Drupal 6 module to Drupal 7, which would help alleviate the current issue that many significant modules are not yet available for Drupal 7.

State of Drupal 7 contrib (modules)

Good news: Almost all “Top 100” Drupal 6 Modules are ported to Drupal 7

The good news, especially for site builders creating a new Drupal 7 site, is that most of the top 100 modules are ready for use on Drupal 7. Nine of them are redundant (now included in “core”), 43 have “stable” releases, 23 have beta or RC, 11 have an alpha release, and 9 are in “dev” status, while a couple others recommend using another module which performs similarly.

Depending on your use case and needs, it is likely that for common needs a site builder will be able to build a Drupal 7 site without any worries about whether Drupal 6 might have been a better choice at this time. As more of the bugs are fixed, this should only get better. Of course it’s worth pointing out that there is no general consensus about what defines a “stable” release, release candidate, beta, alpha, or dev version. I’ve personally used dev versions on sites I’ve built which seemed to be rock solid releases and have attempted to use “stable” releases which did not seem to work at all. It often comes down to the particulars of a use case, so additional complications such as multilingual support or random compatibility issues with the other 100 modules activated on a site can be factors which make one user’s “stable” another user’s headache. In any case, as far as the most-used modules are concerned, things are looking reasonably good, especially considering that since the time of Drupal 7’s release, developers have also had to get over the learning curve associated with the Drupal.org migration to Git (migration from CVS to Git was finished in May 2011, a few months after Drupal 7 was released).

Many other popular modules are not yet ported to Drupal 7

On the other hand, many useful modules may never be ported if they aren’t used on a significant percentage of Drupal sites, and many sites doing anything at all unique are likely using at least one module which is not ready for Drupal 7. The median D6 module is used by about 0.1% of sites*, but such modules still account for A LOT of Drupal installations and many may be critical for a particular use case, such as the “Project” module used on Drupal.org, which has not yet even got a dev release for Drupal 7.

*How did I reach the conclusion that most Drupal 6 modules are used by fewer than 1 in 1,000 sites?
If you search Drupal modules by full-projects compatible with Drupal 6, sorted by decreasing popularity, you will find that there are about 257 pages of results with 25 modules per page, currently 6410 modules. (By the way, in December, when I started putting together this article, that figure has gone up by about 100 modules since I started researching this article back in December). Everything past about page 200 is probably obsolete, experimental, or no longer used by any sites or just a handful of sites. So let’s consider about 5,000 modules are viable projects. Around page 70, you’ll find there are 300 or fewer sites using the module (less than 1 in 1,000). So the majority of the viable modules available for Drupal 6 are used by less than one in a thousand sites. While the need for a significant number of modules is obviated by the additional functionalities added to Drupal 7 “core” modules, this is probably not more than a couple percent of all the viable Drupal 6 modules. They say that “necessity is the mother of invention”, but it applies equally to “re-invention”. If a developer has many other modules they have written for Drupal 6 and have a personal need or a lot of people begging them for the Drupal 7 port, and do not, themselves, have a particular need for some of their “niche need” modules, these modules may never get ported to Drupal 7. If you take a random sampling of modules on or past page 70 of these results, you’ll probably see, as I did, that there are very few modules ported to Drupal 7, and this number steadily decreases as you move through the pages. At this time, more than a year after the release of Drupal 7, I found exactly three stable Drupal 7 releases out of the 25 modules on page 70, with some development for Drupal 7 on about a third of them. This is actually significantly better than the picture only a couple months ago (when I saw only one stable Drupal 7 release), but it will likely be some time yet before many other useful modules have been ported by the community.

You have to go to about page 18 to get to about the 1% use rate (about 3,000 sites for each module). At this point, my examination shows that two of those modules won’t be ported since their functionality is in the D7 core, and eight of the other 23 have a beta, release candidate, or “stable” release, and of the remaining modules, there is at least some development (dev version or a port in progress) for another 7, so the majority of modules used by at least 1% of sites seem to have at least some level of Drupal 7 support, even if somewhat less than half are really ready for production use; and these are modules in the “top 500”. Looking at the next page, the numbers are somewhat lower, but there are at least 10 (of 25) modules which are partially or fully ported to Drupal 7, discussion of a Drupal 7 version for a couple of others, one which is redundant (functionality now in “core”), and proposed alternatives for some of the rest. Again, as you move to modules used by fewer sites, the tendency is that development has either been abandoned or there is less work on a D7 port.

To sum it up: if you are building a new site, and need the functionality of a particular Drupal 6 module, you’ll probably find that there are alternative modules which do something similar and where development for Drupal 7 is further along, so for most Drupal 7 sites and use cases, you may be able to build a great site without the need for much (if any) custom coding or waiting for module availability.

State of the Drupal 6 to Drupal 7 upgrade path

Personal case study of an attempt to upgrade a Drupal 6 site to Drupal 7

Background: Before I started working for Cocomore, I built a personal Drupal project (which unfortunately has not received much of my attention since I launched the site). My year of full-time work, with Cocomore on Drupal projects, first an internship and then as a developer trainee, and additional Drupal self-study in that time helped keep me from working on that project while also helping me reach the conclusion that there was a lot more that the site needs besides what I already knew about (even when I first put it “live”, I considered it “under construction”). Since I hadn’t yet had the opportunity to work on a “major upgrade” from Drupal 6 to Drupal 7, I figured this was a good place to start, since I was the “client” and could decide to trim features or revamp functionality as I saw fit. Of course I decided it was best to start with a “local” installation of the site, so I took a database dump from the live site and a copy of all the Drupal 6 files and followed all the upgrade instructions to get an impression of how ready Drupal 7 is for migration of a semi-complex Drupal 6 site (the site uses quite a lot of “contrib” modules, but currently no real custom ones.)

The site uses most of the common modules, of course: CCK, Views, and a lot of CCK extension modules like Email Field, Link, FileField, ImageField, Node Reference URL Widget, among others. It also uses Content Profile, almost all optional D6 “core” modules, the Date and Calendar modules, Image, ImageCache, a number of input filter modules (e.g. BBCode and Smileys, now Smiley in D7) without which, some of the site’s content will look broken. The GMap and Location, and Nodewords (now Metatag) modules are particularly important for my use case. The site also uses the Advanced help, Advanced Forum, Better Formats, Comment Notify, Frequently Asked Questions, and Global Redirect modules, though they are lower priority for my current needs. Other modules include the HTML Purifier, IMCE, Login Security, LoginToboggan, Modr8, Mollom, Password Policy, Password Strenth, Pathauto, Search 404, Sections, ShareThis, Site map, Taxonomy title, Thickbox, jQuery UI and jQuery Update, Wywiwyg, External Links, Taxonomy Manager, Taxonomy Menu, Captcha, Bad Behavior, Service Links, Page Title, Rules, Weather, Yahoo Weather Forecast, Token, and a few other modules. Phew!

I was prepared to do without the benefits of some of these modules since their use, in many cases, was “educational”, i.e. mostly to get some experience while adding and experimenting with a cool feature that I might want to use on another project. Others were more critical for my data and I don’t want to do without them (or at least a similar module). And then there are yet others which are sitting dormant on my D6 site and which I do want to use, at least at some point. I list all these modules simply to illustrate that a site with a reasonable feature-set has a lot of dependencies for upgrading and obviously a lot of interdependencies which need to be met in a particular order or you will have problems. And ironically, many of the modules whose functionality is now “in core” are the ones causing issues for my upgrade. Go figure.

I want to keep this story short, so I’ll simply say that, even with many modules removed from the site or not yet enabled, I started getting site-crashing fatal errors when activating certain modules (and would need to restore my database and then avoid activating that module again as I continued with the task of carefully activating a module at a time, running update and backing up my database if everything still worked). Some simple CCK fields, inexplicably, were not imported when I used D7’s CCK module to try to migrate the data to Drupal 7 fields; I say “inexplicably” because in one case we are talking about a simple integer field, exactly like others which were migrated without any issues, but I would get an error on attempting to import it. All of my Views displays were non-functional with “broken or missing handler” -type errors for many fields, so the displays had to be rebuilt, mostly “from scratch”, and then changes to data in one field on a node sometimes seemed to result in loss of data in other fields. I couldn’t seem to get my Nodewords data to properly migrate to either Meta tags quick or the new Metatag module, Fivestar ratings were showing up in some node views, but not in Views displays (perhaps I should not be surprised since Fivestar is only released as alpha and dev, neither of which seem to work quite as expected), and other features of the site just generally seemed to fail in odd ways, even after more than a week of experimentation, thinking I was getting close, and then renewed frustration.

In summary: I realized, at a certain point, that the same amount of effort spent on simply improving my Drupal 6 site (content, structure, improvements to configuration, etc) would have been far more useful, since after much effort, I still did not have a Drupal 7 site that I was happy with. On the other hand, it was a good learning experience and I made enough progress and learned enough that I won’t completely scrap the Drupal 7 upgrade effort. But, for now, I will work on getting some of my improvements I added to the Drupal 7 site “back-ported” to my live (Drupal 6) site. And at some point I might try other approaches, such as using the Migrate module to help with getting my data into Drupal 7, learning some new Drush tricks and possibly trying other modules or new releases, as they become available, and of course working on improving my Drupal 7 theme, which was not my initial priority.

My advice for Drupal 6 to Drupal 7 upgrades

If you are working on a Drupal 6 site with quite a lot of contributed modules and wish to avoid a headache and a lot of wasted time, you might want to wait a while longer, but it really depends on the use case and particulars of modules used. People with a lot more Drupal development knowledge may be able to more easily overcome the issues I resolved or find solutions to ones that had me frustrated, but you will likely not find it “quick and easy” if you have a site that’s at all complex. And if you have custom modules to port to Drupal 7, the effort needed could easily outweigh any benefit. Bear in mind that several modules have changed names between Drupal 6 and Drupal 7, so are now considered new projects. You will need to take a close look if you don’t immediately see a Drupal 7 version of a popular Drupal 6 module. For example, the Nodewords module in Drupal 6 is now known as Metatag, Path Redirect is simply Redirect, and several other modules have been forked since the original maintainers took no action to put out a Drupal 7 version before someone else worked on it and created a new project from their efforts, so you will likely find links to the fork in the respective Drupal 6 modules’ issue queues. Time spent reading the issue queues, beforehand and carefully reading all notes from maintainers will certainly be time well spent if you want to avoid frustration, but if you really want to avoid frustration, you might just want to stick with Drupal 6.

Enable the ‘Check for updates of disabled modules and themes’ option

For lengthy upgrade processes, like the one I’m undergoing, or for modules which a site builder or developer might turn off most of the time, but turn on for troubleshooting, theming etc, it is helpful to turn on the option to have the “available updates” page display information for modules and themes which are not activated. This is especially important if certain modules are not yet stable enough to run without crashing a site, but their use is desired as soon as a more stable release is available. I forgot this setting was available and turning it on during the lengthy process of getting a somewhat complex site upgraded can be very helpful. Do this at: admin/reports/updates/settings

Some final thoughts about the current state of Drupal

• Given the fact that the median module popularity is only about 100 sites, and many Drupal projects of significant complexity, uniqueness, and size either use some of these less popular modules which have not been ported and/or have several custom modules, perhaps not fully documented and possibly written by external consultants, freelancers, or ex-employees, it seems likely that at least half of the current Drupal 6 sites may never migrate to Drupal 7.
• We have a high demand for official support for more than two major versions whenever Drupal 8 is released (probably still at least 18 months from now). It’s likely that many of us in the greater Drupal community will still have a stake in Drupal 6 and will want to see security issues maintained for it into the future (past Drupal 8’s release). I would not be surprised if there are still 200,000 Drupal 6 sites at that time.
• It seems to indicate to me that we are reaching a point where non-version-specific support for module compatibility would generally be popular. Perhaps we would have far fewer Drupal 6 sites now than Drupal 7 if upgraded sites could run with some Drupal 6 modules (e.g. by turning on a theoretical “core” API compatibility module to automatically run as a Drupal 6/7 “hybrid” site, perhaps using data sharding, until stable Drupal 7 -native modules were available).

Briefly revisiting this issue of API backward compatibility

I don’t know what is realistic or likely to come of this situation, but I think a long-range plan to have some kind of compatibility API available to run version-prior modules could be a solution in 5 years or so, when people want to migrate Drupal 8 sites to Drupal 9, assuming the differences between Drupal 7 and 8 are too great to allow for such a solution. From the comments left on my recent article on the topic, it’s clear that backward compatibility in the API is a topic that produces heated debate and I am not sure which side I stand on. There are clear disadvantages to retaining backward compatibility, but as Drupal matures, I don’t think that breaking backward compatibility with every major release is a viable long-range strategy for retaining a loyal user-base.

Maintaining backward compatibility may not be possible with every major version of Drupal or it might create too many complications, so my thoughts are that such API backward compatibility could be included as an “API compatibility layer”, comprised of extra API definitions to allow version-prior modules to run (i.e. to seamlessly run a hybrid Drupal X / Drupal Y site while Drupal Y contrib modules are still becoming ready for production use), then perhaps alternating versions of Drupal would be a major re-write with little or no backward compatibility, possibly released about three years after the previous version; then there could be two major core releases which share more similarity at the core and fully support backward compatibility, e.g. what is being developed for Drupal 8 may be too great a departure from Drupal 6 and Drupal 7 to easily support any kind of backward compatibility without losing too many of the benefits of Drupal 8 and the benefit of “clean slate” innovation. But Drupal 9 could extend Drupal 8 and provide for full backward compatibility, and with fewer people inconvenienced by a shorter release cycle, Drupal 9 could come 1–2 years after Drupal 8. And then Drupal 10, if deemed necessary, could be another full re-write, released 3 years, or so, after Drupal 9… and so on. But this is just wild brainstorming on my part, so please don’t quote this back to me in two years as my “suggested approach”. ;-)

State of Drupal contrib themes

Drupal 7 themes — and the improvements in core which relate to “the display layer” — are truly awesome! You’ll find improved support for mobile devices and increased granularity of control over the display of individual fields due to render arrays, among other core improvements. If you have done much custom theming on a Drupal 6 site, however, getting a new theme customized for Drupal 7 will likely be a lot of work and you should plan accordingly. If you are building a new Drupal site, you will likely find that improvements to theming is one of the major benefits of Drupal 7, even if all the contributed modules (and some of the new themes) are not yet quite perfect. Of course, by building your new site on Drupal 7, you will also save yourself a lot of the potential headaches of upgrading it at some future time.

Drupal is a moving target and YMMV

Please bear in mind that the downsides and statistics I provided are going to change and possibly quite rapidly. With any luck, I hope to be able to complete my own Drupal 7 migration in the next month or so, either by persevering and finding workarounds or from newer versions of critical contributed modules being released, probably both. I have come far enough in the process and am curious enough and interested enough in the learning experience that continuing to spend some time on this process is of value to me, but for now I will focus in fixing what needs to be fixed on the Drupal 6 version of the site and building up the stale content. From the comments I’ve read in the greater web, it seems I am not alone in finding that the major version upgrade is probably best left alone if you have many contrib modules and wish to retain their data and functionality. But if you have a project with very experienced developers and a generous time budget allocated to the migration (not just a small personal project), you might find the upgrade process goes smoothly by “team project” standards.