6 August 2012
New spam content posted

Rules is an especially useful Drupal module for all kinds of tasks. One use you might want to put it to is providing admin notifications of certain events on your site, e.g. user registrations (covered in the previous post) and the creation of new comments and content by “untrusted” users (assuming your use case allows them to create any content at all). For some use cases, you may wish to put strict limits on the creation of user accounts and content, but for the purposes of this article we are assuming you are administering a Drupal site where you want to encourage growth and community involvement, so you might allow anonymous users to comment on posts (albeit likely with the rel="nofollow"* attribute added to their links). And you might also allow users to create and confirm their own accounts and then create some types of content (e.g. forum posts, bug reports, etc). The downside is that you’ll need to be vigilant about squashing all the spam this policy invites (or the flow of new spam will quickly increase and be damaging to your SEO efforts), but on a site with only moderate traffic you should be able to manage this without a lot of trouble. This post covers using Rules to provide notifications of new comments and content. If you keep a close eye on user registrations and immediately block the user accounts which follow a pattern like other spam accounts you’ve removed (i.e. accounts likely created with the help of a “spambot”), you can eliminate almost all of the spam that requires use of an authenticated user account.

Adding rel equals nofollow to Drupal Links

*Adding rel="nofollow" to links posted in “Filtered HTML”, presumably the only text format you allow for “untrusted” users, is simple, but well worth doing if you allow “anonymous” or newly-registered “authenticated” users to make any kinds of posts on your site. In Drupal 6, go to admin/settings/filters/1/configure and select the “Spam link deterrent” checkbox. In Drupal 7, go to admin/config/content/formats/filtered_html and find the vertical tab near the bottom labeled “Limit allowed HTML tags”, where the same feature is enabled with a checkbox labeled “Add rel="nofollow" to all links”.

Getting back on topic, the three events we want to create Rules for are:

  1. New user registered (done, see previous article)
  2. New comment posted by “untrusted” user (described below)
  3. New content posted by “untrusted” user (also described below)
In each case, we simply want to send an HTML email* to notify at least one member of staff (anyone with the admin role and, in the case of comments on blog posts, we might want to also email the article’s author.) As with the past installment in this two-part series, this article does not cover configuring your server for sending mail nor setting up HTML mail, but we use Mime Mail for the “send HTML mail” versions of the exported code you’ll find attached. (See attachments to this article for code you can use to import these Rules into your own system—you’ll possibly need to tweak them a bit, but they could save you some time.) Where screenshots are included, we’ll display the Rules administration interface in both Drupal 6 and Drupal 7, side-by-side, since there are some differences that might otherwise lead to confusion. So this article should be helpful for administrators of both Drupal 6 and Drupal 7 sites.

In every case, creating a new rule starts by going to the “add rule” page:
D6: admin/rules/trigger/add
D7: admin/config/workflow/rules/reaction/add

Notify Admin (and/or content authors) when new comments are posted

This rule sends an (HTML or plain text) email which includes the comment title and body, along with a link to quickly edit it. Here at Cocomore, rather than delete comment spam, we’ve been unpublishing the comments so we can still observe patterns. So we don’t get an email for each of our own responses to comments, we configure the rule to only notify us when “untrusted” users (i.e. any users who don’t have a “staff” or admin role) post comments.

5 July 2012
New Spammer Registered

Rules is an especially useful Drupal module for all kinds of tasks. One use you might want to put it to is providing admin notifications of certain events on your site, e.g. user registrations and the creation of new comments and content by these “untrusted” users (assuming your use case allows them to create any content at all). I recently created such rules to help monitor the creation of users, content, and comments on drupal.cocomore.com/.de. Since we use the Project module (and supporting code) to host and track issues on some Drupal modules, we allow users to create accounts and “Issue” nodes. But there hasn’t been much recent change to the modules we host, so most of the “users” turn out to be spamming scumbags who post “issues” with links to questionable sites (you know the type). Since we allow anonymous users to comment on our blog posts, we also get our fair share of comment spam, but a tricky Captcha (we’re using Riddler, these days, to filter out visitors who don’t know or can’t take the time to search the answers to simple Drupal trivia questions) helps keep comment spam to a minimum. Keeping vigilant about stomping out spam is important since leaving spam published looks unprofessional and is bad for SEO… and since it also attracts more spam (spammers see that your site leaves spammy links in place); but of course it’s also important to keep an eye on the valid posts, too, and to respond to them in a timely fashion.

So we will assume that you have a site without a massive flow of new user registrations or new content and that you want to be alerted with some useful information whenever these events occur so that you can take appropriate action (block users and clean out the spam… or respond to valid content/comments). This article will lead you, step-by-step, through the creation of three different rules on both Drupal 6 and Drupal 7 -based sites, identifying particular set-up differences between these versions of Drupal/Rules. The three events we want to create Rules for are:

  1. New user registered
  2. New comment posted (by non-staff user or “untrusted” user)
  3. New content posted (again, by some kind of “untrusted” user)
In each case, we simply want to send an HTML email* to notify at least one member of staff (anyone with the admin role and, in the case of comments on blog posts, we want to also email the article’s author.) This article does not get into the various particulars of configuring your server to be able to send mail; there are a number of factors which might differ from server to server and it’s not really within the scope of a Drupal-related article.
*Note: This article also does not cover setting up HTML mail, but some modules, such as Mime Mail help make this a relatively pain-free process and provide a “send HTML mail” action for Rules. Adding specialized modules is probably not justifiable if you don’t plan to use HTML mail for anything more than admin notifications, but if you want to email users, such modules can help you create much more attractive and useful emails.

In every case, creating a new rule starts by going to the “add rule” page:
D6: admin/rules/trigger/add
D7: admin/config/workflow/rules/reaction/add

Notify admin when a new user registers

This is a simple rule which sends an HTML email with a link to a new user’s profile, along with their username. If you allow users to register themselves on your site, you will likely notice patterns that persistent spammers follow and be alert enough to just block the most suspicious user accounts before they even start spamming your site. I won’t specify the suspicious patterns I’ve been reacting to here (I don’t want to teach spammers how to be sneakier or more effective), but if you have a spam problem, you probably already know the patterns or will quickly recognize them.

19 June 2012

It’s been a busy past several days in Barcelona (for the Drupal Developer Days) and most of us who’d been sprinting during the week before seemed to be in the same condition by Sunday—rapidly running out of energy from progressive sleep deprivation from an increasingly later return to our hotels. But it’s been an exciting week for Drupal core (and contrib) development and significant work has been completed on the Drupal core (mostly building up Drupal 8, but also some for added features in Drupal 7) while a lot of important decisions have been made which will likely shape development in a number of initiatives for the coming months until the sprints at DrupalCon Munich.

In addition to the Sprint I was primarily involved in (I was just trying to get my feet wet with assisting the Drupal 8 core development process by joining the multilingual sprint, but I did write my first committed core patch—admittedly this was a very basic patch), there were also sprints running for “Views in core”, Entity API, Media initiative, Mapping in Drupal 7, configuration management, abstracting social networking, search-related sprints, the Drupal.org upgrade… and possibly more still. I’ll cover some of the highlights of the week that I’m most knowledgeable about.

Multilingual Initiative

The multilingual initiative sprinted all week before the Developer Days sessions, and even continued through the weekend. And a lot of key decisions were made and important code changes committed and pushed to the central Drupal 8.x repository.

New user interface translation improvements in Drupal 8

This is something I got to do a bit with, but Swiss developer, Michael Schmid (Schnitzel on d.o), of Amazee Labs, was the primary developer working on this task during the Sprint. He and his colleague, Vasi Chindris, were among the stars of the week. It was a real privilege to get to look over their shoulders and to get Michael’s support when it came to using Git to manage code in the sandbox we were using for the issue. (Thank you, once again, Michael!) Once everyone was happy with the work, it got committed to core. This new sandbox workflow, used for larger issues, helps avoid a lot of bugs creeping into the main branch, as has happened during previous periods of intense core development. Of course the tests and test bots catch a lot of issues which could otherwise be major headaches for all concerned (automated testing was also a part of Drupal 7 development). If you recall, the long wait for Drupal 7’s release was due to hundreds of critical bugs. Now this should be a thing of the past since we have an established threshold for critical issues; and the core team only commit new patches to the central repository when we are below that threshold (15 “critical” bugs, 100 “major” bugs… among other thresholds specified).

New system for translating Drupal’s user interface

The new user interface translation system allows you to keep imported (community contributed) translations separate from customized translations and search for a particular translation within either or both categories as well as filter by translated strings, untranslated strings, or both. If you have any unsaved translations, they are highlighted to help remind you not to leave the page without saving them and there discussion about providing a dialogue to prevent a site admin from accidentally leaving the page with unsaved changes, too. There is also an issue to allow the string search to be non-case-sensitive (checkbox) to find more strings that contain a particular word or phrase, regardless of text case. Since this feature came up in discussion after the rest of the user-interface changes had already been made, we elected to put the discussion about adding this feature in a separate issue. If you have ideas for what might further improve the Drupal 8 user-interface translation workflow, your input is valued.Customized and imported (community) translations are stored separately

14 June 2012
Morning stand-up meeting at the Drupal 8 Multilingual Sprint

I was supposed to get into Barcelona at 10:30PM on Tuesday evening, but with delays in my flight, it wasn’t till after midnight that our plane landed; it was after 1 a.m. by the time I reached my hotel. Normally travel, when it runs late and long, makes me feel exhausted, but I was excited to be joining my first Drupal core sprint. I’ve been wanting to do a bit more to help build Drupal and it’s great to not only be somewhat aware of what’s coming in Drupal 8, but to also know that I’ve at least played a small part in making it happen.

I wasn’t sure I would attend the Drupal Dev Days in Barcelona till a couple of weeks ago, but I’m glad I’m here. We have a fairly sizable group of developers here at the Citilab helping work on cutting through the issues for Drupal 8 Multilingual Initiative (D8MI). I’ve been helping with some user interface quirks and since it had been long enough since I’d actually done string translations of the user interface, I started out yesterday as a “tester”… at least trying to look at the problem of translating the interface (e.g. translating “Add content” to German) as if I had never done anything like that before. And we did find some issues and, even better, we were able to address and correct those issues during yesterday’s coding. Others have been working on multilingual issues related to the new configuration management system, and a number of other issues which you, too, can help with, if you’d like to join us remotely (or in person, if you happen to already be in Barcelona — the Sprints continue through Friday, too). There are currently about 40 of us in the IRC channel for i18n and I'd say that at least half of those are working on the Sprint. There are about a dozen (give or take, since people are working on other sprints, too) who are here in Barcelona working on D8MI.

You can help make Drupal 8 better, too!

30 March 2012

We have a major upcoming project here at Cocomore which is in the initial planning phase. It’s too early to provide the finer details of the project, but it involves creating a product database for a large publishing house and Drupal 7 has been chosen as the project framework. By mid-June, we will likely have four developers working on it, full-time. Of course, anyone who deals with software development almost certainly knows the problems that tend to occur during the planning of large-scale, long-term projects like this one: in the beginning, the client is often not yet 100% certain of their needs or desired end results, so new requirements and ideas arise in the middle of project development. This means that projects built to the initial specifications often fail to completely meet the client’s needs, which can be disappointing for everyone involved.

Hoping to avoid this scenario, we (our software development team, and project and senior management) proposed implementing the project using Scrum. This allows us to flexibly respond to changes in requirements and keeps the customer closely tied to the project development process so we can avoid unpleasant surprises at the end of development. At a recent Cocomore “KnowledgeLab”, I presented a brief overview of Scrum methodology, a topic which is certainly too broad to cover, in-depth, in an hour-long presentation. So I limited the scope of my presentation to the most essential elements of the Scrum process: roles, events, and artifacts; then used Lego Scrum to try to better illustrate the model. This article provides an overview of the same basic concepts covered in our workshop session and describes how I helped immerse team members who had not previously worked with Scrum in the concepts and processes. We have been using Scrum for other “ambitious” Drupal projects and plan to provide in-depth case studies for some of them, with details about more specifics related to Drupal; this article provides a general foundation for understanding these upcoming case studies.

Scrum: a process model for agile software development

Agile software development is characterized primarily by an iterative procedure with alternating planning and development phases. The advantage this provides is that parts of the system are developed early on and can be tested before implementation of other parts. This reduces the risk that project development heads in the wrong direction. Rather, responding quickly and flexibly to changes in the requirements, the components of a system can be redefined to best meet a client’s real needs.

agile_workflow_with-url_03.png

13 July 2011

We recently decided to add a blog to our Drupal distribution site, and I jumped at the opportunity to help get it set up. Since it's a new feature for our site, I decided to document the steps I took in my development environment so that we can be sure to replicate what works on our production server.