Packaged install profiles ready for testing on d6.project.drupal.org

3281d Consulting is pleased to announce that fully packaged installation profile releases are now available for testing on http://d6.project.drupal.org. This feature has enormous potential to expand Drupal adoption to new audiences by significantly lowering the barrier for end users trying to get a customized distribution up and ready for the kind of site they're building. Instead of separately downloading Drupal core, an installation profile release, and then manually cobbling together all the module and theme dependencies, installation profiles can now be packaged with core and all of their dependencies already included.

This system is all going to go live on drupal.org on Tuesday 2009-12-01 2am-3am UTC. However, it'd be extremely helpful if people who maintain installation profiles (either currently or planning to do so in the near future) would be willing to test the new packaging system to provide feedback on how it all works, and flesh out any problems before deployment on the main site.

The rest of this message only applies to people who maintain an installation profile and want to test the new packaging system. If that's you, please read on! Otherwise, stay tuned to the drupal.org front page for an update when the system goes live with details about what it means for end users.

RTFM ;)

The first step to testing the new system is reading the new How to package a profile on drupal.org handbook page. If you find anything that's confusing or missing, please leave a comment on that page and we'll clarify the docs as needed. Some of the details might change, so don't consider the draft there as the final word on how everything will always work in the future. But, most of it will remain true and we'll announce any major changes that might arise.

Setting up drush make

The profile packaging system on drupal.org is built on top of drush make which is a drush add-on command you can install. There's already momentum in the community around using this wonderful tool for bringing together all the pieces you need to get a functional Drupal site running, so it made natural sense to use the same technology for drupal.org and improve it, instead of building our own parallel system. One of the many benefits is that you can actually install drush make locally and run it yourself, just like the packaging script is going to, so you can ensure that your .make file is valid and it's going to do what you want. A detailed tutorial on installing and configuring drush and drush make is outside the scope of this post, but there are a lot of resources at the drush project page.

Unfortunately, due to some legal hurdles (and security concerns), the drush make that runs on drupal.org itself will only allow a subset of all the possible commands that drush make supports. In particular, for now, releases packaged and hosted on drupal.org can only contain code from projects on drupal.org itself -- you can't have your installation profile pulling in 3rd party libraries and code (yet). So, there's a drupal.org-specific drush make command that the packaging script uses to enforce this extra validation. This is included in the contrib folder inside drush make itself. So, if you want to run exactly what drupal.org will, you'll need the following:

And then invoke drush make with the --drupal-org command-line option.

Working with the d6.project.drupal.org test site

Next, you're going to need to setup a development environment to work with the d6.project.drupal.org test site. This site is a sanitized clone of the drupal.org site and database as of about a week ago. All the email addresses have been wiped clean, so resetting your password, issue subscription emails, etc, won't work. However, you should be able to log in using your drupal.org username and password as of a week ago. There's also a clone of the cvs.drupal.org CVS repository, so you'll be able to commit files and tag releases on the test site without touching your real profile code on cvs.drupal.org.

Instead of a CVSROOT that looks like this:
:pserver:dww@cvs.drupal.org:/cvs/drupal-contrib

You'll want to change the host name and use something like this:
:pserver:dww@d6.project.drupal.org:/cvs/drupal-contrib

Everything else about your CVS workflow should be the same. Once you setup your CVSROOT, you can checkout a copy of your installation profile from the test CVS repository and create the drupal-org.make file as explained in the handbook.

To test your drupal-org.make file with drush make, you're going to want to tell drush make to query d6.project.drupal.org looking for releases, not the live drupal.org site. So, in the the drush make directory, open up the drush_make.drush.inc file and change this constant:

define('DRUSH_MAKE_UPDATE_DEFAULT_URL', 'http://updates.drupal.org/release-history');

To this:

define('DRUSH_MAKE_UPDATE_DEFAULT_URL', 'http://d6.project.drupal.org/release-history');

(It's lame you have to change this via a define in the code, see #645746: provide a switch for default download url for more).

After that, you can invoke drush make like so:

drush make --drupal-org drupal-org.make /path/to/build/directory

If the .make file is happy and does what you expect, you can commit the .make file (only the .make file -- not all the stuff it built for you!) and add an official release CVS tag to your profile. At that point, you'll be able to create a release node on d6.project.drupal.org from your profile's project page. Select the new tag you added, fill in the release notes, etc, just like you do on drupal.org. Then, sit back and wait at most 5 minutes for the next packaging script cron run. If all goes well, you should see something like one of these:

http://d6.project.drupal.org/node/643523
http://d6.project.drupal.org/node/643528
http://d6.project.drupal.org/node/643533

What else is changing?

Although we're extremely happy with the progress so far, there are a lot of things we're going to be fixing and improving over the next few days and in the immediate aftermath of deploying it live on drupal.org. For starters, the UI when you're viewing a release node of a packaged installation profile is going to undergo a lot of improvement over the next two days, so don't be alarmed or disappointed with how it looks now. ;) We know. Among other things, the Package items listing isn't going to be on a separate tab, but will be displayed inline on the installation profile release itself. We'll also clean up how all the files are listed so they're not just an ugly wall of text, but a nicer table where you can clearly see the different files available for download. The default download from the project page is the full distribution, including core, the profile, and all its dependencies. However, for people building a multi-site installation who don't want a whole separate copy of core, there's a version with just the module and theme dependencies. There's also a version with just the contents of the installation profile itself, if you want to run drush make yourself.

We're also hard at work improving the usability of drush make, making it easier to test locally, and adding a command to convert a .make file without specific versions into a .make file with specific version info for all the currently recommended releases for your included projects.

There's a lot more to say, but we mostly just want to get folks started playing with the new system. Please let us know what you think!

Thanks,
-Derek and Chad

Comments

Brilliant

This is the most auspicious change in d.o. since ... well, ever! Thanks for the dedication and hard work.

One question. It's clear that our make files will only be able to do the --drupal-org stuff, but will the rest get ignored? Or is it problematic to even have anything that falls outside of the --drupal-org scope? If we can make elaborate make files that get all 3rd party dependencies and they get packaged with --drupal-org on d.o., that's great, because people can still download the make file and run it themselves. If it's not at all allowed (technically) to have non --drupal-org stuff, that'd be a shame.

Support for 2 .make files per distribution if you need it...

Not sure if this was clear enough on the handbook page when you read it, since we've been editing that page a lot. ;) But, there's a whole section about maintaining two .make files if you need to. The idea is that whatever you give to the packaging script has to be valid. We thought it'd be confusing for things to be silently ignored, which your profile might actually depend on. But, we know that some distributions are going to have more complicated needs than what drupal.org can provide for now. So, you put the complicated stuff in {profile}.make, and run that though drush convert makefile {profile}.make {profile}-drupalorg.make and it'll convert {profile}.make into something drupal.org can handle in {profile}-drupalorg.make and tell you what it did. Does that address your concerns?

Thanks for the encouraging words! ;) I'd like to think of this as the most auspicious chance since the release system, which is really the foundation that even makes this (and countless other things) possible...

Cheers,
-Derek

w00t! External libraries next!

Applause!

This is a huge step towards making install profiles on Drupal.org really useful.

I'll test with http://drupal.org/project/managingnews as soon as I get some air.

Of course, the missing piece now is external libraries packaging :-) I'm optimistic that we'll find a solution for that.

Thanks for your work on making packaging happen on Drupal.org

Alex

THANK YOU!!!

THANK YOU!!!