DrupalCamp NYC 2008

2008-01-08 4-minute read

Drupal scalability

I wasn’t taking notes during the CRM/Drupal workshop - but I did with Drupal tuning. Unfortunately, the focus was on hardware tuning. Boo. Much less useful I think. However, we did spend a short period of time on software tuning, which I think is the most useful.

The first question: What’s the bottleneck?

The workshop presenter seemed to think the Database was the biggest bottleneck (I’m much more familiar with php/processor being the bottleneck).

If the bottleneck is the database, the Devel module will show which queries are the slowest, which would allow you to add index or find the modules that are killing the site.

In some cases, switching tables that are high read, low write from the default MySQL table MyISAM to Innodb can help.

From the web server perspective, the Firefox Why Slow? Plugin can provide a lot of good ideas, such as aggregating css and javascript.

Other resources to try include 2bits.com and Tag 1 Consulting.

Clustering

Aha! An audience member finally pointed out the MySQL feature I’ve been dying for: MySQL proxy. MySQL proxy allows you to configure your web application to use just one database - the mysql proxy. The mysql proxy, in turn, can direct write requests to one database and read requests to other databases. In other words, that means you can use traditional MySQL cluster approach (with one central database that replicates to many distributed, read-only databases).

Memcache

Most of the session was about memcache - which, IMO, seems to be a solution that works best in combination with throwing a lot of hardware at the problem.

Reverse Proxy

Another option is reverse proxy, which caches incoming connections. Nobody reported using Squid, but one member reported success with nginx.

Multi-Media

Big news for me: in the pre-workshop chatter one of the presenters said that the image module is eventually going to be replaced by imagefield.

Example video sites:

It’s not hard to do local video, but the presenters emphasized using YouTube or BlipTV to host the video. This is pretty unfortunate. It was not a politically-based presentation, so the reasons discussed were technically-based without political thought.

If you do host locally, you must use ffmpeg for encoding the video.

What do use?

Video module is pretty much dead.

Embedded media field allows you to drag and drop videos from YouTube, BlipTV or another provider to your submit form in Drupal. View slide show gives you a lot of options for how to display the video (apparently the plain ole’ views module has some ajaxy stuff for displaying as well). With embedded media field, you will be using the provider’s player. You can use local media files and your own player as well, however, it’s a little buggy.

A better way to do local media is to use Filefield plus jQuery Media player. jQuery Media player takes a link and automatically converts it into a player.

And, Drupal Media Player is being developed that will allow you to pull in sources from other providers but play with your own player. The player is being developed in OpenLaszlo, a free language that outputs in either javascript or flash. Unfortunately, to play flash video, you need to output in flash, so Drupal Media Player will be a flash application.

Media Mover will pull in media from other locations (like an email box or an ftp site) so you don’t have to use the web upload interface. It will also push media to other locations.

Combined with a service like TubeMogul allows you to push your video to multiple sites.

All of this makes me think that the best public video strategy might be to scatter your video to various corporate sites, all the while keeping a copy yourself. Then, visitors view video served by the corporations (and they get hit with the bandwidth). Meanwhile, you always have your own copy and provided there are enough providers with your video you can easily recover if one site goes down. You can always revert to serving the copy on your own server as a last resort.

Unit Testing

Ah. This is good. The subject of this session was how to build tests that can be run automatically after you make a change to your code base to see if you broke anything.

It seems to be based on Simple Test module. There’s a manual page which appears to be based on Drupal 6/7.

The answer to: Can I use this with Drupal 5 is: yes, but it doesn’t work so well. It should work well with Drupal 6 and it’s in core for Drupal 7. I did get distracted by the Simple Test and Simple Test Documentation, which suggests simple and quick ways to setup testing for non-Drupal projects.