Testing and code review is something that has been on my mind a lot lately as our shop has been shifting its focus from boutique, one-off projects, to building upon frameworks maintained by other organizations. As these code bases continue to grow, we need to ensure that subtle changes to the core functionality of the underlying systems do not propagate into bugs in our code. We also need a way to handle this situation quickly and efficiently when this does arise. This was especially reinforced by two recent projects our group undertook to migrate nearly decade-old software on to new servers.

If you ask anyone in the office, they will most likely roll their eyes when I start beating the testing drum. These are great tools for not only generating pretty green and red bar charts, but also documenting the intention of the programmer in writing the code, and zeroing in on bugs where they occur without weeks of hunting. However, this is only one of the tools in the chest for writing solid code, sans bugs. In fact, there are a lot of sophisticated, freely available, automated tools that help programmers of all skill levels not only write more consistent code, but also zero in on potential performance issues and just plain smelly code (that they obviously wrote just to get running and fully intended to go back and fix later).

Over the years, tools that measure code complexity (like PMD, PHPMD, and flog), code dependency analyzers (JDepend, PHPDepend, and rcov), copy/paste detection (in PMD, flay, and phpcpd), and enforcing coding standards (a la PHPCode Sniffer and rails_best_practices), along with not only unit and integration tests (in whatever style you choose), but a code coverage analysis reports that provides feedback on which lines were executed, go a long way in reducing the number of bugs in code. These tools are really pre-emptive step in writing stronger, more elegant, and ultimately more sustainable code, all before once gets to the point of performing a human code review.

While I don’t need to be building software per-se, I have started experimenting with the Hudson continuous integration server as a dashboard to get a quick snapshot of how these different metric tests all play together in the code that our team writes. It is no longer good enough to simply have code functioning, we need the code to pass certain thresholds of quality and sustainability before we can release. Where we find issues in the code, like test coverage, high cyclomatic complexity, lots of copy-n-pasted code, or high volatility in dependency scans, we can sit down and perform a rather focused mini code review (resembling the pair-programming idiom) on that section of code to refactor a better solution or approach To this end, we’re currently working on a set of baseline testing and reporting tools for our projects. Currently, we have Ant scripts for our PHP and Java projects, and a gem bundle for Rails and Sinatra projects.

While we take this approach in the Scholars’ Lab, we were wondering if there were others out there that had opinions or experiences to share about code review during development? If you do, leave a comment, write a post, or tweet at us (@scholarslab, @nowviskie, @wayne_graham) — and at @paregorios, who started the conversation in the first place. We’d love to hear about your best practices (and even horror stories) and philosophy on what constitutes good software and useful code reviewing, including whether you think current trends in open source development constitute a good-enough review for DH projects.

Further Resources

Java

• PMD
• JDedend
• junit
• Clover
• Javadoc
• Checkstyle

PHP

• PHP Depend
• PHPMD
• phpcpd
• PHP Codesniffer
• phpDocumentor

Ruby

• Flog
• Flay
• metric_fu
• rdoc

Javascript

• QUnit
• JSCoverage

Continuous Integration

• Hudson
• Phing
• CruiseControl

Issue Tracking

• Google Code
• GitHub
• Project Kenai
• Jira
• Redmine

Qualification

People often think of the Scholars’ Lab as a Ruby (and Rails) shop, which isn’t quite the case. We code in many different languages. In the past several moths, we have written The Mind is a Metaphor in Ruby (with Rails). For Better For Verse uses WordPress with TEI and JSP, and some more recent work on a William Faulkner audio archive employs Cocoon with Solr. In addition to those collaborations with UVa faculty, we’ve been writing plugins for Omeka (and dusting off our PHP skills) and have created a discovery interface for our GIS infrastructure in Ruby (with Sinatra). If you analyze the technologies we currently deploy, it turns out we use Cocoon + Solr more than anything else, though we’re starting to move away from that particular stack as our approach for tool development.

The Scholars’ Lab has a lot of experience with all types of languages, and depending on the circumstances, we choose different tools to accomplish any given task. However, after quite a bit of time helping people get started on a programming path, I’ve come to appreciate some of the features Ruby provides in getting new programmers up to speed.

Learning Curve

Every language has a learning curve. However, once you get the hang of some of the basics of computer languages (flow control, data structures, objects, etc.), the biggest differences come from syntax. All web languages make certain programming exercises easy, and once you buy in to the way in which that language handles programming constructs, moving between languages for experienced programmers becomes a simpler exercise in exploring syntax and built-in functionality.

As one of my computer science professors posited, generally the first language you learn governs the way you code until something significantly better comes along. For a lot of folks getting in to programming for the first time, this usually means either taking a class or finding someone to show you the basics. If you’re in higher education, this has typically meant the tool of choice is PHP. However, having seen the look of bewilderment on the faces of enough graduate students and faculty members as I attempt to explain the difference between sprintf and printf (printf returns the length of the formatted String and sprintf returns the formatted String), I’ve come to believe that the syntax of a programming language (and it’s readability) is an exceptionally important part of a language, especially when teaching basics of software construction.

Method Chaining

Without getting into how the PHP and Ruby duck-type primitive data types and data structures, one big difference in syntax between the two is how one combines multiple method calls together. Ruby uses method chaining for objects whereas PHP uses “bolted-on” functions (think of these as order-of-operations from your high school Algebra class). Let’s look at this brief example of addressing and sorting an associative array in PHP and its equivalent in Ruby:

$projects = array("solr" => 4, "php" => 1, "rails" => 2, "jsp" => 3);
$keys = array_keys($projects);
sort($keys);
$sorted = array_slice($keys, 0, 3);

If you’re a little more advanced, you might refactor (rewrite) the code to look more like this (methods anonymously “bolted-on” to one another):

$projects = array("solr" => 4, "php" =>  1, "rails" => 2, "jsp" => 3);
$sorted = array_slice(sort(array_keys($projects)), 0, 3);

Now, the same examples in Ruby syntax:

projects = {"solr" => 4, "php" =>  1, "rails" => 2, "jsp" => 3}
sorted = projects.keys.sort.slice(0,3)

Or, even more concisely:

projects = {"solr" => 4, "php" =>  1, "rails" => 2, "jsp" => 3}
sorted = projects.keys.sort[0..3]

I’ve found that Ruby’s method chaining syntax makes more sense to new programmers than the more mathematical “bolted-on” syntax.

Blocks

Ruby has a neat construct that you use all over the place to create anonymous functions (a technical term for creating specific functionality without defining a new function to define the action). Let’s take a function to sort an array of projects. First, in PHP:

function sort_projects_by_count($a, $b)
{
    if($a -> counts == $b -> counts)
    {
        return 0;
    }
    return($a -> counts > $b -> counts) ? +1 : -1;
}

usort($projects, "sort_projects_by_count");

And the same thing in Ruby:

projects.sort do |a, b|
    a.counts <=> b.counts
end

Ok, so this is a bit of an unfair comparison, but here is an analogous version of the Ruby code in PHP:

usort($projects, create_function($a, $b, 'if($a->counts == $b->counts){
    return 0;}return ($a->counts > $b->counts ? +1 : -1));

No matter how you slice it, Ruby syntax just feels more human. Even if you don’t know exactly what’s going on, looking up one operator in the Ruby syntax as opposed to following the logic flow and determining what “? +1 : -1″ means (it’s shorthand for an if-then statement) makes the act of reading code much easier.

Monkeypatching

If you’re not familiar with the term, “monkeypatching” is what programmers call changing or extending a base class (like an array or string object) to add functionality or change the way it works. Let’s say you really need to be able to test a string to see if it looks like an integer, you could create a monkeypatch along these lines:

class String
    def is_int?
        self =~ /^[-+]?[0-9]*$/
    end
end

This code snip extends the String class and uses a Regular Expression (regex) to test if a given String is an integer (number) by simply calling “is_int?” (notice the question mark at the end of the definition; this is used for methods that return a Boolean value). That’s a little advanced, but it does show off a very useful piece of functionality of the language that allows you to do a better job dealing with a duck-typed language.

Frameworks

Many people when talking about Ruby associate its use in web development with the Rails MVC framework. Just as PHP has Zend, CodeIgniter, CakePHP, symfony, etc., Ruby has Rails, Merb, Sinatra, Camping, and many more. Rails is the 900-pound gorilla of Ruby frameworks, and has a lot of nice features to get new applications off the ground quickly and some really great online guides to setting things up (I frequent RailsGuides). Since we often suggest Rails to our collaborators I’ll focus on this framework, but there are several other frameworks out there to choose from.

Think of Ruby (or PHP for that matter) as a pile of building materials: you can to build anything you want if you know how to put everything together. Rails, on the other hand, is like a prefab house where workers pour a foundation, set the house up, and then leave you to add the drywall, siding, windows, and roof. If you need a new component for your prefab house, a sales representative is standing by to help immediately ship you what you need.

Generators

To extend the previous metaphor, generators are like sales representatives that allow you to place orders for new aspects of your site.  To create all the erb templates (the default templeting language for Rails), controller methods, model, database migrations, routes, and tests for a new project with a single line, you might run something like the following:

script/generate scaffold topic title:string description:text

I moved away from using scaffolding pretty quickly, but it does provide a nice starting point for new programmers to build interactions with data models.

Templates

The default templating engine in Rails is erb which provides a convenient method for generating views of  your models. One of erb’s most important features is the use of “partials,” pieces of code that are used in multiple views by calling the render method. I often replicated this behavior in PHP by calling an include somewhere in a view.

ActiveRecord

The key to database interactions in Rails is ActiveRecord. As an SQL expert, I have to admit this part of the Rails framework drove me a bit batty at first, but then again I’ve been writing SQL for over 10 years (my colleagues often roll their eyes when I start writing it with relational algebra nomenclature), so allowing a framework to abstract this particular piece took some getting used to. If you’re new to programming, though, this means you don’t have to learn SQL but instead can use Ruby-style syntax to interact with your database without necessarily needing to care what your RDBMS back-end happens to be.

Take this example of looking up a book review where you have both a “book” and “review” table. In PHP you would do something like this (this snip will only work with a MySQL connection but has some sub-selection stuff going on):

function query($sql)
{
    global $conn;
    return mysql_query($sql, $conn);
}

function recent_reviews($count)
{
    $query = query(sprintf('SELECT b.book_title, r.review_id, r.created_at, r.id, review_counter.review_total
        FROM reviews r, books b,
            (SELECT count(*) AS review_total FROM reviews) AS review_counter
        WHERE r.book_id = b.book_id
        ORDER BY created_at DESC
        LIMIT %d', $count);

    return $query
}

print_r(recent_reviews(5));

Now, for the ActiveRecord equivalent:

reviews = Review.find(:limit => 5,  rder => "created_at DESC");
puts reviews.inspect

Because of the way in which ActiveRecord sets up its model associations, you’ll have access to the different name scopes to print out the same information, just in far less code. However, if you really want to, you can pass your SQL to get more granular control over the syntax:

reviews = Review.find_by_sql("SELECT b.book_title, r.review_id, r.created_at, r.id, review_counter.review_total
        FROM reviews r, books b,
            (SELECT count(*) AS review_total FROM reviews) AS review_counter
        WHERE r.book_id = b.book_id
        ORDER BY created_at DESC
        LIMIT ?", count);

One of the real beauties of the ActiveRecord methods is that as long as you’re using the generic ActiveRecord syntax, your data persistence layer can be pretty much any RDBMS and be changed with a couple lines in the configuration file. The trade-off however, is that you lose a few things and can make slightly more work for yourself than you might anticipate. One important caveat is that ActiveRecord doesn’t create foreign keys when you set up reference fields. This is actually by design as it’s using an object-oriented idiom (an object should validate the presence of another, without the underlying persistence layer enforcing any type of constraint), but I find myself adding these in to ensure that the RDBMS takes advantage of the pre-calculated indexes to improve overall performance.

I should also mention that I think the ActiveRecord model has some real limitations. As you develop your models, you will most like be tweaking its fields, which in turn requires new migrations, and you may forget which fields are actually in your models. There are plugins that help with this, but you do need to take additional steps to have this information placed somewhere convenient (I use a pre-commit git hook that calls the annotate gem to dynamically annotate my model schemas).

Security

You’ll notice in the last examples I was doing some funny stuff in both the PHP and Ruby examples to protect against SQL injection attacks. If you’re using the ActiveRecord methods of addressing objects, Rails will take care of this for you. If you’re using PHP, you’ll either need to do this yourself (sprintf is commonly used) or rely on a framework to parametrize your statements (you don’t want someone deleting everything in your database).

You also need to protect yourself from Cross-site Scripting Attacks (XSS) by escaping HTML from fields with dynamic content. erb has a helper function html_escape (h is the shorthand) which escapes this data. In Rails 3, this will change slightly and erb will automatically html_escape model output unless you explicitly tell it not to escape the field. One less thing to remember!

Testing

Testing is important, and I try to preach its virtues every chance I get. After finishing a project, the typical programmer won’t touch the code again until the application breaks. Good testing will save you (or the person that inherits the code) a lot of time discovering exactly what broke the application.

Let’s face it: testing is a pain in PHP, and I rarely see it done well. What I’ve really enjoyed about Ruby development is the fact that no matter how you code there is an appropriate testing framework available (I use rspec). There’s a strong emphasis on not only unit testing, but also integration and acceptance testing. There are also libraries that give you an idea of how well your code is tested, something I sorely missed from my Java coding endeavors. One other huge plus is that every gem on the rubygems.org site includes test-coverage metrics to give you an idea of how well the code you want to install is tested.

PHP also has testing frameworks with PHPUnit and PHPSpec being rather popular. I won’t say too much about the PHP testing frameworks other than to say that there are analogous frameworks for writing and running tests in PHP and Ruby. However, I’ve noticed a slightly more concerted effort to think through the inclusion of the tools in Ruby and their integration into the coding workflow than I’ve experienced with PHP. With the latter language, I’ve often fell in to the trap of writing the code, getting it to where I want it, and then, really as an afterthought, writing basic unit tests to get rather skimpy code coverage.

As a case in point, a mantra in Rails development is TATFT.

BryanL on TATFT from Bryan Liles on Vimeo.

Deployment

Two typical objections raised when contemplating Ruby development are that “Ruby doesn’t scale” and that server setup is a real pain. These are valid concerns, but as with many open source projects with a large number of fanatical supporters, the Ruby community has steadily made improvements in these areas. Actually, for the vast majority of our readership, these issues can be filed away in the solved category.

Scaling

I’ve always found objections to scaling a bit troubling. Scaling is one of those over-used terms that means different things to different people, but most of the “Rails doesn’t scale” comes from Twitter’s experience with the framework. They found, as they scaled horizontally (adding more servers) to handle loads of 11,000+ requests per second, that a bottleneck existed at the data persistence level as Rails doesn’t, by default, provide a mechanism to to address multiple databases. Twitter has since moved parts of their code base to Scala but has retained the majority of their code in Rails and has developed some rather ingenious messaging capabilities to talk to the appropriate abstraction layers that one needs in very large enterprise applications.

While Twitter shows that Rails is capable of scaling (with lots of work), quite honestly the likelihood of any of our applications needing this level of engineering is slim. I will say, however, that there are relatively simple methods of scaling with your infrastructure should you start running into performance issues. We have, for example, an application written in pure Ruby on Rails deployed as a Tomcat application (the details of which are completely outside the scope of this article, but the application gets all the benefits of an Enterprise class Java environment with the ease of Rails development).

Server Support

The Ruby language is included in most (if not all) modern Linux package systems and makes installation a snap. The other “major” piece of software you’ll need is RubyGems (a package manager for Ruby libraries), which is also generally available as a managed package.

Note: There is a major change occurring with the development of Rails 3. Rails is moving from a system of system-wide gems to application-level gems with the introduction of GemBundler. This approach is a more stable method of deploying application requirements which not only allows you to ensure that application libraries are properly resolved, but also provide better granular control over which libraries are deployed in specific contexts.

There was a time where deploying Rails applications was a real bear. Then along came Phusion Passenger (aka mod_rails). This allows you to run Rails (actually any Rack-based application) through Apache and Nginx without any other port management, service process monitoring, file cleanup, etc. As long as Apache is running, so is your Rails app!

Community of Support

The Rails community is pretty great in getting folks off the ground. As with any technology there are a fair number of curmudgeons, but leaders in the community as quick to remind people to be nice (see Yahuda Katz’s The Blind Men and the Elephant: A Story of Noobs). There are several corporations backing Rails development (EngineYard is a big one) and when the Google Summer of Code for Rails program wasn’t continued, the Rails community was able to raise $100,000 in three days to support a Ruby Summer of Code.

There are a number of really good podcasts (Ruby5 and The Ruby Show are good), vodcasts (Railscasts), tutorial sites (ACSIIcasts), blogs (RailsDispatch, ThoughtBot, EngineYard), open source projects (lots on github), open source books (Rails Tutorials), and some really good reference books from The Pragmatic Programmers. There’s even some humor…

Summary

As much as it drives language purists crazy when I say it, there’s nothing that you can do in Ruby that you can’t replicate in PHP (and vice-versa). In my experience, getting people set up with a functioning web application is far easier with Ruby than it is with PHP (and less prone to spaghetti code), and the deployment options make Ruby (plus a framework) a great starting point for new programmers to get their feet wet with web application development (check out Heroku). If you’re an experienced developer, does it make sense to drop PHP and rewrite your code base? Absolutely not. However, at some point, you will be faced with the prospect of needing to migrate a legacy application where Ruby may make a lot of sense. As someone who has spent quite a bit of time de-tangling spaghetti code from Perl CGI and mixed HTML and PHP pages, I’m hoping that the people that will eventually be migrating my Ruby code will not need to perform the level of coding archaeology we’ve needed to perform.

Like most things in life, choosing the correct tool for the job needs some careful consideration and planning. Ruby makes a lot of sense for getting applications off the ground quickly and reinforcing good practices like testing, code separation, and readability that I find important in forming new digital humanities programmers. Web development over the last decade has become exceptionally complex (AJAX, web services, web standards, multiple browsers, etc.) and the real hope is that by using Ruby and Rails as an approach, people will be inspired to continue down a development path to both enrich their own scholarship and impact the larger digital humanities community without becoming frustrated by syntax. This is a bit of an experiment which we and other digital humanities shops are undertaking, and in which we’re inviting everyone to participate. No matter the language, we should all be engaged in teaching best practices in project design and management, in software development techniques, in the construction of usable and elegant interfaces, and in the application of these things to humanities scholarship, through which everyone wins!

Other Resources

• Rails for PHP Developers
• 7 Reason I switched back to PHP after 2 years on Rails
• RubyGuides
• JRuby

Enter Capistrano…If you’ve not used this before, essentially this automates the deployment of web applications to your server environment.  It’s written in Ruby, but allows you to deploy ANY type of web application (we use it for Cocoon, Rails, Java, and PHP applications in the Scholars’ Lab). If you’ve got a larger shop, you may also take a look at a web interface called Webistrano which allows non-programmer types to deploy software through a web interface.

To show off the power of this software, I thought I’d write up how we use capistrano to deploy Omeka in various environments. The setup can be a little complex, but there are some good tutorials for getting started (see Setting up a Rails Server and Deploying with Capistrano on Fedora from Scratch and the Capistrano Getting Started). The following code snips assume you have successfully installed capistrano and use Subversion as your SCM (if you need SVN hosting, you can start a new project on Google Code; you can also use Github if you declare the git scm in the code).

The first step in getting your Omeka project automated for capistrano is ensuring both the capistrano and railsless-deploy gems are installed (if you’re not a ruby-ist, gem is a package manager for Ruby applications and libraries):

sudo gem install capistrano railsless-deploy

Capistrano installs a new command on your system called “capify” which sets up the boilerplate for capistrano. Just execute the capify script in your project directory:

cd /path/to/project/trunk
capify .

This will create two files, Capfile in your root directory and a config/deploy.rb. You’ll need to edit the Capfile ever so slightly to add the requirement for the railsless-deploy gem. It should read as follows:

load 'deploy' if respond_to?(:namespace) # cap2 differentiator
# Dir['vendor/plugins/*/recipes/*.rb'].each { |plugin| load(plugin) }

require 'rubygems'
require 'railsless-deploy'
load    'config/deploy'

Now we just need to do some setup in the config/deploy.rb file to tell capistrano about Omeka. This is where you need to know a little about how your server is set up and you may need to slightly change your server set up in order to use capistrano. The way capistrano works is that it creates a releases directory on your path that holds “deployments” of you project. The latest version of the project is then symlinked the project directory into the releases. This allows you to very quickly undo a deployment that goes awry.

As an example, we deploy projects to /usr/local/projects, so our omeka project would get deployed to /usr/local/projects/omeka. Capistano will create a few directories in /usr/local/projects/omeka:

• releases (timestamped directories of your application)
• shared (for log files, SCM cache, files you don’t want to be overwritten)
• current (symlink to latest directory in the releases directory)

If you’re setting up Omeka, you will need to redirect the base Directory to the “current” symlink. Here’s the vhost entry we use for Omeka as an example (this is an Ubuntu server; you may need to change the log file path if you are deploying to another operating system).

<VirtualHost *:80>
ServerName your.server.org
DocumentRoot /usr/local/projects/omeka/current/
<Directory "/usr/local/projects/omeka/current">
Options FollowSymLinks
AllowOverride All
Order allow,deny
Allow from all
</Directory>

ErrorLog /var/log/apache2/omeka_error.log
TransferLog /var/log/apache2/omeka_access.log
</VirtualHost>

Now, to edit the config/deploy.rb file to set things up for automated deployments.

# You must always specify the application and repository for every recipe. The
# repository must be the URL of the repository you want this recipe to
# correspond to. The deploy_to path must be the path on each machine that will
# form the root of the application path.

set :application, 'omeka'
set :repository, 'http://your.svn.path/repo/trunk'

set :deploy_to, "/usr/local/projects/#{application}"
set :deploy_via, :remote_cache
set :user, 'deployer'
set :runner, user
set :run_method, :run

default_run_options[:pty] = true

ssh_options[:username] = user
ssh_options[:host_key] = 'ssh-dss'
ssh_options[:paranoid] = false

role :app, 'www.coolomekaapp.org'
role :web, 'www.coolomekaapp.org'
role :db, 'www.coolomekaapp.org'

# ===============
# Custom Tasks
# ===============
namespace :deploy do
desc 'Make sure the archives directory has the proper permissions'
task :chmod_archive_dir, :except=>{:no_release => true} do

run "chmod g+rw #{current_path}/archives"
end
desc 'Sets up the intitial db.ini config'
task :upload_database_config, :except=>{:no_release => true} do
db_config = <<-INI
[database]
host     = "your.db.host"
username = "db_user"
password = "db_password"
name     = "db_name"
prefix   = "omeka_"
;port     = ""
INI

put db_config, "#{current_path}/db.ini"

end

desc 'Move archives directory so it doesn\'t get overwritten during deployments'
 task :move_archive_dir, :except=>{:no_release => true} do
 run "mv #{current_path}/archives #{shared_path}"
 end

 desc 'Just svn up the directory'
 task :svn_up, :except => {:no_release => true} do
 run "svn up #{current_path}"
 end

 desc 'Link archives folder to shared directory'
 task :link_archives_dir, :except=>{:no_release => true} do
 run "cd #{current_path} && ln -snf #{shared_path}/archives archives"
 end

end

# ======================
# Task Event Hooks
# ======================

after 'deploy:symlink', 'deploy:upload_database_config', 'deploy:link_archives_dir'
after 'deploy:cold', 'deploy:chmod_archives_dir', 'deploy:move_archives_dir'
after 'deploy', 'deploy:cleanup'

Ok, there’s a lot going on here. I’ll briefly explain what’s going on, but there should be enough here for you to start hacking. But let’s see what tasks capistrano know about and you can call. If you are still in your project directory, just type cap -T to list all the capistrano tasks. Your output should look similar to this:

cap deploy                        # Deploys your project.
cap deploy:check                  # Test deployment dependencies.
cap deploy:chmod_archive_dir      # Make sure the archives directory has the ...
cap deploy:cleanup                # Clean up old releases.
cap deploy:cold                   # Deploys and starts a `cold' application.
cap deploy:migrate                # Run the migrate rake task.
cap deploy:migrations             # Deploy and run pending migrations.
cap deploy:pending                # Displays the commits since your last deploy.
cap deploy:pending:diff           # Displays the `diff' since your last deploy.
cap deploy:restart                # Restarts your application.
cap deploy:rollback               # Rolls back to a previous version and rest...
cap deploy:rollback:code          # Rolls back to the previously deployed ver...
cap deploy:setup                  # Prepares one or more servers for deployment.
cap deploy:start                  # Start the application servers.
cap deploy:stop                   # Stop the application servers.
cap deploy:symlink                # Updates the symlink to the most recently ...
cap deploy:update                 # Copies your project and updates the symlink.
cap deploy:update_code            # Copies your project to the remote servers.
cap deploy:upload                 # Copy files to the currently deployed vers...
cap deploy:upload_database_config # Sets up the intitial db.ini config
cap deploy:web:disable            # Present a maintenance page to visitors.
cap deploy:web:enable             # Makes the application web-accessible again.
cap invoke                        # Invoke a single command on the remote ser...
cap log:tail_log                  # Tail the rails log file
cap shell                         # Begin an interactive Capistrano session.

Some tasks were not listed, either because they have no description,
or because they are only used internally by other tasks. To see all
tasks, type `cap -vT'.

Extended help may be available for these tasks.
Type `cap -e taskname' to view it.

To use a specific capistrano task, you just type the command listed. But let’s get back to the actual script and go over that briefly. Part of the installation process of Omeka requires you to reset the permissions of the archives directory. This is handled by the chmod_archive_dir. However, because of the way that capistrano deploys applications, the archives folder would get overwritten in every deployment. To get around this, we move the archives directory to the shared folder, then create a symlink from the current directory to the shared/archives directory.

There’s a task to upload_database_config that you can have in your cap script (we deploy out of private repos), but if you’re deploying out of a public repo, you may want to just put the db.ini file on the server in the shared directory and symlink it into the current_path. Lastly, there are times where you just need to do an “svn up” (or git pull) to update something small and not need to do a full deployment. This is where the cap deploy:svn_up helps out….guess what it does

Capistrano also provides task even hooks to execute specific tasks after specific events. In this script, when you do a cold deployment (when you are setting things up for the first time), the script will change the permissions on the archives directory, then move the archives directory to the shared directory. When you do a normal deploy (after the directory has gotten a proper symlink), the script will upload the database config, then symlink the archives directory and run a cleanup (keep the last 5 versions you deployed).

While there’s still a bit of up-front set up to do on your server, capistrano significantly speeds up your ability to to consistently deploy software, quickly roll-back if (that changes to “when”, if you write code long enough) problems occur, and reinforces a development process that involves SCM!

Resources

• Capistrano
• Automated PHP Deployment with Capistrano
• Railsless-deploy
• Capistrano Tutorials

The specifications for the Timeline plugin are wonderfully documented and explained on the Omeka wiki. This actually illustrates a great practice that is all too often ignored…explicitly stating what some software should do. Taking the time to think through the “what” a piece of software should do will save  you time in the long run as it forces you to think about how everything fits together, alleviating ambiguity and allowing you to focus on the task at hand.

For this specification, there were two requirements:

• The plugin should create a helper function for creating a SIMILE Timeline widget from an array of items. The helper function should allow you to specify which metadata elements the time data should come from, as well as the element that specifies the caption (by default, the Dublin Core Title element).
• The timeline should allow for time intervals (start and end dates) and points in time (singe date).

While there are two requirements, it’s a good idea to break this us a little more into individual (atomic) tasks. First, we need to take a look at the SIMILE Timeline Widget documentation. Taking a quick look at their Getting Started guide, this seems pretty straight forward

1. Include the Timeline javascript
2. Create a div container in the HTML view (e.g. <div id=”timeline”></div>
3. Format the items from Omeka as Timeline Events
4. Add the Timeline.create() call to the Omeka HTML view

The specification states that this needs to be a helper function. If you’re not familiar with this term, a helper function is a block of code that does some of the computation for another piece of code. In many frameworks, helper functions aren’t actual objects, but pieces of procedural code that can be accessed from across the application that help add functionality that isn’t exactly proper to place in a model or controller; essentially these are statically accessible functions (in coding terms) that can be called from properly instantiated objects (in this case a view).

Now that we have good idea of what the code should do, let’s turn to some actual code. Omeka uses the Zend Framework to keep from doing a lot of repetitive programming, so most of the syntax of what we need to do is driven by how Zend handles PHP. On top of the Zend Framework, Omeka implements its own plugin infrastructure, so there are a few things we need to take in to account in our design.

The Zend Framework is a model-view-controller (MVC) framework designed to organize code for maintainability and DRY-ness (Don’t Repeat Yourself). One the the hallmarks of most MVC applications is its physical separation of files and functionalities. In the case of Omeka plugins, the hierarchy is generally split into model, view, controller, and tests directory. For the Timeline plugin, since we are developing a helper function, we use a slightly modified directory structure:

Timeline
|__helpers
|__tests
|__views

We also need some mechanism to tell Omeka about a plugin. This metadata is currently provided in a file called plugin.ini. This file is pretty straight forward, but  let’s go over it briefly:

[info]
name="Timeline"
author="Scholars' Lab"
description="SIMILE Timeline for Omeka"
link="http://omeka.org/codex/Plugins/Timeline"
omeka_minimum_version="1.0"
omeka_tested_up_to="1.2"
version="1.1"
tags="Timeline, simile, chronology, time, temporal"

This file is what the Omeka admin interface uses to display information about your plugin. Two things that may not be initially obvious are the omeka_minimimum_version and omeka_tested_up_to lines. One fact you’ll learn about software development, especially with API development, is as projects mature, the needs of the API grow along with them. You want to be able to mitigate potential issues should the plugin API change by explicitly setting the minimum revision number that your plugin is tested against (you can get older revisions from the SVN tags repo at https://omeka.org/svn/tags/).

Note: you can run multiple versions of Omeka on your machine for testing by checking out separate versions of the software in you web tree. For instance, you can have localhost/omeka1.0, localhost/omeka1.1, localhost/omeka_trunk. Setting this up is beyond the scope of this post (be sure to set up separate databases), but if you have questions, leave a comment.

The next thing we need to do is tell Omeka what to do with our plugin. The top-level plugin.php file contains instructions (hooks) to tell the Admin interface what to do when a user installs or uninstalls the plugin. This is where we let Omeka know that items tagged with “Timeline” should use the Timeline Plugin, to set up some logging to help us debug when something goes wrong, and some default routes to help make “pretty” URLs.

Now, with all the preliminary setup taken care of, now we can start developing the helper function. First, let’s examine the Controller which tells Omeka what to do when an action is requested by the framework. This is actually a fairly straight forward:

<?php
class Timeline_TimelinesController extends Omeka_Controller_Action

{
	private $logger;

	public function init()
	{
		$this->_modelClass = 'Item';
		$writer = new Zend_Log_Writer_Stream(LOGS_DIR . DIRECTORY_SEPARATOR . "timeline.log");
		$this->logger = new Zend_Log($writer);
	}

	public function showAction()
	{
		$this->view->item = $this->findById();
	}

}

Let’s go over briefly what’s going on here. There are some semantics in the way in which these Controller objects are named which are inherited from the way in which Zend handles Controllers. The Timeline_TimelinesController follows the convention of the “package” (the plugin) name, underscore, plural controller name (to handle multiple controllers), and finally “Controller” (which explicitly tells a programmer what function the Object performs). Because this is a Framework, we also want to be able to inherit a lot of behaviors without needing to code them ourselves, which is handled by the “extends Omeka_Controller_Action” (this is the base CRUD class for Omeka which overrides and extends the Zend_Controller_Action object). The “important” part of the Controller code is really the showAction function which sets a variable named “item” in the view which contains a reference to the ID of an Omeka object. The rest just sets up a logger to keep track of what’s going on.

Note: If you run in to problems with this plugin, it is most likely related to logging. For more on this, see the documentation on Retrieving Error Messages.

Now we can get into the guts of the actual helper function. When you boil down the code, most of this is JavaScript with some strategically placed PHP. What we did is create a helper function named “createTimeline” which actually does the work for us. This takes two required items, a div reference to associate the Timeline on your page, and an array of Omeka Items with which to populate the Timeline.

function createTimeline($div, $items = array(), $captionElementSet = "Dublin Core", $captionElement =  "Title", $dateElementSet = "Dublin Core", $dateElement =  "Date" ) {
		echo js("prototype");
		global $mets;
		$mets = array($captionElementSet, $captionElement, $dateElementSet, $dateElement);
		?>
		<!--  we have to load the script in this funny way because we need to get the tag into the head of the doc
			because of the the funky way Simile Timeline loads its sub-scripts  -->
		<script type="text/javascript">
			scripttag = document.createElement("script");
			scripttag.src = "http://static.simile.mit.edu/timeline/api-2.3.0/timeline-api.js?bundle=false";
			scripttag.type = "text/javascript";
			$$("head")[0].insert(scripttag);

			if (typeof(Omeka) == "undefined") {
				Omeka = new Object();
			}

			if (!Omeka.Timeline) {
				Omeka.Timeline = new Object();
			}

		</script>

		<script type="text/javascript" defer="defer">
			Omeka.Timeline.timelinediv = $("<?php echo $div;?>");

			Omeka.Timeline.events = [
			<?php
				function event_to_json($item) {
					global $mets;
					return "{ 'title' : '" . getMet($item, $mets[0], $mets[1]) . "',
					'start' : '" . getMet($item, $mets[2], $mets[3]) . "',
					'description' : '" . getMet($item, "Dublin Core", "Description") . "',
					'durationEvent':false }";
				}
				echo implode(',',array_map('event_to_json', $items));
				?>
				];

		</script>
		<?php
	     echo js("createTimeline");
		?>
		<script type="text/javascript">
			Event.observe(window, 'load', onLoad);
			Event.observe(document.body, 'resize', onResize);
		</script>

		<?php
}

There’s a lot going on here, and there is a mix of PHP in the JavaScript. The first thing is making sure the prototype.js library is included, then declaring a variable named “mets” in the global scope (to make it available to other variable scopes). After we’ve declared $mets, get in to the JavaScript to include on the page and introducing a new JavaScript Namespace (Omeka.Timeline) which allows you to extend this code in other views.

The second script block actually formats Omeka items that you’ve called as Timeline Events in the JSON format calling a helper method we also include in the code:

function getMet($item, $elementSet, $element) {
	 $tmp = $item->getElementTextsByElementNameAndSetName($element, $elementSet);
	 return addslashes( $tmp[0]->text ) ;
}

This function returns the metadata for an Omeka item, which is then used the createTimeline’s sub-method of event_to_json to properly construct an event for Timeline. After all the JSON strings are created, we “glue” all the array elements with a comma with the implode function.

As you can see, not a lot of code actually needs to be written to add functionality to Omeka. With a little research, and some pointers on syntax, extending Omeka can be done quite quickly and doesn’t require a degree in computer science. If you’re interested in getting started on a plugin, I highly recommend the Omeka dev list; the community is growing and questions are answered quickly (usually by folks on the Omeka development team) and is a great way to learn about the technical issues surrounding developing software using the Omeka platform.

Resources

• Omeka Plugin API
• Omeka Developer List
• Timeline Source Code
• Timeline Documentation
• Zend Framework
• Zend Framework Documentation

Since Omeka runs on PHP, this is actually a PHP configuration issue and not something you can currently tweak in Omeka. Basically, you just need to tell PHP to allow larger files sizes that are larger than the default. A very easy way to do this is to edit the .htaccess file that Omeka ships with along the following lines:

php_value upload_max_filesize 20971520
php_value post_max_size 20971520

I’ll note here that doing things this way only affects your Omeka project. Another way to go about this is to add the above to the Apache configuration that defines from where Omeka should be served. For example:

<VirtualHost *:80>

ServerName www.coolomeka.org
DocumentRoot /var/www/omeka

<Directory "/var/www/omeka">
    Options FollwSymLinks
    AllowOverride All
    Order allow,deny
    Allow from all
</Directory>

    ErrorLog logs/omeka_error_log
    TransferLog logs/omeka_transfer_log

    php_value upload_max_filesize 20M
    php_value post_max_size 20M
</VirtualHost>

Lastly, you can edit the php.ini file (usually in /etc/php.ini or /etc/php5/apache2/php.ini). Just do a search in the file and change the following settings:

  memory_limit = 32M
  post_max_size = 20M
  upload_max_size = 20M

You typically don’t need to reload Apache (as long as you did not edit the Apache configuration file) to get these settings to work.

For more info on this, check out these resources

• Description of core php.ini directives
• PHP Upload Configuration
• PHP Directives Related to (Large) File Upload