First, unlike previous versions of Ubuntu, you need to do nothing to get a recent version of Ruby. First you will need to install the latest version. Along with it we'll include some extras that you'll need to make things go smoothly as you install the gems.
sudo apt-get install ruby1.9.1-full galternatives libxml2 libxml2-dev libxml2-doc libxml2-utils sqlite3 libsqlite3-dev libxslt1.1 libxslt1-dev
Next, especially if you had Ruby already installed, you'll want to set it as the default in the alternatives system. The galternatives package makes this simple. However, make sure you update gem to the 1.9 version or your gems will install in the wrong place.
Once you have the the proper ruby packages installed and configured, you can use the gem system to do the rest.
sudo gem install sqlite3 -v '1.3.6' sudo gem install rails sudo gem install spree
Finally, create your new store. The -A argument on the spree bundle install accepts all prompts, which should create an admin user email@example.com/spree123.
rails new my_store spree install my_store -A cd my_store rails s
You can now see your storefront at http://localhost:3000 and the admin environment at http://localhost:3000/admin
Please feel free to post any comments or corrections to this guide in the comments.
For more info on customizing Spree, visit http://guides.spreecommerce.com/