kyle conarro

This post was originally published on the Rigor Web Performance blog. It is based on a talk I gave at the Atlanta Web Performance Meetup. Here are the slides from that talk.

Modern websites make a lot of requests. And I mean a lot. And many of these requests are to third-party resources. As this trend continues, it is important to routinely analyze the performance cost of your site’s resources to identify areas for optimization.

One approach to such an analysis would be to aggregate requests at the domain-level. Using raw HAR data, the data that underlies the popular waterfall chart, we can calculate the performance cost of each domain that our site uses.

Using this HAR as an example, our domain analysis for the five slowest domains would look like this:

CNN Domain Analysis

This approach makes it obvious which domains contribute the most to our overall load time. But now what? One option is to eliminate requests to a given domain to reduce its cost. For example, let’s remove all requests to z.cdn.turner.com. A quick scan of the page source reveals nine references to this domain:

https://docs.google.com/a/rigor.com/uc?id=0B4OqDVTQ1tMPQ1d3WmFYZHRNT3c

CNN Page Source

Removing all nine of these should do the trick, right? Unfortunately, no. Looking back at our domain analysis, there are actually 30 requests being made to this domain. So where are the other 11 requests coming from?

Tracking down requests with HTTP Referer

To find the 11 other requests, we can use the HTTP Referer request header to reevaluate our HAR data. This header identifies the resource responsible for making a given request. Here is what the referer analysis looks like for our example HAR:

https://docs.google.com/a/rigor.com/uc?id=0B4OqDVTQ1tMPTU5GVUFMcFE2LTg

CNN Referer Analysis

Instead of aggregating requests by domain, we can now see the resources responsible for the majority of the site’s requests. Not surprisingly, the base page (cnn.com, in this case) is often the main referer. But scanning the table reveals other expensive components, one of which is a resource loaded from z.cdn.turner.com. Expanding this referer reveals several requests to z.cdn.turner.com that we weren’t able to find in the page source:

https://docs.google.com/a/rigor.com/uc?id=0B4OqDVTQ1tMPN2R2TjZNSWcwQnc

Second referer

To make this new analysis even more powerful, we can search for all resources referering to or from z.cdn.turner.com. Any resource matching the search is either requesting additional resources from that domain or is hosted on that domain. Here is what our search results would look like using our same example HAR:

https://docs.google.com/a/rigor.com/uc?id=0B4OqDVTQ1tMPVHVBN0JjX2cxWms

Referer search

Using the power of HTTP Referer, we can now assign costs to each component we add to our site by seeing how many requests it makes. Instead of treating a new JavaScript library as a single resource, for example, we can now include all the dependent resources it requests in our cost analysis, giving us more insight into the cost of a given file.

To simplify this type of analysis, we’ve created a simple tool at insights.rigor.com. Simply upload a HAR file, and the tool will generate domain and referer reports to help you identify costly components. Next time you are adding resources to your website, consider using HTTP Referer to combat bloat and slow load times.

Iterating over collections in Ruby is fun. Methods like [each](<http://ruby-doc.org/core-2.0/Array.html#method-i-each>), [map](<http://ruby-doc.org/core-2.0/Array.html#method-i-map>), and [inject](<http://ruby-doc.org/core-2.0/Enumerable.html#method-i-inject>) are intuitive and easy to use. When I find myself doing the same iteration logic over and over, though, I get frustrated. Duplicate iteration logic is a sign of poor design, and loop-happy methods can be harder to read and harder to test.

To avoid these issues, I’ve started creating Collection classes to wrap collections of objects. For example, say we have a Document class that looks like this:

class Document
	attr_reader :filename
	
	def initialize filename
		@filename = filename 
	end
	
	def read
		@content ||= File.open(filename).read
	end
end

Suppose a user can have many documents. Let’s write a method to read all documents for a user:

class User
	def documents
		# returns an array of all the user's Document objects
	end
	
	def read_documents
		self.documents.map do |doc|     
			doc.read    
		end.join  
	end
end

While this does the job, what if we have a Team class that can also have documents? If we want a read_documents method for Team, we’ll be rewriting the exact same loop.

Let’s see how a DocumentCollection class could help us avoid this duplication.

Creating the Collection

First, we create a simple class to wrap a collection of documents:

class DocumentCollection
	def initialize documents=[]   
		@documents = documents  
	end
end

With our base class at hand, let’s add a read_all method to mimic our User#read_documents method.

class DocumentCollection  
	def read_all    
		@documents.map do |doc|      
			doc.read    
		end.join  
	end
end

Great! Now we have a read_all method defined in a single place. Now we can update our User#read_documents object to use the new collection, like so:

class User  
	def read_documents    
		DocumentCollection.new(self.documents).read_all  
	end
end

Adding this logic to a Team class is now easy, and we are only defining the read_all logic in one place. So we’re done, right?

Not exactly. Although our DocumentCollection class meets our current needs, how easy will it be to extend? For example, finding documents of a certain type might mean defining a matching_type method:

  def matching_type type    
		@documents.select do |doc|      
			doc.type == type    
		end  
	end

This works fine, but notice how all our methods are just loops on our @documents collection. How could we clean this up? By using Ruby’s wonderful [Enumerable](<http://ruby-doc.org/core-2.0/Enumerable.html>) module, that’s how!

Improving with Enumerable

Including Enumerable in a class gives you all the iterative powers of classes like Array without any extra work. All we have to do is define an each method to tell Enumerable how we want to iterate over our class.

Here’s how our DocumentCollection class might look after including Enumerable:

class DocumentCollection  
	include Enumerable  
	
	def initialize documents=[]    
		@documents = documents  
	end  

	def each &block    
		@documents.each(&block)  
	end  

	def read_all    
		map {|doc| doc.read}.join  
	end  

	def matching_type type    
		select {|doc| doc.type == type}  
	end
end

See how much cleaner that is than our initial version? Thanks to Enumerable, adding iteration logic is a snap. Now we can encapsulate any collection logic (and testing) in our collection class and share it across our codebase. Oh, the magic of Enumerable!

A couple months ago I stumbled upon the Bullet Journal, a paper-based note-taking system for getting things done. Having jumped around from to-do list app to to-do list app, I figured I’d scrap the digital approach and give the ol’ notebook another chance. After receiving my Moleskin four weeks ago, I got started with the system.

At its core, the Bullet Journal is just a framework for organizing the contents of a notebook. You can read the full details on the website, but here are the aspects of the framework that I’ve found most useful:

• Organizing items by day or by category (e.g. a collection of notes from a meeting)
• Using bullet point styles to identify types of list items (an empty square for a to do, a circle for an idea, etc.)
• Maintaining a table of contents for quick navigation

Beyond the elements of the Bullet Journal system, I’ve tried to adopt some of the ideas from David Allen’s TED talk on productivity. For example, in order to keep my head clear and stay focused, I start each day by jotting down my tasks and thoughts for that day. As the day goes on, I capture anything that I need to remember in that day’s list, pushing any salient thoughts out of my head and onto the paper to keep my focus.

So far I’ve enjoyed my time using the Bullet Journal, and although my simple notebook doesn’t provide any usage statistics or automated reminders, I’ve found that the flexibility of pen and paper is worth the tradeoff. If you’ve been looking for a new way to capture and track your thoughts, I’d definitely recommend checking out the Bullet Journal.

Building a CSV import option into your application can be very helpful for getting a lot of records into your database in one step. While building out some of our recent reporting additions at Rigor, I wanted to include the option to import CSV data to help our team migrate records from one service to another.

Using Ruby’s standard CSV library makes reading CSV files a breeze. Implementing the import function in a Rails-like way, however, can be more difficult. In general, Rails controller actions look like:

def index
  @my_model = MyModel.new(params[:my_model])
  if @my_model.save
    # yay, it worked! render some success page
  else 
    # womp, render some helpful error messages
  end
end

Massaging a CSV import into a controller action of this format isn’t too terribly difficult, but it may not be completely obvious at first. For my import, I opted to lean on the ActiveModel::Model module (say that five times fast) in Rails 4 to create a model-like wrapper around the CSV import functionality.

I started by creating a class in my models directory and including the module:

class MyAwesomeImporter
  include ActiveModel::Model
end

To get the Rails model-like behavior, we have to define a few model methods:

class MyAwesomeImporter
  include ActiveModel::Model

  def persisted?
    # since this model isn't ever persisted
    # just return false
    false
  end
	
  def valid?
    # logic to determine if import is valid
  end
end

For the valid? method, define what a valid import should look like and test it there, returning true or false. For example:

def valid?
  record_attributes = read_stuff_from_csv
  import_records = record_attributes.map {|attrs| MyModel.new(attrs)}
  import_records.map(&:valid?).all?
end

With the model-y parts out of the way, now we just have to set up our model with access to the CSV file and define read_stuff_from_csv and then we can use our new class in a controller action just like we always do:

# my_awesome_importer.rb
def initialize(file)
  @file = file
end

def read_stuff_from_csv
  CSV.new(@file, headers: :first_row).each do |row|
    # do stuff with the row data
  end  
  # return some useful data for making records
end

# somewhere in my controller
def import
  @my_awesome_importer = MyAwesomeImporter.new(params[:csv_file])

  if @my_awesome_importer.save
    # let the user know the import worked
  else
    # boo, return some errors
  end
end

In addition to being easy to read and understand, using a symbolic model to wrap our import makes it easy to add helpful errors to users when an import fails. Since we included the ActiveModel::Model, adding validation errors is simple. For example, if we want to make sure the imported CSV isn’t empty, we can just add the following:

def csv_empty?
  if CSV.new(File.open(file2), headers: :first_row).to_a.empty?
    # add a helpful error message for the user
    errors.add :base, "CSV is empty"
    true
  end
end

Then we can update our valid? definition to check if the file is empty first:

def valid?
  # original validations
  return false if csv_empty? 
end

By leveraging Rails ActiveModel::Model module, we can incorporate the helpful features of a model into our simple Ruby class. When working with objects that span the MVC pattern, consider creating a model-like object to keep your controllers Railsy and keep form error-handling simple.

This week I attended the first-ever Great Wide Open Conference (GWO) in Atlanta. The conference, which focuses on open source software, featured a lot of great speakers from top tech companies around the country. After two days chock-full of content, here are a few of the highlights:

• Sustainable Government opening keynote by Clay Johnson, CEO at DOBT.co. Clay talked about how closed government hurts us, and how developers + open data can scale democracy.
• NoSQL Databases in the Cloud by Ines Sombra, Lead Data Engineer at Engine Yard. Ines gave a high-energy rundown of MongoDB and Riak and the pros and cons of each (talk on Github).
• Intro to Chef by Nathen Harvey, Technical Community Manager at Chef. Nathen talked about the need for configuration management tools and gave an overview of how Chef Server works.
• Overview of Firefox OS by Jason Weathersby, Technical Evangelist at Mozilla. Jason demoed the developer tools for Firefox OS, an impressive, web-inspired mobile operating system from Mozilla.

As developers, it’s easy (and important) to focus on the backlog and pound away at our keyboards. That said, attending a conference is a good excuse to step out of the hustle-and-bustle, rub shoulders with fellow developers, and think about tools and technologies from a different perspective. It was great to see so many people come out and support open source (700+ registrants), and I hope to see the GWO come back again next year!

There are plenty of articles discussing the health implications of sitting down for long periods of time. I’ve been experiencing some minor lower back pain as of late, so I decided to explore the possibility of using a standing desk while I work. In order to save money, I avoided the more sophisticated desk options and opted to put one together myself.

As with most DIY projects, I started by heading over to Bing Google and seeing what ideas were already out there. A few clicks in, I came across Colin Nederkoorn’s solution. To build his standing desk, Colin elected to build a monitor and keyboard platform and stack it on his existing desk. Since this approach was extremely cheap and easy, I figured I’d give it a shot.

Supplies

As Colin’s blog points out, the only supplies necessary are some screws and a few cheap IKEA pieces.

IKEA stuff:

• LACK side table – $7.99
• EKBY VALTER bracket (x2) – $8
• EKBY VICTOR shelf – $5.99

Other stuff:

• Drill / screwdriver
• 1” wood screws (x4)
• 1 ¾” wood screws (x4)

Directions

1. Assemble the side table

2. Using the 1” screws, attach the shelf brackets to the front legs of the table

3. Using the 1 ¾” screws, attach the shelf to the top of the brackets

4. Place the table on your desk and move all your stuff

5. Stand up!

Results

Here are a few shots of my new setup:

IKEA standing desk

IKEA standing desk

The jury is still out on whether or not standing up will ease my back pain, but if you’re in the market for an inexpensive standing desk, I would definitely recommend this solution.

Note: 20-minute setup time does not include the hours spent wandering through IKEA trying to find the three small items needed for this project

A fellow villager recently asked me how we had implemented custom domain support on Rigor’s public status pages. I tried to find a decent resource that walked through the steps, but I noticed that it was hard to find relevant search results, so I figured I’d write about it.

To clarify, the question at hand is this:

As a Rails developer, how can I let my users point their custom domains to my app?

There are three main steps necessary for adding custom domain support to your Rails project:

1. Have your users add a CNAME record pointing their domain to yours

2. Update your routes to handle any custom domains.

3. Add the controller logic to find resources using the custom domain

For this example, let’s assume your user wants to use their domain at blog.company.com to point to their blog hosted on your Rails app at myapp.com/blogs/:id.

Add a CNAME record

Although this step actually occurs last (once you’ve implemented the logic in your app), it serves as a more logical starting point for this walkthrough. Think about it: for a custom domain to go to your app, the first step is to connect the two domains. Adding a CNAME record does just that.

Have your user add a CNAME record for their domain pointing to your domain. For this example, we’ll point blog.company.com to your app domain myapp.com.

Here is what the CNAME record looks like on DNSimple:

Update your routes

In order for your users to be directed to the right place when they visit their custom domain, you’ll need to update the routes in your Rails app. In this case, we can add another root route that sends requests to your BlogsController, but constrain it to the blog subdomain.

# config/routes.rb
# requests to custom.myapp.com should go to the blogs#show action

root to: 'blogs#show', constraints: { subdomain: 'blog' }

# keep your regular resource routing (you probably already have some version of this part in place)

resources :blogs

This would work fine for users using blog as their subdomain, but what if we want to support any custom subdomain? Enter advanced routing constraints.

Use advanced routing constraints

Rails advanced constraints allow for more powerful routing logic. In our case, we can use an advanced constraint to add support for any custom domain. To use an advanced constraint:

1. Define an object that implements the matches? method:

# lib/custom_domain_constraint.rb

class CustomDomainConstraint
  def self.matches?(request)
    request.subdomain.present? && matching_blog?(request)
  end

  def self.matching_blog?(request    )
    Blog.where(:custom_domain => request.host).any?
  end
end

1. Pass the object to the constraint in your routes.rb:

root to: 'blogs#show', constraints: CustomDomainConstraint

# or use the newer constraint syntax

constraints CustomDomainConstraint do
  root to: 'blogs#show'
end

Add the controller logic

With the new CustomDomainConstraint in place, any request that has a subdomain and a matching Blog record will get routed to the BlogsController#show action. To finish our implementation, we need to add logic in BlogsController that finds the correct blog to render.

Assuming your Blog model already has a custom_domain field, adding the logic is easy:

# app/controllers/blogs_controller.rb

def show 
  @blog = Blog.find_by(custom_domain: request.host)
  # render stuff
end

For this to work properly, your user will need to set their blog’s custom_domain to blog.company.com in your app. With that in place, the request flow looks like this:

1. A user visits blog.company.com, which points to myapp.com

2. Your app handles the request from the custom subdomain, routing the request to the #show action in BlogsController

3. Your controller looks up the blog with blog.company.com as the custom_domain and renders it

And just like that, your Rails app now supports custom domains!

Note for Heroku users:

If you’re using Heroku to host your Rails app, you’ll need an additional bit of logic to make this work. Heroku’s routing requires every domain to exist as a ‘custom domain’ in your Heroku app’s settings. This post outlines a way to automate this step via Heroku’s API and Rails background workers.

Going above and beyond

Keep standard route support

To make sure the standard blog routes still work (/blogs/:id), make sure your BlogsController still supports finding blogs by id:

# app/controllers/blogs_controller.rb

def show
  @blog = Blog.find_by(custom_domain: request.host) || Blog.find(params[:id])
  # render stuff
end

To clean things up a bit, you might consider moving this into a before_filter:

# app/controllers/blogs_controller.rb
before_filter :find_blog, only: :show

private

def find_blog
  # find the blog by domain or ID
end

While these tweaks aren’t required for custom domains to work, they do improve the BlogsController logic to be cleaner and more intuitive.

I recently started using the handy Saddle gem by Airbnb’s Mike Lewis to speed up the development of Ruby API clients. Since some of our code runs on Ruby 1.8, I quickly got to work adding 1.8 support for Saddle.

While working out compatibility issues, I came across a strange error when running the test suite. After digging through Faraday (the underlying HTTP client used in Saddle), and FaradayMiddleware, I finally uncovered the issue:

The splat operator handles nil differently in Ruby 1.8 and Ruby 1.9

A quick test

To confirm my suspicion, I did a quick test in both Ruby versions:

def test_splat *args
  args
end

Ruby 1.8.7

irb> test_splat(nil)
=> [nil]

Ruby 1.9.3

irb> test_splat(nil)
=> []

Saddle originally relied on Ruby 1.9’s treatment of *nil to return an empty array when passing arguments to FaradayMiddleware. Adding 1.8 support revealed this dependency and forced me to handle *nil in the Saddle code instead.

Conclusion

When using Ruby’s handy splat operator, be sure to account for differences in Ruby versions. In my case, the fix was simple, but finding the culprit took some time.