Faster display of web pages with ads (from Amazon or Google)

As I’ve mentioned, I’m hoping to make a little money from adding advertising to a web site I’m putting together. A while ago I added both Amazon Associates ad “widgets” and Google AdSense ad “units” to my pages. The addition process we quite straight-forward and painless. I even put together a little MediaWiki extension for embedding Amazon widgets in wiki pages.

But since then, I’ve been frustrated at how slowly my pages have loaded. Both Amazon and Google ads integrate into web pages by having “the publisher” (you and me) include a block of HTML/JavaScript that they generate into our pages at the locations where the ads are supposed to appear. Unfortunately, when most browsers run into a tag that tells them to fetch a remote JavaScript file in the middle of the page, they stop laying out that page on the screen until they’ve finished fetching the requested JavaScript. So, even though my pages are relatively small and download quickly, only a tiny bit of them was actually being drawn to the screen before the browser tried to fetch an ad. It was especially bad in pages where I’d tried to “float” an ad into the upper-right corner: the screen would blank (my page had been received from my server) and then not a single character of text would be shown for the half-second or second it took for the ad to be fetched. And to compound problems, a page containing multiple ads would be blocked in its rendering separately for each ad—the browser wouldn’t fetch them in parallel.

So, eventually I got frustrated, decided that ads wouldn’t do me any good if their delays drove people from my site, and got down to the business of trying to restructure things so that I wouldn’t pay the ad-fetch penalty until my page’s actual content had been displayed. This turned out to be trickier that I’d planned. I spent quite a while messing around with techniques where I’d cause the ads to be downloaded by adding their HTML tags to the page’s Document Object Model (DOM) only after the basic structure of the page was done. I got several versions of this that almost worked, but it seemed that there were scoping problems caused by the difference between a JavaScript node being inserted into a web page after the fact rather than being parsed on the first pass through, and though they downloaded, I couldn’t get either Amazon’s or Google’s ad code to run when inserting it into the page “late.”

Eventually, I found a trick that did work: putting the ad code in HTML <div> blocks at the very end of the page. Of course, this isn’t where I actually wanted the ads to appear, so I also put empty <div>s at the places within my pages where I did want the ads to appear. Also at the bottom of the page, right between the actual ads and the </body> tag, I included a little bit of JavaScript that takes each ad, removes the DOM object representing the <div> that contains the ad from the page, and then re-inserts the ad’s div object within the empty place-holder div that I’d put earlier in the page. With a tiny bit of polish (setting display:none on the ad-loading divs, and setting the height and width on the place-holder divs so that the page wouldn’t re-layout when they were moved up), I had a solution that had my pages being displayed in full as quick as they were before the ads were included, and with the ads popping into view even quicker.

As for the details, I ended up with a single standard piece of JavaScript for moving ads around within a page, regardless of how many there are or where they’re located:

    function moveAndDisplayDelayedDivs(){
      if (typeof divsToMove !== 'undefined')
	for (var c =0; c < divsToMove.length; c++){
	  var targetDiv = document.getElementById( divsToMove[c] );
	  var divWithAd = document.getElementById( divsToMove[c] + '-content' );

	  // remove the ad div from the main document
	  var parent = divWithAd.parentNode;
	  parent.removeChild( divWithAd );
	  // put the ad div into its target location on the page
	  targetDiv.appendChild( divWithAd );
	  // and make it visible
	  divWithAd.style.display = 'block';
	}
    }

This code relies on a couple of conventions. First, somewhere earlier in the pages, there must have been code that made divsToMove an array containing the IDs of the HTML div tags into which the ads must be moved. To avoid having to have to arrays, I established the convention that if id="foo" is the div where an ad is ultimately supposed to be displayed, then the div containing the third-party markup for that ad would always be id="foo-content". After that, it’s a straight-forward bit of DOM manipulation to move the -content divs up the page into their intended display locations.

I chose to build up the divsToMove array incrementally, making one addition to it in each place in the page where I wanted an ad to be located. So, I made sure the following trivial function is defined at the top of my pages:

    function addToDivsToMove(divId){
      if (typeof divsToMove === 'undefined')
	divsToMove = new Array();
      divsToMove[ divsToMove.length ] = divId;
    }

And then the declaration of a point that will eventually display an ad looks like:

    <div id="google-ad" style="width: 160; height: 600px;"></div>
    <script>
      addToDivsToMove('google-ad');
    </script>

With the third-party ad code itself appearing near the bottom of the page like this:

    <div id="google-ad-content" style="display: none;">
      <script type="text/javascript"><!--
	google_ad_client = "my publisher_id";
	google_ad_slot = "my page_slot";
	google_ad_width = 160;
	google_ad_height = 600;
	//--></script>
      <script type="text/javascript"
	      src="http://pagead2.googlesyndication.com/pagead/show_ads.js" >
      </script>
    </div>

The one thing that is worth repeating/noting is that, in order to avoid the page’s layout being changed when the browser reaches the bottom of the page and executes the code to move the ads up to their final locations, the divs that the ads are going to be moved into need to have their size explicitly set to match the ad. I don’t find this to be problematic, as both Amazon and Google require you to choose from a handful of fixed sizes for their advertisements, anyway. So, as long as the size is fixed anyway, it’s only a minor inconvenience to have to repeat it between the first div‘s declaration and the third-party markup.

And that’s that. But, I have the great good fortune to be implementing my web application in Ruby-on-Rails. So, while people unlucky enough to be putting their page templates together by hand might have a problem making sure that the “moving parts” of the two divs that the ads are split between match up, I can actually put those parts in the same source file right next to each other, and depend on Rails to separate them out for me. Below is the equivalent source for including an Amazon “Omakase” ad block into my Rails-generated pages:

    <% unless @this_is_non_information_page or ENV['RAILS_ENV']=='test' %>
      <% if WontoMedia.ads.amazon and
	    WontoMedia.ads.amazon.associate_id %>

	<div id="block-amazon-ad" class="box"
	    style="width: 99%; height: 600px; text-align: center;"></div>
	<script>
	  addToDivsToMove('block-amazon-ad');
	</script>

	lt;% content_for :last_bottom_page_js do %>
	  <div id="block-amazon-ad-content" style="display: none;">
	    <script type="text/javascript"><!--
	      amazon_ad_tag = "<%= WontoMedia.ads.amazon.associate_id -%>";
	      amazon_ad_height = "600";
	      amazon_ad_height = "160";
	      //-->
	    </script>
	    <script type="text/javascript"
		    src="http://www.assoc-amazon.com/s/ads.js"></script>
	  </div>
	<% end %>
      <% end %>
    <% end %>

(Note that this code comes from my live project, WontoMedia, and is licensed under AGPLv3.)

There are several bits of Rails coolness that this takes advantage of or relates to. First, this blob of code and markup is the sole content of a Rails layout partial, so I can incorporate an Amazon ad into any location of any of my pages by rendering that partial into the desired display location within a page layout. Second, Rails allows multiple independent streams of information that will eventually make up a page to be composed in parallel. Most people don’t take advantage of this, and all their templates are composed together linearly in the order in which different parts include each other. However, by using content_for and having a matching yield in my top-level layout, this one file can generate HTML that is part of Rails’ default content flow (the div that the ad will eventually display in when it’s loaded) and part of the flow I’ve labeled last_bottom_page_js. (Yes, I do have other ..._bottom_page_js flows, so what?)

I also use the embedded-Ruby (ERB) that Rails provides in page layouts and partials to:

• manage which of my pages actually have ads displayed (only those that don’t have the Ruby variable @this_is_non_information_page set for them)
• suppress the inclusion of ads when my pages are loaded during unit testing (yes, the ad-loading delays were a pain when visiting my site, but adding a few seconds per page to a few hundred pages loaded during regression testing nearly made me walk off a pier)
• allow for customization of my web application:
  1. The WontoMedia object is a Ruby OpenStruct which I use to hold configuration information for a particular installation of my web application. When I install my app on sites I run, I fill in the advertising config info with the values associated with my Amazon and Google accounts. If someone else installs WontoMedia, they can run it with no advertising (simply by not including these constants in their config file) or set it up to point to their own advertising accounts.
  2. The actual configuration values are then inserted into the boiler-plate code provided by the ad vendor using Rails’ ERB notation for including the result of evaluating a Ruby expression into the output page: <%= expr -%>.

All-in-all, a relatively satisfying solution to my problems with slow page loading due to external ads.

Advertisement