Is Your Website Perfect?
21 Dec 2006

Is Your Website Perfect?

21 Dec 2006

What if you could measure how perfect your website was? If you could definitively say that your website was 100%, pure perfection, wouldn’t you grasp the chance to test how it fares in the test? Well, the Social Issues Research Centre (SIRC) recently released the results of a study commissioned by Rackspace Managed Hosting which claims to provide exactly this: the ‘Perfect Website Formula.’

The formula factors in navigability and speed, among other things, to give a result from 0 to 100%. Although it may seem nice and appealing to say “Oh yeah, well, my site is 100% perfect,” what exactly does this statement mean? What is the definition of perfection? The study never defines it, and I’m sure you will not find anyone who can truly define this for everyone and every website. Beyond an imprecise target, this study suffers from many other pitfalls which debunk the final formula; it is a prime example of drawing erroneous conclusions from bad experiment design, implementation, and analysis. The end result is a formula that claims to predict perfection, but falls far from this illusive goal.

The Formula

Pwebsite = { ((14.14*EaseNav) + (13.56*Speed) + (13.11*CleanDes) + (10.89*Func) + (10.89*Up)) – ((12.63*Pops) + (10.32*Ads) +(5.21*MultiM)) } / 6.26

OR (for sites dependent on security)

Pwebsite = { ((14.14*EaseNav) + (13.56*Speed) + (13.11*CleanDes) + (10.89*Func) + (10.89*Up) + (9.77*Secure)) – ((12.63*Pops) + (10.32*Ads) +(5.21*MultiM)) } / 7.24

Where:
Pwebsite = the degree of perfection of the website
EaseNav = ease of navigation
Speed = the speed at which pages load
CleanDes = clean and simple design
Func = functionality
Up = the site is always live
Secure = the overall security protections
Pops = the site tries to give you pop-ups
Ads = excessive advertising
MultiM = Flash and other multimedia

As evident above, the formula rewards websites that people perceive to be easily navigable, fast, functional, secure, reliable (responsive), and possess a clean, simple design. The other factors negatively taken into account are the use of pop-ups, excessive advertising, flash and other multimedia.

All-in-all, one would say that the formula correctly identifies many perceived positives and negatives of websites. The study was quite extensive, including 2,500+ residents of the UK and thus the formula accurately indicated trends of advantageous and disadvantageous aspects of a website. Unfortunately, the formula goes further to claim to deliver an extremely accurate prediction of a site’s ‘perfection.’ By using coefficients rounded to the nearest hundredth, the formula pretends to be highly accurate and comprehensive with small margins of variability. This arbitrary precision is problematic based on the nature of the study and the study’s analysis suffers from quite a few flaws, including:

  • It is founded on qualitative and subjective questions which cannot efficiently inform such precise calculations of ‘perfection’
  • There is no variance provided for the coefficients. Why are these rounded to the hundredth place?
  • The formula does not account for non-linear relationships. What evidence is there that these relationships are not quadratic or logarithmic?
  • The formula does not show correlations among the variables
  • The formula clearly is not robust and does not account for unknown factors

Applying the Formula

By applying the formula, we can unveil another big issue with the calculations, namely that it assumes that the idea of perfection is the same for all websites. Let us examine one type of website under the microscope of this formula: Video Websites (YouTube, Revver, Blip.tv, etc.).

Not having data for the positive attributes of these sites, let us examine the “negative” aspects. According to the formula, sites are penalized for having pop-ups, excessive advertising and flash. In the case of these sites, they certainly employ pop-ups (full-screen viewing windows) and flash for viewing the content, and would thus be penalized for these features. This is to say then, that even if these sites were to be perfectly accessible, secure, functional and usable, they would never reach perfection because they employ useful features that enhance the user experience and actually make the site work. Does this seem a little off? There are actually times when multimedia integration is necessary and beneficial for a site. See discussion on appropriate use of flash within websites and accessibility tips for Flash (PDF).

When these so-called negative features are suitably integrated in a website, they should increase the level of ‘site perfection’ not diminish it. These features cannot be completely written off as bad and decreasing the site’s positive reception as they can sometimes improve the user experience and overall site value. Moreover, we cannot simply add the good and subtract the bad, and claim that we have reached a perfect balance.

Other types of sites that may misrepresented by this formula include:

  • Portal sites that effectively use pop-ups
  • Marketing sites (due to the ambiguity of excessive advertising)
  • E-commerce sites that effectively use advertising and pop-ups

The Buck Does Not Stop Here

Although this study provides great insight into some important factors for websites they do not cover them all, and certainly not for all websites. Are there not other attributes which could play a factor, such as the accessibility of the site, the customizability, the quality of the content, credibility etc.? Why is there no offset variable which can account for these factors? The exclusion of credibility and accessibility is particular astounding, considering many will not even use a website that does not encourage trust or that they cannot actually use.

Rate Your Site Quiz

Beyond the study, Rackspace offers an automated version to calculate your site’s score. The quiz asks eight questions rated on a scale of 0 to 10 to determine your score. No mention is made here of the viability of score or the subjectivity involved. To be meaningfully used, the information filled in should be based on a large enough sample size representative of the site’s user base. My major concern with the quiz would have to be question 7 which asks “Does your site contain advertising” with the extreme being “a lot.” Unfortunately, this further misconstrues the variable of “excessive advertising” as it assumes that having a lot of advertising is actually excessive (note negative connotation) when it is the way the advertisements are integrated that can cause negative ramifications.

What Does Make a Site Perfect?

Aiming for perfection is more than just meeting a few guidelines set out in general terms, and following the script of what is good and bad. There are and will be sites that score a perfect 100% on the above mentioned formula that may not be meritorious, such as one that forgoes integrating advertisements and thus must be discontinued due to lost revenue or one with great functionality (does what it says), but that doesn’t include tipping point features which give the edge to a competitor. Beyond going through a checklist of items and scoring a high score on an arbitrary scale, perhaps we should measure our sites based upon its practical value. A few questions to examine include:

  • Do users have a positive experience interacting with the site? (admittedly a loaded question)
  • Is the site accessible to all?
  • Can users easily accomplish desired tasks?
  • Are desired click-through rates achieved?
  • Is the marketing effective?
  • Is the content valuable?
  • Is the website readable?

Asking such pointed questions and finding answers for them through investigative processes will yield much more useful information about where a website is successful and where lacking than plugging in numbers into an arbitrary formula. In the end, in-depth investigation will give us more than an abstract number to attach to our websites; it will give us direction toward improving what exists.

Categories All
Leave a comment
More Posts
Comments
  1. Jesse Beach January 27th, 2007 9:09AM

    What a bunch of bunk. If you ran any one of the 22 million MySpace pages through this formula, it probably wouldn’t get above 10% perfect. But then, you have to ask yourself, why are there 22 million MySpace pages? Obviously, there is something about MySpace that cannot be measured with the usual metrics of usability (where it fails miserably), design heuristics (the pages are atrocious), speed (some pages are so littered it takes many seconds for them to load) and functionality. And with 22 million pairs of eyes pouring over those ads on each page, it pays to know what this variable is. Some call it ‘experience’.

Comment

Leave a Reply