Some time ago iVillage commissioned Dynamic Logic to study the effect of page clutter on the effectiveness of advertising on the iVillage web site. The study strove to discern what effect, if any, ‘online clutter’ (defined as the number of text, image, and advertising elements on a page) had on the brand value score (aggregate of the purchase intent generated, brand favorability, brand awareness, message association, and brand attributes) of the actual on-page advertising.
The research was developed and carried out by Dynamic Logic, OgilvyOne, and Jupiter Media Metrix. While some of the results from the study were expected, many were astounding:
- There was a huge disconnect between the actual level online clutter and perceived level of online clutter. In other words, test subjects’ perception of clutter was much lower than the actual online clutter of page variations employed in this study.
- While 300 people were exposed to high-clutter pages, only 79 study participants perceived their web page to be ‘very cluttered’.
- 91% of those that were shown high-clutter pages perceived them to be ‘somewhat cluttered’ or ‘not cluttered’.
- Brand value score of the advertised items showed a statistically significant increase only for those pages that had low actual clutter. Brand value showed no statistically significant change for medium and high-clutter page variations.
- Brand value score of the advertised items did show a statistically significant increase for web page variations that were perceived to be ‘not cluttered’ or ‘somewhat cluttered’.
- Web page variations that were perceived as ‘very cluttered’ had significantly lower purchase intent rates of the advertised products than web pages that were perceived as ‘not cluttered’ or ‘somewhat cluttered’.
- Purchase intent rates were also higher for page variations that actually had low clutter and medium clutter.
- Online clutter did not have significant impacts on brand favorability.
- Brand awareness showed not statistically significant impacts from online clutter.
- Message association was more favorable for page variations that had less actual clutter and were perceived as having less clutter.
The study solicited participants from iVillage that were exposed to three variations of an iVillage web page corresponding to three levels of ‘online clutter’ as defined by the researchers. Subjects were exposed to the page variations and asked to fill out a survey.
The study included both the actual advertisements and the number of text and image elements in its definition of clutter. The corresponding variations were as follows:
- Low-clutter pages contained 230 words, 8 colored elements, and 18 interest areas.
- Medium-clutter pages contained 141 words, 4 colored elements, and 10 interest areas.
- Low-clutter pages contained 119 words, 3 colored elements, and 8 interest areas.
The analysis was carried out using A/B hypothesis testing and ANOVA variation testing on a data set containing 1500 data points. Therefore, many results of this study were statistically significant for a 90% confidence interval.
It is important to note that the study was conducted for one web page, and even though the number of data points was very large, the results obtained do not directly translate to all web pages. However, we can still use the results from the study to serve as loose guidelines when architecting user experiences for websites that rely on advertising for revenue. We can err on the side of ‘more cluttered’ rather than being tied to the notion that minimalist designs are more effective. This is particularly true if the marginal increase in revenue from adding another advertisement is significant.
In addition, this study also has implications for web sites that are aimed at promoting products as well as selling them. If the marginal increase in revenue from cross-promotional modules or from a high-density product listing is great, it is likely better to err on the side of high-density web pages.
However, this study does not address one important question: What is the effect of clutter on customer retention? Intuition would tell us that user retention is inversely proportional to perceived levels of clutter. Despite this, there are no studies that I have seen that support this. Furthermore, the perturbation that this phenomenon would have on the lifetime value of a user is unclear; it may be that the web site in question makes a lot of revenue immediately and does not rely heavily on long-term revenue.
There is one Burts Media ‘study’ that claims that advertisement clutter on web sites decreases users’ lifetime. I call this a ‘study’ loosely because it seems that the findings relied solely on a questionnaire. For anyone that has conducted a marketing or usability study on the web, this is a huge red flag because what users say and actually do tends to greatly diverge.
All in all, the study covered in this article was well implemented and produces some valuable results. Until I see an equally convincing study that disputes the above-mentioned findings or implications, I think we can all forget the erroneous notion that minimalist web site designs are more effective. Just look at MySpace, Craigslist, and portals such as Yahoo.