This is Sci7’s review of an article entitled Free-form searching via web sites: Content and moving observed in the context of personal development by Jarkko Kari of The Department of Information Studies, University of Tampere, Finland. The article was published in the Information Processing and Management Journal Vol. 42 (2006) p.769–784.

The observations and conclusions made in this study were based on a small sample size of only fifteen individuals. Subject’s interactions with websites were monitored by video. The research looked at the way individuals moved between sites on the internet, and also how they accessed local documents on their own machines.

The authors make an interesting and valid criticism of the majority of previous studies in the arena of monitoring individuals using the web - in that idea of a web-site is often, oddly, overlooked as the internet is modelled as a collection of interlinked web pages, and the intermediate layer of organisation - the web-site isn’t considered. It is perhaps worth considering for a moment how search engine algorithms for ranking a page’s importance might deal with the concept of a web-site as opposed to a web page. If I have one popular page with lots of incoming links and an associated high page-rank, should that confer some “importance” onto other pages on my site? In the internet of 2006 it is generally easy to identify pages from a particular “site” as they will all share the same domain, however this was not always the case. During the emergence of the web in the mid to late nineties it was very common for ISP users to host pages in directories off the ISP’s domain, and hosting services such as “Geocities’ held some substantive pages.

The paper under review dates itsself as being a product of 2005 by describing web-pages as “published by an organization or individual”, not giving any recognition of the increasing number of sites developed collaboratively by a collection of individuals or those which contain information derived automatically from various sources.

The title of this research includes the phrase “in the context of personal development”, this is expanded on in the paper which describes essentially how people are becoming more adept at using the web (and broader internet) at the same time as the internet expands and develops. In many ways the two processes are impossible to disambiguate. Would an adept user of the 2006 web be able to make better, more effective use of the 1995 web than those online in 1995 were able to? Personal development is also, confusingly, the topic on which subjects were asked to conduct research during the period in which their behaviour was analysed.

For a piece of research conducted in 2005 when internet research, (albeit not that conducted, like this work in a traditional academic university setting) has become relatively refined and sophisticated the aims of the work had an incredibly and unfeasibly broad scope. The aims were quoted as:

1. What kinds of Web sites do people visit?
   • What entity do the sites focus on?
   • Where do the sites come from?
2. How do the persons move in the WWW?
   • Which pages of a site do they visit?
   • How do they proceed from one site to another?
3. Do the qualities of site and manner of moving correlate?

These questions are addressed by analysing ordinary people’s free-form Web searches recorded in university facilities. The participants were not given tasks to complete - merely told to search for information for personal development.

A degree of disorganisation, and unprofessionalism comes across in the description of the research procedures in which participants were offered the opportunity to participate in a number of venues, all but one were later deemed unsuitable by the researchers due to the lack of portability of their video equipment. Not only was any data derived participation of the unvideoed subjects excluded from the research, but these individuals were, perhaps jokingly described as “dissidents” by the researcher. There are other apparent misplaced attempts at humour throughout the document which seriously affect its credibility including on users moving between web-pages: : “sometimes a keyboard shortcut or the computer’s Reset switch was a more viable solution”, and on describing redirects/automatic forwarding: “it was not a matter of artificial intelligence acting or the machine getting muddled…” Later in the paper more amateurish errors are revealed: “Moreover, the resolution of the video recordings left something to be desired, as the body text on the screen often remained unreadable.” The paper was submitted for publication in 2005, despite the research being conducted mainly in the period November 2001 - January 2002, no explanation for this massive delay is offered.

The techniques used for data collection were interviewing, asking the subject to think aloud while performing tasks, observation and videoing of the display on the computer monitor. Overall 687 sites were viewed by the participants, one of the few potentially interesting outcomes was that 29.1% of these sites were search engines, this perhaps is indicative of this research being conducted at a time prior to the emergence of the major search engines, Google, Yahoo, Altavista, MSN etc. Around one in ten of the page views in this study were reported to be of a word processing document on the local computer in which notes were being made or to which material was being copied and pasted.

In this Finnish study, Finnish sites featured around 50% of the time, the authors of the work suggest this shows a national, rather than global or national orientation to web searching, taken independently of language it is not however a credible conclusion from the data obtained.

Overall the lofty ambitions of this research have in no way been met by the research conducted. The small sample size, both in terms of subjects, and pages/sites/time period analysed was insufficient to draw meaningful comparisons from, the dated nature of the research made it much less valuable at the time of publication than it may have been if published as soon as practicable after it was conducted. Where the work may be of interest to future researchers is in prompting them to consider aspects of a “web session” that may be easily over looked such as concurrent interaction with local applications, as well as the consideration of “websites”, as obvious as that may sound. It would be interesting to see overtime how the importance of local/national/global derived pages changes with respect to how those seeking information use them, however this paper does not offer the 2001 benchmark which it might have done which future researchers considering a post “Google Local” internet might have referred back to.

DOI of document discussed here: doi:10.1016/j.ipm.2005.04.001

Tags: Usability.

This entry was posted on Thursday, December 29th, 2005 at 2:24 am and is filed under Resources. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.