home purpose join itg ITG Publication job bank itg listserv hf/Web conferences library

ARTICLE Predicting the Penetration of Intranet Products: Developing the Model Eric Grose, emgrose@sandia.gov Chris Forsythe, jcforsy@sandia.gov Sandia National Laboratories In a previous paper, Predicting Penetration of Intranet Content and Applications, we presented factors that were theorized to affect "penetration" of Intranet products. By "penetration" we mean the extent to which a product is adopted and used by a population. Through this research we seek to explain why certain web products are accessed frequently while others are not, in spite of the designers' intent. Figure 1 illustrates this point.. Note that the top 10 applications account for more traffic than all other sites combined, even though many of the top 100 sites were more costly to develop. Figure 1. Monthly "Hits" to the Top 100 Sites on the Intranet at Sandia National Laboratories (SNL) We proposed the following 13 factors to explain the penetration of Intranet products (Grose, Varner, & Forsythe, 1998). Population Forced Compliance Tactical Advantage Productivity Improvement Mistrust Compatibility Social Connectedness Entertainment Awareness Habit of Predecessor Competition Novelty Exploitive or degrading work Method Eight of the original 13 factors were chosen for testing. The remaining factors were omitted due to concerns that for these factors, the available data did not supply a sufficient range of values (e.g., the entertainment value was approximately equal for all the web products for which data was available). Following is a list of the factors used in the test and the rating scale employed for each factor. Forced compliance I don’t need this for my job. My job is slightly harder without it. My job is harder without it. My job is very hard without it. I must have this for my job OR my job is not possible without it. Productivity I am markedly slower by using this app compared to the old (non web) version. I am slightly slower by using this app. Neutral I am slightly faster by using this app. I do things faster by using this app. Awareness This app is never presented to me (a typical member of the population). Rarely presented Sometimes presented Often presented This app is regularly presented to me. Tactical Advantage Compared to the old nonWeb way I am markedly less effective. Slightly less effective Neutral Slightly more effective Compared to the old nonWeb way I am markedly more effective. Habit of Predecessor There is no predecessor Used predecessor infrequently and w/ short duration Used predecessor frequently w/ short duration OR infrequently w/ extended duration Used predecessor frequently over extended duration Competition No alternatives Inferior or few alternatives Equal alternatives Better alternatives Many equal or better alternatives The following seventeen web products were selected from the SNL corporate intranet: SNL Weekly Bulletin Weather Facility Space Chargeback Query Office Administrative Assistants Weekly Newsletter What's New on the SNL Web Microsoft's System Management Server Financial Manual Occurrence Management Reports Webmail Employee Recognition Award Vacation Balance Query On-Line Maps of SNL Corporate Training Home Page SALUD Employee Health and Recreation Home Page Just In Time Query Computer Security Desk Reference SNL Web Favorites In selecting these web products, the intent was to assure that the sample included a range of values for each of the eight factors. For each web product, ratings were derived using the aforementioned eight factors. These ratings were based on the expert judgement of two raters. Afterward, usage data for the months of January, February, and March, 1998, was obtained from server logs for each web product. Results Using SAS statistical software, a General Linear Models procedure was utilized to test the effects of each factor on the dependent variable, Usage. The initial run revealed that only three factors obtained statistical significance: Population with F(1)=12.89, p < 0.001; Habit with F(1)=17.14, p < 0.001; and Competition with F(1)=11.62, p <0.001. The overall model was also significant with F(10,37)=8.38, p < 0.001 and R-Square=0.69. Next, interactions were selected for testing that were believed to affect usage. These interactions were tested and a second linear model was analyzed to test main effects, with the interactions in the model. Table 4 provides the results of this analysis. Incorporating the interactions between factors, the overall model was significant with F(6,40)=46.98, p < 0.001 and R-Square=0.88. As shown in Table 1, when interactions were considered, Competition was no longer statistically significant, whereas each of the interactions proved to be statistically significant. Table 1. Results of General Linear Model Analysis Factor F Value p < Population 17.52 0.001 Habit with Predecessor 15.44 0.001 Competition 0.66 0.420 Pop x Habit 41.16 0.001 Pop x Competition 6.27 0.010 Habit x Competition 17.90 0.001 Based on the results of this study, the model: Penetration = 7530 - 4.04(Population) + 5440(Habit) - 3963(Competition) + 0.764(Population * Habit) + 1.11(Population * Competition) - 2507(Habit * Competition). Discussion Initially, it was presumed that to predict usage, a relatively complex model incorporating as many as a dozen factors would be required. A conscientious effort was undertaken to identify potential factors. However, the limited testing reported here suggests that the model may be far simpler than anticipated. In particular, 88% of the variance in usage was predicted employing three factors and their interactions. The two factors that had the largest effect were Population and Habit with Predecessor. Regarding Population, the result is not surprising; web products that are relevant to a large number of potential users should receive more usage. However, it had been hypothesized that Habit of Predecessor would operate in a direction opposite than found in this study. Originally, it was reasoned that if there was a non-web predecessor that had been used frequently for an extended duration, users would be reluctant to adopt a web-based version of the same product. Instead, the presence of a well-entrenched non-web predecessor increased the likelihood that users would adopt a web-based counterpart. This suggests that where a strong habit exists, there are genuine user needs being met by the predecessor that may also be met by a web-based alternative. Competition also played a role, but as a factor in two interactions. In general, where there are few alternative means of accomplishing a task (i.e., low competition), a more pronounced effect is observed for both the Population and Habit factors. If one has limited funds for development of web products for a corporate intranet and must choose between several alternatives, usage offers one measure of the potential return on investment. In such a case, three questions might be asked: To how large of a population is the product relevant? Does there exist a non-web predecessor that has been used frequently for an extended duration? Are there equal or better alternatives (web or non-web) available for accomplishing the same tasks? The results reported here represent only an initial test in developing a model to predict usage of web products. While the results are very promising and carry a certain degree of face validity, caution is encouraged. In particular, the study employed a very small sample of web products residing on a single intranet. Additional work is needed to first replicate the findings reported here with a more extensive sample of web products, and then, validate the findings with data from the intranets at other organizations. Reference Grose, E., Varner, L., and Forsythe, C. (1998). Predicting Penetration of Intranet Content and Applications. Proceedings of the 4th Conference on Human Factors and the Web: Our Global Community, June 5, 1998, Basking Ridge, NJ. © Internet Technical Group Last update: December 5, 1998 URL: http://www.sandia.gov/itg/newsletter/dec98/penetration.html hosted by Sandia National Labs Disclaimer: Neither Sandia Corporation, the United States Government, nor any agency thereof, nor any of their employees makes any warranty, express or implied, or assumes any legal liability or responsibility for the accuracy, completeness, or usefulness of any information, apparatus, product, or process disclosed, or represents that its use would not infringe privately-owned rights. Reference herein to any specific commercial product, process, or service by trade name, trademark, manufacturer, or otherwise does not necessarily constitute or imply its endorsement, recommendation, or favoring by Sandia Corporation, the United States Government, or any agency thereof. The views and opinions expressed herein do not necessarily state or reflect those of Sandia Corporation, the United States Government or any agency thereof.