evaluation

Evaluation Questionnaires

This appendix presents example of questionnaires (see Sec. 8.7) that were used during the evaluation (see Chapter 8) to gather users opinion, measure satisfaction and collect users answers in the question-answering tasks (see Sec. 8.2.3). Since participants using DSpace were asked to fill in slightly different questionnaires than those using JeromeDL we present both sets of questionnaires.

Additional and Detailed Evaluation Results

This appendix presents additional and more detailed results gathered during the evaluation (see Chapter 8).

Details of the Users Satisfaction Results

This section presents six tables with the details on user satisfaction metrics, gathered with the questionnaires during the evaluation. Table D.1 presents comparison of opinions of JeromeDL and DSpace users on various features implemented by both digital libraries. Since these tables deliver only more detailed view on results presented in Chapter 8 we do not describe them in details.

Table D.1: Tasks-based User Satisfaction Metrics
Evaluation task ease of understanding ease of execution intuitive- ness σ P
JeromeDL register 27.46 ( σ=25.01) 25.08 ( σ=22.47) 17.54 ( σ=24.42) 23.95  
DSpace register 32.69 ( σ=18.76) 29.77 ( σ=20.71) 22.31 ( σ=24.71) 28.89  
Δregistering
σ
-16.00% -15.76% -21.38% -17.12% 0.555
JeromeDL adv.search 33.15 ( σ=19.29) 15.46 ( σ=29.53) 11.15 ( σ=28.99) 21.81  
DSpace adv.search 32.62 ( σ=17.44) 17.46 ( σ=28.37) 13.69 ( σ=33.02) 22.88  
Δadv.search
σ
1.65% -11.45% -18.54% -4.66% 0.980
JeromeDL search 24.38 ( σ=24.65) 19.23 ( σ=23.54) 16.46 ( σ=23.91) 20.65  
DSpace search 31.08 ( σ=19.03) 21.08 ( σ=24.16) 13.85 ( σ=29.39) 23.3  
Δsearch
σ
-21.53% -8.76% 18.89% -11.37% 0.837
JeromeDL bookmark 32.00 ( σ=20.23) 20.46 ( σ=19.82) 20.31 ( σ=28.04) 25.36  
DSpace bookmark 19.85 ( σ=20.8) -3.08 ( σ=34.79) -7.23 ( σ=32.74) 5.56  
Δbookmarking
σ
61.24% -765.00% -380.85% 356.13% 0.029
JeromeDL task 1 24.38 ( σ=19.76) 10.77 ( σ=31.18) 7.15 ( σ=28.83) 15.57  
DSpace task 1 28.62 ( σ=24.43) 17.08 ( σ=22.74) 17.85 ( σ=20.55) 22.24  
Δtask.1
σ
-14.78% -36.94% -59.91% -29.99% 0.228

Table D.2: Tasks-based User Satisfaction Metrics (cont.)
Evaluation task ease ease intuitiveness σ  
  of understanding of execution     P
JeromeDL task 2 34.85 ( σ=16.55) 24.77 ( σ=22.54) 25.08 ( σ=21.85) 29.18  
DSpace task 2 11.00 ( σ=33.88) 1.00 ( σ=36.2) 3.00 ( σ=26.24) 5.86  
Δtask.2
σ
216.78% 2,376.92% 735.90% 398.12% 0.029
JeromeDL task 3 34.00 ( σ=19.08) 25.23 ( σ=22.9) 23.08 ( σ=22.01) 28.37  
DSpace task 3 10.38 ( σ=22.78) 18.15 ( σ=23.83) 11.46 ( σ=23.79) 12.91  
Δtask.3
σ
227.41% 38.98% 101.34% 119.74% 0.040

Table D.3 presents details of users satisfaction related to search and browsing mechanisms implemented by JeromeDL and DSpace systems.

Table D.3: Search-related Satisfaction
search task ease of use simplicity intuitiveness interestingness attractiveness usefulness
JeromeDL (1) 17.85 σ=22.06 17.46 σ=21.21 11.69 σ=20.1 17.31 σ=23.86 14.23 σ=21.89 20.77 σ=26.19
DSpace (1) 28.85 σ=19.81 16.15 σ=20.32 15.85 σ=18.52 -6.31 σ=17.61 -11.46 σ=20.18 9.38 σ=17.97
Δtask.1
σ [%]
-38.13% 8.10% -26.21% -374.39% -224.16% 121.31%
JeromeDL (2) 27.46 σ=19.58 21.77 σ=18.22 20.08 σ=21.5 17.92 σ=17.34 14.54 σ=21.08 21.77 σ=22.92
DSpace (2) 10.08 σ=28.01 6.38 σ=28.22 -0.85 σ=23.6 -14.08 σ=17.1 -16.92 σ=17.07 4.08 σ=24.62
Δtask.2
σ [%]
172.52% 240.96% -2,472.73% -227.32% -185.91% 433.96%
JeromeDL (3) 28.62 σ=15.71 23.46 σ=19.77 15.31 σ=21.44 22.38 σ=22.61 14.46 σ=23.13 24.00 σ=20.47
DSpace (3) 23.69 σ=25.75 15.85 σ=24.67 9.00 σ=25.64 -10.46 σ=17.75 -11.46 σ=17.9 4.62 σ=21.94
Δtask.3
σ [%]
20.78% 48.06% 70.09% -313.97% -226.17% 420.00%
           

Table D.4 compare satisfaction of JeromeDL related to semantic, social and recommendation features offered by the system.

Table D.4: Satisfaction Metrics for the Enhanced Features
Feature ease of use simplicity intuitiveness interestingness attractiveness usefulness σ
NLQ 7.85 σ=33.25 11.08 σ=26.54 8.69 σ=29.19 24.92 σ=27.26 12.31 σ=27.6 5.77 σ=27.6 9.8
TTM 4.54 σ=24.48 9.38 σ=20.75 0.62 σ=23.36 12.54 σ=19.51 3.85 σ=20.76 1.92 σ=24.23 4.65
Exhibit 9.38 σ=26.02 12.92 σ=17.38 5.23 σ=23.13 8.77 σ=21.7 10.08 σ=15.72 12.38 σ=22.55 10.04
MBB -6.38 σ=16.73 -2.85 σ=9.52 -3.23 σ=20.24 15.15 σ=20.16 13.54 σ=16.25 12.08 σ=16.29 2.8
SSCF 18.77 σ=28.69 16.31 σ=20.03 17.38 σ=22.13 19.54 σ=17.73 14.54 σ=19.46 18.46 σ=19.34 17.76
Collaborative Browse 6.92 σ=23.35 10.31 σ=14.11 10.23 σ=10.58 15.15 σ=17.38 8.77 σ=18.04 12.54 σ=15.81 10.28
Blog 14.92 σ=13.78 13.31 σ=12.65 13.00 σ=16.57 10.69 σ=14.07 9.77 σ=16.16 14.46 σ=18.39 13.44
Ranking 14.69 σ=23.16 13.85 σ=20.38 10.69 σ=21.46 8.46 σ=12.38 7.23 σ=14.17 13.62 σ=15.84 12.47
Recommend (Resource) 18.54 σ=21.77 21.77 σ=15.3 18.85 σ=19.74 20.08 σ=17.28 13.08 σ=18.45 20.62 σ=15.64 19.32
Recommend (SSCF) 20.85 σ=19.54 16.31 σ=21.43 10.23 σ=19.22 15.00 σ=18.56 10.92 σ=20.82 16.15 σ=17.26 15.83

Table D.5 compares users overall satisfaction measured before and after the evaluation.

Table D.5: Comparison between Overall Impression Before and After Using the Library
  ease of use simplicity intuitiveness interestingness attractiveness usefulness  
JeromeDL pre 13.46 ( σ=23.8) 8.54 ( σ=19.03) 8.15 ( σ=21.83) 11.54 ( σ=19.8) 14.54 ( σ=21.52) 24.15 ( σ=17.19)  
JeromeDL post 18.38 ( σ=22.52) 14.38 ( σ=18.28) 6.62 ( σ=24.71) 24.00 ( σ=18.77) 14.85 ( σ=21.38) 22.62 ( σ=19.87)  
DSpace pre 11.92 ( σ=24.36) 7.54 ( σ=18.43) 5.15 ( σ=22.02) -11.46 ( σ=22.73) -8.38 ( σ=24.26) 4.00 ( σ=19.7)  
DSpace post 16.69 ( σ=23.51) 5.92 ( σ=21.5) 6.77 ( σ=26.21) -12.85 ( σ=26.57) -14.85 ( σ=24.06) 9.92 ( σ=24.2)  
Δpre
σ [%]
12.90 13.27 58.21 200.67 273.39 503.85  
Δpost
σ [%]
10.14 142.86 -2.27 286.83 200.00 127.91  
             

Table D.6 compares overall satisfaction of JeromeDL users from using semantic and social features.

Table: General Satisfaction ( σ) of Semantic and Social Features
Enhancement type ease of use ease of understanding ease of execution simplicity intuitiveness
Semantic 4.54 σ=21.61 2.85 σ=21.38 4.46 σ=24.81 5.85 σ=17.58 1.08 σ=22.27
Social 25.62 σ=16.01 23.92 σ=12.61 15.23 σ=19.46 10.31 σ=19.73 11.46 σ=24.21
Enhancement type interestingness attractiveness usefulness σ  
Semantic 25.38 σ=18.18 12.77 σ=21.99 21.31 σ=25.21 8.65  
Social 26.77 σ=19.36 22.85 σ=23.45 29.77 σ=17.05 21.27  

Finally, Table D.7 compares satisfaction of JeromeDL and DSpace users during the memory task.

Table D.7: Task-based Satisfaction for the Memory Task
  ease of understanding ease of execution intuitiveness σ
JeromeDL 29.11 ( σ=18.95) 2.00 ( σ=27.11) 21.11 ( σ=16.13) 19.08
DSpace 10.89 ( σ=23.90) -17.22 ( σ=30.89) -1.00 ( σ=24.90) -0.54
σ 167.35% 111.61% 2,211.11% 3,635.37%

Users Opinion on Particular Search and Browsing Features

After the evaluation we have asked JeromeDL users to select up to three (3) features which they liked, did not liked, found useful, and found useless (see Fig. D.1).

Most of the JeromeDL users liked the simple search and bookmarking feature; only a few less liked SSCF (collaborative filtering), natural language queries (NLQ), TTM, and resource recommendations. Other features were not so highly rated, and some of them, e.g., Exhibit and MBB, were actually disliked by many participants. Surprisingly equally many participants did not like NLQ, TTM, and advanced search. This can be caused by the complexity of the latter two, and limited set of queries supported by NLQ during the evaluation (only those that come with the default JeromeDL distribution).

Figure D.1: Average Number of References




From the perspective of usability of features implemented in JeromeDL, the search and bookmarking (including SSCF) were the most favorable. TTM, MBB, NLQ, and Exhibit were found pretty useless. In our opinion, it was mainly caused by the complexity of these solutions compared to the tasks users were asked to perform, i.e., answer three questions.

Scores Assigned During the Evaluation

This appendix presents scores which were assigned to articles from the reference database (RefDB, see Table A.1) and other articles (see Table A.2) cited by the participants of the evaluation (see Chapter 8). The scores were assigned based on how well each article answered given questions; we used the Score(ref, q) function (see Eq. 8.2) defined in Chapter 8.

Table A.1: Scores Assigned to the Referenced Articles
Article Q1 Q2 Q3 Q4 Q5 Q6 Q7
Morahan-Martin and Schumacher (2003) 0 0 1 0 0 2 0
Metzger et al. (2003a) 2 2 0 0 2 0 0
Wathen and Burkell (2002) 2 2 0 0 2 0 0
Amiel and Sargent (2004) 0 0 2 0 0 2 0
Hills and Argyle (2003) 0 0 2 0 0 2 0
Liu (2004) 0 1 0 0 0 0 0
Landers and Lounsbury (2006) 0 0 2 0 0 2 0
Leung (2003) 0 0 0 1 0 0 1
Klein et al. (2003) 0 0 0 2 0 0 0
Flanagin and Metzger (2001) 2 2 0 0 2 0 0
Hebert and Vorauer (2003) 0 0 0 2 0 0 0
Sanders et al. (2000) 0 0 0 2 0 1 0
Amichai-Hamburger (2002) 0 0 2 0 0 2 0
Bierhoff and Vornefeld (2004) 0 0 0 1 1 0 0
McKenna and Green (2002) 0 0 0 1 0 0 0

Article Q1 Q2 Q3 Q4 Q5 Q6 Q7
Hamburger et al. (2004) 0 0 2 0 0 2 0
Swickert et al. (2002) 0 0 2 0 0 2 0
Jackson et al. (2001) 0 0 2 0 0 2 0
Nithya and Julius (2007) 0 0 2 0 0 2 0
Tidwell and Walther (2002) 0 0 0 2 0 0 0
Anolli et al. (2005) 0 0 2 0 0 2 0
Burgoon et al. (2000) 2 2 0 0 1 0 0
McKenna and Bargh (2000) 0 0 1 0 0 1 0
Kaye and Johnson (2004) 1 1 0 0 1 0 0
Metzger et al. (2003b) 2 2 0 0 2 0 0
Hamburger and Ben-Artzi (2000) 0 0 0 2 0 0 0
Kiousis (2001) 1 1 0 0 2 0 0
Flanagin and Metzger (2000) 2 2 0 0 2 0 0
Schrodt and Turman (2005) 0 0 0 0 0 0 2
Tseng and Fogg (1999) 1 1 0 0 0 0 0
Metzger et al. (2003c) 1 1 0 0 1 0 0
Kraut et al. (2002) 0 0 2 0 0 2 0
Bonebrake (2002) 0 0 2 0 0 2 0
Stanton and Stanton (2001) 0 0 1 0 0 2 0
Fogg and Tseng (1999) 0 2 0 0 0 0 0

Table A.2: Scores Assigned to the Non-Referenced Articles Cited by Participants
Article Q1 Q2 Q3 Q4 Q5 Q6 Q7
Enguix (2005a) -2 -2 -2 -2 -2 -2 -2
Enguix (2005b) -2 -2 -2 -2 -2 -2 -2
Dervan et al. (2006) -2 -2 -2 -2 -2 -2 -1
Hall et al. (2007) -2 -2 -2 -2 -2 -2 -1
Dervan and McDaniel (2007) -2 -2 -2 -2 -2 -2 -1
Duval et al. (2007) -2 -2 -2 -2 -2 -2 -1
Dobrzanski (2007) -2 -2 -2 -2 -2 -2 -2
Nagle and Golden (2006) -2 -2 -2 -2 -2 -2 -2
Eysenck and Keane (2005) -2 -2 -2 -2 1 -2 -2
Kruk et al. (2007) -2 -2 -2 -2 -2 -2 -2
Westerski et al. (2006) -2 -2 -2 -2 -2 -2 -2
Jankowski (2007) -2 -2 -2 -2 -2 -2 -2


Articles Correctly References in the Evaluation

  1. Amichai-Hamburger, Y. (2002).
    Internet and personality.
    Computers in Human Behavior, 18(1):1-10.
  2. Amiel, T. and Sargent, S. L. (2004).
    Individual differences in internet usage motives.
    Computers in Human Behavior, 20(6):711-726.
  3. Anolli, L., Villani, D., and Riva, G. (2005).
    Personality of people using chat: An on-line research.
    Cyberpsychol Behav., 8(1):89-95.
  4. Bierhoff, H.-W. and Vornefeld, B. (2004).
    The social psychlogy of trust with applications in the internet.
    Trust and Distrust in Online Communities: Theoretical and Methodological contributions from Social Psychology, 26:48-62.
  5. Bonebrake, K. (2002).
    College students' internet use, relationship formation, and personality correlates.
    CyberPsychology & Behavior, 5(6):551-557.
  6. Burgoon, J., Bonito, J., Bengtsson, B., Cederberg, C., Lundeberg, M., and Allspach, L. (2000).
    Interactivity in human - computer interaction: a study of credibility, understanding, and influence.
    Computers in Human Behavior, 16(6):553-574.
  7. Flanagin, A. J. and Metzger, M. J. (2000).
    Perceptions of internet information credibility.
    Journalism and Mass Communication Quarterly, 77(3):515-40.
  8. Flanagin, A. J. and Metzger, M. J. (2001).
    Internet use in the contemporary media environment.
    Human communication research, 27(1):153-181.
  9. Fogg, B. and Tseng, H. (1999).
    The elements of computer credibility.
    In Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, pages 80 - 87. ACM New York, NY, USA.
  10. Hamburger, Y. and Ben-Artzi, E. (2000).
    The relationship between extraversion and neuroticism and the different uses of the internet.
    Computers in Human Behavior, 16(4):441-449.
  11. Hamburger, Y., Fine, A., and Goldstein, A. (2004).
    The impact of internet interactivity and need for closure on consumer preference.
    Computers in Human Behavior, 20(1):103-117.
  12. Hebert, B. G. and Vorauer, J. D. (2003).
    Seeing through the screen: Is evaluative feedback communicated more effectively in face-to-face or computer-mediated exchanges?
    Computers in Human Behavior, 19(1):25-38.
  13. Hills, P. and Argyle, M. (2003).
    Uses of the internet and their relationships with individual differences in personality.
    Computers in Human Behavior, 19(1):59-70.
  14. Jackson, L. A., Ervin, K. S., Gardner, P. D., and Schmitt, N. (2001).
    Gender and the internet: Women communicating and men searching.
    Sex Roles, 44(5-6):363-379.
  15. Kaye, B. K. and Johnson, T. J. (2004).
    A web for all reasons: uses and gratifications of internet components for political information.
    Telematics and Informatics, 21(3):197-223.
  16. Kiousis, S. (2001).
    Public trust or mistrust? perceptions of media credibility in the information age.
    Mass Communication & Society, 4(4):381-403.
  17. Klein, E. E., Clark, C. C., and Herskovitz, P. J. (2003).
    Philosophical dimensions of anonymity in group support systems: Ethical implications of social psychological consequences.
    Computers in Human Behavior, 19(3):355-382.
  18. Kraut, R., Kiesler, S., Boneva, B., Cummings, J., Helgeson, V., and Crawford, A. (2002).
    Internet paradox revisited.
    Journal of social issues, 58(1):49-74.
  19. Landers, R. N. and Lounsbury, J. W. (2006).
    An investigation of big five and narrow personality traits in relation to internet usage.
    Computers in Human Behavior, 22(2):283-293.
  20. Leung, L. (2003).
    Impacts of net-generation attributes, seductive properties of the internet, and gratifications-obtained on internet use.
    Telematics and Informatics, 20(2):107 - 129.
  21. Liu, Z. (2004).
    Perceptions of credibility of scholarly information on the web.
    Information Processing and Management: an International Journal, 40(6):1027 - 1038.
  22. McKenna, K. Y. A. and Bargh, J. A. (2000).
    Plan 9 from cyberspace: The implications of the internet for personality and social psychology.
    Personality and social psychology review, 4(1):57-75.
  23. McKenna, K. Y. A. and Green, A. S. (2002).
    Group dynamics: Theory, research, and practice.
    Group Dynamics: Theory, Research, and Practice, 6(1):116-127.
  24. Metzger, M. J., Flanagin, A. J., Eyal, K., Lemus, D. R., and Mccann, R. M. (2003a).
    Credibility for the 21st century: Integrating perspectives on source, message, and media credibility in the contemporary media environment.
    Communication Yearbook, 27(1):293-335.
  25. Metzger, M. J., Flanagin, A. J., and Zwarun, L. (2003b).
    College student web use, perceptions of information credibility, and verification behavior.
    Computers & Education, 41(3):271 - 290.
  26. Metzger, M. J., Flanagin, A. J., and Zwarun, L. (2003c).
    An experimental examination of readers' perceptions of media bias.
    Journalism and Mass Communication Quarterly, 80(2):282-294.
  27. Morahan-Martin, J. and Schumacher, P. (2003).
    Computers in Human Behavior, 19(6):659-671.
  28. Nithya, H. and Julius, S. (2007).
    Extroversion, neuroticism and self-concept: Their impact on internet users in india.
    Computers in Human Behavior, 23(3):1322-1328.
  29. Sanders, C. E., Field, T. M., Diego, M., and Kaplan, M. (2000).
    The relationship of internet use to depression and social isolation among adolescents.
    Adolescence, 35(138):237-242.
  30. Schrodt, P. and Turman, P. D. (2005).
    The impact of instructional technology use, course design, and sex differences on students' initial perceptions of instructor credibility.
    Communication Studies, 56.
  31. Stanton, A. D. and Stanton, W. W. (2001).
    To click or not to click: Personality characteristics of internet versus non-internet purchasers.
  32. Swickert, R. J., Hittner, J. B., Harris, J. L., and Herring, J. A. (2002).
    Relationships among internet use, personality, and social support.
    Computers in Human Behavior, 18(4):437-451.
  33. Tidwell, L. C. and Walther, J. B. (2002).
    Computer-mediated communication effects on disclosure, impressions, and interpersonal evaluations: Getting to know one another a bit at a time.
    Human Communication Research, 28(3):317-348.
  34. Tseng, S. and Fogg, B. J. (1999).
    Credibility and computing technology.
    Communications of the ACM, 42(5):39 - 44.
  35. Wathen, C. N. and Burkell, J. (2002).
    Belive it or not: Factors influencing credibility on the web.
    Journal of the American Society for Information Science and Technology, 53(2):134 - 144.

Non-Referenced Articles Cited by Participants

  1. Dervan, S., McCosker, C., McDaniel, B., and O'Nuallain, C. (2006).
    Educational multimedia.
    In Proceeding of m-ICTE 2006.
  2. Dervan, S. and McDaniel, B. (2007).
    Practice perfect - an intelligent music tutoring system for assisting and improving practice and exam preparation.
    In ED-MEDIA-World Conference on Educational Multimedia, Hypermedia & Telecommunications.
  3. Dobrzanski, J. (2007).
    Employing social semantic information sources for e-learning.
    In Proceedings of the Faculty Research Day, NUIG.
  4. Duval, E., Klamma, R., and Wolpers, M. (2007).
    Creating New Learning Experiences on a Global Scale, volume 4753 of Lecture Notes in Computer Science.
    Springer.
  5. Enguix, C. F. (2005a).
    D03.02 semantic web data versioning methodology and implementation.
    Technical Report 1.03, DERI.
    DERI Lion Deliverable 1.03.
  6. Enguix, C. F. (2005b).
    Semantic web data versioning methodology and implementation.
    Technical report, DERI.
    DERI Lion Deliverables 1.03.
  7. Eysenck, M. and Keane, M. (2005).
    Cognitive Psychology - A Student's Handbook.
    Psychology Press Ltd.
  8. Hall, T., McCauley, V., and Dervan, S. (2007).
    Getting something ''out of school'' children's informal learning with novel technology.
    In Proceedings of American Educational Research Association.
  9. Jankowski, J. (2007).
    Copernicus - virtual encyclopedia.
    Deri elite project deliverable 1.6.11, DERI.
  10. Kruk, S. R., Gzella, A., Dobrzanski, J., McDaniel, B., and Woroniecki, T. (2007).
    E-learning on the social semantic information sources.
    In Second European Conference on Technology Enhanced Learning.
  11. Nagle, T. and Golden, W. (2006).
    E-learning bussiness models: a multiple case study.
    In The Irish Academy of Management Conference (2006).
  12. Westerski, A., Kruk, S. R., Samp, K., Woroniecki, T., Czaja, F., and O'Nuallain, C. (2006).
    E-learning based on the social semantic information sources.
    In Proceedings of LACLO 2006.

Semantic Digital Libraries - Improving Usability of Information Discovery with Semantic and Social Services by Kruk

This book is based on Sebastian's PhD thesis. He reviews architectures, abstract models, metadata standards and various technologies for building digital library management systems. He derives requirements for advanced digital libraries and proposes an architecture model and a set of ontologies for semantic digital libraries.

This book presents information discovery services using the semantic and social technologies, and the prototype implementation of a semantic digital library that answers these requirements. The claim presented in this book is that the semantic and social technologies applied a digital library management system delivers more efficient information discovery solutions, while the library users become more satisfied and can remember more information from what they have learned when using the library. This book presents the essence of Sebastian's research on the Semantic Digital Libraries in years 2002-2009.





There are 2 content-wise versions of this book: with and without appendix. Each can be purchased at lulu.com as hardcover, paperback or e-book (full version as PDF, lite version as ePUB).

Hardcover Paperback E-Book
Full version ISBN: 978-1-4452-7770-7

Support independent publishing: Buy this book on Lulu.
ISBN: 978-1-4452-8243-5

Support independent publishing: Buy this book on Lulu.
PDF

Support independent publishing: Buy this book on Lulu.
Lite version* ISBN: 978-1-4452-8864-2

Support independent publishing: Buy this book on Lulu.
Lite Paperback version

Support independent publishing: Buy this book on Lulu.
ePUB**

Support independent publishing: Buy this book on Lulu.
Amazon Kindle
Purchase at Amazon

* NOTE: Lite version does not contain appendix

** Open standard format, compatible with, e.g., iPad DRM-enabled, compatible with Adobe Digital Editions

Syndicate content