Australasian Journal of Educational Technology
2009, 25(4), 559-580.
AJET 25

The topic of instructional design in research journals: A citation analysis for the years 1980-2008

Zehra Ozcinar
Ataturk Teacher Training Academy, North Cyprus

This study examines research publication and trends in instructional design, as found in selected professional journals during the period 1980-2008. Citation analysis was used to investigate documents relating to instructional design, as indexed in the Web of Science. Instructional design; instruction design; educational design; learning design; teaching design; ID, etc, were the key phrases used for searching the documents, and a total of 758 analysed documents were judged relevant to the field of instructional design. The results of the study were explored according to the journal of publication; document type; year of publication; author; author affiliation; country of publication; most frequently used words and phrases; subject; language; cited by years and h index rating. Notable findings include the most frequently used key words and phrases (after instructional design, these are cognitive load theory and worked examples); regarding subject area, education, educational research, and educational psychology dominate. With regards to country of origin, 87% of authors came from only 7 countries, with countries in the Far East being poorly represented.


Introduction

In this study, instructional design is defined as the systematic development of instructional specifications, using learning and instructional theory derived from behavioural, cognitive and constructivist theories, in order to ensure the quality of instruction. It is the entire process of the analysis of learning needs and goals and the development of a delivery system to meet those needs, including development of instructional materials and activities, and testing and evaluating all instruction and learner activities.

Citation analysis has been used in the social sciences for investigating the research contributions of individuals, institutions and professional journals (Brown & Gardner, 1985). It allows researchers to examine how frequently a work has been cited by other authors, providing one measure of the influence of a writer or of a particular article. The use of citation analysis as a research tool began during the mid-1950s, when Garfield (1955) proposed citation indexing.

Content analysis and citation analysis of published articles in academic journals has been conducted in a variety of professional fields, such as psychology, science education, and instructional technology. For example, in the field of psychology, Howard, Cole and Maxwell (1987) and Smith et al. (1998) reviewed research papers published in selected American Psychological Association (APA) journals, whilst Eybe and Schmidt (2001) and Tsai and Wen (2005) examined research papers from selected science education journals. Aylward, Roberts, Colombo and Steele (2008) investigated and identified the top 100 cited 'classic' articles in the Journal of Paediatric Psychology from 1976 to 2006, and Uzunboylu & Ozcinar (2009) examined researches and trends in computer assisted language learning published in selected professional documents during the period 1990-2008. As a result, publications do appear periodically that identify and discuss patterns and trends in instructional design and instructional technology (e.g. Hew & Kale, 2007; Reiser & Dempsey, 2006; Ely, 2002; Maushak, Price & Wang, 2000; Gall, Ku, Gurney, Tseng & Yeh, 1999; Klein, 1997).

Building upon Dick and Dick's (1989) system of categorising journal articles, Reeves (1995) created a 5 x 6 grid for categorising research in instructional technology, combining five research methods (quantitative, qualitative, critical theory, literature review, and mixed methods) with six research goals (theoretical, empirical, interpretive, post-modern, developmental, and evaluation). Hew, Kale and Kim (2007) noted that Reeves' review was outdated, and that many new technologies have subsequently emerged in instructional design. Thus, they reviewed and categorised empirical studies relating to instructional technology, and the results were published in three prominent journals between 2000 and 2004: Educational Technology Research and Development (ETR&D), Journal of Instructional Science, and the Journal of Educational Computing Research. Approximately 8% of these articles addressed instructional design.

Klein (1997) examined 100 articles published in the development section of ETR&D between 1989 and 1997, and a limitation of this analysis was that the author's review was of articles published in a single journal. Maushak, Price and Wang (2000) identified and analysed broad categories of topics, based on leading journals in educational technology: categories included research design, student attributes, teachers, and educational environments. Rourke and Szabo (2002) analysed articles that were published in the Journal of Distance Education from 1986 to 2001 and, more recently, Lee, Driscoll and Nelson (2004) examined research topics, methods, and citation trends in 383 articles, published in four prominent distance education journals between 1997 and 2002. Because these reviews were limited to distance education, the results may not reveal the direction of current research or trends in instructional technology (Hew, Kale & Kim, 2007). Basically, all studies relating to instructional design analysis conducted a journal based content analysis, in contrast to the document based citation analysis conducted hitherto.

Brown and Gardner (1985) observed that citation analysis has been used in social sciences for investigating the research contributions of individuals, institutions, and professional journals. Aylward, Roberts, Colombo and Steele (2008) concluded that citation analysis, whilst potentially useful in identifying documents with a large number of citations in a given journal, should also be an indicator of an article's overall influence in a given field. Also, citation analysis allows researchers to examine how frequently a work has been cited by other authors and to measure the influence of a writer or of a particular article. The use of citation analysis as a research tool began in the mid-1950s, when Garfield (1955) proposed citation indexing. With the introduction of the Social Sciences Citation Index (SSCI), the Science Citation Index - Expanded (SCIE), the Arts and Humanities Citation Index (AHCI), and The Institute for Scientific Information (now Thomson Scientific), an avenue opened for the systematic analysis of research trends and the influence of scholarly works.

The citation analysis technique was used by Gall, Ku, Gurney, Tseng and Yeh (2004) in an analysis of ETR&D articles published during the period 1990-1999. These researchers identified and analysed 260 articles, in addition to more than 1600 citations of these articles in other journals. Gall et al. used citation analysis as the basis for understanding the 'influence and influencers' of published works (the approach used in this report), permitting readers to observe 'the social construction of knowledge... publication represents successfully completing the peer review process, but citation of one's work is a greater indicator of the influence of the work' (p. 315).

Ely (2002) described trends as 'snapshots [of educational technology] taken at one point in time, thus reflecting the current status' (p. 7). Sometimes, a citation analysis that is presented reports trends and current topics, based on a review of documents from sources such as professional journals, papers presented at conventions, dissertations, and documents from various databases.

From the framework of findings related to instructional design, as explained above, and the content analyses or journal based citation analyses that have been undertaken by researchers, it can be established that document based citation analysis studies have not been carried out so far. This revealed a research gap for citation analysis in the area of instructional design.

The purpose of this study was to examine research and trends in instructional design, published in selected professional journals during the years 1980-2008. Factors examined in the publications were the journal of publication; the most frequently used words or phrases; document type; year of publication; authors; authors' affiliation; country of publication; subject of document; language of document and h index.

Methodology

Citation analysis was used in this study, to investigate documents relating to instructional design and indexed by the Web of Science, EBSCOhost, and ERIC. Using the keywords educational design, learning design, instruction design, teaching design, ID, etc, to search documents published and indexed in these databases during the period 1980-2008, more than 7,000 documents were found (13 February 2009). Most of the documents in the ERIC and EBSCOhost databases were not peer reviewed journal documents, and not related to the instructional design field. In addition, the instructional design documents indexed in the Web of Science were almost completely indexed in the ERIC and EBSCOhost databases. For these reasons, the researchers decided that only those instructional design documents indexed in the Web of Science would be included; this constitutes the main limitation of the study.

The Web of Science includes three citation indexes, SSCI, SCIE and AHCI, and over 9,800 scholarly, peer reviewed journals, published from 1980 to the present, are included in the indexing. SSCI is a multidisciplinary index to the journal Literature of the Social Sciences, and offers access to bibliographic information, abstracts, and cited references from more than 2,000 journals in the social sciences. SCIE provides access to bibliographic information, author abstracts, and cited references from approximately 6,700 scientific and technical journals, whilst AHCI is a multidisciplinary index covering the journal Literature of the Arts and Humanities.

The keywords relating to instructional design were used to search documents published and indexed in SSCI, SCIE and AHCI during the period 1980-2008, and 795 documents were found (see Figure 1). Using these documents, citation analysis was carried out by referring to article abstracts and publication information indexed in the Web of Science. In cases where publication information was insufficient or missing from the Web of Science, the researchers analysed a full document. The selection of documents was processed by two doctoral researchers in educational technology and two in curriculum and instruction, and then further validated by an associate professor in the field. This procedure identified 758 documents that were judged to be relevant to the field of instructional design.

Figure 1

Figure 1: The instructional design search page in the Web of Science

Documents identified were analysed according to document type; language of document; source of document; year of publication; author(s); most frequently used keywords; citation by year, and most frequently cited. Keywords from relevant documents in the Web of Science during the years 1980-2008 were classified and accumulated, and these can be used to show trends in the field of instructional design. Trend lines help researchers to understand their own field. They clarify information such as the year researches were conducted, in accordance with area of interest, the extent of citations, and they also acknowledge relationships and the fact that an increase in the area of interest or field plays an important role. Trend lines provide the number of citations received in a year, regardless of the publication date of the cited document, divided by the total number of documents published in that year. This formula for calculating trend lines was used by Scopus (Uzunboylu & Ozcinar 2009).

The most frequently cited documents were analysed according to the cited authors, focus, subject area, and whether they were self cited or not. Hirsch (2005) has used the notion of citation analysis to develop a means for ranking the research output of academicians within the disciplines, based on the number of papers published, and the number of times each paper is cited in professional literature. According to Hirsch, 'A scientist has index h if h of his or her Np papers [number of papers] have at least h citations each, and the other (Np - h) papers have ≤ h citations each' (Gill, 2006: 16569). Thus, the academician who writes 50 papers has an h index of 20 if 20 papers have been widely cited by other authors, and each of the remaining 30 papers has been cited fewer than 20 times. Put simply, an author with very few high impact papers or many low impact papers will have a weak h index; the h index helps distinguish between a 'one hit wonder' and an 'enduring performer'. The h index documents were also analysed on the basis of content analysis (topic and research design) methodology, and descriptive statistics were used to analyse and report this data.

Results

Overall instructional design document analysis

Documents taken from journals during the years 1980-2008
A total of 758 documents related to the instructional design field, and all were taken from professional sources (see Table 1). 379 (50%) of the documents were published in 11 journals (each publishing 9 or more documents) and the most frequent publishers of documents relating to instructional design were ETR&D (n = 136, 17.94%); Instructional Science (n = 56, 7.39%); Computers in Human Behaviour (n = 29, 3.83%); British Journal of Educational Technology (BJET) (n = 27, 3.56%); Lecture Notes in Computer Science (n = 20, 2.64%); Computers & Education (n = 19, 2.51%); Educational Technology and Society (n = 19, 2.51%); Learning and Instruction (n = 17, 2.24%); Journal of Educational Computing Research (n = 12, 1.58%); Educational Psychology Review (n = 12, 1.58%); Medical Teacher (n = 12, 1.58%), Applied Cognitive Psychology (n = 11, 1.45%) and Educational Psychologist (n = 9, 1.19%).

Table 1: Number of published documents by journals

RankTitle of journalNTotal docs (%)
1Educational Technology Research and Development13617.94
2Journal of Instructional Science567.39
3Computers in Human Behavior293.83
4British Journal of Educational Technology273.56
5Lecture Notes in Computer Science202.64
6Computers & Education192.51
6Educational Technology and Society192.51
8Learning and Instruction172.24
9Journal of Educational Computing Research**121.58
9Educational Psychology Review121.58
9Medical Teacher121.58
12Applied Cognitive Psychology111.45
13Educational Psychologist91.19

Other journals37950.00

Total758100
** This journal did not index in the Web of Science after 2001.

379 documents (50%) were found in journals that published fewer than 9 documents, that is about 1% or less of the total documents; 3 journals published 7 documents; 2 journals published 6 documents; 8 journals published 5 documents; 8 journals published 4 documents; 25 journals published 3 documents; 45 journals published 2 documents; and 141 journals published 1 document each.

Year of publication
Between 1980-2008, the number of documents published annually in journals relating to instructional design increased: in 1980, 4 documents were published, and in 2008 this figure had risen to 60 (see Figure 2). The least documents were published in 1983 (n = 1), and the most in 2006 (n = 82). Interestingly, published documents increased greatly in 1992 (n = 36), 2004 (n = 57) and 2006 (n = 82). Of all the documents, 383 (54.02%) were published after the year 2000.

Figure 2

Figure 2: Number of published documents (1980-2008)

Language of documents
English was the most frequently used language in the journals relating to instructional design during the years 1980-2008 (n = 743, 98.02%); eight documents (1.05%) were published in German and 3 documents in Turkish, while two were published in French and two in Spanish.

Table 2: Number of published documents by document type

RankDocument typeNTotal docs (%)
1Articles57475.73
2Book reviews10714.12
3Proceedings papers405.28
4Editorial materials151.98
5Meeting abstracts81.06
6Software review70.92
7Discussion40.53
8Reprint, correction, note30.40

Publication by author's affiliation
There were 971 institutions represented in the author affiliations for the 758 documents (see Table 3). The top 12 universities were named 200 times, and were located in the United States (n = 8), Europe (n = 3), and Australia (n = 1) (see Table 3); this represents about 21.6% of the 971 affiliations. Two institutions each provided affiliations for nine documents; 2 institutions each provided affiliations for eight documents; 10 institutions each provided affiliations for seven documents; 13 institutions each provided affiliations for six documents; 8 institutions each provided affiliations for five documents; 16 institutions each provided affiliations for four documents; 28 institutions each provided affiliations for three documents; 56 institutions each provided affiliations for two documents; and 289 institutions recorded affiliations for one document only.

Table 3: Number of published documents by author's affiliation

RankInstitutionNo. of institutional
affiliations
% of institutional
affiliations (n = 971)
1Open University (NL)282.88
2Open University (UK)262.68
3University of New South Wales (AU)232.37
4Penn State University (USA)191.96
5Florida State University (USA)171.75
5Arizona State University (USA)151.54
7Indiana University (USA)151.54
7Utah State University (USA)151.54
9University of Georgia (USA)111.13
9University of Minnesota (USA)111.13
11University of Colorado (USA)101.03
11Erasmus University (NL)101.03

Other institutions77179.40

Total971100
* 45 documents (5.66%) do not contain data in the field being analysed.

Documents by author
A total of 1586 authors published 758 documents in the journals indexed by SSCI, SCIE, and AHCI. Authors publishing 10 or more documents included Jeroen Van Merriënboer, Fred (F. G. W. C) Paas and Paul Kirschner (all from the Open University of the Netherlands), and John Sweller and Paul Chandler from the University of New South Wales: the ranking of these authors was affected by their frequent collaboration and research endeavours. Liesbeth Kester and M. David Merill separately published 9 documents relating to instructional design, and, in addition to these, 7 authors wrote or co-wrote 5 documents, 15 authors wrote or co-wrote 4 documents, 25 authors wrote or co-wrote 3 documents, 109 authors wrote or co-wrote 2 documents, and 1060 authors wrote or co-wrote 1 document. The majority of journal documents relating to instructional design were co-authored.

Table 4: Number of published documents by authors

RankAuthorn% of total author
namings (n = 1586)
1Van Merriënboer, Jeroen301.89
2Paas, Fred (F.G.W.C)241.51
3Sweller, John201.26
3Kirschner, Paul130.82
5Chandler, Paul100.63
6Kester, Liesbeth90.57
6Merrill, M. David90.57
8Hannafin, Michael80.5
9Jonassen, David60.38
9Martens, Rob L.60.38
9Mayer, Richard E.60.38
9Tennyson, Robert D.60.38

Other authors143990.73

Total1586100

Author by country
Almost 85% of the authors (n = 688) were from 7 major countries (see Table 5), whilst the other authors (n = 119, 14.75%) of documents were from 39 other countries. The majority of the authors of the 758 documents reviewed were from the United States of America (n = 438, 53.9%); seventy-eight (9.67%) were from the Netherlands; 53 (6.57%) were from Australia; 44 (5.45%) were from Germany; 38 (4.71%) were from Canada; 28 (3.47%) were from England, and 12 (1.49%) were from Spain.

Table 5: Number of namings of author's country of origin

RankCountryn% of total namings
of country (n = 807)
1United States43853.90
2Netherlands789.67
3Australia536.57
4Germany445.45
5Canada384.71
6England283.47
7Spain121.49

Other country11914.75

Total807100
* 42 records (5.28%) do not contain data in the field being analysed.

The most frequently used key words and phrases
The keywords and phrases used to search documents relating to instructional design are presented in Table 6: there were a total of 1021 keywords and phrases used in the documents relating to instructional design during the years 2002-2008. Nineteen different words and phrases (n = 768, 75.22%) were mostly used (see Table 6), and the rest (n = 253, 24.78%) were used in documents. Instructional design was the most frequently used phrase (n = 162, 15.87%); cognitive load theory was the second most frequently used (n = 131, 12.83%), and worked examples was the third most frequently used (n = 57, 5.58%).

Table 6: Frequency of the most used key words and phrases

RankKey words2002200320042005200620072008Total
1Instructional design972612413532162
2Cognitive load theory57109353233131
3Worked examples237416111457
4Students02351291546
5Multimedia1224981238
6Skills317186733
7Problem based learning1221971032
8Information033593528
8Online learning333046928
10Performance305294427
10Distance education153416727
12Self esteem0131106526
13Strategies213275424
14Design245133422
15Instruction242133520
16Models223442118
17Acquisition132153217
17Knowledge141043417
19Split attention114132315
Total
39559458192154176768

Table 6 reveals an increased use of the following key words and phrases: instructional design; cognitive load theory; worked examples; students; multimedia; skills; problem based learning; information; performance; self esteem; strategies, and distance education.

Subject area of documents
The subject area in the journals relating to instructional design is shown in Table 7. More than half (n = 416) of the 758 documents addressed education or educational research. Educational psychology was the second most frequently addressed subject (n = 130), and education, educational research, and educational psychology were found in a total of 682 documents (61.33%). However, 357 documents in the Web of Science addressed more than one category.

Table 7: Subject area of published documents

RankSubjectn% of total subject
namings (n = 1112)
1Education and educational research41637.41
2Psychology, educational13011.69
3Psychology, multidisciplinary474.23
3Psychology, experimental474.23
5Education, scientific disciplines423.78
6Information science and library science282.52
7Computer science, theory and methods262.34
7Health care sciences and services262.34
7Computer science, interdisciplinary applications262.34
10Communication221.98
11Computer science, information systems211.89

Other subjects28125.27

Total1112100.00
* 2 records (0.28%) did not contain data in the field being analysed

Figure 3

Figure 3: Number of citations according to authors (1980-2008)

Citation by year
Citation by year, with regards to the journals relating to instructional design, can be seen in Figure 3. A total of 5660 works were cited in the 758 documents relating to instructional design during the period 1980-2008; the lowest citation occurred in 1981 (n = 2) and the highest in 2008 (n = 1092). More than half (n = 3584) of the citations occurred between the years 2004-2008. Figure 3 shows a gradual increase in the number of citations for the years 1980-2003, transitioning into larger increases after about 2003. This may be attributed to the spread of information and communication technologies, and the authors' improved access to literature search resources such as ERIC, EBSCOhost, Scopus, The Web of Science, etc.

Analysis of highly cited articles

To identify articles of higher quality or impact, the 758 documents relating to instructional design were further analysed by citation counts in the SSCI, SCIE, and AHCI databases (as at 13 February 2009). Using the h index (see Appendix), the 35 most frequently cited articles published between 1980-2008 and their authors were identified; the citation counts of these articles ranged between 277 and 40. According to Shih, Feng and Tsai (2008), "articles with more citation frequencies are usually those that are better recognised by others in related fields. They probably present more fundamental ideas about the issues for future research" (p.960). The most cited articles were analysed according to research design type, their participants, research setting, and research methods (data collection and data analysis). However, one of the articles was excluded because it was not related to the field of instructional design. The top five articles were selected for detailed analysis.

The top five documents in the h index
Ranked first was Cognitive architecture and instructional design by Sweller, Merriënboer and Paas (1998). This was cited 277 times and also ranked highest in the mean number of annual citations (M = 23.08). The focus of the article was worked examples; programming instruction; schema acquisition; goal specificity; students' learning; load approach; high school; strategies; memory, and generation. The article's most frequently cited top ten authors were Paas, F. (30 times); Van Merriënboer (29 times); Sweller (25 times); Kirschner (15 times); Renkl (13 times); Chandler (12 times); Kester (12 times); Atkinson (9 times); Van Gerven (9 times), and Kalyuga (8 times); the remaining were cited 115 times. Also, the article was self cited 84 times (30.32%) by its authors. The article's subject category was educational psychology.

The second most cited article was Application of cognitive, skill based, and affective theories of learning outcomes to new methods of training evaluation by Kraiger, Ford and Salas (1993). This was cited 164 times by the authors and also ranked fourth in the mean number of annual citations (M = 9.65). The focus of the article was self efficacy; individual differences; performance; knowledge; students; motivation; management; attitudes; task, and participation. The article's most cited authors were Salas (15 times); Bowers, Brown and Burke (cited 6 times); Ford (cited 5 times) and Cannon-Bowers, Gully and Kraiger (cited 4 times). The article was self cited 20 times (12.2%) by its authors, and the subject category was applied psychology.

In third was the article Why some material is difficult to learn by Sweller and Chandler (1994), which was cited 145 times by authors. This article had a low mean number of annual citations (M = 9.06), which were ranked eighth in 35 highly cited documents. The focuses of the article were cognitive load theory; information; instruction; consequences; strategies; attention; pictures; words, and text. The article's most cited authors were Sweller (26 times), Chandler (15 times), Van Merriënboer (8 times), Van Der Meij (7 times) and Kalyuga (6 times). This article was self cited 39 times (26.9%) by its authors, and its subject category in the Web of Science was educational psychology and experimental psychology.

Ranked fourth was the Expertise reversal effect (2003) by Kalyuga, Ayres, Chandler and Sweller, which was cited 119 times by authors; the article also ranked second in the mean number of annual citations (M = 17). The focuses of the article were cognitive load approach; worked examples; split attention; individual differences; working memory; architecture; acquisition; statistics, and science, and the article's most cited authors were Sweller (26 times), Chandler (15 times), Van Merriënboer (8 times), Van Der Meij (7 times) and Kalyuga (6 times). The article was self cited 14 times (11.76%) by its authors, and the subject category was educational psychology.

Ranked fifth was Cognitive load theory and instructional design: Recent developments by Paas, Renkl and Sweller, which was cited 95 times by the authors. The article, an editorial document, also ranked third in the mean number of annual citations (M = 13.57), and its focus was cognitive load approach and instructional design. The most cited authors were Paas (17 times), Sweller (11 times), Van Merriënboer (6 times) and Van Gog (5 times). The document was self cited 23 times (24.21%) by its authors, and the subject category was educational psychology.

Research design
In terms of the type of research design, the papers were classified into three major types, in accordance with the classification used by Shih, Feng, and Tsai (2008). These three major research types, as described in the Handbook of research for educational communication and technology (Jonassen, 1996), are experimental research, which typically involves an experimental group and a control group to test hypotheses regarding certain treatments (Ross & Morrison, 1996); descriptive research, which gathers data from events or participants' responses in describing, explaining, validating, or exploring a particular issue (Kunpfer & McLellan, 1996) and developmental research, which systematically studies the design, development, and evaluation process of certain educational interventions (Richey & Nelson, 1996); the development of several instructional design systems is also included in this type. The majority of h index studies were classified as developmental research (24 of the 32 documents), experimental research was ranked second (n = 6), while the descriptive research method was ranked third (n = 2).

Discussion

In this study, the researchers analysed documents relating to instructional design, published in journals between the years 1980-2008 and indexed in the SSCI, SCIE, and AHCI databases. A citation and content analysis of the studies revealed that 758 published documents related to instructional design; almost 50% of the documents were published in 13 journals, with each journal publishing 9 or more documents. ETR&D was found to contribute the most to the instructional design field, with a publishing rate of almost 20%, and The Journal of Instructional Science attained a publishing rate of 8%. This was an expected result, as both journals can be accessed via the website of the Association for Educational Communication and Technology (AECT) (http://www.aect.org/), acknowledged as a pioneer institution in the field of instructional design and instructional technology (Kirby, Hoadley & Carr-Chellman, 2005). Moreover, the fact that research in the instructional design field is spread over 268 journals indicates that the field has a wide span: this corresponds to the information gathered regarding the authors and subject areas. Reiser and Dempsey (2006) note that the field of instructional design is being increasingly appreciated as a developing interdisciplinary field that helps improve other fields; this development can be observed from the indexes in the Web of Science. 758 documents relating to instructional design have been categorised under 91 subject area categories, with a combination of education, educational research, and educational psychology subject areas accounting for almost 60% of instructional design documents. Thus, almost half of the instructional design documents addressed more than one subject category in the Web of Science. The notion of instructional design is multidisciplinary, which is applied in a wide scope of subjects including education, engineering, ergonomics, psychology, computer science, and health sciences.

A total of 574 (75.73%) documents were articles, and English was the most frequently used language (98.02%) in journals relating to instructional design. This result is similar to Nederhof's (2006) study, which states that the Web of Science index holds documents predominantly in English (93-95%), with the remainder being shared between German (2-3%), French (1%), and other languages. However, in the past few years, educational journals (e.g., Hacettepe University Journal of Education, Teoria De La Educacion, Education Science in Theory and Practice) in other foreign languages (Turkish, German, Spanish, etc.) have been included in the SSCI, SCIE, and AHCI indexes, which has resulted in an increase in the articles on instructional design in different languages. For example, 25 journals have been listed in Thomson Reuters' Master Journal List, which start with the words 'Turk' or 'Turkish' (Thomson Reuters, 2008): this indicates an increase in the number of Turkish scientific documents in the Web of Science (Uzunboylu & Ozcinar, 2009), and this increase will naturally be within the instructional design field that enables interdisciplinary applications.

A total of 1586 authors were named in the 758 documents in the journals relating to instructional design and were indexed by the SSCI, SCIE, and AHCI databases. The top five authors were Jeroen Van Merriënboer (Open University of the Netherlands), Fred Paas (Open University of the Netherlands), John Sweller (University of New South Wales), Paul Kirschner (Open University of the Netherlands), and Paul Chandler (University of New South Wales). These authors appear to be the leading researchers in the instructional design field; however, the ranking of these authors is affected by their frequent collaboration and research endeavours, as the majority of the journal documents relating to instructional design were co-authored by two or more authors. Similar results were found by Kirby, Hoadley, and Carr-Chellman (2005) who observed that almost 70% of the instructional system design and learning science documents were co-authored. Latchem (2006) found that 56% of BJET documents were co-authored. Based on these results, it can be said that there is a consistent trend for articles on instructional design to be based on the collaborative work of groups of researchers.

Almost 85% of the authors of the documents were from 7 countries; and the remaining authors (15%) of the documents were from 40 countries. In more than half of the 758 documents reviewed, almost 54% of the authors originated in the United States of America. The second ranked country of origin of authors was the Netherlands, and third ranked, Australia. The other four high ranking countries of origin of authors were Germany, Canada, England, and Spain, in that order. Documents published were written by authors affiliated with 971 institutions. Regarding the top 12 universities, 8 were from the USA, 3 were from Europe, and 1 was from Australia, which represented about 20% of the 971 institution namings. The top three universities, all outside the United States, were the Open University Netherlands, The Open University of the United Kingdom, and The University of New South Wales (Australia), respectively. This result is very interesting: the two open universities are the world's leading institutions for sourcing instructional design documents. However, the majority of journal documents relating to instructional design had authors' affiliations in the USA.

Anglin and Towers (1992) noted that authors could be cited for affecting a particular area of inquiry, theory, perspective, or research design; for example, the most frequently cited articles in this review have addressed cognitive load theory (see Appendix). Sweller and Chandler (1994), authors of the third most frequently cited article in this review, cited four experiments in their study, in order to support the assumptions underlying cognitive load theory. The following terms were increasingly used during 2002-2008: instructional design; cognitive load theory; worked examples; students; information; skills; performance; multimedia; problem based learning; self esteem; strategies, and distance education.

There were a total of 5660 citations in the documents relating to instructional design during the years 1980-2008 by the authors. The lowest citation occurred in 1981 (n = 1) and the top citation occurred in 2008 (n = 1092). More than half (n = 2584) of citations occurred between the years 2004-2008, and, for these years, an increase in instructional design citations has been identified. This increase resulted from the increasing availability of databases such as ERIC, ScienceDirect, EBSCHOhost and Web of Science, and the fact that it has become easier for researchers to access them. It is expected that document numbers and citation numbers relating to instructional design will increase in the coming years.

Based on the h index, the 35 most frequently cited articles and their authors, published in journals during the years 1980-2008, were related to instructional design. The most frequently cited article in this review addressed Cognitive load theory and instructional design (Sweller, Van Merriënboer & Paas, 1998), and the vast majority of articles relating to instructional design were focused on cognitive load theory, work example, performance, instruction, student, and strategies. The majority of h index studies were classified as developmental research; second ranked was experimental research, and third was the descriptive research method. This is a very interesting result: Ross and Morrison (1996) claimed that experimental research was the traditional research method in the field of studying learning and psychology related issues. Also, if we look at the topics of the h index documents, about two-thirds are based on the psychology of learning, and thus it would be expected that most would be based on experimental research: this result is also very interesting and should be investigated by instructional design researchers. There was only one descriptive research method used in the h index documents; this also suggests that researchers who intend to study learners' cognitive psychology characteristics are likely to utilise an experimental research design, in order to gain better research outcomes. As descriptive research combines qualitative and quantitative research design features, it is more suited to describing the complicated relationships between the different variables involved in specific situations (Shih, Feng & Tsai, 2008). Therefore, most instructional design studies use this approach to investigate the different sub-domains of the field, and the research design types of these 35 articles are a valuable reference for researchers interested in studies relating to the area of instructional design.

Conclusions

Figure 4 shows the citation trend line (by year) in journals relating to instructional design. The low trend line (0.33) occurred in 1981 and 1982 and the top trend line occurred in 2008 (18.2). With the exception of the years 2004 and 2006, there is an increasing trend in the number of articles on instructional design from about 1997 onwards.

Figure 4

Figure 4: Citation trend line for instructional design (1980-2008)

Although the nature of this study is descriptive, it is hoped that the results can provide educators and researchers in the field of instructional design with some insightful ideas about publishing trends for research studies in instructional design related journals. This type of citation analysis is also very beneficial for young scholars in the field (Tsai & Wen, 2005). The analysis can help them to identify contemporary research topics, methods, and trends, and to understand the influence and influencers in their major subjects (Gall et al. 2004). Understanding trends in recent research studies can help policymakers to make future plans, in terms of these issues. The documents identified in the current study have contributed to important advances, not only in the field of instructional design or instructional technology, but also in other important fields. With the continued scholarly submissions to instructional design journals, new citation analyses will emerge and continue to impact this expanding field and other fields.

While the present study offers a descriptive approach to identifying instructional design documents, some limitations should be noted. For example, citation analyses do not provide information regarding how or why a specific work was cited (Everett & Pecotich, 1993; Hoffman & Holbrook, 1993). Also, the instructional design impact of a given article cannot be measured in this manner: although the instructional designer could be using the information from a study in practice, this would not necessarily result in a citation of the article. Citation analyses can be limited by their 'snapshot' approach in examining the citation impact of a given article and, given a more suitable lag time since publication, it is likely that more recent articles would appear as hi-index. Despite these limitations, citation analyses provide a direct, objective, and reliable means of defining the documents in a field (Baltussen & Kindler, 2004; Terajima & Aneman, 2003). Although only part of the bigger picture, the current findings highlight some of the influential works in the field of instructional design, as it is reflected in the field's flagship publications. In addition to this, the current study highlights potential areas of future research. For example, future studies could examine several issues, such as the availability of research grant funding, advances in instructional design models, or the use of more sophisticated statistical techniques (which affect the topics published within related journals), and, in turn, how these aspects influence citation rates.

Although citation analysis is considered as one of the objective measurements of document evaluation, there were still some disadvantages with this method, such as negative citation and gratuitous citation (Shih, Feng & Tsai, 2008; Chu, Hsu & Yu, 1997). Therefore, it is suggested that a combination of citation analysis and other analytical techniques should be used in future research studies, to obtain better results.

This study analysed documents relating to instructional design, published and indexed in SSCI, SCIE and AHCI during a specific period. As instructional design is a multidisciplinary, global field, it would be interesting to see investigations concerning other publications or databases (ERIC, Scopus, Science Direct, etc.) in the future. To understand the continuous trends and patterns in this topic, it is also recommended that similar, journal based studies should be conducted, and should be repeated at least every five years. For example, future studies could focus on citation analysis in the different sub-domains of educational technology, such as e-learning, computer assisted learning, multimedia, distance education, and mobile learning.

The widespread support for the h index is a recent development, and has increased the value of publications in peer reviewed journals. For example, Scopus, a large online database of peer reviewed literature that includes about 16,500 peer reviewed academic journals has incorporated the h index, and Thomson Scientific, formerly known as The Thomson Institute for Scientific Information, now includes the h index as part of its citation report in the Web of Science.

Of particular interest in this review was the absence of the use of instructional design in Asia, a region that is experiencing a convergence of engineering, science, and technology: the number of graduates in engineering has exponentially increased in countries like China, India, Japan, and South Korea. According to Jischke (2006), more than 90% of the world's engineers and scientists will reside in Asia by 2010. Yet, less than 5% of the frequently cited documents in instructional design were authored by academicians from this region.

Acknowledgment

Advice from Dr Hüseyin Uzunboylu, Associate Professor of Educational Technology in the Department of Computer Education and Instructional Technology, Faculty of Ataturk Education, Near East University, is acknowledged with appreciation.

References

Angling, G. J. & Towers, R. L. (1992). Reference citations in selected instructional design and technology journals, 1985-1990. Educational Technology Research and Development, 40(1), 189-194.

Aylward, B. S., Roberts, M. C., Colombo, J. & Steele, R. G. (2008). Identifying the classics: An examination of articles published in the Journal of Pediatric Psychology from 1976-2006. Journal of Paediatric Psychology, 33(6), 576-589.

Baltussen, A. & Kindler, C. H. (2004). Citation classics in critical care medicine. Intensive Care Medicine, 30, 902-910.

Batista, P. D., Campiteli, M. G., Kinouchi, O. & Martinez, A. S. (2006). Is it possible to compare researchers with different scientific interests? Scientometrics, 68(1), 179-189.

Brown, L. D. & Gardner, J. C. (1985). Using citation analysis to assess the impact of journals and articles on contemporary accounting research (CAR). Journal of Accounting Research, 23(1), 84-109.

Chu, C. Y. Cyrus, Hsu, S. K. & Yu, R. R. (1997). Proceedings of the National Science Council (Part C: Humanities and Social Sciences), 7(3), 445-473.

Dick, W. O., Carey, L. & Carey, J. O. (2004). The systematic design of instruction (6th ed.). Boston: Allyn & Bacon.

Dick, W. & Dick, W. D. (1989). Analytical and empirical comparisons of the "Journal of Instructional Development" and "Educational Communication and Technology Journal". Educational Technology Research and Development, 37(1), 81-87.

Egghe, L. (2006). Theory and practice of the g-index. Scientometrics, 69(1), 131-152.

Ely, D. P. (2002). Trends in educational technology (5th Edition). Washington, DC: Association for Educational Communications and Technology.

Everett, J. E., & Pecotich, A. (1993). Citation analysis mapping of journals in applied and clinical psychology. Journal of Applied Social Psychology, 23, 750-766.

Eybe, J. & Schmidt, H. J. (2001). Quality criteria and exemplary papers in chemistry education research. International Journal of Science Education, 23, 209-225.

Gagne, R. M., Wagner, W. W., Golas, K. & Keller, J. (2004). Principles of instructional design. Pacific Grove, CA: Wadsworth Publishing.

Gall, J. E., Ku, H. Y., Gurney, K., Tseng, H. W. & Yeh, H. T. (2004). An analysis of citation patterns in ETR&D. In Proceedings of Association for Educational Communications & Technology (AECT) conference, Chicago, IL, October 19-23, 2004.

Garfield, E. (1955). Citation indexes for science: New dimension in documentation through association of ideas. Science, 122(3159), 108-111.

Gill, M. (2006). The 'h' index: A measure of quality of scientific research. Chowk, 3 January. [viewed 5 Jul 2007 from http://preview.tinyurl.com/yurrhk, verified 6 Sep 2009 at http://www.chowk.com/articles/10107]

Harzing, A. (2007). Publish or perish. [viewed 11 Jul 2007, verified 6 Sep 2009] http://www.harzing.com/pop.htm

Hew, K. F., Kale, U. & Kim, N. (2007). Past research in instructional technology: Results of a content analysis of empirical studies published in three prominent instructional technology journals from the year 2000 through 2004. Journal of Educational Computing Research, 36(3), 269-300.

Hirsch, J. E. (2005). An index to quantify an individual's scientific research output. Proceedings of the National Academy of Sciences, 102(46), 16569-16572. [verified 6 Sep 2009] http://www.pubmedcentral.nih.gov/articlerender.fcgi?tool=pubmed&pubmedid=16275915

Hoffman, D. L. & Holbrook, M. B. (1993). The intellectual structure of consumer research: A bibliometric study of author cocitations in the first 15 years of the Journal of Consumer Research. Journal of Consumer Research, 19, 505-517.

Howard, G. S., Cole, D. A. & Maxwell, S. E. (1987). Research productivity in psychology, based on publication in the journals of the American Psychological Association. American Psychologist, 42, 975-986.

Jin, B. (2007). The AR-index: Complementing the h-index. ISSI Newsletter, 3(1), 6.

Jonassen, D. H. (Ed.) (1996). Handbook of research for educational communications and technology. (2nd ed). Mahwah, NJ: Lawrence Erlbaum Association.

Kirby, J. A., Hoadley, C. M. & Carr-Chellman, A. A. (2005). Instructional systems design and the learning sciences: A citation analysis. Educational Technology Research and Development, 53(1), 37-48.

Klein, J. D. (1997). ETR&D - Development: An analysis of content and survey of future direction. Educational Technology Research and Development, 45(3), 57-62.

Kunpfer, N. & McLellan, H. (1996). Descriptive research methodologies. In D. H. Jonassen (Ed.), Handbook of research for educational communications and technology (2nd ed.) (pp. 1196-1212). Mahwah, NJ: Lawrence Erlbaum.

Latchem, C. (2006). A content analysis of the British Journal of Educational Technology. British Journal of Educational Technology, 37(4), 503-511.

Lee,Y., Driscoll, M. P., & Nelson, D. W. (2004). The past, present, and future of research in distance education: Results of a content analysis. The American Journal of Distance Education, 18(4), 225-241.

Maushak, N. J., Price, R. & Wang, H. C. (2000). Identification and analysis of the leading journals in the field of educational technology: A guide for reading and writing about educational technology. Proceedings of the Association for Educational Communications and Technology Conference, Denver, CO, 25-28 October, 2000. [verified 6 Sep 2009] http://www.eric.ed.gov:80/ERICWebPortal/contentdelivery/servlet/ERICServlet?accno=ED455780

Nederhof, A. (2006). Bibliometric monitoring of research performance in the social sciences and the humanities: A review. Scientometrics, 66(1), 81-100.

Reeves, T. C. (1993). Pseudoscience in computer-based instruction: The case of learner control research. Journal of Computer-Based Instruction, 20(2), 39-46.

Reeves, T. C. (1995). Questioning the questions of instructional technology research. In Proceedings of the 1995 Annual National Convention of the Association for Educational Communications and Technology (AECT), Anaheim, CA. [verified 6 Sep 2009] http://it.coe.uga.edu/~treeves/edit6900/deanlecture.pdf

Reiser, R. A. & Dempsey, J. V. (2006). Trends and issues in instructional design and technology (2nd Edition), Prentice Hall.

Richey, R. & Nelson, W. (1996). Development research. In D. H. Jonassen (Ed.), Handbook of research for educational communications and technology (2nd ed.) (pp. 1213-1245). Mahwah, NJ: Lawrence Erlbaum Association.

Ross, S. & Morrison, G. (1996). Experimental research methods. In D. H. Jonassen (Ed.), Handbook of research for educational communications and technology (pp. 1170-1213). Mahwah, NJ: Lawrence Erlbaum Association.

Rourke, L. & Szabo, M. (2002). A content analysis of the Journal of Distance Education 1986-2001. Journal of Distance Education, 17(2), 63-74.

Scopus (2008). Scopus Journal Analyser. [viewed 13 July 2008] http://info.scopus.com/journalanalyzer/

Shih, M., Feng, J. & Tsai, C. C. (2008). Research and trends in the field of e-learning from 2001 to 2005: A content analysis of cognitive studies in selected journals. Computers & Education, 51(2), 955-967.

Sidiropoulos, A., Katsaros, D. & Manolopoulos, Y. (2006). Generalised h-index for disclosing latent facts in citation networks. Scientometrics, 72(2), 253-280.

Smith, M. C., Locke, S. G., Boisse, S. J., Gallagher, P. A., Krengel, L. E., Kuczek, J. E., et al. (1998). Productivity of educational psychologists in educational psychology journals, 1991-1996. Contemporary Educational Psychology, 23, 173-181.

Tennyson, R. D. (1994). The big wrench vs. integrated approaches: The great media debate. Educational Technology Research and Development, 42(3), 15-28.

Terajima, K. & Aneman, A. (2003). Citation classics in anaesthesia and pain journals: A literature review in the era of the Internet. Acta Anaesthesiologica Scandinavica, 47, 655-663.

Thomson Reuters (2008). Thomson Reuters Master Journal List. [viewed 29 June 2009] http://scientific.thomsonreuters.com/cgi-bin/jrnlst/jloptions.cgi?PC=master

Tsai, C .C. & Wen, L. M. C. (2005). Research and trends in science education from 1998 to 2002: A content analysis of publication in selected journals. International Journal of Science Education, 27, 3-14.

Ullmer, E. J. (1994). Media and learning: Are there two kinds of truth? Educational Technology Research and Development, 42(1), 21-32.

Uzunboylu, H. & Ozcinar, Z. (2009). Research and trends in computer-assisted language learning during 1990-2008: Results of a citation analysis. Eurasian Journal of Educational Research, 34, 133-150. [verified 6 Sep 2009, abstract only] http://www.ejer.com.tr/index.php?git=22&kategori=66&makale=616

Appendix: The most cited articles on h index

RStudyAuthorsYrSourceTopicRes.
design
Tot
cit.
Self cit.Mean
ann. cit.
1Cognitive architecture and instructional designSweller, J., Van Merriënboer, J. & Paas, F.1998Educational Psychology Review, 10(3), 251-296.Cognitive load theoryDevelop-mental2775323.08
2Application of cognitive, skill-based, and affective theories of learning outcomes to new methods of training evaluationKraiger, K., Ford, J. & Salas, E.1993Journal of Applied Psychology, 78(2), 311-328.Learning outcomesDevelop-mental164209.65
3Why some material is difficult to learnSweller, J. & Chandler, P.1994Cognition and Instruction, 12(3), 185-233.Cognitive load theoryExperi-mental145399.06
4The expertise reversal effectKalyuga, S., Ayres, P., Chandler, P., et al.2003Educational Psychologist, 38(1), 23-31.Cognitive load theoryDevelop-mental1191417
5Cognitive load theory and instructional design: Recent developmentsPaas, F., Renkl, A. & Sweller, J.2003Educational Psychologist, 38(1), 1-4.Cognitive load theoryDevelop-mental952313.57
6The split attention effect as a factor in the design of instructionChandler, P. & Sweller, J.1992British Journal of Educational Psychology, 62, 233-246.Cognitive load theoryExperi-mental84274.67
7The regulation of constructive learning processesVermunt, J.1998British Journal of Educational Psychology, 68, 149-171.Constr-uctivist learningDescri-ptive79156.58
8Enhancing teaching through constructive alignmentBiggs, J.1996Higher Education, 32(3), 347-364.Constr-uctivist learningDevelop-mental73235.21
9When two sensory modes are better than oneTindallford, S., Chandler, P. & Sweller, J.1997Journal of Experi-mental Psychology - Applied, 3(4), 257-287.Cognitive load theoryExperi-mental71245.46
10Instructional-design for situated learningYoung, M.1993Educational Technology Research and Development, 41(1), 43-58.Situated learningDevelop-mental7174.18
11An empirically based instructional design theory for teaching conceptsTennyson, R.D. & Cocchiarella, M.J.1986Review of Educational Research, 56(1), 40-71.TeachingDevelop-mental7102.96
12The teaching of concepts - a review of instructional design research literatureTennyson, R.D. & Park, O.C.1980Review of Educational Research, 50(1), 55-70.TeachingDevelop-mental7002.33
13First principles of instructionMerrill, M.D.2002Educational Technology Research and Development, 50(3), 43-59.LearningDevelop-mental6908.62
14Levels of expertise and instructional designKalyuga, S., Chandler, P. & Sweller, J.1998Human Factors, 40(1), 1-17.Cognitive load theoryExperi-mental68145.67
15When less is more: Meaningful learning from visual and verbal summaries of science textbook lessonsMayer, R., Bove, W. & Bryman, A.1996Journal of Educational Psychology, 88(1), 64-73.Meaning-ful learningExperi-mental6794.79
16The structure of design problem spacesGoel, V. & Pirolli, P.1992Cognitive Science, 16(3), 395-429.Structure of designDevelop-mental6633.67
17The promise of multimedia learning: Using the same instructional design methods across different mediaMayer, R.E.2003Learning and Instruction, 13(2), 125-139.Multimedia learningDevelop-mental6509.29
18Toward a design theory of problem solvingJonassen, D.H.2000Educational Technology Research and Development, 48(4), 63-85.Problem solvingDevelop-mental6558.12
19Taking the load off a learner's mind: Instructional design for complex learningVan Merriënboer, J., Kirschner, P. & Kester, L.2003Educational Psychologist, 38(1), 5-13.Cognitive load theoryDevelop-mental63239
20Learning from examples: Instructional principles from the worked examples researchAtkinson, R., Derry, S., Renkl, A. & Wortham, D.2000Review of Educational Research, 70(2), 181-214.Instruction-al principlesDevelop-mental63207.88
21Instructional design models for well-structured and ill-structured problem-solving learning outcomesJonassen, D.H.1997Educational Tech-nology Research and Development, 45(1), 65-94.Instruction-al design modelsDevelop-mental6264.77
22Behavioral fluency: Evolution of a new paradigmBinder, C.1996Behavior Analyst, 19(2), 163-197.Behavior-al fluencyDevelop-mental5924.54
23Cognitive load theory and complex learning: Recent developments and future directionsVan Merriënboer, J. & Sweller, J.2005Educational Psychology Review, 17(2), 147-177.Cognitive load learning/ complex learningDevelop-mental561511.20
24Aids to computer-based multimedia learningMayer, R. & Moreno, R.2002Learning and Instruction, 12(1), 107-119.Multimedia learningDevelop-mental5637
25Empirically-based guidelines for the design of interactive multimediaPark, I. & Hannafin, M.1993Educational Technology Research and Development, 41(3), 63-85.Multimedia learningDevelop-mental5623.29
26The foundations and assumptions of technology-enhanced student-centered learning environmentsHannafin, M. & Land, S.1997Instructional Science, 25(3), 167-202.Student centred learningDevelop-mental5464.15
27Why minimal guidance during instruction does not work: An analysis of the failure of constructivist, discovery, problem-based, experiential, and inquiry-based teachingKirschner, P.A., Sweller, J. & Clark, R.E.2006Educational Psychologist, 41(2), 75-86.TeachingDevelop-mental52913
28Seriously considering play: Designing interactive learning environments based on the blending of microworlds, simulations, and games sourceRieber, L.P.1996Educational Technology Research and Development, 44(2), 43-58.Interactive learning environ-mentsDevelop-mental5063.57
29Assessing professional competence: From methods to programmesVan der Vleuten, C.P.M. & Schuwirth, L.W.T.2005Medical Education, 39(3), 309-317.EvaluationDevelop-mental4659.2
30Use of the critical decision method to elicit expert knowled-ge: A case study in the methodology of cognitive task analysisHoffman, R., Crandall, B. & Shadbolt, N.1998Human Factors, 40(2), 254-276.Cognitive task analysesDevelop-mental4533.75
31Cognitive load while learning to use a computer programChandler, P. & Sweller, J.1996Applied Cognitive Psychology, 10(2), 151-170.Cognitive load theoryExperi-mental44163.14
32When problem solving is superior to studying worked examplesKalyuga, S., Chandler, P., Tuovinen, J., et al.2001Journal of Educational Psychology, 93(3), 579-588.Problem solvingExperi-mental4284.67
33An international virtual medical school (IVIMEDS): The future for medical education?Harden, R. & Hart, I.2002Medical Teacher, 24(3), 261-267.Educat-ional programDevelop-mental4195.12
34Multimedia learning environments: Issues of learner control and navigationLawless, K.A. & Brown, S.W.1997Instructional Science, 25(2), 117-131.Multi-media learningDevelop-mental4143.15
35Multimedia learning environments: Issues of learner control and navigationLawless, K.A. & Brown, S.W.1997Instructional Science, 25(2), 117-131.Multi-media4025

Author: Dr Zehra Ozcinar, Assistant Professor, Ataturk Teacher Training Academy (http://www.aoa.edu.tr/) and Faculty of Ataturk Education, Near East University, North Cyprus (http://www.neu.edu.tr/?q=en/node/665). Email: zehra.ozcinar@aoa.edu.tr

Please cite as: Ozcinar, Z. (2009). The topic of instructional design in research journals: A citation analysis for the years 1980-2008. Australasian Journal of Educational Technology, 25(4), 559-580. http://www.ascilite.org.au/ajet/ajet25/ozcinar.html


[ PDF version of this article ] [ AJET 25 ] [ AJET home ]
HTML Editor: Roger Atkinson [rjatkinson@bigpond.com]
This URL: http://www.ascilite.org.au/ajet/ajet25/ozcinar.html
Created 15 Sep 2009. Last revised 15 Sep 2009.

Page access count