|Australasian Journal of Educational Technology
2009, 25(4), 559-580.
The topic of instructional design in research journals: A citation analysis for the years 1980-2008
Ataturk Teacher Training Academy, North Cyprus
This study examines research publication and trends in instructional design, as found in selected professional journals during the period 1980-2008. Citation analysis was used to investigate documents relating to instructional design, as indexed in the Web of Science. Instructional design; instruction design; educational design; learning design; teaching design; ID, etc, were the key phrases used for searching the documents, and a total of 758 analysed documents were judged relevant to the field of instructional design. The results of the study were explored according to the journal of publication; document type; year of publication; author; author affiliation; country of publication; most frequently used words and phrases; subject; language; cited by years and h index rating. Notable findings include the most frequently used key words and phrases (after instructional design, these are cognitive load theory and worked examples); regarding subject area, education, educational research, and educational psychology dominate. With regards to country of origin, 87% of authors came from only 7 countries, with countries in the Far East being poorly represented.
Citation analysis has been used in the social sciences for investigating the research contributions of individuals, institutions and professional journals (Brown & Gardner, 1985). It allows researchers to examine how frequently a work has been cited by other authors, providing one measure of the influence of a writer or of a particular article. The use of citation analysis as a research tool began during the mid-1950s, when Garfield (1955) proposed citation indexing.
Content analysis and citation analysis of published articles in academic journals has been conducted in a variety of professional fields, such as psychology, science education, and instructional technology. For example, in the field of psychology, Howard, Cole and Maxwell (1987) and Smith et al. (1998) reviewed research papers published in selected American Psychological Association (APA) journals, whilst Eybe and Schmidt (2001) and Tsai and Wen (2005) examined research papers from selected science education journals. Aylward, Roberts, Colombo and Steele (2008) investigated and identified the top 100 cited 'classic' articles in the Journal of Paediatric Psychology from 1976 to 2006, and Uzunboylu & Ozcinar (2009) examined researches and trends in computer assisted language learning published in selected professional documents during the period 1990-2008. As a result, publications do appear periodically that identify and discuss patterns and trends in instructional design and instructional technology (e.g. Hew & Kale, 2007; Reiser & Dempsey, 2006; Ely, 2002; Maushak, Price & Wang, 2000; Gall, Ku, Gurney, Tseng & Yeh, 1999; Klein, 1997).
Building upon Dick and Dick's (1989) system of categorising journal articles, Reeves (1995) created a 5 x 6 grid for categorising research in instructional technology, combining five research methods (quantitative, qualitative, critical theory, literature review, and mixed methods) with six research goals (theoretical, empirical, interpretive, post-modern, developmental, and evaluation). Hew, Kale and Kim (2007) noted that Reeves' review was outdated, and that many new technologies have subsequently emerged in instructional design. Thus, they reviewed and categorised empirical studies relating to instructional technology, and the results were published in three prominent journals between 2000 and 2004: Educational Technology Research and Development (ETR&D), Journal of Instructional Science, and the Journal of Educational Computing Research. Approximately 8% of these articles addressed instructional design.
Klein (1997) examined 100 articles published in the development section of ETR&D between 1989 and 1997, and a limitation of this analysis was that the author's review was of articles published in a single journal. Maushak, Price and Wang (2000) identified and analysed broad categories of topics, based on leading journals in educational technology: categories included research design, student attributes, teachers, and educational environments. Rourke and Szabo (2002) analysed articles that were published in the Journal of Distance Education from 1986 to 2001 and, more recently, Lee, Driscoll and Nelson (2004) examined research topics, methods, and citation trends in 383 articles, published in four prominent distance education journals between 1997 and 2002. Because these reviews were limited to distance education, the results may not reveal the direction of current research or trends in instructional technology (Hew, Kale & Kim, 2007). Basically, all studies relating to instructional design analysis conducted a journal based content analysis, in contrast to the document based citation analysis conducted hitherto.
Brown and Gardner (1985) observed that citation analysis has been used in social sciences for investigating the research contributions of individuals, institutions, and professional journals. Aylward, Roberts, Colombo and Steele (2008) concluded that citation analysis, whilst potentially useful in identifying documents with a large number of citations in a given journal, should also be an indicator of an article's overall influence in a given field. Also, citation analysis allows researchers to examine how frequently a work has been cited by other authors and to measure the influence of a writer or of a particular article. The use of citation analysis as a research tool began in the mid-1950s, when Garfield (1955) proposed citation indexing. With the introduction of the Social Sciences Citation Index (SSCI), the Science Citation Index - Expanded (SCIE), the Arts and Humanities Citation Index (AHCI), and The Institute for Scientific Information (now Thomson Scientific), an avenue opened for the systematic analysis of research trends and the influence of scholarly works.
The citation analysis technique was used by Gall, Ku, Gurney, Tseng and Yeh (2004) in an analysis of ETR&D articles published during the period 1990-1999. These researchers identified and analysed 260 articles, in addition to more than 1600 citations of these articles in other journals. Gall et al. used citation analysis as the basis for understanding the 'influence and influencers' of published works (the approach used in this report), permitting readers to observe 'the social construction of knowledge... publication represents successfully completing the peer review process, but citation of one's work is a greater indicator of the influence of the work' (p. 315).
Ely (2002) described trends as 'snapshots [of educational technology] taken at one point in time, thus reflecting the current status' (p. 7). Sometimes, a citation analysis that is presented reports trends and current topics, based on a review of documents from sources such as professional journals, papers presented at conventions, dissertations, and documents from various databases.
From the framework of findings related to instructional design, as explained above, and the content analyses or journal based citation analyses that have been undertaken by researchers, it can be established that document based citation analysis studies have not been carried out so far. This revealed a research gap for citation analysis in the area of instructional design.
The purpose of this study was to examine research and trends in instructional design, published in selected professional journals during the years 1980-2008. Factors examined in the publications were the journal of publication; the most frequently used words or phrases; document type; year of publication; authors; authors' affiliation; country of publication; subject of document; language of document and h index.
The Web of Science includes three citation indexes, SSCI, SCIE and AHCI, and over 9,800 scholarly, peer reviewed journals, published from 1980 to the present, are included in the indexing. SSCI is a multidisciplinary index to the journal Literature of the Social Sciences, and offers access to bibliographic information, abstracts, and cited references from more than 2,000 journals in the social sciences. SCIE provides access to bibliographic information, author abstracts, and cited references from approximately 6,700 scientific and technical journals, whilst AHCI is a multidisciplinary index covering the journal Literature of the Arts and Humanities.
The keywords relating to instructional design were used to search documents published and indexed in SSCI, SCIE and AHCI during the period 1980-2008, and 795 documents were found (see Figure 1). Using these documents, citation analysis was carried out by referring to article abstracts and publication information indexed in the Web of Science. In cases where publication information was insufficient or missing from the Web of Science, the researchers analysed a full document. The selection of documents was processed by two doctoral researchers in educational technology and two in curriculum and instruction, and then further validated by an associate professor in the field. This procedure identified 758 documents that were judged to be relevant to the field of instructional design.
Figure 1: The instructional design search page in the Web of Science
Documents identified were analysed according to document type; language of document; source of document; year of publication; author(s); most frequently used keywords; citation by year, and most frequently cited. Keywords from relevant documents in the Web of Science during the years 1980-2008 were classified and accumulated, and these can be used to show trends in the field of instructional design. Trend lines help researchers to understand their own field. They clarify information such as the year researches were conducted, in accordance with area of interest, the extent of citations, and they also acknowledge relationships and the fact that an increase in the area of interest or field plays an important role. Trend lines provide the number of citations received in a year, regardless of the publication date of the cited document, divided by the total number of documents published in that year. This formula for calculating trend lines was used by Scopus (Uzunboylu & Ozcinar 2009).
The most frequently cited documents were analysed according to the cited authors, focus, subject area, and whether they were self cited or not. Hirsch (2005) has used the notion of citation analysis to develop a means for ranking the research output of academicians within the disciplines, based on the number of papers published, and the number of times each paper is cited in professional literature. According to Hirsch, 'A scientist has index h if h of his or her Np papers [number of papers] have at least h citations each, and the other (Np - h) papers have ≤ h citations each' (Gill, 2006: 16569). Thus, the academician who writes 50 papers has an h index of 20 if 20 papers have been widely cited by other authors, and each of the remaining 30 papers has been cited fewer than 20 times. Put simply, an author with very few high impact papers or many low impact papers will have a weak h index; the h index helps distinguish between a 'one hit wonder' and an 'enduring performer'. The h index documents were also analysed on the basis of content analysis (topic and research design) methodology, and descriptive statistics were used to analyse and report this data.
|Rank||Title of journal||N||Total docs (%)|
|1||Educational Technology Research and Development||136||17.94|
|2||Journal of Instructional Science||56||7.39|
|3||Computers in Human Behavior||29||3.83|
|4||British Journal of Educational Technology||27||3.56|
|5||Lecture Notes in Computer Science||20||2.64|
|6||Computers & Education||19||2.51|
|6||Educational Technology and Society||19||2.51|
|8||Learning and Instruction||17||2.24|
|9||Journal of Educational Computing Research**||12||1.58|
|9||Educational Psychology Review||12||1.58|
|12||Applied Cognitive Psychology||11||1.45|
|** This journal did not index in the Web of Science after 2001.|
379 documents (50%) were found in journals that published fewer than 9 documents, that is about 1% or less of the total documents; 3 journals published 7 documents; 2 journals published 6 documents; 8 journals published 5 documents; 8 journals published 4 documents; 25 journals published 3 documents; 45 journals published 2 documents; and 141 journals published 1 document each.
Year of publication
Between 1980-2008, the number of documents published annually in journals relating to instructional design increased: in 1980, 4 documents were published, and in 2008 this figure had risen to 60 (see Figure 2). The least documents were published in 1983 (n = 1), and the most in 2006 (n = 82). Interestingly, published documents increased greatly in 1992 (n = 36), 2004 (n = 57) and 2006 (n = 82). Of all the documents, 383 (54.02%) were published after the year 2000.
Figure 2: Number of published documents (1980-2008)
Language of documents
English was the most frequently used language in the journals relating to instructional design during the years 1980-2008 (n = 743, 98.02%); eight documents (1.05%) were published in German and 3 documents in Turkish, while two were published in French and two in Spanish.
|Rank||Document type||N||Total docs (%)|
|8||Reprint, correction, note||3||0.40|
Publication by author's affiliation
There were 971 institutions represented in the author affiliations for the 758 documents (see Table 3). The top 12 universities were named 200 times, and were located in the United States (n = 8), Europe (n = 3), and Australia (n = 1) (see Table 3); this represents about 21.6% of the 971 affiliations. Two institutions each provided affiliations for nine documents; 2 institutions each provided affiliations for eight documents; 10 institutions each provided affiliations for seven documents; 13 institutions each provided affiliations for six documents; 8 institutions each provided affiliations for five documents; 16 institutions each provided affiliations for four documents; 28 institutions each provided affiliations for three documents; 56 institutions each provided affiliations for two documents; and 289 institutions recorded affiliations for one document only.
|Rank||Institution||No. of institutional|
|% of institutional|
affiliations (n = 971)
|1||Open University (NL)||28||2.88|
|2||Open University (UK)||26||2.68|
|3||University of New South Wales (AU)||23||2.37|
|4||Penn State University (USA)||19||1.96|
|5||Florida State University (USA)||17||1.75|
|5||Arizona State University (USA)||15||1.54|
|7||Indiana University (USA)||15||1.54|
|7||Utah State University (USA)||15||1.54|
|9||University of Georgia (USA)||11||1.13|
|9||University of Minnesota (USA)||11||1.13|
|11||University of Colorado (USA)||10||1.03|
|11||Erasmus University (NL)||10||1.03|
|* 45 documents (5.66%) do not contain data in the field being analysed.|
Documents by author
A total of 1586 authors published 758 documents in the journals indexed by SSCI, SCIE, and AHCI. Authors publishing 10 or more documents included Jeroen Van Merriënboer, Fred (F. G. W. C) Paas and Paul Kirschner (all from the Open University of the Netherlands), and John Sweller and Paul Chandler from the University of New South Wales: the ranking of these authors was affected by their frequent collaboration and research endeavours. Liesbeth Kester and M. David Merill separately published 9 documents relating to instructional design, and, in addition to these, 7 authors wrote or co-wrote 5 documents, 15 authors wrote or co-wrote 4 documents, 25 authors wrote or co-wrote 3 documents, 109 authors wrote or co-wrote 2 documents, and 1060 authors wrote or co-wrote 1 document. The majority of journal documents relating to instructional design were co-authored.
|Rank||Author||n||% of total author |
namings (n = 1586)
|1||Van Merriënboer, Jeroen||30||1.89|
|2||Paas, Fred (F.G.W.C)||24||1.51|
|6||Merrill, M. David||9||0.57|
|9||Martens, Rob L.||6||0.38|
|9||Mayer, Richard E.||6||0.38|
|9||Tennyson, Robert D.||6||0.38|
Author by country
Almost 85% of the authors (n = 688) were from 7 major countries (see Table 5), whilst the other authors (n = 119, 14.75%) of documents were from 39 other countries. The majority of the authors of the 758 documents reviewed were from the United States of America (n = 438, 53.9%); seventy-eight (9.67%) were from the Netherlands; 53 (6.57%) were from Australia; 44 (5.45%) were from Germany; 38 (4.71%) were from Canada; 28 (3.47%) were from England, and 12 (1.49%) were from Spain.
|Rank||Country||n||% of total namings |
of country (n = 807)
|* 42 records (5.28%) do not contain data in the field being analysed.|
The most frequently used key words and phrases
The keywords and phrases used to search documents relating to instructional design are presented in Table 6: there were a total of 1021 keywords and phrases used in the documents relating to instructional design during the years 2002-2008. Nineteen different words and phrases (n = 768, 75.22%) were mostly used (see Table 6), and the rest (n = 253, 24.78%) were used in documents. Instructional design was the most frequently used phrase (n = 162, 15.87%); cognitive load theory was the second most frequently used (n = 131, 12.83%), and worked examples was the third most frequently used (n = 57, 5.58%).
|2||Cognitive load theory||5||7||10||9||35||32||33||131|
|7||Problem based learning||1||2||2||1||9||7||10||32|
Table 6 reveals an increased use of the following key words and phrases: instructional design; cognitive load theory; worked examples; students; multimedia; skills; problem based learning; information; performance; self esteem; strategies, and distance education.
Subject area of documents
The subject area in the journals relating to instructional design is shown in Table 7. More than half (n = 416) of the 758 documents addressed education or educational research. Educational psychology was the second most frequently addressed subject (n = 130), and education, educational research, and educational psychology were found in a total of 682 documents (61.33%). However, 357 documents in the Web of Science addressed more than one category.
|Rank||Subject||n||% of total subject |
namings (n = 1112)
|1||Education and educational research||416||37.41|
|5||Education, scientific disciplines||42||3.78|
|6||Information science and library science||28||2.52|
|7||Computer science, theory and methods||26||2.34|
|7||Health care sciences and services||26||2.34|
|7||Computer science, interdisciplinary applications||26||2.34|
|11||Computer science, information systems||21||1.89|
|* 2 records (0.28%) did not contain data in the field being analysed|
Figure 3: Number of citations according to authors (1980-2008)
Citation by year
Citation by year, with regards to the journals relating to instructional design, can be seen in Figure 3. A total of 5660 works were cited in the 758 documents relating to instructional design during the period 1980-2008; the lowest citation occurred in 1981 (n = 2) and the highest in 2008 (n = 1092). More than half (n = 3584) of the citations occurred between the years 2004-2008. Figure 3 shows a gradual increase in the number of citations for the years 1980-2003, transitioning into larger increases after about 2003. This may be attributed to the spread of information and communication technologies, and the authors' improved access to literature search resources such as ERIC, EBSCOhost, Scopus, The Web of Science, etc.
The top five documents in the h index
Ranked first was Cognitive architecture and instructional design by Sweller, Merriënboer and Paas (1998). This was cited 277 times and also ranked highest in the mean number of annual citations (M = 23.08). The focus of the article was worked examples; programming instruction; schema acquisition; goal specificity; students' learning; load approach; high school; strategies; memory, and generation. The article's most frequently cited top ten authors were Paas, F. (30 times); Van Merriënboer (29 times); Sweller (25 times); Kirschner (15 times); Renkl (13 times); Chandler (12 times); Kester (12 times); Atkinson (9 times); Van Gerven (9 times), and Kalyuga (8 times); the remaining were cited 115 times. Also, the article was self cited 84 times (30.32%) by its authors. The article's subject category was educational psychology.
The second most cited article was Application of cognitive, skill based, and affective theories of learning outcomes to new methods of training evaluation by Kraiger, Ford and Salas (1993). This was cited 164 times by the authors and also ranked fourth in the mean number of annual citations (M = 9.65). The focus of the article was self efficacy; individual differences; performance; knowledge; students; motivation; management; attitudes; task, and participation. The article's most cited authors were Salas (15 times); Bowers, Brown and Burke (cited 6 times); Ford (cited 5 times) and Cannon-Bowers, Gully and Kraiger (cited 4 times). The article was self cited 20 times (12.2%) by its authors, and the subject category was applied psychology.
In third was the article Why some material is difficult to learn by Sweller and Chandler (1994), which was cited 145 times by authors. This article had a low mean number of annual citations (M = 9.06), which were ranked eighth in 35 highly cited documents. The focuses of the article were cognitive load theory; information; instruction; consequences; strategies; attention; pictures; words, and text. The article's most cited authors were Sweller (26 times), Chandler (15 times), Van Merriënboer (8 times), Van Der Meij (7 times) and Kalyuga (6 times). This article was self cited 39 times (26.9%) by its authors, and its subject category in the Web of Science was educational psychology and experimental psychology.
Ranked fourth was the Expertise reversal effect (2003) by Kalyuga, Ayres, Chandler and Sweller, which was cited 119 times by authors; the article also ranked second in the mean number of annual citations (M = 17). The focuses of the article were cognitive load approach; worked examples; split attention; individual differences; working memory; architecture; acquisition; statistics, and science, and the article's most cited authors were Sweller (26 times), Chandler (15 times), Van Merriënboer (8 times), Van Der Meij (7 times) and Kalyuga (6 times). The article was self cited 14 times (11.76%) by its authors, and the subject category was educational psychology.
Ranked fifth was Cognitive load theory and instructional design: Recent developments by Paas, Renkl and Sweller, which was cited 95 times by the authors. The article, an editorial document, also ranked third in the mean number of annual citations (M = 13.57), and its focus was cognitive load approach and instructional design. The most cited authors were Paas (17 times), Sweller (11 times), Van Merriënboer (6 times) and Van Gog (5 times). The document was self cited 23 times (24.21%) by its authors, and the subject category was educational psychology.
In terms of the type of research design, the papers were classified into three major types, in accordance with the classification used by Shih, Feng, and Tsai (2008). These three major research types, as described in the Handbook of research for educational communication and technology (Jonassen, 1996), are experimental research, which typically involves an experimental group and a control group to test hypotheses regarding certain treatments (Ross & Morrison, 1996); descriptive research, which gathers data from events or participants' responses in describing, explaining, validating, or exploring a particular issue (Kunpfer & McLellan, 1996) and developmental research, which systematically studies the design, development, and evaluation process of certain educational interventions (Richey & Nelson, 1996); the development of several instructional design systems is also included in this type. The majority of h index studies were classified as developmental research (24 of the 32 documents), experimental research was ranked second (n = 6), while the descriptive research method was ranked third (n = 2).
A total of 574 (75.73%) documents were articles, and English was the most frequently used language (98.02%) in journals relating to instructional design. This result is similar to Nederhof's (2006) study, which states that the Web of Science index holds documents predominantly in English (93-95%), with the remainder being shared between German (2-3%), French (1%), and other languages. However, in the past few years, educational journals (e.g., Hacettepe University Journal of Education, Teoria De La Educacion, Education Science in Theory and Practice) in other foreign languages (Turkish, German, Spanish, etc.) have been included in the SSCI, SCIE, and AHCI indexes, which has resulted in an increase in the articles on instructional design in different languages. For example, 25 journals have been listed in Thomson Reuters' Master Journal List, which start with the words 'Turk' or 'Turkish' (Thomson Reuters, 2008): this indicates an increase in the number of Turkish scientific documents in the Web of Science (Uzunboylu & Ozcinar, 2009), and this increase will naturally be within the instructional design field that enables interdisciplinary applications.
A total of 1586 authors were named in the 758 documents in the journals relating to instructional design and were indexed by the SSCI, SCIE, and AHCI databases. The top five authors were Jeroen Van Merriënboer (Open University of the Netherlands), Fred Paas (Open University of the Netherlands), John Sweller (University of New South Wales), Paul Kirschner (Open University of the Netherlands), and Paul Chandler (University of New South Wales). These authors appear to be the leading researchers in the instructional design field; however, the ranking of these authors is affected by their frequent collaboration and research endeavours, as the majority of the journal documents relating to instructional design were co-authored by two or more authors. Similar results were found by Kirby, Hoadley, and Carr-Chellman (2005) who observed that almost 70% of the instructional system design and learning science documents were co-authored. Latchem (2006) found that 56% of BJET documents were co-authored. Based on these results, it can be said that there is a consistent trend for articles on instructional design to be based on the collaborative work of groups of researchers.
Almost 85% of the authors of the documents were from 7 countries; and the remaining authors (15%) of the documents were from 40 countries. In more than half of the 758 documents reviewed, almost 54% of the authors originated in the United States of America. The second ranked country of origin of authors was the Netherlands, and third ranked, Australia. The other four high ranking countries of origin of authors were Germany, Canada, England, and Spain, in that order. Documents published were written by authors affiliated with 971 institutions. Regarding the top 12 universities, 8 were from the USA, 3 were from Europe, and 1 was from Australia, which represented about 20% of the 971 institution namings. The top three universities, all outside the United States, were the Open University Netherlands, The Open University of the United Kingdom, and The University of New South Wales (Australia), respectively. This result is very interesting: the two open universities are the world's leading institutions for sourcing instructional design documents. However, the majority of journal documents relating to instructional design had authors' affiliations in the USA.
Anglin and Towers (1992) noted that authors could be cited for affecting a particular area of inquiry, theory, perspective, or research design; for example, the most frequently cited articles in this review have addressed cognitive load theory (see Appendix). Sweller and Chandler (1994), authors of the third most frequently cited article in this review, cited four experiments in their study, in order to support the assumptions underlying cognitive load theory. The following terms were increasingly used during 2002-2008: instructional design; cognitive load theory; worked examples; students; information; skills; performance; multimedia; problem based learning; self esteem; strategies, and distance education.
There were a total of 5660 citations in the documents relating to instructional design during the years 1980-2008 by the authors. The lowest citation occurred in 1981 (n = 1) and the top citation occurred in 2008 (n = 1092). More than half (n = 2584) of citations occurred between the years 2004-2008, and, for these years, an increase in instructional design citations has been identified. This increase resulted from the increasing availability of databases such as ERIC, ScienceDirect, EBSCHOhost and Web of Science, and the fact that it has become easier for researchers to access them. It is expected that document numbers and citation numbers relating to instructional design will increase in the coming years.
Based on the h index, the 35 most frequently cited articles and their authors, published in journals during the years 1980-2008, were related to instructional design. The most frequently cited article in this review addressed Cognitive load theory and instructional design (Sweller, Van Merriënboer & Paas, 1998), and the vast majority of articles relating to instructional design were focused on cognitive load theory, work example, performance, instruction, student, and strategies. The majority of h index studies were classified as developmental research; second ranked was experimental research, and third was the descriptive research method. This is a very interesting result: Ross and Morrison (1996) claimed that experimental research was the traditional research method in the field of studying learning and psychology related issues. Also, if we look at the topics of the h index documents, about two-thirds are based on the psychology of learning, and thus it would be expected that most would be based on experimental research: this result is also very interesting and should be investigated by instructional design researchers. There was only one descriptive research method used in the h index documents; this also suggests that researchers who intend to study learners' cognitive psychology characteristics are likely to utilise an experimental research design, in order to gain better research outcomes. As descriptive research combines qualitative and quantitative research design features, it is more suited to describing the complicated relationships between the different variables involved in specific situations (Shih, Feng & Tsai, 2008). Therefore, most instructional design studies use this approach to investigate the different sub-domains of the field, and the research design types of these 35 articles are a valuable reference for researchers interested in studies relating to the area of instructional design.
Figure 4: Citation trend line for instructional design (1980-2008)
Although the nature of this study is descriptive, it is hoped that the results can provide educators and researchers in the field of instructional design with some insightful ideas about publishing trends for research studies in instructional design related journals. This type of citation analysis is also very beneficial for young scholars in the field (Tsai & Wen, 2005). The analysis can help them to identify contemporary research topics, methods, and trends, and to understand the influence and influencers in their major subjects (Gall et al. 2004). Understanding trends in recent research studies can help policymakers to make future plans, in terms of these issues. The documents identified in the current study have contributed to important advances, not only in the field of instructional design or instructional technology, but also in other important fields. With the continued scholarly submissions to instructional design journals, new citation analyses will emerge and continue to impact this expanding field and other fields.
While the present study offers a descriptive approach to identifying instructional design documents, some limitations should be noted. For example, citation analyses do not provide information regarding how or why a specific work was cited (Everett & Pecotich, 1993; Hoffman & Holbrook, 1993). Also, the instructional design impact of a given article cannot be measured in this manner: although the instructional designer could be using the information from a study in practice, this would not necessarily result in a citation of the article. Citation analyses can be limited by their 'snapshot' approach in examining the citation impact of a given article and, given a more suitable lag time since publication, it is likely that more recent articles would appear as hi-index. Despite these limitations, citation analyses provide a direct, objective, and reliable means of defining the documents in a field (Baltussen & Kindler, 2004; Terajima & Aneman, 2003). Although only part of the bigger picture, the current findings highlight some of the influential works in the field of instructional design, as it is reflected in the field's flagship publications. In addition to this, the current study highlights potential areas of future research. For example, future studies could examine several issues, such as the availability of research grant funding, advances in instructional design models, or the use of more sophisticated statistical techniques (which affect the topics published within related journals), and, in turn, how these aspects influence citation rates.
Although citation analysis is considered as one of the objective measurements of document evaluation, there were still some disadvantages with this method, such as negative citation and gratuitous citation (Shih, Feng & Tsai, 2008; Chu, Hsu & Yu, 1997). Therefore, it is suggested that a combination of citation analysis and other analytical techniques should be used in future research studies, to obtain better results.
This study analysed documents relating to instructional design, published and indexed in SSCI, SCIE and AHCI during a specific period. As instructional design is a multidisciplinary, global field, it would be interesting to see investigations concerning other publications or databases (ERIC, Scopus, Science Direct, etc.) in the future. To understand the continuous trends and patterns in this topic, it is also recommended that similar, journal based studies should be conducted, and should be repeated at least every five years. For example, future studies could focus on citation analysis in the different sub-domains of educational technology, such as e-learning, computer assisted learning, multimedia, distance education, and mobile learning.
The widespread support for the h index is a recent development, and has increased the value of publications in peer reviewed journals. For example, Scopus, a large online database of peer reviewed literature that includes about 16,500 peer reviewed academic journals has incorporated the h index, and Thomson Scientific, formerly known as The Thomson Institute for Scientific Information, now includes the h index as part of its citation report in the Web of Science.
Of particular interest in this review was the absence of the use of instructional design in Asia, a region that is experiencing a convergence of engineering, science, and technology: the number of graduates in engineering has exponentially increased in countries like China, India, Japan, and South Korea. According to Jischke (2006), more than 90% of the world's engineers and scientists will reside in Asia by 2010. Yet, less than 5% of the frequently cited documents in instructional design were authored by academicians from this region.
Aylward, B. S., Roberts, M. C., Colombo, J. & Steele, R. G. (2008). Identifying the classics: An examination of articles published in the Journal of Pediatric Psychology from 1976-2006. Journal of Paediatric Psychology, 33(6), 576-589.
Baltussen, A. & Kindler, C. H. (2004). Citation classics in critical care medicine. Intensive Care Medicine, 30, 902-910.
Batista, P. D., Campiteli, M. G., Kinouchi, O. & Martinez, A. S. (2006). Is it possible to compare researchers with different scientific interests? Scientometrics, 68(1), 179-189.
Brown, L. D. & Gardner, J. C. (1985). Using citation analysis to assess the impact of journals and articles on contemporary accounting research (CAR). Journal of Accounting Research, 23(1), 84-109.
Chu, C. Y. Cyrus, Hsu, S. K. & Yu, R. R. (1997). Proceedings of the National Science Council (Part C: Humanities and Social Sciences), 7(3), 445-473.
Dick, W. O., Carey, L. & Carey, J. O. (2004). The systematic design of instruction (6th ed.). Boston: Allyn & Bacon.
Dick, W. & Dick, W. D. (1989). Analytical and empirical comparisons of the "Journal of Instructional Development" and "Educational Communication and Technology Journal". Educational Technology Research and Development, 37(1), 81-87.
Egghe, L. (2006). Theory and practice of the g-index. Scientometrics, 69(1), 131-152.
Ely, D. P. (2002). Trends in educational technology (5th Edition). Washington, DC: Association for Educational Communications and Technology.
Everett, J. E., & Pecotich, A. (1993). Citation analysis mapping of journals in applied and clinical psychology. Journal of Applied Social Psychology, 23, 750-766.
Eybe, J. & Schmidt, H. J. (2001). Quality criteria and exemplary papers in chemistry education research. International Journal of Science Education, 23, 209-225.
Gagne, R. M., Wagner, W. W., Golas, K. & Keller, J. (2004). Principles of instructional design. Pacific Grove, CA: Wadsworth Publishing.
Gall, J. E., Ku, H. Y., Gurney, K., Tseng, H. W. & Yeh, H. T. (2004). An analysis of citation patterns in ETR&D. In Proceedings of Association for Educational Communications & Technology (AECT) conference, Chicago, IL, October 19-23, 2004.
Garfield, E. (1955). Citation indexes for science: New dimension in documentation through association of ideas. Science, 122(3159), 108-111.
Gill, M. (2006). The 'h' index: A measure of quality of scientific research. Chowk, 3 January. [viewed 5 Jul 2007 from http://preview.tinyurl.com/yurrhk, verified 6 Sep 2009 at http://www.chowk.com/articles/10107]
Harzing, A. (2007). Publish or perish. [viewed 11 Jul 2007, verified 6 Sep 2009] http://www.harzing.com/pop.htm
Hew, K. F., Kale, U. & Kim, N. (2007). Past research in instructional technology: Results of a content analysis of empirical studies published in three prominent instructional technology journals from the year 2000 through 2004. Journal of Educational Computing Research, 36(3), 269-300.
Hirsch, J. E. (2005). An index to quantify an individual's scientific research output. Proceedings of the National Academy of Sciences, 102(46), 16569-16572. [verified 6 Sep 2009] http://www.pubmedcentral.nih.gov/articlerender.fcgi?tool=pubmed&pubmedid=16275915
Hoffman, D. L. & Holbrook, M. B. (1993). The intellectual structure of consumer research: A bibliometric study of author cocitations in the first 15 years of the Journal of Consumer Research. Journal of Consumer Research, 19, 505-517.
Howard, G. S., Cole, D. A. & Maxwell, S. E. (1987). Research productivity in psychology, based on publication in the journals of the American Psychological Association. American Psychologist, 42, 975-986.
Jin, B. (2007). The AR-index: Complementing the h-index. ISSI Newsletter, 3(1), 6.
Jonassen, D. H. (Ed.) (1996). Handbook of research for educational communications and technology. (2nd ed). Mahwah, NJ: Lawrence Erlbaum Association.
Kirby, J. A., Hoadley, C. M. & Carr-Chellman, A. A. (2005). Instructional systems design and the learning sciences: A citation analysis. Educational Technology Research and Development, 53(1), 37-48.
Klein, J. D. (1997). ETR&D - Development: An analysis of content and survey of future direction. Educational Technology Research and Development, 45(3), 57-62.
Kunpfer, N. & McLellan, H. (1996). Descriptive research methodologies. In D. H. Jonassen (Ed.), Handbook of research for educational communications and technology (2nd ed.) (pp. 1196-1212). Mahwah, NJ: Lawrence Erlbaum.
Latchem, C. (2006). A content analysis of the British Journal of Educational Technology. British Journal of Educational Technology, 37(4), 503-511.
Lee,Y., Driscoll, M. P., & Nelson, D. W. (2004). The past, present, and future of research in distance education: Results of a content analysis. The American Journal of Distance Education, 18(4), 225-241.
Maushak, N. J., Price, R. & Wang, H. C. (2000). Identification and analysis of the leading journals in the field of educational technology: A guide for reading and writing about educational technology. Proceedings of the Association for Educational Communications and Technology Conference, Denver, CO, 25-28 October, 2000. [verified 6 Sep 2009] http://www.eric.ed.gov:80/ERICWebPortal/contentdelivery/servlet/ERICServlet?accno=ED455780
Nederhof, A. (2006). Bibliometric monitoring of research performance in the social sciences and the humanities: A review. Scientometrics, 66(1), 81-100.
Reeves, T. C. (1993). Pseudoscience in computer-based instruction: The case of learner control research. Journal of Computer-Based Instruction, 20(2), 39-46.
Reeves, T. C. (1995). Questioning the questions of instructional technology research. In Proceedings of the 1995 Annual National Convention of the Association for Educational Communications and Technology (AECT), Anaheim, CA. [verified 6 Sep 2009] http://it.coe.uga.edu/~treeves/edit6900/deanlecture.pdf
Reiser, R. A. & Dempsey, J. V. (2006). Trends and issues in instructional design and technology (2nd Edition), Prentice Hall.
Richey, R. & Nelson, W. (1996). Development research. In D. H. Jonassen (Ed.), Handbook of research for educational communications and technology (2nd ed.) (pp. 1213-1245). Mahwah, NJ: Lawrence Erlbaum Association.
Ross, S. & Morrison, G. (1996). Experimental research methods. In D. H. Jonassen (Ed.), Handbook of research for educational communications and technology (pp. 1170-1213). Mahwah, NJ: Lawrence Erlbaum Association.
Rourke, L. & Szabo, M. (2002). A content analysis of the Journal of Distance Education 1986-2001. Journal of Distance Education, 17(2), 63-74.
Scopus (2008). Scopus Journal Analyser. [viewed 13 July 2008] http://info.scopus.com/journalanalyzer/
Shih, M., Feng, J. & Tsai, C. C. (2008). Research and trends in the field of e-learning from 2001 to 2005: A content analysis of cognitive studies in selected journals. Computers & Education, 51(2), 955-967.
Sidiropoulos, A., Katsaros, D. & Manolopoulos, Y. (2006). Generalised h-index for disclosing latent facts in citation networks. Scientometrics, 72(2), 253-280.
Smith, M. C., Locke, S. G., Boisse, S. J., Gallagher, P. A., Krengel, L. E., Kuczek, J. E., et al. (1998). Productivity of educational psychologists in educational psychology journals, 1991-1996. Contemporary Educational Psychology, 23, 173-181.
Tennyson, R. D. (1994). The big wrench vs. integrated approaches: The great media debate. Educational Technology Research and Development, 42(3), 15-28.
Terajima, K. & Aneman, A. (2003). Citation classics in anaesthesia and pain journals: A literature review in the era of the Internet. Acta Anaesthesiologica Scandinavica, 47, 655-663.
Thomson Reuters (2008). Thomson Reuters Master Journal List. [viewed 29 June 2009] http://scientific.thomsonreuters.com/cgi-bin/jrnlst/jloptions.cgi?PC=master
Tsai, C .C. & Wen, L. M. C. (2005). Research and trends in science education from 1998 to 2002: A content analysis of publication in selected journals. International Journal of Science Education, 27, 3-14.
Ullmer, E. J. (1994). Media and learning: Are there two kinds of truth? Educational Technology Research and Development, 42(1), 21-32.
Uzunboylu, H. & Ozcinar, Z. (2009). Research and trends in computer-assisted language learning during 1990-2008: Results of a citation analysis. Eurasian Journal of Educational Research, 34, 133-150. [verified 6 Sep 2009, abstract only] http://www.ejer.com.tr/index.php?git=22&kategori=66&makale=616
|1||Cognitive architecture and instructional design||Sweller, J., Van Merriënboer, J. & Paas, F.||1998||Educational Psychology Review, 10(3), 251-296.||Cognitive load theory||Develop-mental||277||53||23.08|
|2||Application of cognitive, skill-based, and affective theories of learning outcomes to new methods of training evaluation||Kraiger, K., Ford, J. & Salas, E.||1993||Journal of Applied Psychology, 78(2), 311-328.||Learning outcomes||Develop-mental||164||20||9.65|
|3||Why some material is difficult to learn||Sweller, J. & Chandler, P.||1994||Cognition and Instruction, 12(3), 185-233.||Cognitive load theory||Experi-mental||145||39||9.06|
|4||The expertise reversal effect||Kalyuga, S., Ayres, P., Chandler, P., et al.||2003||Educational Psychologist, 38(1), 23-31.||Cognitive load theory||Develop-mental||119||14||17|
|5||Cognitive load theory and instructional design: Recent developments||Paas, F., Renkl, A. & Sweller, J.||2003||Educational Psychologist, 38(1), 1-4.||Cognitive load theory||Develop-mental||95||23||13.57|
|6||The split attention effect as a factor in the design of instruction||Chandler, P. & Sweller, J.||1992||British Journal of Educational Psychology, 62, 233-246.||Cognitive load theory||Experi-mental||84||27||4.67|
|7||The regulation of constructive learning processes||Vermunt, J.||1998||British Journal of Educational Psychology, 68, 149-171.||Constr-uctivist learning||Descri-ptive||79||15||6.58|
|8||Enhancing teaching through constructive alignment||Biggs, J.||1996||Higher Education, 32(3), 347-364.||Constr-uctivist learning||Develop-mental||73||23||5.21|
|9||When two sensory modes are better than one||Tindallford, S., Chandler, P. & Sweller, J.||1997||Journal of Experi-mental Psychology - Applied, 3(4), 257-287.||Cognitive load theory||Experi-mental||71||24||5.46|
|10||Instructional-design for situated learning||Young, M.||1993||Educational Technology Research and Development, 41(1), 43-58.||Situated learning||Develop-mental||71||7||4.18|
|11||An empirically based instructional design theory for teaching concepts||Tennyson, R.D. & Cocchiarella, M.J.||1986||Review of Educational Research, 56(1), 40-71.||Teaching||Develop-mental||71||0||2.96|
|12||The teaching of concepts - a review of instructional design research literature||Tennyson, R.D. & Park, O.C.||1980||Review of Educational Research, 50(1), 55-70.||Teaching||Develop-mental||70||0||2.33|
|13||First principles of instruction||Merrill, M.D.||2002||Educational Technology Research and Development, 50(3), 43-59.||Learning||Develop-mental||69||0||8.62|
|14||Levels of expertise and instructional design||Kalyuga, S., Chandler, P. & Sweller, J.||1998||Human Factors, 40(1), 1-17.||Cognitive load theory||Experi-mental||68||14||5.67|
|15||When less is more: Meaningful learning from visual and verbal summaries of science textbook lessons||Mayer, R., Bove, W. & Bryman, A.||1996||Journal of Educational Psychology, 88(1), 64-73.||Meaning-ful learning||Experi-mental||67||9||4.79|
|16||The structure of design problem spaces||Goel, V. & Pirolli, P.||1992||Cognitive Science, 16(3), 395-429.||Structure of design||Develop-mental||66||3||3.67|
|17||The promise of multimedia learning: Using the same instructional design methods across different media||Mayer, R.E.||2003||Learning and Instruction, 13(2), 125-139.||Multimedia learning||Develop-mental||65||0||9.29|
|18||Toward a design theory of problem solving||Jonassen, D.H.||2000||Educational Technology Research and Development, 48(4), 63-85.||Problem solving||Develop-mental||65||5||8.12|
|19||Taking the load off a learner's mind: Instructional design for complex learning||Van Merriënboer, J., Kirschner, P. & Kester, L.||2003||Educational Psychologist, 38(1), 5-13.||Cognitive load theory||Develop-mental||63||23||9|
|20||Learning from examples: Instructional principles from the worked examples research||Atkinson, R., Derry, S., Renkl, A. & Wortham, D.||2000||Review of Educational Research, 70(2), 181-214.||Instruction-al principles||Develop-mental||63||20||7.88|
|21||Instructional design models for well-structured and ill-structured problem-solving learning outcomes||Jonassen, D.H.||1997||Educational Tech-nology Research and Development, 45(1), 65-94.||Instruction-al design models||Develop-mental||62||6||4.77|
|22||Behavioral fluency: Evolution of a new paradigm||Binder, C.||1996||Behavior Analyst, 19(2), 163-197.||Behavior-al fluency||Develop-mental||59||2||4.54|
|23||Cognitive load theory and complex learning: Recent developments and future directions||Van Merriënboer, J. & Sweller, J.||2005||Educational Psychology Review, 17(2), 147-177.||Cognitive load learning/ complex learning||Develop-mental||56||15||11.20|
|24||Aids to computer-based multimedia learning||Mayer, R. & Moreno, R.||2002||Learning and Instruction, 12(1), 107-119.||Multimedia learning||Develop-mental||56||3||7|
|25||Empirically-based guidelines for the design of interactive multimedia||Park, I. & Hannafin, M.||1993||Educational Technology Research and Development, 41(3), 63-85.||Multimedia learning||Develop-mental||56||2||3.29|
|26||The foundations and assumptions of technology-enhanced student-centered learning environments||Hannafin, M. & Land, S.||1997||Instructional Science, 25(3), 167-202.||Student centred learning||Develop-mental||54||6||4.15|
|27||Why minimal guidance during instruction does not work: An analysis of the failure of constructivist, discovery, problem-based, experiential, and inquiry-based teaching||Kirschner, P.A., Sweller, J. & Clark, R.E.||2006||Educational Psychologist, 41(2), 75-86.||Teaching||Develop-mental||52||9||13|
|28||Seriously considering play: Designing interactive learning environments based on the blending of microworlds, simulations, and games source||Rieber, L.P.||1996||Educational Technology Research and Development, 44(2), 43-58.||Interactive learning environ-ments||Develop-mental||50||6||3.57|
|29||Assessing professional competence: From methods to programmes||Van der Vleuten, C.P.M. & Schuwirth, L.W.T.||2005||Medical Education, 39(3), 309-317.||Evaluation||Develop-mental||46||5||9.2|
|30||Use of the critical decision method to elicit expert knowled-ge: A case study in the methodology of cognitive task analysis||Hoffman, R., Crandall, B. & Shadbolt, N.||1998||Human Factors, 40(2), 254-276.||Cognitive task analyses||Develop-mental||45||3||3.75|
|31||Cognitive load while learning to use a computer program||Chandler, P. & Sweller, J.||1996||Applied Cognitive Psychology, 10(2), 151-170.||Cognitive load theory||Experi-mental||44||16||3.14|
|32||When problem solving is superior to studying worked examples||Kalyuga, S., Chandler, P., Tuovinen, J., et al.||2001||Journal of Educational Psychology, 93(3), 579-588.||Problem solving||Experi-mental||42||8||4.67|
|33||An international virtual medical school (IVIMEDS): The future for medical education?||Harden, R. & Hart, I.||2002||Medical Teacher, 24(3), 261-267.||Educat-ional program||Develop-mental||41||9||5.12|
|34||Multimedia learning environments: Issues of learner control and navigation||Lawless, K.A. & Brown, S.W.||1997||Instructional Science, 25(2), 117-131.||Multi-media learning||Develop-mental||41||4||3.15|
|35||Multimedia learning environments: Issues of learner control and navigation||Lawless, K.A. & Brown, S.W.||1997||Instructional Science, 25(2), 117-131.||Multi-media||40||2||5|
|Author: Dr Zehra Ozcinar, Assistant Professor, Ataturk Teacher Training Academy (http://www.aoa.edu.tr/) and Faculty of Ataturk Education, Near East University, North Cyprus (http://www.neu.edu.tr/?q=en/node/665). Email: email@example.com
Please cite as: Ozcinar, Z. (2009). The topic of instructional design in research journals: A citation analysis for the years 1980-2008. Australasian Journal of Educational Technology, 25(4), 559-580. http://www.ascilite.org.au/ajet/ajet25/ozcinar.html