Australian Journal of Educational Technology
1986, 2(2), 103-109. 		AJET 2

Evaluating LOGO: Use of the SOLO Taxonomy

Wayne Hawkins
Division of Teacher Education
Darwin Institute of Technology

John G Hedberg
University of New South Wales

This article highlights the problem for researchers and classroom teachers who use LOGO and who wish to maintain a free exploratory classroom atmosphere, yet evaluate student responses in a disciplined and systematic manner. The Solo Taxonomy is put forward as a useful and effective way to overcome this problem. The various levels of the Solo Taxonomy are discussed and examples of LOGO procedures from students' work are provided to illustrate each of the levels.

The computer language LOGO was developed in 1967 by Papert and his colleagues working at the Artificial Intelligence Laboratory, Massachusetts Institute of Technology (MIT). The emphasis of the LOGO language is particularly on assisting children to learn the basics of computer programming, mathematical concepts and important skills for problem solving. In the twenty years since it was developed, it is still considered to be a relatively new innovation within the field of educational computing. Research studies concerned with LOGO are steadily increasing, although many focus on its use as part of the curriculum rather than the effects of its use on problem solving or the transfer of learning to other areas of the curriculum. Krasnor and Mitterer (1983) made the observation that "the entire LOGO literature merits considerable criticism from an experimental point of view", (p.11). They continued by suggesting that the lack of objective measurement makes the work currently available "useful only as a first step in evaluating the effects of LOGO" (p. 11).

Much of the literature concerning LOGO is of a descriptive nature explaining how LOGO works and providing explanations of how it can be used. In September, 1977, the Artificial Intelligence Laboratory at MIT produced a document discussing the development of LOGO to that time and also suggesting areas where further research and documentation would be helpful. It is admitted within the document that certainly up until the time of writing there were a number of weaknesses about the work done with LOGO. The main points raised were:

  1. Until that time there had been little effort to implement what was being done in the laboratory into actual school environments.

  2. There seemed to have been insufficient attention given to teacher training. If it were to be introduced into classrooms both in-service training and pre-service training of teachers was needed to prepare them for teaching with LOGO.

  3. The concept held by the researchers of "teaching materials" was not consistent with that of practising teachers.

  4. The organisation within the laboratory "had a looseness which was appropriate for pure research but not well adapted to an operational project such as running a school or preparing materials for general dissemination". (A.I.L. LOGO Memo #48, 1977, p.iv).

  5. The research team had been vague about evaluation of their work thus far. Much developmental work had been done with little effort to research into the effectiveness and measure the impact LOGO was having.
The document states in a later section:
Our first goal in the work we want to do next is to gather such data by having small classes taught systematically by a disciplined, observant teacher while observers collect as much data as possible. (1977, p.16)
However, there is very little evidence of this kind of research particularly within refereed journals. One reason for this is attributed to the concept of the LOGO "microworld" where the emphasis is on play and exploration and where the child is encouraged to experiment to find out what works. This makes it difficult for "experimental studies" to be carried out as free exploration and classes taught systematically by a disciplined, observant teacher are not easily achieved.

Figure 1. General description of Solo levels with matching skills acquired while using LOGO

Solo levelGeneral description
using the computer		Skill acquisition
and LOGO
1. Pre-structural 1. Students do not fully comprehend
the problem eg "Dunno" etc		 1. Does not attempt the problem.
2. Blankly stares at the screen.
3. Types in nonsense commands
2. Unistructural 1. Student is unable to consider alternatives.
2. Gives up after first attempt.
3. Can't give reasons for responses.		 1. Uses only display mode.
2. Uses only single commands.
3. Clears the screen as soon as a mistake is made and either starts again or leaves problems
3. Multistructural 1. Students work by trial and error.
2. Is able to see more than one
adequate strategy but they are not necessarily connected or inter-related.		 1. Uses display mode but types in more than one command.
2. Uses text mode and attempts to program.
3. Has no ability to edit. If wrong deletes entire program and starts again.
4. Relational 1. Student has ability to close off closures while decisions are made.
2. Integrates all data presented and
then focuses on appropriate responses.		 1. Writes programs in text mode.
2. If a mistake is made - edits
5. Extended
Abstract
Responses		 1. Student has the ability to consider more than one answer to any item.
2. Relationships, systems, variables
are handled easily.		 1. Writes programs in text mode and edits if necessary.
2. Ability to introduce variables.

This task has been made somewhat easier however since the development of the Solo Taxonomy (Biggs and Collis, 1982). Biggs and Collis maintain that there is an "overwhelming dominance" of educational evaluation which is quantitative. Even where open ended responses are required, points are awarded and qualitative assessment usually gives way to quantitative. Yet, they argue, the term evaluation contains the root word "value". Biggs and Collis therefore have developed a qualitative model of evaluation based on the belief that there are natural stages in the growth of learning information and skills. Attention should then be given to students' responses representing these various stages. They have called this interest in the structure of the responses - the structure of the Observed Learning Outcome or SOLO. They have identified five basic categories of student responses. These Solo levels describe a particular performance at a particular time and are not intended as labels for students. The emphasis of the Solo taxonomy is on qualitative assessment of the students' responses. The fact that quality is assessed retrospectively in an objective and systematic manner allows exposure to LOGO to still take place in the desired exploratory manner.

The five levels identified by Biggs and Collis are explained briefly in Figure 1. In the final column of that Figure are the various types of responses that can be expected from students working with LOGO. These LOGO responses or skills have been listed so as to match Biggs and Collis' Solo levels as closely as possible.

Biggs and Collis also refer to "transitional responses". These transitional responses occur when a student is nearly at the next level but doesn't quite make it. This is usually indicated by a certain amount of confusion or inconsistency. These transitional stages have not been included in Figure 1 as separate levels.

In a recent study (Hawkins, 1985) students' problem solving strategies were investigated while using LOGO. The Solo Taxonomy served as a useful tool for categorising the students' responses to problems in a qualitative way. To illustrate the Solo levels as applied to skill acquisition while using the computer and LOGO the following responses were obtained from students involved in the study mentioned above.

Pre-structural Responses

The following program is one written by a Year 6 boy, Kevin. while intending to draw a circle. He commenced with the turtle in HOME position and used the computer in immediate mode (that is, each command was executed as it was typed into the computer). 
RT 90     Turtle turns right 90 degrees
FD 2      Turtle moves forward 2 steps
RT 45
FD 4
RT 45
FD 4
RT 32
FD 4
RT 30
FD 4
RT 28
FD 4
RT 45
FD 4
RT 30
FD 4
RT 28
FD 4
RT 45
FD 4
RT 30
FD 4
RT 30
RT 45
FD 6
FD 6
RT 1
FD 1
RT 1
FD 1
PU        Pen is raised up
RT 1
FD 8
RT 120
FD 6
RT 45
FD 6
A quick glance over these commands will enable the reader to appreciate that in fact they are virtually "nonsense" commands. It is not clear why towards the end he included the PU command followed by another six commands of which he could not have seen the results.

Unistructural Responses

The following set of commands taken from Monica's file is representative of this level. The task was simply to draw two geometric shapes that were separated ie. the need to use PU (Pen up) command. She elected to draw two squares
Drawing of two squares
She consistently worked in immediate mode, although she had been introduced to writing programs. She felt more secure giving single commands and immediately seeing the computer respond.

Multistructural Responses

Sharon consistently typed in several commands in succession before pressing the return key to execute them. The following program illustrates this and is representative of then work of others working at this level: 
FD 50    RT 60    FD 50 RT 60
FD 50    RT 60    FD 50 RT 60
FD 50    RT 60    FD 50 RT 60
Each line above represents a set of commands entered before being executed, resulting in a hexagon being drawn in three stages.

Relational Responses

A Year 7 boy, Greg, provided a good example of operating at the Relational level. The following programs for a six point star illustrates this: 
TO TRIANGLE
REPEAT 3 (FD 10 RT 120)
END
This program was then followed by another: 
TO STAR
RT 30
REPEAT 6 (TRIANGLE RT 60 FD 10)
EN
This in itself was an interesting approach to this problem in that Greg did not need to use the PU command. Most other students visualised the Star as over lapping triangles and ran into difficulties using PU and relocating the turtle for the positioning of the second triangle. The important difference is of course Greg's ability to write programs and even when mistakes appeared his ability to detect where the error was and successfully edit.

Extended Abstract Responses

Perhaps this level is best illustrated by using the programs Greg used to make a star. The major differences here would be that if a star of variable size was wanted then a variable needs to be introduced into the program TRIANGLE. 
TO TRIANGLE : SIDE
REPEAT 3 (FD :SIDE RT 120)
END
Now by entering a numeric value after the triangle eg. TRIANGLE 15, a triangle with sides 15 units will be drawn. The Star can then be drawn in varying sizes by the program: 
TO STAR : X    where X provides the length of the side of the triangle
RT 30
REPEAT 6 (TRIANGLE :X RT 60 FD :X)
END

Conclusion

The use of Biggs and Collis' Solo Taxonomy has indicated that children do in fact work at various levels and even while two or more students may each successfully solve the same problems they do so at their own level of cognitive development. The Solo Taxonomy has demonstrated an effective system for describing and classifying student responses when using LOGO. Using this system it can be hoped that more rigorous study of the role of the LOGO language can be undertaken. It certainly provides for the multi- structured response and allows the creativity and exploration available with computers to be retained while evaluating problem solving skills of the students. Indeed, it could be the key to maintaining a freedom for exploratory learning while providing researchers with an effective system of collecting useful qualitative data about student response levels.

References

Artificial Intelligence Laboratory Memo#48 (1977). Assessment and Documentation of a Children's Computer Laboratory, September, Cambridge: Massachusetts Institute of Technology.

Biggs, J. B. and Collis, K. F. ( 1982). Evaluating the Quality of Learning - The SOLO Taxonomy. New York: Academic Press.

Hawkins, W. J. (1985). An Investigation into Students' Problem solving Strategies while using LOGO and Cognitive Factors to Predict Their Success. Unpublished MEd Thesis, W. A. Institute of Technology.

Krasnor, L. R. and Mitterer, J. O. (1983). LOGO and the Development of General Problem Solving Skills. Unpublished paper, Brock University.

Please cite as: Hawkins, W. and Hedberg, J. G. (1986). Evaluating LOGO: Use of the SOLO Taxonomy. Australian Journal of Educational Technology, 2(2), 103-109. http://www.ascilite.org.au/ajet/ajet2/hawkins.html

[ AJET 2 ] [ AJET home ]
HTML Editor: Roger Atkinson [rjatkinson@bigpond.com]
This URL: http://www.ascilite.org.au/ajet/ajet2/hawkins.html Last revision: 24 Sep 2002.
Previous URL 30 Nov 1996 to 24 Sep 2002: http://cleo.murdoch.edu.au/gen/aset/ajet/ajet2/su86p103.html