A spreadsheet based simulator for experiential
learning in production management

This paper presents a spreadsheet based simulation game for teaching and learning production management concepts of forecasting, material requirements planning, order review and release. In this game the student plays the role of a production planner managing two products, for which customer orders are placed in variable quantities throughout the week. The student builds forecasting and material requirement planning systems to help in the tasks of production and vendor order release. In parallel with this, we have run a small learning awareness program, to test and stimulate the skill of reflection. Initial student responses to the game have been favourable, but the proportion of time spent on reflection is low. Contemplated refinements are presented.

Introduction

While there are many top management games available for students and teachers of management, the number of games aiming to teach/learn specific management skills is small. There is also a dearth of games that enhance detailed modelling and decision making capability. The primary goal of the research presented in this paper is to develop a simulation game that meets this void, in the specific area of management of production planning and control, and to explore the learning implications of the game. The paper describes the game that we developed. This game is able to impart an understanding of the issues in order-release; and as a secondary benefit, the spreadsheeting skills of the students are enhanced. The paper also describes the learning experiences of the students, with an emphasis on reflection, and their evaluation of the simulation game.

Literature review

There is now a wide range in the complexity of simulation games: from board games to computerised simulations. Even the simple act of walking may serve as a simulation for instructional purposes (Wu, 1988)! While a simulation mimics reality and is often used to predict what would happen in a given scenario, the word "game" suggests playfulness and competition. Simulation games combine these two characteristics. Games that explore business strategies for the entire organisation are called top management games, and games that have their primary focus on a selected functional area of business are called functional games. These functional games are available in the areas of accounting/finance, marketing, production, and human resource management.

The model of experiential learning provides a theoretical underpinning of simulation games as a learning/teaching tool. Kolb's (1984) experiential learning model is shown in Figure 1. According to this model, concrete experience of a phenomenon in the real world triggers the learning cycle. This event is observed/experienced, and causes/encourages reflection in the student. The student forms/uses abstract concepts and models/hypotheses to make sense of reality. This leads to experimentation and hypothesis testing that provides concrete experience, which starts the cycle again. Simulation games provide the concrete experience needed in Kolb's model, and are a good platform for stimulating learning awareness in students, encouraging them to better understand their own learning processes (Scott, 2002).

Figure 1: Experiential learning (EL) cycle model

Raia (1966), in his often cited study, carried out an experimental comparison between a simple game, a complex game, and readings. Boseman and Schellenberger (1974) used modified Raia's instruments to compare students who did only cases against students who did cases and games. Attempts were made to equalise the workload. They did not find the interest of game players any higher than non-game players. Their attitudes towards cases, management, course, and the instructors were not significantly different. Similar results were obtained for perceived or actual learning.

Wolfe and Guth (1975) made an experimental comparison between case only and game only approaches to teaching business policy. In their study, the game only students achieved a higher level of examination scores than the case only students. Students in the game only section achieved a higher degree of principle and concept mastery, but the differences in fact mastery were not significant. However, their games only section had a lot of structure - periodic reporting, class discussions of events, review sessions, self appraisals, and diary of events. This structure and guidance essentially closed the loop of experiential learning (Figure 1) and may have contributed to the positive result as evidenced by a subsequent study (Wolfe, 1975), which did not have this guidance, and had a negative result.

Parsuraman (1981) classifies the existing evaluation of simulation games into three methodologies:

1. Experimental evaluation, where the students are split into different groups which are exposed to different educational techniques. The efficacy of the techniques is then judged by a common examination at the end. Most of this type of evaluation has been inconclusive, and Parsuraman criticised it on the grounds that the common examinations test cognitive learning of the students, while simulation games actually teach the "process" of decision making (and not cognitive learning).

2. Correlation between students' performances in the simulation games and their performances in other examinations and assignments in the course. Here, Parsuraman questions the simulation game performance as a measure of learning.

3. Surveys of student self reports. Generally, these surveys have shown simulation games on a positive light, but the ability of the students to judge the worth of games is questionable. Parsuraman concludes that the evaluations should actually test the appearance of reality in the games, and the worth of the types of decisions made in the games. He suggests that experienced practitioners, not college students, are better judges.

According to Ruohomäki, simulation games provide: 1) cognitive learning outcomes - information, principles, critical thinking, 2) attitude changes toward the subject matter, society, and oneself, 3) increased motivation and interest towards the subject, for doing research in that field, and 4) positive effects on groups - better communication, interactional skills, empathy for those in other roles. Simulation games provide active learning (versus passive learning in lectures). Thus they provide student centred learning, and emphasise learning by doing.

Finally, Lane (1995) in a review of simulation games has crystallised some of the cautions to be exercised in the educational use of games:

1. Learning objectives. It is easy to be sold by the gimmicks and fun in the games, but what will be learnt?
2. Supporting materials. There should be enough learning materials to support the game's objectives.
3. Other pedagogical tools. It cannot be expected that simulation games will serve all the needs of a course. At best they will supplement a well-designed course.
4. Bells and whistles. So much seductive technology is available that it is possible for the designers to be looking for appropriate topics for the technology, rather than finding suitable technology for the teaching objectives in hand. The use of animation, multimedia, and virtual reality may provide more fun than education.
5. Complexity. It is better to have a simple game serving a specific learning objective, than a complex game satisfying a number of objectives. Students may find it hard to capture the desired experience.
6. Briefing and debriefing. Evaluation of simulation games has shown time and again the importance of proper briefing and debriefing. Students cannot be left to decide on the rationale of the game, and to reflect on what happened.
7. Facilitators. The facilitators need to understand the simulations and learning objectives thoroughly. Games cannot teach by themselves.
8. Resources. Computer software is famously known for underestimating the resource requirements. Games designers need to beware.

Production management games

Beginning students of production planning and control (PPC) often struggle with the technical concepts in PPC such as bill of materials, order review and release, and action buckets, to name just a few. The students need to see how forecasting leads to master production schedule, and to material requirements planning, and finally to order release. An important objective for the students is to appreciate the variability and dynamics of the production environment, where for example even the forecast is not a static input to production planning. A simulation game is ideal for this appreciation. Such a simulation game would be a more specific and simpler game, compared to the larger production games mentioned above, which attempt to give a flavour of the entire production management function and even its strategic significance.

In this paper we present a spreadsheet based production planning simulator called MRP-SIM, designed to meet the above objective. The next section presents a description of the game. This is followed by the learning awareness program we ran in parallel and a student evaluation of the game. Finally, concluding remarks are presented.

The game

Objectives

Figure 2: Bill of materials

The scenario

Figure 3: Routing and processing time information

The production planning is done on a weekly basis. At the beginning of the week, inventories are checked and orders are released both within the company and to the vendors. Through the week the processing takes place. Customer orders arrive based on a stochastic process that simulates seasonality, trend, and randomness. The parameters of this process are of course unknown to the students. As customer orders arrive, the orders are filled from inventory on hand. Customer orders may be filled partially if there is insufficient on hand inventory for the whole order. Unfilled customer orders are placed on file and filled when the product is available. The weekly cycle of activities is:

• release production and vendor orders,
• decide on overtime for the processes,
• simulate for a week (during this time these events occur: production in the facility, order fulfilment from the vendor, order arrival from the customer, and order fulfilment to the customer), and
• monitor the situation.

Profits are accumulated for every item in the customer order that is filled. For every item in the customer order that is late (filled after the day the order arrives), a penalty is charged per day. There are also costs associated with holding inventory (both finished and work in process) and with overtime work.

Student task

User interface

Figure 4: The main screen

Once the decisions are entered, they press the Simulate! button to let the production for the week begin, and to let the time advance to the next week. The queues of the machines, the finishing of the work, the inventory position, the arrival of customer orders, and the filling of the customer orders are animated on the screen. Profits are accumulated for every item in customer orders that is filled. For every late item a penalty is charged per day. There are also costs associated with holding inventory and with overtime work. The details of the model may be viewed by pressing the View Model Details button. At the end of the week the current and cumulative financial performance is shown at the top right of the screen (Figure 4 shows that a net profit of $2062.25 was made at the end of week 21. The students then make decisions for the next week and repeat the cycle.

The assignment

The assignment consisted of three steps:

Step 1.	Familiarisation with the simulator. The students play the game in an ad hoc manner, guessing the decisions.
Step 2.	Playing the game on a re-order point basis. The students try on different levels of re-order points and fixed order quantities.
Step 3.	Playing the game with a DSS, built by the students. To do this, they use a forecasting model to forecast demand of the finished products. This is fed into the master production schedule (MPS), which it explodes into the MRP for the components. The students then create the CRP model from the MRP model. This completes the DSS, which suggests the order quantities for all the items, and the overtime to order for all the processes.

Formal assessment for this assignment consists of reflective essay (50% of the assignment marks) and the DSS (50%). The class was split into self selected groups of 3 students. The development of the DSS was a group activity, but the reflective essay was an individual task. The assessment criteria were reflection breadth (number of activities or items discussed) and reflection depth (thoroughness of discussion, depth being more important than breadth), and completeness and correctness of the DSS work.

Learning and the simulation

Figure 5: Single-loop versus double-loop learning

We decided to:

• track the main activities students used before and during each major step within the MRP-SIM assignment, using activity logs
• create time maps from the activity logs
• record the levels of reflection witnessed
• stimulate learning awareness by discussing, in class, the results with the students, before they finalised their reflective essays, which were part of the assessment scheme.

Creating time maps of the steps was simply done by graphing tasks versus (proportion of) time spent, over the engagement. One class graph for each significant MRP-SIM engagement was created, using different colours for each student group.

With the MRP-SIM assignment requiring students to set levels of production, observe the result/outcome, and revise their estimates, potential levels of reflection were clearly related to Single-loop and Double-loop Learning. Four levels were set: (0) trial and error, to no plan, (1) discussed/predicted what would happen before numbers were entered (Sll), (2) questioned model or process being used to predict the result/outcome (Dll), and (3) questioned the process being used in step (2).

Experiences in using the game in a teaching context

The assignment was distributed in class and the first lab session held, 14 days later. As "entry tickets", the activity logs for ad-hoc and re-order point running of the simulation were collected. The second lab session was held, 2 days later. The two lab sessions were essentially help sessions, offering individual help to the class members in creating their DSS. Students can often be unsure of the concepts emphasised in this assignment, or even the purpose of the DSS. Many students struggle to acquire the level of spreadsheeting skills needed for the assignment. The two lab sessions offered instructions in these matters.

The spreadsheet files and the DSS activity logs were then submitted for assessment. Discussion of the results and feedback was given in class, 21 days before their reflective essay on the assignment was submitted. The reflective essay was for three pages of thoughtful responses to reflective questions we had provided.

There were six students taking this paper in 2002; they were given three activity logs to fill in, corresponding to three ways, or steps, of playing the game: in an ad hoc way, with reorder points for each items, and with the help of the DSS created by them. Students filled in the time they spent on various activities. All possible activities were identified in the logs, including the four dealing with reflection, but the students were not told the learning levels of the logged items until the discussion of results and feedback session. This session was deliberately held before the final reflective essays were submitted.

Figure 6 shows the self-reported total time spent by students on the four levels of learning in carrying out the assignment. Similar graphs were also available for individual steps.

Figure 6: Self reported activity log for all steps

When presented with the learning levels of their activities and the time they spent on activities, most of the students were surprised at the very high time they spent on level 0 activities (trial and error or without a plan), as against single loop or double loop learning.

Student evaluations

Statement	Average agreement rating 1998	Average agreement rating 2002
Simulation gaming was instructive for learning production planning	2.0	1.2
I enjoyed playing the simulation game	1.9	1.6
I put a lot of effort in playing the simulation game	2.1	1.4
The simulation game represents fairly well the decision making faced by real production planners	2.1	1.8
I found the simulation game challenging	1.8	1.4
There was a strong sense of make believe in playing the game	2.2	2.0
I felt the game enhanced my understanding of planning and control	*	1.2
The game helped me improve my use of spreadsheet software	*	1.4
The game assignment should be retained for next year	1.8	1.0
* Not included in the 1998 evaluation

Generally, the students found the assignment quite challenging. But they felt that they learnt the MRP concepts pretty well. Since the students had some familiarity with spreadsheets, the spreadsheet format of the game helped in gaining student acceptance. Their evaluations bore testimony to this. Some student comments, collected in 2002 from their reflective essays, are given below.

Student A:
The next consideration was that the game opened my perspective to the complexity of working within a production environment. It was insightful to see all of the considerations that need to been (sic) thought of such as overtime, holding costs and late delivery costs. Also the fact that at the end of the game it was not full proof (sic, the student meant 'fool-proof') in its recommendations supported the difficult nature of production management.

Student B:
Therefore, using the DSS helped me play the game better and also my profits had increased as well (i.e. started making profits rather than loss). I believe the reason for improved results was due to taking more accurate and precise figures into account when planning the future productions. However, with DSS, decisions were still based on using my own judgment and the game was still played with a great deal of uncertainty due to reliance on forecasting figures. But to make it more realistic in the essence to make the user of the MRP Simulation believe it is the real world and to be able to imagine themselves in that environment, I think adding more financial data such as how much overtime and late delivery is really costing for each component (not just the total), and more costing information would enrich a persons mind.

Student C:
At the beginning I did not know where to start. However, with general knowledge of production planning that I had learned from the lectures, I continuously read and followed the instructions provided. At the same time, I familiarised myself with the simulation software by using MS Excel application to get a sense out of it, which was quite difficult for me. Then, I found there were some parts of MRP that were separated by sheets or tabs like; Bill of Materials, MRP / CRP, Routing, Order Release, and Forecasting, which in the beginning I did not see how I can relate these tasks together.

Suggested improvements included making a competition out of it, with the highest profit making students gaining rewards and high marks. Another student (accounting) suggested more detailed reporting of the financial outcome of the game.

Conclusion

This game enhances the understanding of PPC concepts as well as providing the students with an opportunity to build a decision support system that provide the game players with detailed modelling and decision making capability, and to learn the difference between ad hoc decisions and model based decisions. Learning spreadsheet modelling is an additional educational benefit from the game. Vaszonyi (1993) and Plane (1994) present forceful arguments for using spreadsheets in management science and operational research.

The low proportion of time spent on reflection - making sense of action, connecting theory, and planning for further action - supports the view that our students should benefit from greater understanding of reflection and its praxis before playing the game. There are four aspects we have identified that could be added to the learning awareness aspect in the future:

• investigating if time spent developing spreadsheeting skills correlates with reflection patterns witnessed;
• recording the order in which tasks were done, although this would complicate recoding and analysis;
• relating time maps and ordering of tasks to learning styles;
• requiring more specific reflection on the reflective aspects in the final essay;
• discussing with future classes taking the assignment, the outcomes from this year, thereby seeding interest and awareness of reflection pre-assignment.

The simulation model and the related assignment may be obtained by writing to the authors. The simulation game is included in a list of exemplar learning designs based on information and communication technologies compiled by the Australian Universities Teaching Committee. This can be accessed at the URL:
http://www.learningdesigns.uow.edu.au/exemplars/info/LD14/index.html

References

Biggs, W. D. (1987). Functional business games. Simulation and Games, 18(2), 242-267.

Boseman, F. G. and Schellenberger, R. E. (1974). Business gaming: An empirical appraisal. Simulation and Games, 5(4), 383-402.

Burgess, T. F. (1991). The use of computerized management and business simulation in the United Kingdom. Simulation and Gaming, 22(2), 174-195.

Churchill, G. (1970). Joblot: A Production Management Game. UK: Macmillan.

Faria, A. J. (1987). A survey of the use of business games in academia and business. Simulation and Games, 18(2), 207-224.

Faria, A. J. (1990). Business simulation games after thirty years: Current usage levels in the United States. In J. W. Gentry (Ed), Guide to Business Gaming and Experiential Learning, 36-47. London: Nichols/GP Publishing.

Gilgeous, V. and D'Cruz, M. (1996). A study of business and management games. Management Development Review, 9(1), 32-39.

Kibbee, J. M., Craft, C. J. and Nanus, B. (1961). Management Games: A New Technique for Executive Development. New York: Reinhold.

Kolb, D.A. (1984). Experiential Learning: Experience as the Source of Learning and Development. USA: Prentice-Hall.

Lane, D. C. (1995). On a resurgence of management simulations and games. Journal of the Operational Research Society, 46, 604-625.

McKenna, R. J. (1991). Business computerized simulation: The Australian experience. Simulation and Gaming, 22(1), 36-62.

Mize, J. H, Herring, B. E., Cook, C. L., Chun, M. S. and White, C. R. (1971). roduction System Simulator (PROSIM V): A User's Manual. USA: Prentice-Hall.

Parsuraman, A. (1981). Assessing the worth of business simulation games. Simulation and Games, 12(2), 189-200.

Plane, D. R. (1994). Spreadsheet power. OR/MS Today, 20, 32-38.

Raia, A. P. (1966). A study of the educational value of management games. Journal of Business, 39(3), 339-352.

Ruohomäki, V. (1995). Viewpoints on learning and education with simulation games. In J. O. Riis (Ed), Simulation Games and Learning in Production Management, 13-25. UK: Chapman and Hall.

Scott, J. L. (1990). OR methodology and the learning cycle. OMEGA International Journal of Management Science, 18(5), 551-553.

Scott, J. L. (2002). Stimulating awareness of actual learning processes, Journal of the Operational Research Society, 53(1), 2-10.

Vaszonyi, A. (1993). Where we ought to be going: The potential of spreadsheets. Interfaces, 23, 26-39.

Wolfe, J. and Guth, F. R. (1975). The case approach versus gaming in the teaching of business policy: An experimental evaluation. Journal of Business, 48(3), 349-364.

Wolfe, J. (1975). A comparative evaluation of experiential approach as a business policy learning environment. Academy of Management Journal, 18, 442-452

Wolfe, J. (1993). A history of business teaching games in English-speaking and post-socialist countries: The origination and diffusion of a management education and development technology. Simulation and Gaming, 24(4), 446-463.

Wu, N. L. (1989). Understanding production through human simulation: Experiencing JIC (just-in-case), JIT (just-in-time), and OPT (optimised-production-technology) production systems. International Journal of Operations and Production Management, 9(1), 27-34.

Yeo, G. K. and Nah, F. H. (1992). A participants' DSS for a management game with a DSS generator, Simulation and Gaming, 23(3), 341-353.

Appendix 1: Activity log sample

Individual ACTIVITY LOG for step 1: Playing the game in an ad-hoc manner
ACTIVITY:	Mins spent	Thinking associated (and outcomes)*
Read the assignment
Wrote down questions and thoughts
Played around inside the spreadsheet file
Put in some rough numbers into the spreadsheet, just to see what would happen
Developed expectations about the results of running this simulation
Compared simulation results with your predictions/expectations
Openly questioned processes you were using in forming your expectations
Sought further information about specific things
Discussed the assignment with my group
Worked out numbers before running the simulation
Changed my/our approach ..... times during the 32 weeks
Predicted what would happen before we simulated
Questioned the processes being used to 'Make/alter decisions'
Questioned the entire approach you were using
Any other activities (state here)

* Think of the page as a map of what you did, with enough detail for the reader to be able to understand all the activities you followed and your thinking behind it. Tell it truthfully, as there is no "right answer" here, just a conscientious completion of the picture of your activities.
Profit achieved in your best simulation run in an ad-hoc manner = $ ____

Read through the page now, speaking it out in your head as if you were telling someone what you did, and the thinking behind it.

Authors: Chuda Basnet and John L. Scott Department of Management Systems, University of Waikato Private Bag 3105, Hamilton 2020, New Zealand Telephone: +64 7 838 4562 Fax: + 64 7 838 4270 Email: chuda@waikato.ac.nz, jls@waikato.ac.nz http://www.mngt.waikato.ac.nz/school/staff/staffhome.asp?ident=732&user=JLS Please cite as: Basnet, C. and Scott, J. L. (2004). A spreadsheet based simulator for experiential learning in production management. Australasian Journal of Educational Technology, 20(3), 275-294. http://www.ascilite.org.au/ajet/ajet20/basnet.html