Soapbox: An applet to do an explanation over the web
Nathan Scott and Brian Stone
Department of Mechanical and Materials Engineering
The University of Western Australia
We have written two pieces of software to make it easier for teachers to put explanations on the web. The first piece of software is an authoring tool. In this environment a teacher can record an audio file and simultaneously scribble on one or more picture files. All sound and gestures are recorded in a script. The second piece of software is a Java applet which plays back the script, audio, pictures and scribbling to reconstruct the explanation. It is all very simple to use but the results are most engaging - and, importantly, very compact for quick transmission over a modem line.
In this paper we will describe the software and an experiment we performed in staff development using it.
Background - what makes a good explanation?
Imagine that you are driving somewhere new and you are lost. You stop and ask a stranger for directions. What happens next can be quite amusing. Most people, when asked to give directions, and if they actually know the way, will try very hard to explain. There is something immensely gratifying about having desired knowledge and an opportunity to show it off! The explainer will gesticulate wildly, tracing the shapes of roads and intersections in the air with his or her hands; there are unspoken conventions: a hand raised and pointed means "go a long way away", whereas a hand at chest level means "go nearby". The teaching process can be painful for the lost person because the explainer inevitably uses too much "jargon" in the form of local place names and landmarks which are of course unknown to the newcomer. After the first three new pieces of information the listener can only smile and nod, wishing for a proper map! The point here is that, to be effective, an explanation really needs the right "props". A very crudely drawn map, on the back of an envelope, with a few well chosen words, will be more effective as a teaching tool than ten minutes of unsupported verbal explanation.
Now imagine that you are lost, and you ask for directions, but the person you ask remains silent and only gives you a map. This is better than nothing but is still not ideal. You may not know how to find the important places as represented on the map (where you are and where you want to go) because they are embedded in a context you are not familiar with. To make matters worse, and to make this scenario more like University, the names on the map might well be in a foreign language!
The best explanation is thus a well chosen combination of visual cues (lines drawn on a map) and spoken cues. The sequence of cues is important: the spoken cues and visual cues must be synchronised. This idea by itself is nothing new and in educational literature it is called "dual coding theory". The new thing we wish to present is a group of software tools to make it easy to put such "synchronised" explanations up on the Web.
How to do it without our software
It is possible to put such an explanation on the Web without using our software. One approach (which we were using until recently) is to put the pictures and words into a QuickTime(TM) movie. Here's an approach developed by Scott with help from Dr Moira Maley of the Faculty of Medicine at UWA:
These steps can lead to fairly small file sizes. A one minute explanation can be squeezed into perhaps 1.4 MB. Using a standard modem line at 28.8 kbits/s this file will take about 5 minutes. Will a student wait this long to hear it? Perhaps, if it is considered to be important enough. We were unhappy with this approach because
- Use an ordinary video camera. Point the camera at a whiteboard, a piece of blank paper or perhaps a printout of a diagram.
- The person doing the explanation then draws on the surface (in front of the camera) while explaining. The camera records the drawing but not the face of the speaker.
- Capture a low resolution version of the video (and audio) using a video digitiser, ie. make a QuickTime movie of the explanation. We used an old Macintosh AV model. High resolution images are not important at this stage and the size need only be about 300 by 200 pixels. High frame rates are also not important; 15 frames per second is more than enough.
- The resulting movie would work reasonably well as an explanation except that the file size tends to be HUGE, eg. 20 MB for a one minute explanation. The remaining steps are all to do with making the file smaller so it will go down a modem line quickly.
- One way to make the file smaller is to sample the audio at low quality. We found that "mono, 8 kHz, 16 bit" is scratchy but reasonably clear.
- The audio is typically only a small fraction of a movie's size. To really save space we must simplify the video. One approach we developed was to replace the "moving" image with a series of carefully chosen "still" images. For example, if the speaker has drawn a series of lines, it may be enough to include just the frames of the video where finished lines appear. In other words a given image is shown and then persists on the screen, unchanged, until it is replaced by another still image. These frame changes must happen at the right moments of course! Video editing software such as Adobe Premiere(TM) is needed and it is quite expensive (about $800).
- Changing the image infrequently is only part of the solution. We found that we had to use a good "codec" to achieve any savings in file size. Many of the standard video codecs pay no attention to "frame differences". This means that each frame of the video is compressed as though it were only a static image, with no reference to the frames before it. But if an image appears unchanged on hundreds of consecutive frames, this is terribly inefficient. We found that the Video codec that is part of QuickTime itself worked quite well in this respect. The resulting file size is only slightly larger than the sum of the "static image sizes" used.
It requires expensive software and equipment;
We have invented what we believe to be a much better way to capture - and replay - an explanation. The design goals were
- It is very time consuming for the teacher because so much fiddling about with the video editing software is needed;
- The result tends to be a bit boring because the image only changes when absolutely necessary. All the subtle visual cues and gestures from the movement of the speaker's hand are lost.
The software we wrote is called "Soapbox" because it is traditional to stand on a wooden soap crate when giving a public oration.
- To make the process of capturing the explanation as quick and easy as possible, so that teachers will be inclined to do it frequently;
- To make the resulting performance self contained for easy transport;
- To make the file sizes small for quick access by modem; and
- To make the playback environment platform independent and web based.
The Soapbox authoring tool
The authoring tool is currently only available for Macintosh. However the performances can be played back on any modern computer, eg. Windows(TM). Here is a condensed excerpt from the user manual, describing how to use the authoring tool:
Assemble and convert your images
First assemble a collection of images that you want to talk about in your explanation. Put all the images (or copies of the images) in one folder in the finder. Make sure that the images are in GIF or JPEG form.
Double click the Soapbox application.
Draw on your pictures (pre-drawing)
Open one or more images using the File menu (see Figure 1). At this stage, if you wish, you can draw on the images before you start recording. The drawing that you do will be preserved and will appear in the final performance as the starting condition of the images. For example, if your performance was about the solution to a problem, you might choose to mask off most of the image at the start. Then, during your explanation, you could progressively reveal parts of the solution using the eraser tool or by selecting and clearing.
Figure 1: A selection is filled using the paint bucket tool.
In most paint packages that you will have used, the painting tools change the image itself. In Soapbox the paint tools never change the starting image: they only draw on top of it. It's as though you have a clear transparent sheet to draw on, and the sheet is on top of the image. This is illustrated in Figure 2:
Figure 2: The eraser tool erases the paint but not the underlying image.
Select Record from the File menu. Recording starts immediately and continues until you select Pause from the File menu. While recording, feel free to draw on the picture(s) and swap between them. Your cursor positions and all your drawing is recorded and is synchronised with the audio.
At any time you can Pause the recording. This means that audio recording is suspended and "time is stopped". If you start Recording again, a new audio track is created and it starts from the end of the previous one.
Hint for better recording: stop frequently!
A limitation of Java means that it can only play a sound from the start. So every 10 seconds or so, it is a good practice to pause recording and then start it again. These breaks in the audio become places where the student may start playing back - the more the better.
The stored performance
The performance is stored as a series of files in a single directory:
The script file is a time stamped record of the files used in the performance (the explanation) along with time stamped mouse movements, tool changes and so on. See Figure 3.
- The image files (which are not changed by the authoring tool);
- One or more audio files in Sun "au" (µ-law) format;
- A single script file.
0=swap banner.jpg 0 0
0=tool PEN FF0000 FFFFFF
183=m 158 74
2333=m 158 72
2350=m 157 72
2433=m 157 71
2483=m 156 71
2633=m 156 70
2650=m 155 70
2667=m 155 69
2683=m 154 67
2783=m 153 67
The script file is plain text format and can be edited using an ordinary text editor (although this is not recommended).
Figure 3: A fragment of a script file
The format is line delimited; each line has the form
[time stamp in milliseconds]=opcode arguments
The number and type of arguments depends on the opcode. For example an "au" opcode means "play an audio file now" and the name of the file must follow. A "m" opcode means "move the cursor" and an (x, y) pixel coordinate pair must follow. The author of a performance (the teacher) does not need to know anything about these opcodes: they are only mentioned so that you may see how the performance is stored. The main point here is that the performance script captures everything done during the performance (as far as the computer is concerned) - every mouse movement, click, picture change and sound - but it is stored in a very efficient form. These script files are typically only 50 kB or less for several minutes of explanation - a trivial size compared to the other files involved. The biggest files in a Soapbox performance tend to be the audio files.
Any number of script files may appear in a given directory and may make use of the same source pictures.
The Soapbox applet
Once a teacher has recorded an explanation, it must be put up on the Web. This is fairly straightforward. Let us suppose that at a certain point in a document called thing.html the explanation is desired. At that point in the HTML source, the following tag is included:
<APPLET CODE="CSoapbox.class" WIDTH=400 HEIGHT=320
<PARAM NAME=base VALUE=tutorial>
<PARAM NAME=script VALUE="tutorial.txt">
The relative path to the performance directory (the directory that contains the source images, sounds and script), and the name of the script file, are provided to the Applet as parameters.
When the file thing.html is viewed with Netscape or Internet Explorer(TM), the applet loads, and then in turn it loads the files mentioned in the script (tutorial/tutorial.txt in this example). It then appears as shown in Figure 4.
Figure 4:The Soapbox applet as it appears in Netscape(TM) 4.6 for Macintosh(TM). This applet is part way through playing back.
The applet interface is deliberately simple. A Start and a Stop button are provided, and there is a "progress bar" display between them. The user may click in the progress bar to "fast forward" or "rewind" the performance. It is important to note that the audio can only start playing from certain moments (a limitation of Java) and these are marked as green triangles in the progress bar. If the playback is started at other points in the timeline, the play time "jumps" back in time to the previous green triangle.
Early reports of usage
The test of any educational innovation is whether it is accepted by staff and students, and makes a positive contribution to learning. The software we have described exists and works but has not yet been tried in a genuine teaching situation. We have, however, an early report from a user of the software (Stone). The report is presented in the form of a dialogue:
What did you explain?
Stone: As a first example the construction of a velocity diagram was described. The existing WWW pages had a step by step construction using a QuickTime movie (Figure 5). The relevant stills from this are shown below (Figure 6). This has the advantage that the student can step back and forth. Normally students would only see the final diagram in their notes or text and they would not necessarily remember all the steps. It was considered that one of the essential inputs from a lecture that was missing was the voice and pointing to relevant parts of the diagram. The student has to read the text and then go looking. It was expected that the student would with the Soapbox version be able to watch the screen and follow the pointer/pen while listening to the voice.
Figure 5: A QuickTime movie with 7 frames (shown below in Figure 6). Each frame is a step in the construction. The movie is not meant to be "played" in the normal way: the student is expected to page through.
Figure 6: The frames of the movie of Figure 6.
Was the Soapbox authoring tool easy to use?
Stone: There was a tutorial that was informative. There was no difficulty loading diagrams, pointing/drawing on them and recording the voice (though advice on microphone is important).
Difficulties were encountered starting and pausing. If talking live then a male brain has to stop and think what to do next, ie. hit pause. This gives long silences when playing back. Also a live show is probably not best.
Was it easy to put your performance on the Web?
Stone: The major difficulty was in the step from the text file to playing back on the WWW. Need to revise the tutorial here. Explicit advice is needed, eg. "copy the local template and change the following words..."
How will you use this software?
Stone: We have an extensive set of WWW based lecture notes for Dynamics. This new software will allow us to add buttons or links along the lines of "Listen to the Prof explain this bit". These would then be short clips on topics that have historically been found difficult for some but not all students.
Figure 7: The Soapbox applet playing the explanation of the velocity diagram. Prof. Stone has used the existing sequence of steps but has added some drawn elements.
Tools for creating, storing and replaying an explanation have been written. A voice soundtrack with synchronised cursor motions, drawing and pictures are stored in a compact format. The authoring tool is aimed at the busy academic and is deliberately simple and fault resistant. The player environment is aimed at a typical student user at home with a modem and either a Macintosh or Windows computer. The tools are as yet untested in real teaching but there is some evidence that they are easy for staff to use.
Try the software online!
The Soapbox web site is
Information about our other products, eg. server software and courses in Dynamics, can be found at
|Authors: Dr Nathan Scott|
Prof. Brian Stone
Department of Mechanical and Materials Engineering
The University of Western Australia
Please cite as: Scott, N. and Stone, B. (2001). Soapbox: An applet to do an explanation over the web. In A. Herrmann and M. M. Kulski (Eds), Expanding Horizons in Teaching and Learning. Proceedings of the 10th Annual Teaching Learning Forum, 7-9 February 2001. Perth: Curtin University of Technology.
[ Abstract for this article ]
[ TL Forum 2001 Proceedings Contents ]
[ All Abstracts ]
[ TL Forums Index ]
HTML: Roger Atkinson, Teaching and Learning Centre, Murdoch University [firstname.lastname@example.org]
This URL: http://lsn.curtin.edu.au/tlf/tlf2001/scott.html
Last revision: 8 Feb 2002. © Curtin University of Technology
Previous URL 22 Dec 2000 to 8 Feb 2002 http://cleo.murdoch.edu.au/confs/tlf/tlf2001/scott.html