Computer-Assisted Commenting and Theories of Written Response

One answer to this chore is to abandon it altogether. It makes both pragmatic and pedagogical sense to substitute one-to-one conferences for marginalia that are often misunderstood, sometimes not even read, and that always raise the danger of treating the text as a closed, finished product rather than a candidate for rethinking and revision. This argument has been frequently made in the literature on writing instruction (see for instance Thomas A. Carnicelli, "The Writing Conference," Eight Approaches to Teaching Composition, ed. Timothy R. Donovan and Ben W. McClelland [Urbana: NCTE, 1980] 101-32), and I have made a version of it myself ("Subverting Linear Structures." Inkshed 5.1 (1984): 2). Yet written comments can have a role to play in response, not just because class size can make it impossible to schedule as many conferences as one might like, but also because written comments can be genuinely useful. In Responding to Student Writing (Urbana: NCTE, 1987), Sarah Freedman reports that the students she surveyed preferred written comments to any other type of response (86). This preference may partly result from years of evaluation-oriented conditioning. However, it may also suggest that written comments provide a valuable record that the student can turn back to in order to extract suggestions that she can apply to later assignments. The transience of the spoken word, when used as a complete substitute for rather than a support for written comments, seems unnerving to student and instructor alike. It seems, then, that some form of written comments are bound to remain at least one item in most pedagogical toolkits.

Given this fact, it makes sense to try to streamline this laborious task to try to avoid the end-of-term whiplash collar. The computer has frequently offered itself as a possible means of doing so. Its huge memory, speed of operation, and perhaps most of all, its monumental and inhuman patience‑-surely the single most important qualification of a composition instructor‑-seem to invite applications to the time-consuming task of marking students' papers.

So far, however, attempts at using the computer as a marking aid have had limited success. I wish to argue that this is not because the designers of marking programs have done a bad job of designing the programs themselves, but because they have been trying to do the wrong things. Specifically, they have failed to take account of the mass of literature on the art of responding to student writing, literature that sets a direction for response that runs exactly counter to that of the existing attempts to automate the process. This disjuncture does not mean, however, that computer-assisted marking is inherently doomed to failure. I will close this article with a discussion of ways in which the computer can support the act of responding to student writing without ignoring what modern composition theory tells is most helpful to student writers.

Here I wish to look specifically at marking programs that automate the process of commenting but do not attempt to undertake totally computerized text analysis. Such programs, such as RightWriter, Grammatik III, and the grandfather of them all, Bell Laboratory's Writer's Workbench, have their own problems which I do not wish to examine here. (For a discussion of such programs, see Elray L. Pedersen, "The Effectiveness of Writer's Workbench and MacProof," 1984, ERIC Document 281231, and Geoffrey Sirc, "Responding in the Electronic Medium," Writing and Response [Urbana: NCTE, 1989]: 187-205.) The programs that I am interested in evaluating here are those which seek to blend the abilities of the computer and the human marker, using the computer's speed to maximize the efficiency of the human being's complex sensitivity.

Three examples will illustrate the typical direction of such programs. In "Grading Essays on a Microcomputer" (College English 46 [December 1984]: 797-810), William Marling describes a system designed to take the drudgery out of marking routine errors. The system has three components. WRITER is a basic text editor that students use to compose assignments. GRADER, the heart of the system, allows the teacher to mark directly on the computer, scrolling through the student's prose and adding both individualized marginal commentary and, with the touch of a function key, abbreviated correction codes. READER allows the student to review the comments and access an online grammar book that expands the error codes into more detailed information.

In "A Partnership of Teacher and Computer in Teaching Writing" (College Composition and Communication 34 [October 1983]: 361-67), Lorne Kotler and Kamala Anandam describe another project. Their program, RSVP (Response System with Variable Parameters), does not require the student to compose or review comments on a computer. Rather, the student hands in hard copy and the teacher comments by marking an optical scanning sheet. From this sheet, the computer generates an "individualized" letter identifying key problems and offering a brief discussion of each, with models and examples. A student whose paper has been found to lack consistent ordering of general and specific ideas, for instance, may receive this sort of advice:

In a paragraph you should use one main generalization (usually called the topic sentence). This generalization tells the reader what the rest of the paragraph is about. Because the topic sentence (central idea) is general, the other sentences must directly support and develop that sentence. Use developmental sentences that are as specific and concrete as possible. (363)

This advice is followed by a sample paragraph in which the student is asked to "notice the use of specific and concrete language."

Elray L. Pedersen's program, COMMENTS, takes a similar approach but produces a more global level of commentary on matters such as purpose, organization, and tone ("Computerized Personal Comments for Student Discourse," 1984, ERIC Document 253882). After reading the student's paper, the teacher enters a set of three-letter codes that will generate a page or so of "specific, personalized" comments such as this:

Overall in this paper you show a good, but not outstanding, grasp of purpose. Performing well requires that you know the exact purpose of what you are doing before and during writing. Or you need to rewrite after one draft. (16)

Comments can be generated in twelve categories at five levels ranging from unqualified praise to outright damnation, for a total of sixty possible comments.

The most striking feature of these sorts of programs is that they are not really ways of assisting teachers to write comments. Rather, they are ways of assisting teachers to avoid writing comments. Marling's and Pedersen's systems do provide ways of writing individualized as well as prepackaged comments, but the focus of both articles is on the automated advice. As Marling puts it, "My intent was to create a grading program that would automate the explication of the myriad black-and-white grammatical errors that I encountered, yet leave me a 'window through the program' to deal with complex or subtle errors" (797). The individualized comments, in other words, are supplementary, to be used when the automatic system fails to cover the problem.

In short, then, these programs use an extremely powerful communications device for nothing more than a gee-whiz version of handbook code-marking. Having comments embedded in or stapled to his text saves the student the labor of looking codes up in a book, thus increasing slightly the probability that he will read them. It does nothing to increase the probability that he will find them useful.

The fundamental assumption behind these systems is that students can learn to improve their writing by receiving generalized handbook-style advice. But this assumption has been under heavy fire in the literature for many years. The classic statement on the subject was made in 1982 by Nancy Sommers ("Responding to Student Writing," College Composition and Communication 38 [May 1982]: 148-156). In order to encourage revision, Sommers points out, the teacher must react to and question the text, encouraging the student to take the risk of collapsing a hard-won but second-rate coherence to rethink the ideas and rebuild from the bottom up. This goal cannot be accomplished by vague directives that could be rubber-stamped on almost any text:

[T]o tell [a] student . . . "to be specific" or "to elaborate" does not show our student what questions the reader has about the meaning of the text, or what breaks in logic exist, that could be resolved if the writer supplied specific information; nor is the student shown how to achieve the desired specificity. (153)

Or as Geoffrey Sirc acerbicly puts it, "When can a fortune cookie ever tell you anything about your life, except accidentally?" (195)

Current theories of response insist that rather than issuing vague directives, our comments must engage the text itself. We need to respond in the manner that Robert E. Probst calls "transactional" ("Transactional Theory and Response to Student Writing," Writing and Response [Urbana: NCTE, 1989] 68-79). Using Louise Rosenblatt's terms of reference, Probst points out that reading is not simply "decoding" a text; it is participating in a transaction with the text, in which the reader uses it as a guide to the creation of her own meaning. Our job as sensitive readers of a student's text is to provide for her a window on this transaction, showing her what her text actually does for a reader. By doing so we defeat her sense of closure, encouraging deep revision and offering concrete suggestions as to how she can reopen the text, making it do other things that may be closer to her intentions. We will never achieve this goal with acontextual fortune cookies.

Perhaps the designers of prepackaged commenting systems are unaware of, or disbelieve, the avalanche of research that suggests that they are doing exactly the wrong thing. Or possibly they are willing to compromise personalized comments because, determined to use the computer to save them from the whiplash collar, they see no other way of pressing it into service. One of the computer's best talents, after all, is slugging in prepackaged material, and perhaps it seems as if this is the only way to exploit it as an aid to response.

However, an alternative exists. All we have to do is to be willing to use the computer in the more humble capacity in which it is actually much more familiar to us: as a writing tool, rather than as a not-writing tool. Many of us now compose extended documents directly on the computer, quickly becoming addicted to the ease with which we can enter and revise text. Anyone with even minimal keyboard skills‑-and those who do not will quickly acquire them after a year or two in front of a computer‑-will quickly find that he can experiment, rattle off ideas quickly and polish later, make both deep revisions and cosmetic improvements far more easily than with a pencil and paper. Yet ironically, when we come to compose comments on our student's papers, we somehow fail to see the activity as a word-processing task. We see it as a task for the ubiquitous red pen.

There is no reason, however, not to use the computer as a supremely powerful composing device with which to write responses to students as well as letters and articles. We do not need an elaborate system such as Marling's to add comments directly into our students' text, thus forcing students both to compose and to read our comments on a computer. All we need is our favorite word processing program with which we can compose comments and print them on a separate page, keyed to the student's text with reference numbers handwritten at appropriate places in the paper. Instead of turning his paper sideways and deciphering our scrawled advice, the student turns to a neatly printed page of comments stapled to the end of a relatively clean paper.

The question is, are there any significant advantages (other than aesthetic) to bringing this power of the computer to bear on the process of composing the hundreds of mini-documents that we normally crowd into the margins of our students' texts?

The advantages of the computer used this way are the same as the advantages of the computer used to compose other forms of discourse. The first of these is simply the speed and ease of entering text; it is possible to enter text two or three times faster on a keyboard than to enter it in longhand. It also results in less physical pain: sitting upright at a keyboard is far less demanding than bending over papers trying to crowd comments into margins that are never big enough.

I have found that writing comments this way does not in fact save me a lot of time, but it does improve the quality of the comments. The labor of writing in longhand frequently tempts me to respond in vague growls and truncated directives‑-to write "Obscure," "What do you mean here?" or even "Come again?" in the margins. But once I have written a reference number in the margin and begun filling a comfortably blank screen with words, I can usually enter three or four lines of genuine interactive response in the time it would take me to scrawl a few words of illegible longhand. I can therefore afford to be more expansive, more humane.

While writing an essay on the subject of voice mail, for instance, one of my students got off on a slight tangent, discussing first voice synthesis and then the touch-tone phone without explaining the relationship of either to the subject. Instead of growling "Relevance?" I found myself writing this:

I have been waiting for some time now for you to tell me how voice synthesis actually works with the touch-tone phone to send and receive messages. You are making too many assumptions here; you need to pause to give me a little walk-through of how the technology works so that I can understand the importance of these two components.

In this comment I was not just telling the student to "improve coherence" and adding a general discussion about the nature of coherence. I was showing him how the essay's coherence had affected me, and suggesting specific ways to make it affect me more positively.

Rapid text entry also makes it easier to write praise. Both empirical evidence and common sense insist that it is as important to praise as to blame (see for instance Donald A. Daiker, "Learning to Praise," Writing and Response [Urbana: NCTE, 1989] 103-13). However, a few scattered instances of "good paragraphing" and "nice word choice" simply won't do; the student must know what is good about his paragraphing and nice about his word choice. On a computer it is easy to write brief but specific remarks such as "This is an excellent example of a technical discussion‑-just enough to make the technology clear without becoming bogged down in detail that is not relevant to this sort of paper," or "This paragraph proceeds smoothly and logically from an overview of voice mail technology to the uses that can be made of it." Comments such as these suggest principles that the writer can apply later, but grounds those principles in his practice. They show the writer what he does right.

In addition to comments of praise and blame, I can also make value-neutral observations on content. These comments have a more or less phatic function: they remind the writer that her paper is part of a conversation, that there is someone out there who occasionally cares what she has to say as well as how she says it. For instance, in an essay on electronic music, a student mentioned that some people find electronic music dehumanizing. This stuck me as an interesting idea, and triggered the following brief response:

This feeling that electronic music is dehumanizing may partly be a holdover from the early days of beeps and twiddles. However, it is true that a synthesizer still cannot produce the rich overtones of vibrating wood or metal.

This is not a particularly exciting insight, nor does it reflect an idea that I thought should have been addressed in the essay. It was simply a response, an indication that I was thinking about what the writer was saying.

None of this attitude to response is new; theories of response have been recommending it for years. My point is that before I began using the word processor to help me write them quickly, I simply could not afford the time or energy to respond in this manner. The computer acts as a facilitator that allows me to do more easily what I have long believed to be right.

Aside from the relative ease of initial text entry, commenting by computer can also engage another more general advantage of the computer as a composing tool: ease of revision. Admittedly, few teachers are likely to revise and polish their comments the way they would a journal article. But I often find that my first impression of what is wrong or right with a passage is not quite on; after a moment's thought, I see more, or different, suggestions that I could make. I frequently use the computer's revising capabilities to fine-tune my comments, making them more accurate, toning down criticism that on reflection seems too harsh, going back and inserting praise where I find that I have succumbed to a long stretch of blame. Sometimes I even take the opportunity to correct my own spelling and sentence structure.

Used as a word processor, then, the computer can bring to the process of writing comments some of the general advantages of computer-assisted writing. The energy it saves can be invested in putting into practice some of the principles of written response that our profession has developed.

Is the computer's ability to prepackage text then a total liability? Not necessarily. When a piece of discourse begins to approach at least tentative closure, the writer does need to attend to surface matters. I do not intend to rehearse here the ongoing debate over whether or not students can profit by explicit editorial advice when they reach this point in their composing process. However, if a teacher considers it desireable to explain editorial matters explicitly when a student's text is nearing a final version, the power of the computer to enter text automatically can expedite this job. Though seldom black and white, these matters are apt to involve basic conventions that can be discussed in roughly the same words time after time. But it is only possible to do so productively when the automatic comments are used to support individualized comments, not the other way around.

This requires a system that allows prepackaged text to be freely intermingled with individualized text. There are many ways of doing this, but I have found that the most efficient is to use keyboard macros. Macros allow the user to prepackage text and enter it automatically with a single keystroke or a short series of keystrokes. No elaborate custom-written program is needed. There are many macro programs readily and cheaply available: PROKEY, SUPERKEY and NEWKEY are common examples. You simply run one of these programs before running your regular word processor. Many word processors already have a macro function built in.

Once the macro program is installed, you can use it to build up your own library of abbreviations for text that you find yourself commonly re-using. When you are about to type a string that you think you will use again, you touch a key that puts you into a "record mode." Then everything you type, until you turn record mode off, is saved and assigned to whatever key or key combination you select.

The macro function hovers invisibly behind the main word processor, waiting patiently for you to hit one of these preset combinations of keystrokes. When you do, the text string attached to that combination appears on the screen as if ghost fingers had typed it. Then you can go on entering text manually as before.

This shortcut means that I can insert generalized advice into the middle of a specific comment. I might, for instance, be writing comments on a writer's ideas and organization, and want to alert her to a fragmentary "being" phrase as well. I simply type "beingfrag," and the following advice is rubber-stamped into my comment:

The word "being" traps a lot of people into sentence fragments. Used as an auxiliary with a main verb, it can support a sentence: "She was being a pest." But take away the subject and the main verb and you have a sentence fragment: "Being a pest." Your sentence . . .

This is highly generalized advice (although not as generalized as a code-mark that directs the student to three pages in a handbook). But having stamped this advice, I am still in my comment with my word processor engaged. I do not have to cut to another function of the program to enter specific advice; I have never left it, except for the instant that my friendly ghost assistant has been entering four or five line of text. It is easy to continue writing in order to tie the advice to the specific context. I fact I force myself to do so by setting up the prepackaged text with an incomplete tag ("Your sentence . . .). I can complete the idea by relating it to the student's text:

Your sentence, "Being a very new application of this technology," is just as much a fragment as "Being a pest," but you probably did not notice because you mentally attached it to the following sentence. Just change the period to a comma and the two parts of the sentence will work together: "Being a very new application of this technology, the voice mail system has not yet been completely accepted."

There is a fairly limited number of instances in which I can effectively prepackage this much text: elementary matters of grammar, punctuation, and format largely exhaust the possibilities. However, I often find the shorthand function useful for short introductory leads. Instead of growling "WW," I can enter a few keystrokes that automatically type in the string, "I don't think this is exactly the word you want. It means . . ." Then I can complete the string with a definition. I can point out coherence problems by entering an automatic string such as "This idea seems more logically to follow . . ." and then manually entering a description of the idea that it more logically follows. I can even encourage myself to praise more by setting up abbreviations for "I really like the way you . . ." and "This passage works really well because . . ." In short, prepackaged text can be useful as long as it is prepackaged in relatively small bits and the teacher takes the trouble to assemble those bits into text-specific advice.

This use of the computer is the most sophisticated of the ones that I have discussed in this paper, but it, too, is really only a variant on an ordinary word processing function. I use it frequently to speed up the writing not only of student comments but of other documents. For example, when I was writing a paper in which I had frequent recourse to the bulky phrase "discourse processing theories of comprehension," I set up a macro that would ghost-type it for me whenever I entered the abbreviation "dispro." The use of a standard macro system means that I can streamline any writing task by setting up the macros that I need, when I need them.

In short, all the functions I have described are variants of the computer used as a writing tool, not as an electronic Harbrace Handbook. Rather than trying to minimize the writing of comments, they maximize it, streamlining the mechanics of writing so the instructor has more time and physical stamina to respond extensively and sensitively.

We have, then, been expecting too much of the computer as an aid to response. We have been asking it to do too much of our work for us, thereby trapping us into prepackaged and insensitive modes of response that we have long known to be unhelpful. We should no more ask the computer to generate comments for us than we should ask it to generate letters and articles for us. Instead, we should do the generating, using the computer as a writing tool rather than an automated marking system. Like any writing tool, the best that the computer can do is to make it easier for us to do our best.