Figure 1. Example of One Student's Web Page and Links to Coursework
The Software Engineering Research Network has experimented with methods for delivering graduate software engineering courses at a distance. This paper describes our experiences with instruction, evaluating student learning environments, and technology tools and requirements. The methods used to prepare the courses for distance delivery are described. Several technological tools to facilitate computer mediated communication were used: WebCT, electronic mail, a list server, and NetMeeting. Observations about workload requirements for teaching, support, instructional design, and maintenance are described for the benefit of faculty interested in moving towards distance delivery methods. The methods used to evaluate the distance learning environments experienced by graduate students are outlined and explained. An analysis and report on the data and information collected from students and instructors is presented, and web-based data collection methods for educational and psychological research on distance learning are discussed. Finally, recommendations are made for future research and practice with distance learning environments.Keywords: Distance Education, Graduate Software Engineering, Web-based Instruction
The program philosophy is based on experience with constructivist approaches to undergraduate courses over the last five years where all course material and student assignments have been made publicly available on the web. This has involved the restructuring of existing pedagogical approaches, de-emphasizing received wisdom and authority of the instructor, and emphasizing collaborative learning and meta-reflection on the learning process. We have seen evidence of the positive feedback processes that are the basis of major innovations in the way that our students are using resources provided by other students world-wide and are increasingly seeing themselves as contributing to those resources rather than merely 'taking a course.' Students who have graduated continue to return electronically to use the list server associated with a course, either because they have come across an interesting idea or technology to contribute, or because they wish to raise a question that might be answered within the class. One of the goals of the MSc program is to make material available to software engineers and small companies who cannot afford the time or cost of formal graduate education. The industry sponsors agreed from the start that putting everything in the public domain, that is on the web, might help to achieve this end.
The required courses emphasize collaboration and cooperation with students working in groups to research a topic and present their findings to the others. They are continually changing groups for each assignment, and get to be familiar with many working styles and others' widely varying background experience. They are encouraged to take advantage of work done by others, acknowledging help where appropriate, and to submit their work on the web then revise it according to feedback in class and on the list until they feel that they have done as well as they can. This puts the emphasis on mastery of concepts and skills rather than "getting it right first time" which is the experience many have had in their earlier programs.
Tom Russell, from North Carolina State University, has compiled a large bibliography of research that demonstrates that there is no significant difference related to the media used for distance learning. His well-known "no significant difference" study (i.e., a 50+ year retrospective analysis of the difference between distance and local education, available online at http://teleeducation.nb.ca/nosignificantdifference/) suggests that student learning is not disadvantaged by distance education models. We believe that computer mediated distance methods offer several learning advantages to students that go beyond "anytime, anywhere" (Daniel, 1997) access: 1) learning about networks and computers, 2) interaction and contact with experts and others outside of their immediate community or workplace setting, 3) increased self-reliance, and 4) developing independent learning approaches. The use of technology in teaching and learning has the potential to produce gains in productivity and efficiency. Although there is an initial increase in resources needed to research, develop and implement a distance model, which will be discussed in detail later in this paper, a distance education model can be cost-effective in the long run (Daniel, 1997; Bates, 1995; Rumble, 1997).
We are currently addressing the challenge of making the MSc program formally available to students using the second approach, on-line course delivery, while maintaining current levels of interaction and ongoing participation. First steps have included developing and offering parallel sections of SENG 611, a compulsory course in Requirements Engineering. One section of Requirements Engineering was delivered on campus, and the other delivered at a distance using facilities provided by WebCT. A second step was to offer SENG 609.04, an optional course in Software Design Patterns, as a distance education course that was supported with WebCT and weekly chats using NetMeeting.
Trialability is the degree to which an innovation may be experimented with on a limited basis (Rogers, 1995), and represents less uncertainty to individuals who are considering it for adoption. We chose to conduct our first distance education experiments with graduate rather than undergraduate students because class sizes are smaller, and graduate students are less "grade-driven" and more independent in their approach to learning, and as a result, tend to be more forgiving of the fits and starts that tend to accompany changed learning environments. Thus, although on-line course delivery approaches and software tools were novel, we deliberately chose a group that would have lower levels of uncertainty with this type of experiment. It is important to note that all of our graduate students were local, and had convenient access to the instructor and other students at all times.
This paper reports on our experience to date with these two distance education graduate courses in software engineering. Described first are the methods used to prepare the courses for distance delivery. This work experimented with several technological tools to facilitate computer mediated communication: WebCT, electronic mail, a list server, and NetMeeting. Second, the paper outlines and explains the methods used to evaluate the distance learning environments experienced by the graduate students. An analysis and report on the information collected is presented. Web-based data collection methods for educational and psychological research on distance learning are discussed. Finally, recommendations for future research and educational practice in distance learning environments are summarized.
The overall aim of the graduate program is to provide a collaborative learning environment in which experience and knowledge can be constructed and shared, ignorance displayed and errors made without censure and with ready access to diagnostic help. It becomes clear to students that, while there are no easy answers to the core questions of industrial practice in software engineering, there are many useful perspectives and that simplistic, surface level answers and approaches generally have very limited applicability. Several resources distinguish the program from more traditional programs:
All of these extensions to the courses took a considerable amount of time, and would not have been possible without additional help with these projects, given that one instructor was also teaching other courses and involved in other research activities, while the other is a full-time Software Engineering Manager in local industry. A post-doctoral research fellow provided assistance and support with the repurposing of course content for distance delivery. The post-doctoral assistant also provided technical assistance in investigating features of WebCT, adapting the WebCT scripts as needed, and coordinating with the WebCT developers. Evaluation of the learning environments for both courses was done by a post-doctoral researcher (with a background in Educational Psychology) who was not involved in student evaluation, and protected student anonymity with respect to information gathered throughout the project. A graduate student repurposed the original web notes for the second course and posted these in the WebCT environment, and also participated in the course as an informal peer-advisor. A network manager was available at all times to address technical concerns, install and implement new software, and react to technological breakdowns and failures. Although these first experiments with distance offerings of two courses demanded a large infusion of time and additional support, from the instructors and other assistants, future offerings of the courses will require less time and effort. Learned lessons and hands-on experience with distance education has provided insight into refining and improving the processes.
As a result of outcomes from the first experiment, we improved the design of our distance model in a number of ways. Concurrent sections of the Requirements Engineering course were offered for the purpose of directly comparing the two learning environments. The Design Patterns course was offered only as a distance section rather than offering concurrent distance and local sections. This was done for two reasons: 1) it is a conservative estimate to suggest that instructor workload was doubled because they were, in effect, teaching two courses to two different classes of students with different needs, and 2) if the "distance" students are not truly at a distance (students in our graduate program tend to be located in Calgary), they tend to gravitate towards campus to meet anyway. It was difficult to keep the distance students in the first course at a distance to increase the authenticity of the experiment. It was discovered that students in both sections wanted to meet face-to-face to work on assignments, and usually met on campus prior to and after the lecture.
A third reason for offering only a distance section of the Design Patterns course was that students in the first experiment felt somehow disadvantaged no matter which group they were in -- the local groups felt that allocating attention to creating a lecture summary reduced their opportunity to fully participate in the classroom discussion. The distance group felt they were missing vital information by not being present for classroom discussions and the face-to-face contact with instructors. Students in the local section of the course were assigned the task of summarizing the in-class discussion and interaction, and posting this information on the course web page for the benefit of the distance students. Unfortunately, this did not work well for distance students because of the variable quality of the transcripts and the delayed posting times. Also, according to local students, summarizing the lecture reduced the quality of their participation because their attention was focused on "catching every word" rather than on contributing to discussion. Finally, both local and distance students perceived that they spent more time per week on course-related activities, although the average time spent by each group per week was not significantly different (see Figure 5). Thus, it was decided to minimize perceived equity problems by offering only a distance section of the second course.
For the Design Patterns course, the first and last lecture were face-to-face, and other classes were at a distance. The course was project-based; students "presented" their projects on the web, then led web-based discussions about their work for the subsequent week. NetMeeting, a computer-based conferencing tool with video, audio, and application sharing features, was used to experiment with real-time, synchronous class discussions at designated "lecture times" employing . The final lecture of the course was a face-to-face seminar, where students presented and discussed their work in a conference-like setting.
An extensive summary of our evaluation of WebCTs features is available online (Jacobsen, Wijngaards, Kremer, Shaw and Gaines, 1999). In brief, WebCT is designed for non-technical faculty who are new to creating web-based documents. We found that the WebCT designer interface often hampered the HTML-knowledgeable instructor's efficiency in building and maintaining the course material. For example, sequencing course materials requires the designer to set up a "path", which involves a number of intermediate steps that would not be required if the designer was using only HTML. However, for an instructor who is new to creating web-based documents, the structure and tool set offered by WebCT provide valuable supports as they construct course materials.
It was found that WebCT was not completely applicable to the current offerings of the two graduate courses. However, based upon research that compared WebCT to other web-based course tools (e.g., Kristapiazzi, 1998), we expect that other web-based course tools are also not completely applicable and would also require customization for these courses. Courses in the research-based M.Sc. program have been developed within an open architecture philosophy and the belief that there should be public access to past and present course materials and student work. Inherent to the design of WebCT (and most web-based distance education tools) is a closed architecture philosophy -- courses are password protected, and information in courses is not easily accessible to the general public. WebCT's student tools, in particular note-making and group presentation areas, offer useful tools for online courses. The email and chat facilities in WebCT were not used for either course; an external, publicly accessible list server remained in use. The Software Engineering courses share a common list server that allows the students to enjoy input from students in other courses, former students and industry partners who subscribe to the list. An advantage of the list server is that each message is delivered to subscribers (push technology), as opposed to the student having to seek out the postings of others by accessing a separate web page "discussion list", threaded discussion or newsgroup, or bulletin board (pull technology). WebCT's quizzes, which employ selected response and open-ended questions, and so on, were not useful for graduate-level courses. For a brief review of the efficacy of WebCT's online testing tools for undergraduate computer science, see Jacobsen, Kremer, and Flores (1999).
Microsoft NetMeeting was used as a substitute for face-to-face class discussions. The instructor and students participated in weekly discussions about course topics using a version of NetMeeting running on a local server. The potential of this on-line chat environment was not fully realized in this course. For example, at the end of a sixty or ninety minute session, the instructor observed that very little material was actually covered. The level of interaction and discussion productivity did improve during the course as students adapted to the limits of the technology. Although it offers a multimedia discussion environment using audio and video, NetMeeting offers little more than a standard text-based chat tool as it allows only point-to-point audio and video and its shared drawing tool was found to be inadequate. The one thing that NetMeeting does add over a text-based chat tool is shared application viewing, which some students used to great advantage in making network "presentations" using PowerPoint. Other factors that seemed to limit the efficacy of NetMeeting chats were:
The list server proved to be an important communications element in the first course, but appeared to be under-used in the second course. For both courses, the instructors were participant observers on the list server, rather than active and directive moderators (except when students appeared to be involved in a non-productive debate or required expert advice). The instructors assessed list server participation during the first course, and found the sheer volume of discussion was greater than in previous, classroom-only offerings of the course. There are a number of possible explanations for this trend. First, a large proportion of the local students speak and write English as their second language, and seemed to prefer to participate more in the list server than in the lectures. It may be that these students are more confident when they have time to reflect and respond to a written discussion, rather than speaking out in class. Second, distance students actively participated in list server discussions, possibly because they were not present for the classroom discussions. A third possibility is that students actively participated because this was a graded expectation for the course, although this expectation was unchanged from previous classroom-only offerings of the course.
Participation in list server discussions was not as active during the second course. A possible explanation for this result is that attention may have been diverted from the list server as a result of the emphasis on real-time weekly interaction using NetMeeting. Students may have saved their questions for the weekly chat sessions rather than posting them to the list server. As a result of the experiments, we have concluded that the list server offers several advantages over the weekly NetMeetings including:
Figure 1. Example of One Student's Web Page and Links to Coursework
By the second course, students were accustomed to the expectation that
they post all of their assignments and coursework on the web, and reported
very few difficulties with achieving this requirement. An example of one
student's course web page with links to their coursework is presented in
Figure 1. An example from a student's work for their textual and graphical
presentation on a design pattern is presented in Figure 2. Finally, Figure
3 provides an example of a different student's presentation materials posted
on the web.
Figure 2. Example of Student Assignment on Design Patterns
Figure 3. Example of Student Presentation Materials on the Web
Two unconventional data collection procedures, along with a conventional end-of-course evaluation, were used to explore and evaluate the distance learning environments offered to students in the two courses. Students emailed weekly learning logs and a final biography of learning to a distance education researcher who took no part in the grading of student achievement in the course. Students recorded three types of information for their learning log:
The Biography of Learning is a retrospective narrative account through which the student questions and reflects upon their educational biographies and learning experience throughout the course. The learning logs proved to be a rich data source from which students could conduct a comprehensive review and analysis of their personal learning experience in the course.
- reflections on the outcomes of lecture/sessions - main objectives of lesson, what was important new learning, what was confirmed, what changed for the student, value of topics discussed to student's work life or future career goals),
- notes on what worked particularly well, and what did not - perhaps according to preferred learning methods, individual learning styles or personality types), and
- time spent on various course-related tasks - such as reading the textbook, writing responses, participating in project work and group meetings, reading and contributing to the course list server, and constructing and uploading HTML documents.
The use of open-ended questions allowed students provide feedback in their own words on the nature of their learning experience. The questions provided an opportunity for students to highlight any ideas, concerns, problems, or suggestions which they considered of relevance to teaching and learning at a distance. Student responses were condensed to reflect emergent categories and themes, and subsequent reporting includes verbatim selections from the source data.
When using this data collection technique, it is important to keep this task open-ended rather than deriving categories or imposing structure on student's time reporting (e.g., by setting up form-fill-in web pages with defined categories for response). With preset categories, one risks that students will only attend to the amount of time it takes for the identified tasks, rather than constructing a genuine account of time spent on tasks (many of which may not have been anticipated by the researchers).
"This is the first time that I am taking a class specialized in professional computer industry. You can understand that there are lots of excitement and yet also lots of strange terminology as well. I was surprised that two chapters of the textbook must be summarized due next week. This is going to be a fresh experience for me since I have been working as a mechanical engineer and most of the time I spent has been on design and calculations, rarely I have spent time on book summarization. I will have to cultivate my fast reading skill in order not to fall out the class".Students would often use the forum offered by the weekly learning log to reflect upon their learning discoveries and comment on the value of topics investigated throughout the course.
"During the reading assignments and writing the paper, I found that classifying design patterns depending on their relationships with other patterns can help designers to select patterns for solving problems. This is interesting because I think learning how to use patterns is more important than just knowing what patterns are. Writing this paper makes me think differently and I start to wonder about why people are using different schemes to classify the design patterns".It is a rare opportunity to get insight into the thinking and discovering that accompanies a reading and writing assignment in graduate school. We will discuss later how students viewed the Learning Logs and the Biography of Learning as valuable opportunities both for increased self-awareness as a learner, and to reflect upon their own learning processes as they worked through the course.
"The main objective in the week #4 was implementation of the selected pattern and a class presentation. Since my pattern is hard to implement, I wrote a paper on it. It required lots of reading, and from reading references, I learned more. I also spent a lot of time preparing for the presentation. The presentation went well, except that I had some problems with presentation skills. I need more practice to improve that skill. I also learned something from other students' course work. Since I didn't do programming, it is good to see patterns in code from other people's work"This student was not required to write a paper on the design pattern but made an independent learning decision to actively seek out more information, and organize this into a form that was appropriate for their peers. The student identifies an individual learning goal related to presentations, and outlines the benefits of participating in other students' presentations.
Students were given the option to participate in either the local or distance offering of the first course. This student was motivated in their choice of the local section, not because of their philosophical beliefs about learning environments, but as a result of not having the necessary computer equipment for participation.
"The decision to take the course by attending the class rather than distance group was motivated by the fact that I don't have computing facilities at home".Planners must take into consideration the requirements, both hidden and visible, for students to be able to access a distance program. Though this is a graduate program in software engineering, we cannot assume that all students have adequate computing facilities at home that will enable them to consider online, or web-based, instruction. Although many students work at home, the distance model was originally designed to be useful to students who are employed full time, to increase accessibility for students working in industry. One way to describe the interaction that we hoped to facilitate is the "coffee-break model", whereby students access course materials during natural or scheduled breaks in their work day. Instead of trying to schedule large blocks of time for coursework, the student learns to accomplish certain tasks and goals in shorter periods that contribute to larger goals.
Both local and distance students regarded the list server discussions as an important extension of classroom discussions, and all students agreed that group work was a valuable component of the course. Students criticized some technical aspects of the current experiment. In particular, unreliable accessibility to the course material, which was related to network access issues that were beyond the instructors' immediate control. Students also experienced difficulties generating and translating their assignments as HTML documents; some lacked prior experience with HTML, and others encountered problems accessing the server.
"Writing summaries forces me to read and understand the material. Course participation forces me to think deeply about the material and come up with questions and answers. I had problems posting my summaries on WebCT. A peer helped me. It was not clear where to post my stuff. This was not clearly stated in the course pages".The weekly learning logs were a valuable formative evaluation of the course that enabled us to learn about problems, omissions, concerns, and student needs as they occurred, rather than waiting to find out from a summative, end of course evaluation. As a result of the weekly feedback from each student, we were able to refine and improve the course materials on WebCT, respond to technical difficulties, and address concerns immediately, which enabled us to better meet student needs.
Using NetMeeting in the second course for weekly discussions was not always successful. The instructor was online for each session, and at times it proved to be a challenge to sustain interest and participation in certain topics. On the other hand, some topics really stimulated active and profuse discussion, which led to rapid fire comments appearing on the screen. The active discussions in NetMeeting were hard to follow as several responses to a question would appear at once, which in turn set off another round of responses and new questions.
"Based on the online chat we had on Tuesday afternoon, I find that the discussion through email is more fruitful than the chat room (i.e., NetMeeting). Basically, I found that during chat session, there were many topics discussed at the same time, and focusing (or following) the topics at the same time were rather difficult. I think we have to find a way to moderate the discussions. When a topic starts I think everybody should focus on that until it concludes, then the next topic starts".Students made a number of suggestions about how to improve the organization of discussions using NetMeeting. First, generate a list of discussion topics before the meeting using the list server to elicit suggestions during the week. An student-constructed agenda may reduce the time lags in participation in the meeting. Second, assign a student moderator each week who will pose questions during lags, attempt to focus the group discussion, and guide participants through the agenda items. A student moderator may help to reduce the number of items being discussed at any point in time, and because each student would have an opportunity to participate in this role, the whole group may develop useful strategies to facilitate discussions using this medium."This week the discussion was not very productive. First of there were fewer people than there normally are. Secondly, the discussion was not on topic. I prefer that the discussion remains focussed on one topic before moving on to the next one. Hopefully, this will work better next time".
In both courses, students were required to post all of their coursework online using HTML and WebCT.
"Writing web notes is a good learning experience. Because I was not familiar with web page posting and editing, I spent a lot of time on web page construction. I also had some problem in uploading the notes. I lost all of my images. But I finally got it fixed".Although students were keen to experiment with the distance model, a number of them were relieved to gather together at the end of the course for the group presentations and class discussions.
"It was nice being back in the classroom and being able to interact normally".It is becoming increasingly difficult to identify what is a "normal" learning environment given the growing diversity of the student body. We no longer have traditional students who start post-secondary programs in their late teens right after high school. Students are starting or returning to post-secondary institutions who are all ages, have all types of background experiences, and have different expectations and needs. Institutions need to be flexible in the programs they offer, when they offer these, and the methods they use to construct learning environments, both local and distance. All learners are different. Some students work full time, and others are at home with young children, and can only attend classes in the evening. Others prefer to be on campus full time and taking classes during the day. Some people enjoy coming to class, and feel that they benefit most from discussions with other students. Other students enjoy or prefer working on their own, in short intervals, and rely less on the groupings and class gatherings for their learning and interaction. It may be that some of our students have only experienced the traditional classroom-based campus learning experience, and are tentative and unsure about the suitability of other learning environments based upon this learned preference."I enjoyed being back in a traditional classroom setting. I still feel that that is the best learning environment for me".
Weekly time estimates for course-related tasks were collated for each
of the five weekly learning logs. An aggregate summary of the total and
average number of hours per week per task, and the average number of hours
spent on each task per student is presented in Figure 4.
Figure 4. SENG 611 - Percentage of Time Spent Per Task Per Week
Per Student (Local vs. Distance Cohorts)
Local students reported spending a larger proportion of their time in the classroom, reading course material, and conducting on-line research and reading documents online. Distance students report spending slightly more time on chapter summaries, and a large proportion of their time interacting using email and the listserver, constructing and transfering HTML documents, and meeting with their group to work on projects.
Surprisingly, local and distance students spent a similar number of
hours per week on average for all course-related tasks (Figure 5). This
finding contradicts the distance students' perception that they spent more
time overall than local students on course-related requirements.
An explanation for this difference in perception may be found in how local
and distance students spent their time on various tasks (Figure 5).
Figure 5. SENG 611 Percent of Total Time Local and Distance Students
Spent on Various Tasks (including class time).
Because we only offered a distance section of the Design Patterns course,
Figure 6 presents a summary of the average number of hours spent on the
course each week by each student. The average number of hours remains fairly
stable for the first three weeks of the course, with students reporting
an average of 12.9 to 14.1 hours spent each week. As the course moves into
the fourth and fifth week, students report spending much more time, from
19.6 to 22.5 hours on average per week. A ready explanation for this result
is the additional time allocated to preparing their final project for public
presentation during the sixth week. Students report spending an average
of 13.1 hours on the course during the final week of classes.
Figure 6. SENG 609.04 Average Number of Hours Spent On Course Per
Student Per Week (including class time)
In ways that are similar to the first experiment, student estimates
of time spent and descriptions of tasks varied. As such, several categories
were created to organize and summarize results from the second course (Figure
7). Reading the text and WWW documents represents an average of all reported
hours spend on required and elective reading time reported for both media.
The average of 5.3 hours per week spent browsing and creating web pages
includes the construction of HTML documents, reported hours spend searching
and browsing for research documents relevant to their final projects or
weekly readings, and time spent accessing course materials on WebCT. The
average 1.1 hours spent on writing tasks include preparing their learning
log and writing summaries of required readings. Because students were required
to post all of their coursework on the web, the time spent for writing
each week is probably better captured by the "creating web pages" task
than the conventional writing task. Most students created their original
documents as HTML rather than word processing first and converting later.
"Prototyping/code/implentation pattern" arose as a category to record the
average 6.4 hours of time students reported spending per week on their
design patterns and prototyping and coding a variety of programs. The task
that students reported spending the most time on was presentation preparation,
which includes research, writing, and preparing visuals for their public
presentation. The average reported time was 9.2 hours per week, but this
figure is best understood using individual numbers to get a perspective
on how students allocated their time. Students had to prepare presentations
for different weeks throughout the course, and again for the seminar at
the end of the course. One student spent 12 hours on presentation preparation
in the second week, and another 8 hours in week 5. Another student spent
10 hours in week 3 and 7.5 hours in week 5. A third student reported spending
27 hours in the first week.
Figure 7. SENG 609.04 Average Time Per Task Per Student Per Week
(including class time)
A number of students described how writing weekly Learning Logs and a final Biography of Learning raised their awareness about self as learner because of ongoing attention to how much time they were spending on various tasks, what was working and what was not, how many tasks, type of engagement in course, and so on. Students commented on the benefits from time invested in completing the Learning Logs and the Biography of Learning. Many students observed that the requirement to take time to reflect upon their learning processes as they worked through the course created a valuable opportunity to become more self-aware as a learner and direct attention to habits and patterns of tasks and time spent on learning activities.
"Forcing myself to look back on my impressions of the lectures and the time I spent on the different activities was revealing. Normally, I attend lectures and absorb what I can. By looking back in a more structured way, I have probably absorbed a little extra. If I were a more organized person, I would have also been better able to manage my time. It may just help me become more organized in the future".Some students commented on specific aspects of the course that were beneficial, such as using the Myers-Briggs as a technique to better understand self as learner and potential contributions and roles in group work. Others submitted positive descriptions about the value of group work to their learning.
"Since there was a lot of group work, I was fortunate to be in a group that worked well together. Each person was more than willing to do their share which helps create a feeling of good will between the group members. For the most part, I thought the text was very useful and I'm sure that I'll refer to it many times in my career. I enjoyed critiquing the text and found that the process helped a great deal in my understanding".Some students used the Biography of Learning as a forum to reflect on their fluctuating satisfaction with the technologies used in the course, and provided summative views on the efficacy of certain types of interaction. The following two comments provide examples of the range of reactions to the distance technologies.
"As I signed up for SENG609.04 I was very excited to begin the course. However, through the course my excitement has withered. I personally found the NetMeetings were an inappropriate medium for discussing complex issues. I am still open minded towards distance learning through tools similar to videoconferencing. But NetMeeting’s awkward audio/video interface and comparably slow typing versus speaking made the NetMeeting’s largely unproductive when compared to a real meeting".
"I learned a lot by the new method of distance learning that was employed in this course. At times the immaturity of this new technology (NetMeeting) and its users (us) showed up. As the course proceeded, we could make better use of the tools we had at our disposal, and the course was even more fun... There was a lot of research involved and a lot of material had to be read in a relatively short period of time... the instructor was able to devote more time to individual students. Overall I spent more time on this course than any other course this semester. I spent a total of around 110 hours during the five weeks of this course".As instructors interested in refining and improving the learning environments for students, it was instructive to have access to student reflections on what they believed to be important to their experience in a particular course. The following student provides insight into how time gets prioritized and allocated for projects that will be presented to peers.
"Web notes preparation was an important new learning in this course. Because I thought I had to know more to be a class expert on my pattern and to answer other peoples' questions, I spent a lot of time on that, and I found I learned more from reading references and writing web notes than I did from just writing summaries. I also learned some other patterns that are related to my pattern, on which I don't think I would spend time if I just wanted to write a summary".Breaking with traditional post-secondary models, in which student assignments and projects are submitted to and read by only one person, is one way to stimulate and promote exemplary work. When students know that their assignments and projects will be examined by their colleagues and peers, they invest more time and effort to do their best work. The Biography of Learning provided insight into student thinking and planning for public presentations and displays of their work.
Many areas of inquiry can benefit by extending their data collection activities to take advantage of an Internet connection (Mueller, 1997). The use of a web-based interface to collect information for educational research offers several advantages over traditional methods. First, web-based forms save the expense and delay of regular postage and paper-based distribution methods. If sampling error is a concern, researchers have the flexibility to collect data using a local area network configuration (LAN), rather than making the instrument available to "the world" on the Internet. In the present experiment, the learning log form and exit survey were published on the World Wide Web and students were provided with links to the relevant documents along with due dates. Second, instead of using manual or scantron methods for data entry, the data collected using a web-based interface are entered by students and are stored in a cumulative data file that is ready for analysis with SPSS or SAS software. The web-based form reduced the time spent sorting and organizing the student-provided data sets by more than half, which in turn meant the instructor got the anonymous formative evaluations very quickly each week. Third, because the Internet is available to anyone with a computer and a connection, at anytime and from anywhere, students can submit information 24 hours per day without the researcher having to be physically or temporally present and without having to procure laboratory space or photocopy questionnaires.
Anonymous participation was important and was guaranteed to students. In general, when results are submitted using a web-based, HTML-generated response form, the cumulative data file lists the date and time the results were submitted and the IP address of the server from which the survey was completed in a header. For each learning log and the biography of learning, students were asked to include their name so the researcher could keep track. This information was used only to confirm that students had submitted weekly logs, and the instructor was only informed whether or not students were up-to-date with their assignments.
Mechanisms and support systems need to be put in place that allow for cyclical and iterative development and assessment of distance learning environments. Tolerance has to be build into the system for time lags, unexpected barriers, and longer course development periods. Distance education environments and new technological tools constitute a major change in some people’s lives. Such change does not happen quickly or easily. Even in the best of circumstances, teachers and students need high levels of support, training, and access to technology. The instructors for both courses spent a significant amount of additional time on the distance courses than would have been required for classroom-based courses. In addition to the instructors, several individuals provided on-going technical support, instructional design and development, and evaluation and assessment for the distance courses. Therefore, it is recommended that more instructor/design time be allocated for initial distance course offerings. Technical support and help with the technology will be needed if the instructor is not completely familiar with the tools used to support on-line course delivery.
While computer networking can open up possibilities for teaching and learning at a distance, care should be taken to make sure that the chosen technology fits the core values and goals of the program, and not the other way around. The technology tools used in the present experiment were evaluated based upon the fit between core learning objectives and program requirements, rather than designing courses based upon what affordances and constraints were inherent in the technology. For example, a number of WebCT's features and built-in tools were not applicable to graduate courses, and therefore were not used.
Distance students require very explicit, well-organized, and well indexed web materials and resources. It is important to structure carefully the entire course before it starts and to be faithful to that structure throughout the course to provide distance students with a stable context in which to learn. While this finding may seem rather obvious from an instructional design standpoint, it is not the typical approach in SERN's graduate course offerings. Instead, flexibility to change course material and follow "excursions", as dictated by student interests, experiences, and current situations, is favored over rigid structure. This constructivist approach that puts the subject and the student at the center may be at odds with the need for more structure in a distance model. This is worrisome because the more open-ended and responsive approach is highly valued by both students and instructors. We are currently investigating instructional strategies that provide a balance between carefully structured and defined course design, and responsive, flexible learning environments. It may be that our distance students felt they needed more structure because the distance model of learning was new to them. The distance students monitored what the local students were doing in the course, and were fearful of falling behind or missing important information. We will continue to evaluate student learning experiences and concerns with subsequent distance course offerings.
Further information about SERN can be found at its web site - http://sern.ucalgary.ca/
Additional information about these courses, including examples of student coursework, is available from the course pages at SERN:
 |
Software Engineering
Research Network (SERN)
247 MS, Department of Computer Science 2500 University Drive, N.W. University of Calgary Calgary, Alberta, Canada T2N 2N4 |