Global Regional Review (GRR) URL: http://dx.doi.org/10.31703/grr.2019(IV-II).57 Students Perceptions of Effectiveness of Course Contents, Supplementary Material and Tutorial Support in Phonology Course offered By Allama Iqbal Open University, Islamabad Vol. IV, No. II (Spring 2019) | Page: 546 ‒ 563 | DOI: 10.31703/grr.2019(IV-II).57 p- ISSN: 2616-955X | e-ISSN: 2663-7030 | ISSN-L: 2616-955X Ubaidullah Khan * Shaista Zeb † Muhammad Uzair ‡ This study has been conducted to evaluate the Phonology course offered by the Department of English Language & Abstract Applied Linguistics, Allama Iqbal Open University, Islamabad, Pakistan. The research is a descriptive evaluation study that aims to investigate the effectiveness of the course. It is based on the feedback of the students regarding the effectiveness of course contents, supplementary material, and tutorial support. A survey-based questionnaire was designed, which sought students’ opinions about the above three areas. The questionnaire was administered to 24 students studying at one of the 26 study centers. The data have been analysed statistically/quantitatively as well as qualitatively. The study found out that the textbook needs to be revised/rewritten with the latest teaching techniques included in it. The supplementary material needs to be made more relevant, and the quality of printed and recorded material needs to be improved. Key Words: Phonology, Allama Iqbal Open University, Islamabad, English Language, Distance Learning System (DLS) Introduction Distance Learning System (hereinafter referred to as DLS) is a unique context in which learning takes place remotely. Unlike the traditional system where students attend educational institutions and are taught by teachers in classrooms during specified hours, in DLS, learning takes place at home or away from the school or college or university, and it is not necessary that the teacher-student interaction will take place during a specified time period. While different DL universities might have different specifications of their respective systems, the general philosophy remains the same. Greater reliance is on the study material, which is provided to the students through various means such as printed books, electronic books, CDs or online material. The teacher might get an occasional chance of interacting with the student or may never get this chance, and learning goes on with the help of learning material only. Distance Education achieves almost the same outcomes as Traditional Education does, as opined by Hanson and Maushak (1996) in Simonson (1997, p. 86). The learning outcomes achieved during the study period are assessed electronically or with the help of staff who may not necessarily be the ones who have taught the courses being assessed, but some other, temporarily hired staff who have been assigned the task to conduct the exams provided by the institution on a given venue. DLS, in other words, is a flexible system in which the learners are not bound by time, place and schedule of studying particular courses to be taught by particular teachers. The study in DLS is self-paced rather than time-bound. Lessons do not necessarily have a time limit, but modules do. In such a system, it becomes doubly important to assess the quality of instructional material and all kinds of support the students are provided by the system or the institution they are affiliated with to ensure that quality learning takes place and the students do not lag behind and do not remain disadvantaged compared to the ones studying under regular/face to face/traditional system. The mechanism of the delivery of course contents should be such that it makes up for the absence or minimal interaction of the teacher as is the case in a traditional classroom, the content is communicated in such a way as to engage the students and other procedures of the * Lecturer, Department of English Language & Applied Linguistics, Allama Iqbal Open University, Islamabad, Pakistan. Email: ubaidkhan81@gmail.com † Department of English (GS), National University of Modern Languages, Islamabad, Pakistan. ‡ Department of English (GS), National University of Modern Languages, Islamabad, Pakistan. Students Perceptions of Effectiveness of Course Contents, Supplementary Material and Tutorial Support in Phonology Course offered By Allama Iqbal Open University, Islamabad mechanism such as dissemination of the tutorial-help and teacher’s instruction satisfactorily fulfil the students’ learning needs. It is a normal practice in educational intuitions to carry out course evaluation to evaluate the quality of instruction, instructional material and delivery of content, but little effort is made to study the effectiveness of all the above elements in the DLS, where it is of immense importance to ensure that the learning takes place effectively in the absence of a regular instructional mode and a full-time teacher. English language teachers in Pakistan often get admission to Diploma and MA in Teaching of English as a Foreign Language (TEFL) in Allama Iqbal Open University (henceforth referred to as AIOU), Islamabad, which, as its name suggests, is a DL university. These teachers belong to all levels; they include university teachers, college teachers, school teachers and Montessori / KG teachers. All these teachers prefer to do these programs to not only enhance their educational qualification but to develop themselves professionally. And since teaching English involves teaching the skills such as reading, listening, speaking and writing, it becomes a challenge to impart instruction without or with minimal teacher support because, in language classes, a teacher is considered as a model who demonstrates correct use of language while the students learn from him/her. Despite the fact that Diploma TEFL is a teacher training programme, it is meant for the English language teachers, and the importance of teacher/trainer’s presence is as much important as it is in the language classroom. The importance of teaching English in Pakistan is manifold as English is considered lingua-franca across the globe. Although it is not a native language for Pakistanis, its importance as lingua-franca or language of interaction between international communities or countries cannot be denied, and this is the reason why it has acquired a special status in the country and has become an essential part of the curricula at all levels. Learning English is considered essential, which necessitates quality English teaching across the country. Teaching English to the wider public is highlighted so as to increase the scope of its use and improve the chances of its accessibility to more and more non-native speakers using it as a second or foreign language. Teaching Methodology vs Content Teaching is not only concerned with the teacher being skilful in the art of teaching or using appropriate teaching methodologies and methods; it also has a lot to do with the material that he has been assigned with the responsibility of teaching it effectively. Effective teaching is not likely to yield favourable enough results in case it is not backed up by the material that promotes the advancement of knowledge in a given field of study. That is to say, the material a teacher is supposed to teach needs to be ‘appropriate’. Appropriacy in terms of the available study material involves so many considerations. The material needs to be designed keeping in mind the students’ level, their educational and social background, their learning needs, their convenience in terms of covering the given syllabus in the given time, the teacher’s ability and convenience to teach it effectively in the specified time, the extent of the scope of it is a helpful resource in increasing their information, understanding and skills in the given area, and so many other considerations. In such a case where good teaching is not the only way to promote effective learning, the role of the syllabus designers becomes very much important. The teaching process, in such a case, cannot be deemed to be limited to good teaching only. It also involves so many pre-teaching and post-teaching measures such as syllabus designing and course/programme evaluation, both of which, alongside effective teaching, play an important part in the effective implementation of a comprehensive teaching plan. Evaluating a Course In today’s digital age, in which information is abundantly accessible, curriculum designers are always under pressure to design courses that are up to date and fulfil the students’ needs keeping in view the needs of the age. And this pressure is manifold when it is a matter of syllabus designed for the teacher training programme because it must aim to equip the teachers/trainees with the knowledge as well as strategies that best help them impart quality education to their students whose demand for quality education is ever-increasing due to the reason mentioned above. Although the sample of this study is trainee teachers, for the purpose of convenience, we shall refer to them as students in our discussion. Distance Learning courses are definitely dissimilar to the traditional courses in many ways, which increases the importance of course evaluation manifold to gauge the effectiveness of the course for the intended audience. Effective course evaluation keeps in mind the needs of the students, and the purpose of getting feedback is to Vol. IV, No. II (Spring 2019) Page | 547 Ubaidullah Khan, Shaista Zeb and Muhammad Uzair modify or update the syllabus in accordance with the students’ needs. Henckell (2007, 01) cites Benigno & Trentin (2000) and Spooner et al. (1999), who draws our attention to the fact that distance education courses “share many of the common characteristics of traditional, face-to-face courses, yet introduces new variables in teaching and education”. Keeping in mind this difference, it is paramount to evaluate the quality of these differently structured programmes and courses through programme/course evaluation. Students’ performance in the courses cannot be considered a sole indicator of the success of the courses, and it must be complemented with their evaluation of the courses, which informs all stakeholders about their (dis) satisfaction with the courses. As a result of the students' feedback, the institutions amend courses, revise and/improve them with a view to attaining better student satisfaction. Evaluation can be useful both before and after the implementation of the course. Initially, it provided the syllabus designers with an insight into the students’ expectations from the course, and after a course has been launched, it helps us know what parts of the course are effectively working and what areas need revision or updating. This makes evaluation an essential part of pedagogy as it enables the course designers and the administration to take appropriate decisions in the first place and then make adjustments if there are flaws or weaknesses in the courses they have developed and the methodology they have adopted to teach these courses. Aim of the Research The present study investigates the effectiveness of the Phonology course offered by the Department of English Language & Applied Linguistics, AIOU, Islamabad,t the Diploma TEFL level. It intended to obtain students’ feedback about course content/material, the supplementary material, and the tutorial support, which are the three most important elements of the course. Structure of the Programme The Diploma TEFL offered by the Department of English Language & Applied Linguistics at AIOU is a DL programme in which students from across the country enroll themselves and can attend the programme without having to take regular/on-campus classes. The course material in each course consists of a textbook prepared according to self-instructional design, which is accompanied by a reader, supplementary reading material, and audio aids. The teaching methodology consists of the following: the students have to attend 10 classes on Sundays, out of which 70 % are compulsory to pass the course irrespective of their scores in exams. They are assigned a tutor for each of the courses they study, who takes classes at their nearest study centre. During the classes, they prepare one theory based written assignment and another research-based assignment. They also present their research work during the Assignment Presentation Workshop held after the classes. Background of the Course Phonology is one of the Diploma in TEFL courses. The textbook for this course, just like other courses of the programme, has been prepared following self-instructional design, as stated above. It is accompanied by a ‘Reader’ that consists of ‘supplementary reading material’, which is a composition of recommended texts/articles related to the contents of the textbook and the Reader. The teaching methodology of the course includes 18 face to face tutorials/classes (as compared to 10 in other courses, due to the demanding nature of this course) in which the tutor is supposed to cover the whole contents of the textbook as well as provide guidance to attempt two mandatory assignments. Objectives of the Study The purpose of the research is to take students’ feedback about the course material. The specific objectives that this study aims to achieve are the following: 1. To investigate the effectiveness of the course contents/material & tutorial support of Phonology offered by AIOU. 2. To suggest improvements in the course contents/material and tutorial support system in the light of the respondents’ feedback. Research Questions The current research aims to find an answer to the following research questions: Page | 548 Global Regional Review (GRR) Students Perceptions of Effectiveness of Course Contents, Supplementary Material and Tutorial Support in Phonology Course offered By Allama Iqbal Open University, Islamabad 1. What is the students’ perception of the contents, supplementary material and tutorial support of the Phonology course vis a vis their learning needs? Literature Review What is Course Evaluation? Different writers refer to course evaluation as ‘student evaluation’, ‘student rating of instruction’ and ‘student evaluations of teaching (SETs). The overall mean the same thing except for a slight shift of emphasis (Gravestock & Gregor-Greelealf, 2008: 13). These terms have a slightly different connotation as each of them has an emphasis either on student or on course and ratings. Further, Gravestock & Gregor-Greelealf (2008) cite Wright (2008), who suggests that the most appropriate of these is ‘student rating of instructions’ because it most effectively describes how the instrument is used. Evaluation becomes a necessity due to the educators’ responsibility towards the students and their parents to provide quality education. They are responsible for giving the best compensation to the students for the huge fee they pay to the institutions. The students’ feedback helps educators identify the weak links in the chain and bring improvement in the areas of the course that need improvement, and then offer the course next time in a better shape and raise the quality of the programme. As discussed by Achtemeier et al. (2003), in Seven Principles for Good Practice in Undergraduate Education, Chickering and Gramson (2000) argued that good teaching practice must follow the following seven things: (i) encourage student-faculty contact, (ii) encourage cooperation among students, (iii) encourage active learning, (iv) give prompt feedback, (v) emphasize time on task, (vi) communicate high expectations, and (vii) respect diverse talents and ways of learning. Why Course Evaluation? Course evaluation is necessary to improve the quality of the courses the educational institutions offer. Normally, the institutions do not pre-plan evaluation. As Diamond & Sudweeks (1980) suggest, “Often, when projects are planned, too little attention is paid to evaluation. Innovations may be implemented without plans or procedures for formally evaluating the program” (p. 28). It is only after the courses have been offered that those involved in course offering are faced with the task of reporting the success or otherwise implementation of the course plan, and it is then that they require some kind of feedback which they could present as evidence of the success of the course if it is so, or to base their future plan upon it if it has been reported as unsuccessful. The evaluation must aim at achieving certain objectives. Campbell (1998) and Magdziarz et al. (2006), as cited by Dabob (2008:17), put forward six objectives of course evaluation. These are as follows: i. For the purpose of promoting an effective and adequate classes/curriculum by improving the materials, media and tools. ii. To be able to determine if there is justification to continue, extend, minimize, replace or stop the course based on graduate placement iii. To determine if the course was successful in terms of the relationship of the instructional content to the class needs, lecture-performance, students’ achievement, and the application of obtained knowledge and careers on the job in a real-life situation. iv. To fulfil licensing authentication and official approval requirements. The Role of Evaluation Evaluation of a course is one of the important pedagogical practices. It helps inform the administration about the response the course receives from the students and the teachers and help devise strategies accordingly. The decisions about the course contents, structure and revision policies must be based upon some kind of information provided to the decision-makers by the stakeholders, and they must not take these decisions on the basis of their subjective judgment and understanding, which may not necessarily be appropriate in all cases. Diamond & Sudweeks (1980: 28) suggest that “Without the information base that formal, systematic evaluation provides, decisions about content, structure, and revision must be based on hunches, personal preferences, or tradition”. Diamond & Sudweeks (1980) further elaborate upon their point and suggest that evaluation plays a dual role in the process of course offering. The first stage helps in the initial phase of designing the course/programme and layout the detailed plan in terms of what goals to achieve, what content to include and what structure to be given. Vol. IV, No. II (Spring 2019) Page | 549 Ubaidullah Khan, Shaista Zeb and Muhammad Uzair They suggest that “First, it provides information essential to those involved in designing the goals, content, and structure of the program (information about the student, the discipline, the profession-in job-focused pro-gramsthe institutional priorities, etc.) (p.28). The second role of course evaluation is that when the course has been designed and offered, it helps get feedback about what is and is not working in terms of achieving goals set at the time of designing the course. As they further opine, “Second, once the program is underway, evaluation provides information about what is happening, what is working, and what isn't (p.28). Course Evaluation in Distance Learning System (DLS) Although distance education is almost as effective as traditional one (Hanson and Maushak, 1996), the programme or course evaluation in this mode of teaching needs to be done as it determines and adds to the worth of this system (Simonson, 1997, p. 88). The evaluation process must keep the uniqueness of the system in mind while devising a method of evaluation. Simonson gives an example of programme evaluation at Open University, the UK, where it is taken into account curriculum and teaching method. Citing Woodley and Kirkwood (1986), Simonson points out that for evaluation in distance education, OU has merged the traditional approach, which makes use of procedures of physical sciences to evaluation and a more eclectic approach incorporating qualitative and naturalistic techniques. The traditional approach involves an experiment to measure the effectiveness of distance education. It is a carefully structured project with the requirements of the evaluator in mind. In this approach, the samples are carefully matched, controls are established, and the variables to be compared are selected. The objective tests are constructed for the variables, and the data is collected before, during and after the instructional event. (p. 88-89). As a countermovement to this method, “evaluation activities are incorporating more naturalistic methodologies with holistic perspectives. This second perspective for evaluation uses focus groups, interviews, observations, and journals to collect evaluation information in order to obtain a rich and colourful understanding of events” (House, 1986 as cited by Simonson, p. 89). Simonson sums up that now both qualitative and quantitative measures are being used because we need to quantify as well as “need to understand opinions and hear perspectives” (p. 89). Research Methodology This paper presents the quantitative part of the study, which also included semi-structured interviews. The qualitative part, however, is the subject of another paper based on this study. This paper carries out a statistical analysis of the students’ responses to the questionnaire about the three areas of the course under investigation, that is, course content/material, supplementary material, and tutorial support. The study is descriptive evaluation research as it takes reflective feedback of the students about the course they studied in the penultimate semester. The responses to the questionnaire administered to the students have been presented in the form of graphs and charts as well as descriptively. Research Tool Questionnaire A questionnaire was designed in order to get the students’ feedback. The questions comprised three main areas of the course. These areas are the following: • Course Content/material • Supplementary Material • Tutorial Support The respondents were asked to provide demographic information, that is, name, age, gender, educational qualification, teaching experience, if any, and then they were asked to provide email and cell numbers which were optional. Twenty-one (21) questions were asked about the above mentioned three areas. Population & Sample The population of the study was Diploma TFEL students enrolled with the Department of English, AIOU, Islamabad. They are normally aspiring / English language teachers of different levels with different educational Page | 550 Global Regional Review (GRR) Students Perceptions of Effectiveness of Course Contents, Supplementary Material and Tutorial Support in Phonology Course offered By Allama Iqbal Open University, Islamabad backgrounds and experiences and have a good insight into curriculum-related affairs. The sample included all the students who belonged to Islamabad city and district. Data Analysis The statistical data based on the responses of the questionnaire has been presented in the analysis below. The first step presents the numbers of responses to each of the response types in the form of a table. The highest number has been highlighted in the table for the convenience of analysis as well as reading. In the second step, the percentage of responses to each question has been shown with the help of bar graphs as well as explained descriptively. The following key explains the abbreviations. Key: SA: Strongly Agree A: Agree NI: No Idea, DA: Disagree SDA: Strongly Disagree NR: No Response Students’ Responses to the Questionnaire: Part One Following is the summary of the responses to the questions that the students were asked about the contents of the course. The questionnaire consisted of three parts. First one is as follows: Table 1. Section No. 1: The course Contents/Material S. No 1 Statements Course material is sufficient to meet my learning needs. 2 The Material is up-to-date and provides an elaborate study of the latest trends. 3 4 5 6 7 Phonetic symbols used in the book are clear. Phonetic symbols used in the book are up-to-date. The textbook needs to be revised and updated. The scope of researchable topics in Phonology needs to be expanded. The course material is satisfactory and does not need any improvement. SA 0 A 11 NI 5 DA 6 SDA 1 NR 1 0 8 5 7 4 0 5 4 12 9 8 4 1 3 7 5 4 0 3 2 0 1 3 1 10 8 3 3 0 0 0 1 3 9 11 0 The second part of the questionnaire was about supplementary material provided by the department alongside the coursebook for the course under investigation. The following table presents the responses of the students to the questions asked about this topic. Table 2. Section No. 2. Supplementary Material S. No 8 9 10 11 12 13 14 Statements The supplementary reading material is relevant to the content of the text. Supplementary Reading Material is helpful in preparing assignments/research projects. The audio aid provides a good model of standard pronunciation. The audio aid helps understand complicated points of the English language clearly. The quality of printed material and audio aid is satisfactory. Visual aid needs to be provided along with the textbook. Print material and audio aid should be replaced with CD SA A NI DA SDA NR 1 11 5 4 1 2 6 6 3 6 2 1 2 10 6 4 1 1 3 11 4 3 3 0 0 11 10 6 11 6 0 1 4 12 0 4 4 0 0 2 1 0 The third part of the questionnaire consisted of the questions related to tutorial support that the university offers to the students. Vol. IV, No. II (Spring 2019) Page | 551 Ubaidullah Khan, Shaista Zeb and Muhammad Uzair Table 3. Section No. 3: Tutorial Support S. No 15 16 17 18 19 20 21 Statements Tutorial support is effective in tackling the problems faced during the study. The number of tutorials should be increased. Attending the classes should be compulsory to ensure that the learning process is effective and smooth. The number of tutorials for Phonology should be decreased. There should be no compulsory classes or research assignments and presentations in the Phonology course. The number of evaluation components (theory-based assignments, research projects, presentations and written exams) should be decreased. The tutorial support in its current form should be retained, and there is no need for improvement SA A NI DA SDA 3 10 1 6 4 13 7 1 3 0 12 9 1 2 0 1 3 3 7 10 1 6 2 10 5 0 3 0 9 12 1 0 3 10 10 In the following sections, the responses have been analysed one by one to see how the respondents have responded to each question. Each question has been analysed in terms of percentage (shown in bar graphs), the actual number (shown in tables) as well as descriptively. Section 1 (The Course Contents) The first section included seven different questions about the course contents. We asked the respondents opinions about the course content being relevant and up-to-date and tried to get feedback about the phonetic symbols used in the book. Following is one by one analysis of the responses of the participants to each of the questions: Section1: Course Contents The first section of the questionnaire consisted of statements related to the course content/material. SA Course material is sufficient to meet my learning needs. 0% A NI 45.83% 20.83% DA SDA NR 25% 4.16% 4.16% Graph 1: Statement 1 In response to the first statement, 11 (45.83%) students agreed that the course material was sufficient to meet their learning needs. Six (29.16 %) students disagreed with the statement, and one (4.16 %) strongly disagreed and as many did not respond, whereas 5 (20.38 %) chose to stay neutral. A considerable number of students (5 + 1 out of 24) were unclear about their answers, whereas 33% thought that course material was satisfactory for their needs. SA The material is up-to-date and provides elaborate study of latest trends. 0% A NI 33.33% 20.38% DA 29% SDA NR 16.66% 0.00% Graph 2: Statement 2 Page | 552 Global Regional Review (GRR) Students Perceptions of Effectiveness of Course Contents, Supplementary Material and Tutorial Support in Phonology Course offered By Allama Iqbal Open University, Islamabad Regarding the second statement,8 (33.33 %) students said that the material is up-to-date and provides elaborate study of the latest trends, 4 or 29% disagreed, and 16.66 % students strongly disagreed with the statement, whereas 5 (20.33 %) did not have an idea. 40 37.5 35 30 Strongly Agree 25 20.38 Agree 20.38 20 No Idea 15 Disagree 12.5 Strongly Disagree 10 No response 4.16 5 4.16 0 Phonetic symbols used in the book are clear. Graph 3: Statement 3 The third question aimed to elicit their response regarding the issue of Phonetic symbols, whether the symbols used in the book are clear to them or not. Most of the respondents agreed. Five (20.38%) strongly agreed and 9, which makes 37.5 % of the total number, agreed, whereas 20.38 % disagreed and another 12.5 % strongly disagreed. One student (4.16 %) said no idea, whereas the same number of respondents did not respond to this statement. 33.33 35 30 25 Strongly Agree Agree 20 16.66 15 16.66 No Idea 12.5 12.5 Disagree Strongly Disagree 8.33 10 No response 5 0 Phonetic symbols used in the book are up-to-date. Graph 4: Statement 4 The fourth statement asked whether the phonetic symbols used in the book are up-to-date. 16.66 % strongly agreed. 33.33 % agreed, 16.66 % disagreed, and 8.33 % disagreed, whereas 3 respondents said no idea, and many did not respond to this statement. That means that just above 50 % agreed that the phonetic symbols are up-to-date. This trend indicates the non-satisfaction regarding the symbols found among half of the respondents. SA The text book needs to be revised and updated. 50% A NI 16.66% 29.16% DA SDA NR 0% 0.00% 4.16% Graph 5: Statement 5 Vol. IV, No. II (Spring 2019) Page | 553 Ubaidullah Khan, Shaista Zeb and Muhammad Uzair The fifth statement in the questionnaire was that the textbook needs to be revised and updated. In response to this statement, 50 % strongly agreed, 29 % agreed, and 13.33 % said that they had no idea. No one disagreed or strongly disagreed, which means that majority of the respondents, and that is, about 80 % were of the view that it should be revised. 45 41.66 40 33.33 35 Strongly Agree 30 Agree 25 No Idea 20 Disagree 15 12.5 12.5 Strongly Disagree 10 No response 5 0 The scope of researchable topics in Phonology needs to be expanded. Graph 6: Statement 6 This question concerned the topics of research that are included in the second assignment of the course. 10 of the total 24 respondents (41.66%) strongly agreed and said that their number needs to be increased and the scope of the topics may be expanded. Another 8 (33.33%) agreed, making the total vote in favour of this statement 75 %. Three (3) of the respondents said they had no idea, and as many disagreed. This means that the majority of students wanted that more topics may be included in the assignments than the ones that were already included. Graph 7: Statement 7 The last statement of the first part of the questionnaire stated that the course material is satisfactory and does not need any improvement. A majority of the respondents disagreed with the idea (37.5% disagreed, and another 45.3 % strongly disagreed), implying that the course does need improvement. Section 2 (Supplementary Material) The second part of the questionnaire was about the supplementary material that is provided with the course textbook. 50 45.83 45 40 35 Strongly Agree 30 Agree 25 No Idea 20.38 20 Disagree 16.66 15 Strongly Disagree 8.33 10 5 4.16 No response 4.16 0 The supplementary reading material is relevant to the content of the text. Graph 8: Statement 8 Page | 554 Global Regional Review (GRR) Students Perceptions of Effectiveness of Course Contents, Supplementary Material and Tutorial Support in Phonology Course offered By Allama Iqbal Open University, Islamabad The first question asked whether the supplementary material was relevant to the course contents or not. 4.16 %, that is, 1 student strongly agreed, 45.83 % agreed, 20.38 % said they had no idea. Whereas 4 (16.66%) disagreed, 1 (4.16%) strongly disagreed, and another 2 (8.33 %) did not respond to this statement. 30 25 25 25 25 20 Strongly Agree Agree 15 12.5 No Idea Disagree 8.33 10 Strongly Disagree 4.16 5 No response 0 Supplementary Reading Material is helpful in preparing assignments/research projects. Graph 9: Statement 9. The opinion of the students has generally been divided in response to the statement that the supplementary material helped in preparing the assignments or research project. 25 % each strongly agreed and agreed, whereas as much disagreed and another 8.33 % strongly disagreed. 12.5 % said they had no idea and another 4.16 % chose not to respond to the statement. 45 41.66 40 35 Strongly Agree 30 25 Agree 25 No Idea 20 16.66 Disagree 15 Strongly Disagree 8.33 10 No response 4.16 5 4.16 0 The audio aid provides a good model of standard pronunciation. Graph 10: Statement 10 When asked whether audio aid provides a good model of standard pronunciation, 2 out of 24, that is, 8.33 % of respondents strongly agreed, whereas another 10 (45.66 %) agreed. 25 % said they had no idea, and 4 (16.66 %) disagreed, whereas one strongly disagreed and another one chose to not respond. The overall impression from the feedback that emerges is that not all the students are satisfied with the standard of the pronunciation. 50 45.83 45 40 35 Strongly Agree 30 Agree 25 No Idea 20 15 Disagree 16.66 12.5 12.5 12.5 Strongly Disagree No response 10 5 0 The audio aid helps understand complicated points of English language clearly. Graph 11: Statement 11 Vol. IV, No. II (Spring 2019) Page | 555 Ubaidullah Khan, Shaista Zeb and Muhammad Uzair To the question of audio aid helping the complicated points of English language clearly, 3 (12.5%) of the total number of students strongly agreed, 11 (45.83%) agreed, 4 (16.66%) said they had no idea, 3 (12.5%) disagreed and another three strongly disagreed. The overall impression is that the majority views audio aid as helpful for understanding the complicated points in the textbook, which it is based on. 60 50 50 Strongly Agree 40 Agree 30 No Idea 25 Disagree 20 16.66 Strongly Disagree 8.33 10 No response 0 The quality of printed material and audio aid is satisfactory. Graph 12: Statement 12 Only 25 % agreed with the view that the printed material and the recorded text is satisfactory. 50 % disagreed, and added to this are another 4 who disagreed, taking the total percentage of disagreeing respondents to 66.66 %. No one among the participants of the study was indecisive except two who chose not to respond to the statement. No one said they had no idea, nor did someone say that they strongly agreed. 50 45.83 45.83 45 40 35 Strongly Agree 30 Agree 25 No Idea 20 Disagree 15 Strongly Disagree No response 10 4.16 5 4.16 0 Visual aid needs to be provided along with the text book. Graph 13: Statement 13 A majority seemed to demand that the visual aid needs to be provided with the textbooks. This suggests that there is not enough visual support provided with the textbook, and more visual explanations of the sounds (e.g. place and manner of articulation) need to be made part of the textbook. 45.83 % strongly agreed, and as many agreed, making the total number of respondents desiring the inclusion of visual support in the textbook to about 92 %. 45 41.66 40 35 Strongly Agree 30 25 Agree 25 No Idea 20 16.66 16.66 15 Disagree Strongly Disagree 10 No response 5 0 Print material and audio aid should be replaced with CD Graph 14: Statement 14 Page | 556 Global Regional Review (GRR) Students Perceptions of Effectiveness of Course Contents, Supplementary Material and Tutorial Support in Phonology Course offered By Allama Iqbal Open University, Islamabad Out of the total number of 24 respondents, 41.66 % were of the view that the print material and the audio aid can be replaced with the CD. Another 25 % agreed. Only 16.66 disagreed, and as many said, they had no idea. No one strongly disagreed, and no one abstained from responding. This clearly indicates the shifting trend of the students who want more digital-based academic data than the traditional material provided in the form of hard books and audio cassettes. Section 3 (Tutorial Support) Tutorial support is the third area about which the participants were asked questions. As the course is offered in a distance learning system, the students are required to attend the tutorials/ compulsory classes for each course. Due to the demanding nature of the course, Phonology has 18 classes, which is eight more than other courses. The analysis of the questions and the responses is as follows: Tutorial support provided by the university is effective in tackling the problems faced during the study Tutorial support provided by the university is effective in tackling the problems faced during the study. , Agree, 42% Tutorial support provided by the university is effective in tackling the problems faced during the study. , Strongly Agree, 13% Tutorial support provided by the university is effective in tackling the problems faced during the study. , Disagree, 25% Tutorial support provided by the university is effective in tackling the problems faced during the study. , Strongly Disagree, 16.67% Tutorial support provided by the university is effective in tackling the problems faced during the study. , No Idea, 4.17% Graph 15: statement 15 Vol. IV, No. II (Spring 2019) Page | 557 Ubaidullah Khan, Shaista Zeb and Muhammad Uzair As can be seen in the table and the chart, the opinion of the students regarding the tutorial support being effective in dealing with the problem faced during the study was mostly in favour of the statement. 3 strongly agreed, 10 agreed, one said no idea, 6 disagreed, and 4 strongly disagreed. The number of tutorials should be increased. , The number of tutorials should be increased. Strongly Agree, 54.17% The number of tutorials should be increased. , Agree, 29.17% The number of tutorials should be increased. , Disagree, 12.50% The number of tutorials should be increased. , No Idea, 4% The number of tutorials should be increased. , Strongly Disagree, 0.00% Graph 16: statement 16 A majority of the respondents agreed with the idea of increasing the classes for the Phonology course. 54.17 % strongly agreed, whereas another 29.17 % agreed. That means a total of about 84 % were in favour of increasing the number of classes. 12.5 % disagreed, whereas no one strongly disagreed, and one said they had no idea. Due to the demanding nature of the course, they need more classes. Page | 558 Global Regional Review (GRR) Students Perceptions of Effectiveness of Course Contents, Supplementary Material and Tutorial Support in Phonology Course offered By Allama Iqbal Open University, Islamabad Attending the classes should be compulsory to ensure that the learning process is effective and smooth. 60.00% 50.00% 40.00% 30.00% 20.00% 10.00% 0.00% SA A NI DA SDA Graph 17: statement 17 A total of 88 % of students were in favour of the view that the classes should be compulsory. 50 % (of the total number) out of those who were in favour strongly agreed, whereas 37.50 % agreed. No one disagreed or strongly disagreed. Only one respondent was unclear about his / her response. This means that a vast majority thought that it was essential to making the classes compulsory so that the process of learning is effective and smooth. The number for Phonology Course should be decreased. 45.00% 40.00% 35.00% 30.00% 25.00% 20.00% 15.00% 10.00% 5.00% 0.00% SA A NI DA SDA Graph 18: Statement 18 In order to counter check the response of the participants of the study to the question of increasing the number of classes, the researcher asked whether the classes should be decreased. The response reaffirmed the stance the participants took in question number 16, in which they had said that the number of classes should be increased. Total number of 70 % disagreed with the idea of decreasing the classes (29 % disagreed and 41.67 % strongly disagreed). However, 8 % strongly agreed, and 13 % agreed, whereas many were indecisive. There should be no compulsory classes or research assignment and presentation in Phonlogy Course. 45.00% 40.00% 35.00% 30.00% 25.00% 20.00% 15.00% 10.00% 5.00% 0.00% SA A NI DA SDA Graph 19: Statement 19 Vol. IV, No. II (Spring 2019) Page | 559 Ubaidullah Khan, Shaista Zeb and Muhammad Uzair The opinion of the students regarding this matter was largely divided. Whereas 25 % said that they agreed, 29 % disagreed, and another 21 % strongly disagreed. 8 % said that they were undecided. The general picture suggests that they did not approve of the idea of reducing the classes and compulsory assignments and presentations. This trend is consistent with the responses to the previous questions in which they were asked about whether to make the classes compulsory (to which they had strongly agreed). The number of assessment components should be decreased. 40.00% 35.00% 30.00% 25.00% 20.00% 15.00% 10.00% 5.00% 0.00% SA A NI DA SDA Graph 20: Statement 20 This question is consistent with the previous question. The response to this question is similar as was in the previous one, which means that they were in favour of retaining the evaluation system as it is, in which they are asked to submit compulsory assignments and give compulsory presentations. About 90 % of students disagreed with the idea of reducing the number of components in which the students are assessed in terms of their performance. The tutorial support in its current form should be retained as there is no need of improvement. 45.00% 40.00% 35.00% 30.00% 25.00% 20.00% 15.00% 10.00% 5.00% 0.00% SA A NI DA SDA Graph 21: Statement 21 To the question of retaining the tutorial support in its current form, most of the participants disagreed. 42 % strongly opposed the idea, whereas 29 % generally opposed it. Only one person strongly agreed, whereas 3 respondents or 12.5 %, though they had no idea. This concludes the section in which structured questions were asked. Let us now discuss the important findings of the study, and then in the subsequent section, recommendations based on the findings will be given. Page | 560 Global Regional Review (GRR) Students Perceptions of Effectiveness of Course Contents, Supplementary Material and Tutorial Support in Phonology Course offered By Allama Iqbal Open University, Islamabad Findings The important findings of the study based upon responses of the students to the questionnaire are as follows. Section 1: Course Contents i. ii. iii. iv. v. vi. vii. 46 % of students felt that course material was sufficient to meet their learning needs, whereas 43 % had a different opinion. Almost 43 % of students wanted the course material updated. 58 % of students felt that phonetic symbols are clear, whereas about 30 % disagreed. About 50% of students felt that the phonetic symbols were up-to-date, whereas 17 % of students wanted them to be revised and updated. 80 % of students wanted the book to be revised (All of them strongly agreed). About 75 % of students wanted the scope of researchable topics to be expanded. 37 % of students disagreed with the statement that the course material was satisfactory and met the learning needs. Section 2: Supplementary Material viii. ix. x. xi. xii. xiii. xiv. Almost 50 % of students agreed that the supplementary reading material was relevant to the content of the text, whereas 21 % disagreed. 50 % of students agreed and strongly agreed that it was helpful in preparing assignments/research projects, whereas 50 % of teachers disagreed with the suggestion. A little less than 50 % of students and 37 % of teachers felt that the audio aid provided a good model of standard pronunciation, compared to 20 % who disagreed. 58 % of students and 37 % of teachers felt that the audio aid helped understand complicated points of the English language clearly, compared to 25 % of students and 37 % of teachers who were against this opinion. 67 % of students and overall of 66.66 % teachers went against the statement that the quality of printed material and audio aid is satisfactory, meaning thereby that it needs to be improved. Only 25 % of students and 16.66 % of teachers thought that it was satisfactory. 92 % (compared to 0 %) students and 100 % of teachers asserted that Visual aid needs to be provided along with the textbook. 66 % of students (compared to 17 %) and 84 % of teachers favoured the idea of replacing the audio cassette with the CD. Section 3: Tutorial Support xv. xvi. xvii. xviii. xix. xx. xxi. 54 % of students felt that the tutorial support was effective, whereas 42 % of students disagreed. About 84 % of students felt that the number of tutorials should be increased. 88 % of students favoured the idea that classes should be compulsory. About 70 % of students opposed the statement that the number of tutorials should be decreased, hence supporting /reaffirming the demand made in number xvi. 62 % of students opposed the idea of not having compulsory classes. 88 % of students opposed the idea of decreasing the number of components upon which assessment is done. 84 % of students disagreed with the idea that tutorial support should be retained in its current form. Vol. IV, No. II (Spring 2019) Page | 561 Ubaidullah Khan, Shaista Zeb and Muhammad Uzair Recommendations In the light of the findings as presented in the last section, here are certain recommendations which I feel must be taken seriously in order to make the course effective to achieve the purpose of imparting real learning to the students. As discussed in the Literature review, any evaluation study is useful only when some serious corrective measures are taken as a result. i. The coursebook may be revised, and the latest theories on the subject of phonology may be included to give the students exposure to up-to-date knowledge of the field. ii. Special care may be taken in the use of phonetic symbols while writing the new course so that confusion on the part of the students could be avoided. iii. The supplementary reading material may be selected more carefully, and it may be in properly organized form so as it does not to appear to be irrelevant or too much to read. iv. The quality of the supplementary reading material may be taken special care of, and better paper, preferably colour paper, may be used to print it on. v. The audio aid should be re-recorded in standard pronunciation so that the students could fully benefit from it to increase their understanding of Standard English pronunciation vi. A due consideration may be given to the demand of increasing the number of classes for the Phonology course. vii. The scope of researchable topics may be increased, and more topics may be included in the assignments to avoid repetition of the few topics that appear again and again. viii. The cassette provided to the students along with the textbook as audio aid may be replaced with the Compact Disk / CD. ix. The coursebook may be accompanied by Visual aid so as to enable the students to better understand the intricate points of the course. Suggestions for Future Researchers The present study was based on the issues that surface in day to day talk about the course. The researcher being a faculty member as well as a tutor and coordinator of the course, was aware of these issues that were the topic of continuous debate, and that is why it was decided to take students and teachers’ opinions on these issues. The questionnaire was designed with the view to take feedback on these oft-debated issues, that is why it is different from the normal evaluation questionnaires and did not follow any real theoretical guidelines. However, future researchers may go deep into the course evaluation and carry out further studies which may be specifically focused on the course contents, their effectiveness, the inclusion of activities in the book and their understandability. A separate study may also be carried out to look into the questions that have not been asked so far regarding the areas discussed in the questionnaire, or the questions that arise from the feedback of the students itself. Conclusion There is an utmost need to carry out such studies and the process of revision of all Diploma TEFL courses needs to be started with fullest possible commitment and zeal, so that the decades old courses may be replaced with new ones and the students could get the best compensation for their / their parents’ hard earned money. Course evaluation, as has been already suggested, must aim to bring improvement to the courses through concrete corrective steps so that the students as well as other stakeholders such as parents and teachers may be satisfied with the teaching-learning process. It is, therefore, hoped that this study will be beneficial for the purpose of bringing improvement to the course in particular, and to the teaching-learning situation in the Distance Learning System in general. Page | 562 Global Regional Review (GRR) Students Perceptions of Effectiveness of Course Contents, Supplementary Material and Tutorial Support in Phonology Course offered By Allama Iqbal Open University, Islamabad References Achtemeier, S. D. et al. (2003). Considerations for developing evaluations of online courses. JALN , 7(I), 1-13. https://olj.onlinelearningconsortium.org/index.php/olj/article/download/1859/690 Dabob, K. M. A. (2008). Custom online course evaluation for IT master students at UUM. Univesrity Utara Malaysia. http://etd.uum.edu.my/727/ Diamond, R. M. & Sudweeks, R. R. (1980). Comprehensive approach to course evaluation. Journal of Instructional Development. 4(1), 28-34 Published by: Springer http://www.jstor.org/stable/30220636. Gravestock, P. & Gregor-Greenleef, E. (2008). Student course evaluations: Research, models and trends. Higher Education Quality Council of Ontario, Toronto. Canada Hanson, D. & Maushak, N. (1996). Distance education: Review of literature. Research Institute for Studies in Education. Ames, Iowa. (pp. 87-94) Henckell, M. M. (2007). Evaluating distance education: The student perspective. A Dissertation Presented to the Faculty of the Graduate School at the University of Missouri—Columbia Kirkpatrick, A. (2007). World Englishes: Implications for international communication and English language teaching. Cambridge University Press, Cambridge, New York, Melbourne, Madrid, Cape Town, Singapore, São Paulo Mollin, S. (2012), English as a lingua franca: A New variety in the new expanding circle?. http://gupea.ub.gu.se/bitstream/2077/3149/1/5-2-Mollin.pdf on 12-04-2012 Pakir, A. (2009). English as a lingua franca: Analyzing research frameworks in international English, world Englishes, and ELF. World Englishes, 28(2), 224–235. https://doi.org/10.1111/j.1467971X.2009.01585.x Quirk & Widdowson, H. G. (1985). English in the world. Cambridge University Press Shameem, F. (2011). English as the language for development in Pakistan: Issues, challenges and possible solutions. Developing English in development Context. British Council Simonson, M. R. (1997). Evaluating Teaching and Learning at a Distance. New Directions for Teaching and Learning, 1997(71), 87–94. https://doi.org/10.1002/tl.7111 Stevans, P. (1977). New orientations in teaching of English. Oxford University Press. London Vol. IV, No. II (Spring 2019) Page | 563