STUDENT AND TEACHER PERSPECTIVES ON FACTORS TO SUCCESS AND ATTRITION IN ENGINEERING MINI-MOOC USING TECHNOLOGY ACCEPTANCE MODEL (TAM)

Purpose: To keep abreast of the latest educational development and to enrich student learning. Vocational Training Council (VTC) is among the first vocational institutions in Hong Kong for piloting massive open online courses (MOOCs) (which are being referred as mini-MOOCs here) on various disciplines. This paper examines one of the miniMOOCs we have developed for the Engineering discipline. The aim of this study is to gather information and investigate the influencing factors that contribute to the success and attrition in the Engineering mini-MOOC. Methodology: These insights come from the empirical study conducted in a mini-MOOC titled “Unauthorized Buildings Works” (UBWs), which was being implemented as a blended activity in the module. Two groups of second-year students from the higher diploma in construction were selected to participate in non-MOOC (i.e., the control group) and MOOC (i.e., the experimental group) classes over two years. The mini-MOOC subjects were invited to complete questionnaires and attend a focus group interview upon completion of the activity. Main Findings: The analysis yielded striking results, which led to a discussion on the contributing factors to success and attrition in the Engineering mini-MOOC using Technology Acceptance Model (TAM). Implications: Due to limited resources in the research, the methods of data collection were different over two years of study. An implication of this is that an enhanced methodology is needed to determine the academic effectiveness of the mini-MOOC and achieve the consistency of the findings. Novelty/Originality: The present study contributes to significant findings in how TAM factors and external variables influenced the usage of mini-MOOC from the perspectives of both learners and teachers.


BACKGROUND / OBJECTIVES AND GOALS
The rapid growth of technology has dramatically transformed the lives of individuals in the society in the last three decadesfrom having only a landline phone in a household to owning one or multiple cellphones, from the first IBM Personal Computer (PC) with a bulky monitor to a laptop computer or to a tablet that can be carried on daily commutes, and so on. Not only does technology have a huge impact on the 21st century living, but it also plays an important role in teaching and learning. Teachers are no longer confined to the brick and mortar walls of a classroom, learning can take place in an online environment with a great amount of flexibility to study however, wherever, and whenever the learners want. In recent years, Massive Open Online Courses (MOOCs) have attracted millions of learners around the globe to enroll courses from renowned universities, such as Harvard University, Massachusetts Institute of Technology (MIT), etc., through various MOOC platforms, such as edX, Coursera, etc. These MOOCs are generally open to public and the entire learning takes place online. Certificates are rewarded to those who successfully complete the course.
On the back of the emerging trend of MOOCs, Vocational Training Council (VTC), the leading vocational and professional education and training provider in Hong Kong, initiated a three-year project to develop MOOCs for current higher diploma students in various programs as a pilot study. This paper primarily focuses on one of the MOOCs offered to the second year students in the construction program of the Engineering discipline. The MOOCs in the VTC context are different from the aforementioned ones where the whole course is online and has an open access and is offered as a blended activity in the module to those who are currently enrolled in the program, hence the term 'mini-MOOCs'. The main objective of this study is to gather information and investigate the influencing factors that contribute to the success and attrition in the Engineering mini-MOOC. These insights come from the empirical study conducted in the mini-MOOC titled "Unauthorized Buildings Works" (UBWs) over two years from 2016-17 to 2017-18.

LITERATURE REVIEW
In light of the dynamic growth of technologies, many researchers were interested in finding how users accept new technologies and how likely these technologies will be implemented. Myriad theories on technology adoption models, such as the Theory of Diffusion of Innovations (DIT) (Rogers, 1995), the Theory of Reasonable Action (TRA) (Fishbein and Ajzen, 1975), Theory of Planned Behavior (TPB) (Ajzen, 1985(Ajzen, , 1991, Decomposed Theory of Planned Behavior, (Taylor and Todd, 1995), the Technology Acceptance Model (TAM) (Davis, Bagozzi and Warshaw, 1989), Technology Acceptance Model 2 (TAM2) (Venkatesh and Davis, 2000), and Technology Acceptance Model 3 (TAM3) (Venkatesh and Bala, 2008) have been proposed. What these theories and models have in common is that they are proposed in an attempt to address various factors on how users embrace novel technologies. The actual usage of a new technology can be explained by their behavior, intention, and other variables. In this paper, we will focus only on the Technology Acceptance Model (TAM) (Davis, Bagozzi, and Warshaw, 1989) to explain how the model is applied to examine factors to the success and attrition in the Engineering mini-MOOC.
TAM is usually employed to predict user acceptance and adoption of a novel technology; it is also used to help researchers to identify the determinants as to why a particular technology is acceptable or unacceptable by users. Figure  1 depicts the TAM model.  (Davis, Bagozzi and Warshaw, 1989) A number of factors are examined to evaluate the acceptance of a certain system. The actual use of any kind of technology is driven by the user's behavioral intention to use the technology, which is then influenced by the user's attitude towards its usability and its usefulness as perceived by the users. The attitude is linked to two beliefs: Perceived Usefulness (PU) and Perceived Ease of Use (PEOU). "Perceived Usefulness is defined as the potential user's subjective likelihood that the use of a certain system […] improve his/her action and Perceived Ease of Use refers to the degree to which the potential user expects the target system to be effortless" (Davis, 1989). In other words, the usefulness of a technology can have an impact on the user's intention to use the technology. If the system is considered user-friendly and easy to use, it can lead to a positive attitude towards its use. The new system may consequently be perceived as useful. But if the system is difficult to use, then it increases the barrier of using the technology and may cause a negative attitude towards it, which then leads to a decrease in the actual usage of the technology. The belief towards the technology can be influenced by other determinants, such as external variables in TAM.
Owing to the user-friendliness of the MOOC platform, the PEOU is predicted to be positive for teachers and students. Computers and mobile devices are easily accessible by both teachers and students, and 21st century learners are accustomed to seek and acquire new information through digital devices, which is then possessed and transformed into knowledge. It is expected that the PU will be positive, since users are already familiar with the digital way of learning. Other external variables, such as institutional readiness and teaching styles, are predicted to be the determinants to engender an influence of the actual usage of mini-MOOC.

METHODS
Due to limited resources, such as the production time of the mini-MOOC and the number of students enrolled in the program, the methods of collecting data and the sample sizes were different in this ongoing pilot study over two years. In 2016-17, all second year students enrolled into the module (i.e., n = 118) were selected as an experimental group. In order to examine the effectiveness of the mini-MOOC, the academic results of all second year students, who were enrolled in the same module with a traditional teaching approach in 2014-15 (i.e., n = 126), were chosen as a control group. A larger sample size was involved in the first year of the study as a result. However, in 2017-18, two groups of second year students were selected to participate in non-MOOC (control group) and MOOC (experimental group) classes with relatively small sample sizes of 19 in each group. For the non-MOOC class, only the traditional face-to-face teaching was adopted. In the MOOC group, the module involved mostly face-to-face teaching with roughly 10% of online learning as a blended learning that took place in mid-semester. The subjects were asked to watch a series of engineering-related videos (instructed in English with captions) and complete online post-video exercises as a way of assessing their understanding of materials. All mini-MOOC participants were invited to complete questionnaires and attend a focus group interview upon completion of the mini-MOOC exercise. In this study, we are interested in finding whether or not the mini-MOOC can significantly improve the academic performance of students by studying the mean scores of online exercises on the platform. Based on the statistics from the student performance in the mini-MOOC, teacher and student questionnaires, and interviews with students, we scrutinize the influencing factors to the success and attrition of this mode of learning. As all groups were independent of each other, two-sample t-tests were conducted separately for each year to find out whether there is a difference in the mean academic performance of each group at the significance level of 0.05. We postulate the following hypotheses: For 2016/17: Null hypothesis (H0): The mean mark for the "lecture only" and "mini-MOOC" teaching methods is the same Alternative hypothesis (Ha): The mean mark for the "lecture only" and "mini-MOOC" teaching methods is not the same For 2017/18: Null hypothesis (H0): The mean mark for the mini-MOOC and non-MOOC groups is the same Alternative hypothesis (Ha): The mean mark for the mini-MOOC and non-MOOC groups is not the same

FINDINGS
Choosing the significance level (α) of 0.05, two-sample t-tests were performed and calculated by using Excel.

Results in 2016/17
The results for the first year of the study are as follows: A two-tail t-test was performed for the first year of the study. It is found that the t-statistic (i.e. t Stat) equals to -6.70 and the t-critical (i.e. t critical two-tail) are 1.97 and -1.97. If t-statistic < -t-critical or t-statistic > t-critical, we reject the null hypothesis. Because -6.70 < -1.97 and 6.70 > 1.97, we reject the null hypothesis. The infinitesimal P-value of 1.48 × 10-10 is statistically significant at α = 0.05, indicating strong evidence against the null hypothesis. Our statistical analysis shows that P-value ≤ α, the null hypothesis is rejected in favor of the alternative hypothesis. That is, the mean mark for the "lecture only" and "mini-MOOC" teaching methods is different.

Results in 2017/18
The results for the second year of the study are as follows: Similar to the first year of the study, a two-tail t-test was performed. The results show that the t-statistic (i.e. t Stat) is 0.68 and the t-critical (i.e. t critical two-tail) are 2.03 and -2.03. Because -2.03 < 0.68 < 2.03, we fail to reject the null hypothesis. Additionally, the P-value of 0.50 is greater than the significance level (i.e. P-value > α), we fail to reject the null hypothesis and that the result is statistically insignificant. Although our statistical analysis reveals that H0 is not rejected, that does not necessarily indicate the mean mark for MOOC and mini-MOOC is the same. Our findings suggest that we have insufficient evidence to demonstrate a difference between the two teaching methods. It is arguable that the statistical power of this study could be greatly decreased by the much smaller sample size (n = 19) in the second year. However, the size of the sample in the study is directly proportional to the number of students enrolled into the program at the time of the mini-MOOC exercise. This confounding variable cannot be controlled by the researcher but can have an impact on the outcome of the experiment.

Contributing Factors to mini-MOOC Success from Learner's Perspective
Various contributing factors can be taken into account when measuring the success of the mini-MOOC. Hong Kong has an examination-focused and marks-oriented educational system. Therefore, it is tempting to base our conclusion on the effectiveness of the mini-MOOC merely on the academic performances of students. The outcome was then substantially attenuated if only the statistical analyses of the two-sample t-tests were considered. Statistically speaking, we have no evidence to conclude that the mean difference between the mini-MOOC performances for the MOOC and non-MOOC classes is different. Despite that, students by and large expressed positive comments about their perceptions of the mini-MOOC in the post-course surveys as well as focus group interviews. Several dimensions including accessibility, layout and interface, learning content, and outcomes were examined on a six-point Likert scale questionnaire. The results revealed that more than 50% of the mini-MOOC participants (i.e. 63.3% in 2016-17 and 56.1% in 2017-18) were in favor of the mini-MOOC accessibility. Based on the TAM, the PEOU was considered positive by the participants' point of view. In the focus group interview, the subjects stressed that they had experience in learning from online video lectures prior to this study and that they were accustomed to this mode of learning. In addition, the mini-MOOC offered the flexibility to learners and on-the-go learning anytime and anywhere. This is particularly beneficial for those with long commutes and those with long breaks between classes. They can make use of the otherwise wasted time to study the learning materials. Participants also found that the learning content was well-structured (56.6%) and well-explained (58.6%), and their overall knowledge was enriched (53.5%) upon the completion of the mini-MOOC. Therefore, the results suggested that the mini-MOOC participants generally held positive perceptions toward the adoption of MOOC as a learning tool and the PU is regarded as positive in the TAM. This can be explained by the fact that most of the subjects in the study were digital natives, whose age ranged between 16 and 20 years with 62% in 2016-17 and 58% in 2017-18.
There are ample research studies on the characteristics of the digital natives, such as technology infused learning environment, flexible schedule, short attention span, immediate feedback, collaborative learning, active learning, and mobile devices (Sarkar, Ford & Manzo, 2017). Our findings concur with previous studies that a self-paced learning environment that is unconstrained by time and place is favorable by digital natives. Additionally, institutional readiness is considered an external variable in the TAM. As the mini-MOOC exercise is one of the school initiatives, the financial resources, technical support and manpower were already provided for the implementation of the pilot project. On the basis of the above, it can be concluded that the adoption of MOOC was considered successful in terms of learners' readiness to accept the new mode of learning. However, the academic performance failed to reflect the success.

Contributing Factors to mini-MOOC Attrition from Teacher's Perspective
Similar to the student questionnaire, in order to find out the teachers' perception of the implementation of mini-MOOC, teachers who taught the blended version of the module were asked to complete a six-point Likert scale survey evaluating the aspects of general teaching and outcomes, followed by open-ended questions. Three teachers adopted the mini-MOOC in 2016-17, while only one teacher adopted it in 2017-18. The teachers were asked about their views on the accessibility and usefulness of the mini-MOOC. A majority of the teachers were in favor of the use of mini-MOOC, suggesting that both PU and PEOU are positive from the teachers' perspective. However, teachers raised a few concerns about the production and adoption of the mini-MOOC, which are worth delving into in our discussion. External variables, such as video production time, storyboard writing, post-production, language barriers of students, and teaching styles, may cause a negative attitude towards the use of mini-MOOC, reduce the intention to use, and eventually lead to a decrease in the actual usage. It was stressed that the production of a roughly 30-minute video was expensive and timeconsuming. Teachers did not have any experience with the process of video film production from the development of