Measurement of effectiveness of cMOOCs

Here is my response to Christina’s post on difficulties researching cmoocs.

How to measure the effectiveness of a cMOOC?

There are 4 semantic conditions of networks that Stephen Downes has proposed. As Stephen has commented, those properties – openness, diversity, autonomy and connectedness & interactivity is not perfect in cMOOCs. Besides Connectivism as applied in cMOOCs could likely best be based on an informal learning, rather than a traditional institutional model.

I have reiterated that the constraints typically imposed with an institutional model would be huge challenge for administrators and educators to adapt, as is witnessed even in xMOOCs, where a totally new approach (such as flipping the class or flipped learning) as perceived by professors would be at odds with the mass lecture approach typical in mass-education, with a broadcasting model. How to overcome those challenges, and ensure learning is more effective, when cMOOCs are embedded in an institutional model?

Here is my response  that I perceive as a way to measure the effectiveness of cMOOCs – in its

1. awareness of Networked Learning and Connectivism as an “informal learning paradigm”,

2. an adoption and leveraging of the 4 properties- openness, diversity, autonomy and connectedness & interactivity when networking,

3. an achievement of personal goals with immersion in the network and community (and community of practice) on personal basis,

4. adoption of Personal Learning Environment and Network PLE/PLN in pursuit of life-long learning, and

5. a shift of frame of reference and paradigm from knowledge transmission to knowledge sharing and creation model under a knowledge ecology.

John Mak

Social Media, Google + and the Golden Eggs with MOOC

What is social media, Google + and technology’s role in MOOC?

What sort of MOOC model would be useful for online learning?

Is MOOC suited for the education and learning of any subjects and topics?

What would be the future MOOC like?

First, the introduction of Google + has generated lots of discussion, here by George Siemens and here on how to fix Google + circles by Stephen Downes and here about the mother of Google + features.

This led me to think of the goose and golden egg story – The goose that laid the golden eggs. When Google+, FB, Twitter (as a digital goose) is fully leveraged and exploited, the golden eggs could be collected (artifacts and resources – blog posts, videos, articles, conversation, engagement,  etc.). We need to nourish the goose, in order to have healthy golden eggs. But if we are not patient enough and kill the goose (i.e. kill or ban the technology and tools), thinking that what we want are just the golden eggs within then we might not find any eggs inside when the tool is dismantled.

Photo: from wikipedia

So, what do all these mean? The Goose= Google+, FB, Twitter are tools that are part of the Digital Goose.  Learning is that part of the discovery of curated information and distributed knowledge through such a digital goose, where the goose would help us in hatching the eggs.  There is a process involved in this hatching, where thinking and reflection is involved.

How does all these relate to MOOC and the eduMOOC?

Inge de Waard in her recent post on MOOC:

“As educational technology is becoming more mainstream through social media and mobile devices, there is a rising interest to find methodologies that build upon these new technologies to enhance the learning and teaching process. MOOC is one of these emerging formats. A MOOC can boost your institutional, corporate, or NGO knowledge, if you are open to its innovative approach.”

A MOOC is a Massive Open Online Course. It is a gathering of participants, of people willing to jointly exchange information and collaboratively enhance their knowledge.

A useful critique on MOOC.

Here is another MOOC referred to by Stephen in his OLDaily, on e-portfolio. This would surely be another exciting MOOC for educators  and practitioners to consider. It has been put together by a group from the VET E-portfolios Community of Practice (EpCoP) to help vocational education and training (VET) practitioners build their e-portfolio knowledge and skills by developing their own professional e-portfolio.

My previous post with Gordon’s fabulous cow and an adapted version of the original clip art drawing of the duck by one of the CCK participants.

Photo credit: Gordon

Is learning related to the transfer, transmission or replication of information or knowledge in MOOC?  This has been an interesting debate in previous CCKs and there is still no absolute answer to this.  From what I could sense and experience, learning is AN EXPERIENCE WITH THINKING AND REFLECTION and might be embedded in a conversation, an activity, a problem, a project, or reading and commenting.  It may be unintentional as cited by Stephen, especially when learning relates to higher order, critical thinking and reflective learning.  It emerges out of the conversation, and would likely take up a pattern as shown in the COW cartoon above.  It is not easily predictable, as the emerging knowledge would change over time, based on the interaction and engagement amongst the networkers.

George in his post here raised a number of questions on research into MOOC.

The following are some questions George have around MOOCs:

1. What are the habits and patterns of learner self-organization in open online courses?
2. Do learners perform better in open online courses that in traditional courses? By what metrics would we begin to answer that question?
3. Are there any subjects that are not suitable for open online course delivery?
4. How can open online courses be merged with on-campus courses (i.e. blended MOOCs)?
5. What types of skills do educators need to teach MOOCs? How can educators support learner success?
6. What principles or models of instructional design are most effective in developing MOOCs?
7. What types of technologies are needed to a) help learners make sense of complex topics, b) manage information overload, c) maintain instructor presence, d) encourage learner autonomy?
8. How do existing online learning/teaching models relate to MOOCs (i.e. communities of inquiry, communities of practice, connectivism)?

Here is my response to some of his questions:

I came across this post http://www.pontydysgu.org/2011/05/moocs-a-model-for-open-education/ by Attwell referred by Stephen in his OLDaily.

Refer to George’s post on the Mother of MOOC, here are some of my thoughts written near the end of CCK11, where I shared privately with a few others, but haven’t posted them publicly:

In future MOOC, how about?

(a) An orientation session/workshop which would provide an overview on what is involved in the MOOC – the theory and practice part.  An introduction by participants in a fun, enjoyable way.  A one paragraph summary about individuals’ background, needs and expectations.

(b) Development and practice of peer facilitation – allowing participants to be engaged at the start a short MOOC event, with a 1-2 day conference or for the first week, where invited participants would treat the workshops as practice sessions, using Google + Hang out, Elluminate or Skype (for small groups).  This allows participants to “play around with the tools” (the sandpits) without too much worry about their “performance” or “facilitation” skills before an actual final presentation.  The participants would be encouraged to choose their topics, their tools (powerpoints) or videos or podcasts (like voicethreads) and media (Youtube, FB, Twitter, SL, Amplify, Quora) etc.  So this could be a series of workshops with events of short duration, ensuring all people have opportunities in engaging in the activities, which should be built with fun.

(c) Development of mentoring skills – one-on-one or one on many, many on many (peer mentoring) and organisation of learning activities (like b) – provision of a mentoring workshop where every participant could volunteer to become a potential mentor of MOOC (present or future).  In other words, peer mentors could be a good starting point for existing experienced educators to provide support to other less experienced educators or novice learners.  For those people who have less experience in teaching and learning, they could be involved in the program in many ways, like working on small groups, sub-networks, or individual projects and activities of their choice, such as blog post – learning or research, twitter – learning or research, FB – learning or research and exploration and research into various tools and technology – mobile learning, and COPs and research etc.  Conduction of actual interviews with educators, professors, and report on such interviews – via videos, blog posts etc. This could be done in wikis or a forum (a space of their choice).  Some initiation and organisation will be necessary to kick start such projects or activities.

Report on the exploration of some of the media or tools – like Amplify, Diigo, Scribe, Google +, Google Documents, wikis and research articles studied etc. could all be done in wikis, forum, or blog posts sharing.

(d) Development of research skills – this could be integrated with (b) and (c) via a combination of research workshop on Connectivism (Networked Learning) and application (via action research) and presentation of research projects (especially good for those working on PhDs to practise their research writing and trial presentation to a group of audience, and to obtain feedback from such “group” or community).

(e) Development of Network of Practice – in PLE – this could include a short group PLE exercise where participants could work together in various interactive share spaces/media – with digital story, role plays, games etc..  It would be important that participants could choose the media to work on, based on their nominated spaces.

(f) An evaluation session of the course- where the good, bad and ugly could all be discussed, with a SWOT analysis to evaluate the different workshops and forums, blogs openly, connectively and collectively.

(g) A survey research on the course in various forms – small focus groups, Delphi approach – with post it notes on “what and how to improve the course” by the participants and facilitators doing it in a joint function, and a fun way.  Just critique on the issue, not the person.  The current MOOC book initiative could also be a wonderful way to document the research done through the collaboration of various scholars, educators and networkers.

The above are just some of my thoughts (which have now been adapted with updated information) which I think could be useful for a MOOC, though it really depends on who those target educators and learners are, and what they want out of the course.

If it is possible,  why not encourage and support the participants start thinking about design and development of a “mini OOC”, with some of the above elements in mind?  The concept of Connectivism would all be built in if the course could really help the majority of participants to learn more successfully and effectively.

For highly academic scholars, researchers and educators, I reckon MOOC needs to be based on their actual needs.  Most of the participants who have experiences with MOOC know the principles of keeping up-to-date accurate knowledge and information and its importance, and the need to learn with connections with various networks, and the relevant pedagogy in situated learning, based on context.  There is no need to repeat those principles.  Instead, it may be useful to use narrative research to reinforce the applications of some of the principles underlying networked learning, both at work and in study.

So, the actual issues evolved around authority, power and control, an emphasis on openness, a reinforcement of skills or literacy for some less experienced learners, the provision of  guidance for some educators and learners, and an encouragement of autonomy to experiment and teach or learn in a way that best suits them in the MOOC.  This will encourage and support all participants to learn and connect (including all professors and guest speakers to have their involvement in the networks), participate and engage, and having conversations in order to succeed.

Isn’t that the self-organised learning at best?  Or the emergent learning resulting from connective knowledge growth and development in the networks?

This in turn would invert the course design, so that the participants would actually be designing the MOOC, implementing it themselves, and reflecting together on what works and what doesn’t.  Wouldn’t this be more valuable to most educators and learners?  Would this be the biggest challenges for even the most experienced educators and PhDs?

I understand that I have made lots of assumptions here, as learning is complex and all of the above depends on the interaction amongst participants and facilitators in actual case.  However, if we have never tried it, how do we know if it works or not?

In summary, what is more important in MOOC is not just the theory, or the principles as suggested, but the actual projects and community or networks that are created, developed and worked on.  This would take away the often “known” ways of learning with a MOOC.  That is the EMERGENT LEARNING both for the individuals and the networks.  There may be some educators and learners who could feel it too hard to do it in a MOOC, and so instead of doing a whole Connectivism course, why not having it designed in parts, so participants would only choose what they need only?

If we are to ask participants to design courses or sub-networks (with events, workshops, seminars, presentation, activities), then those designs would most likely be refined by the participants, implemented and evaluated more successfully, as they are the master piece of their suggestions, and so learning is built into the design with continuous improvement and review.

There is also an urgent need in gaining a deeper understanding of: Managing uncertainty in social networks as contained in this reference, using social network analysis and learning analytics.

The most successful MOOC that I have witnessed so far seems to be based on the CCK and PLENK model, where structured and unstructured models are blended, and as individuals become active in the conversation and “cross fertilization” in the community of inquiries.  This seems to also come at the intersection of community of practice, landscape of practice, social networks, where the knowledge web, social web, learning web are all important part of the online learning collaboration and cooperation.

I still think the emergent practice based on Cynefin framework would apply to MOOC, where complex situations have always been the most challenging ones for educators to facilitate and steer in the case of huge online course.

This will beg the rhetorical questions: Is MOOC always the most effective way in addressing complex learning situations?  Is Openness at the heart of MOOC?  How would the reality and ideal of MOOC be possibly leveraged under such a learning model?

Photo credit: wikipedia

Dave Cormier shared his views on success on MOOC:

MOOC would likely be more successful if the following conditions are satisfied:

1. The topic of interests offered in MOOC are aligned with those of the target audience.  Such topics need to be “new”, “exciting”, “challenging to some extent” but not overwhelming for most of the participants.  If the topics are related to simple, elementary concepts, then such MOOC may be conducted similar to the traditional online video classroom basis, like the Khan Academy, or what might have been adopted in the high school with a lecturer giving the talk.

Though chaos could be part of the MOOC at various stages of the course, especially during the introduction, a tempering or “intervention” based on curation of information, collaboration and learning clusters formation to share views, feelings, and learning would likely ensure that individual voices are heard, and feedback loop is used to continuously improve and develop the course, through connective networks.

2. The participants are coming from a diverse background (or even from global networks), and that openness, diversity, autonomy, and interactivity and connectivity are encouraged, supported and celebrated, not through a centralized system, but a decentralized network structure based on egalitarian principles.  This would ensure  a healthy growth within and amongst the networkers and networks, which collaborate and cooperate, rather than compete with each others.

3. The MOOC structure needs to be adaptive in nature, and may exhibit the complex adaptive system where the actors and system co-evolve as the course progresses. This means that a breakdown into mini-OOC may be more practical, especially if the interests of the participants are too diverse, leading to fragmentation of MOOC.  Traditional, objective and learning outcomes based online course may need to be changed in order to adapt to a high in flux, highly complex and adaptive sort of MOOC where each participant is developing their own unique PLN and “MOOC” in mind.  This alignment of online course to an emergent structure with MOOC will allow for a decrease in drop out amongst networkers, and an increase in understanding of the netagogy as proposed and problem and project based learning.  It could also be based on lots of fun, as shared by Michael Wesch and his students, producing the artifacts (videos and wikis) under Michael Wesch’s guide on the side when learning in an online environment.

4. That there are open educational resources available and open for access, remix, reuse and repurpose for the creation or feedforwarding of artifacts to the networks, as shared by Stephen Downes.

5. The teaching, social and cognitive presence are all supported throughout the MOOC and beyond.  These could be based on distance education pedagogy.  It would best be based on a learning experience as discussed by Jenny Mackness where the process is open and community based – with an emergent landscape of practice as value proposition and value creation with communities of practice.

This social media and higher education provides a useful insight and models where social media could be considered and used.

Our current eduMOOC is moving towards the 5th week, and I am still reflecting on the design, development and implication of it on elearning.

Finally, may I put these into philosophical propositions?

1. When you don’t see any rigid structure in MOOC, that is good, as MOOC should be personalized, having adaptive and amorphous structures that are all customized to suit the learners, not just the educators needs.

2. When there seems to be a chaotic structure in place, that is good, because such structure would challenge even the most intelligent and talented educators, scholars, professors and learners to sort them out, so everyone has to rethink and reflect about what it means to learn in a chaotic Web and internet based learning environment.  That is the reality that we are facing, in times of flux.

3. Where there are more and more problems emerging out of MOOC design, delivery and development, that is good, because this would give a chance for scholars, researchers, administrators, educators, and learners to change and adapt their teaching and learning, based on a shift in the pedagogy, paradigms. This would challenge each of them to re-think about the importance, significance and implications of online participation (with a participatory culture), collaboration and cooperation, as a network, as a cluster of educators, researchers, and learners throughout the global networks, as an institution, or a partnership of institutional networks.  This would stimulate and promote stakeholders to research, to learn and to improve and innovate altogether,  in order to tackle the challenges ahead of us and that of our next generation.  That is the change and transformation needed to keep abreast of knowledge and learning in an ever changing world.

4. Are we living in an era of disruptive digital media based ecology? The challenge is huge, but the reward is even bigger.  The more we know, the more we know that we don’t know.  And that is learning as growth and development, both individually and as connective and collective wisdom.

This is the time to celebrate the successes and failures, through experimentation, and possible failures of MOOCs, where educators and learners could learn together.  Without trials, we never learn.

Learning Metaphor – Our Digestive System

My philosophy of learning is like the digestive system of our human body.
I digest and assimilate the food (ideas, resources – books, articles, on the net, artifacts, knowledge and information) and absorb those nutrients (which become emergent knowledge) out of it into my body through the blood stream (through plan-do-check-act learning reflection cycle in my connections – nodes and networks).
I will ensure that I take a variety of foods (learning network at neural, conceptual, external – communities, social levels and information sources) to maintain a healthy body and mind.
I will egest any by-products of learning (those obsolete knowledge, SPAMS, distractions, overloading of knowledge and time wasters) to keep my body clear of toxins and wastes.
The ICT and Web 2.0, PLE etc. could act as catalysts (or enzymes) for the digestion.
I would also take extra physical, spiritual and mental exercises (external support, experts’ advice, courses, community or network participation and involvement, action research and learning projects) to ensure a proper balance of my health.

Photos: credit from wikipedia on digestive system

I have also posted this onto our Community Network in http://connectivismeducationlearning.ning.com
Comments?