Application of Game Theory in the design, delivery and assessment in MOOCs

This Lifelong education on Steroids provides an excellent and insightful overview about MOOCs.  What I would add is that MOOCs could be one the game changers in Higher Education, not just online education.  Why?  Higher Education has been a game in business, where each of the game players are playing a fair, though competitive game in a global arena for decades.  

The strategic alliance and partnership is one of the macro approaches in game playing where institutions are working with various other education providers or services in order to enhance the overall education and learning experiences of the learners, or consumers and customers.

How would Game Theory help in the design, delivery and assessment in MOOCs?

There are two main approaches that we could consider – a macro and a micro approach.

Macro approach:

First, to design MOOCs based on Game Theory, on a macro scale. What this involves is to compare and contrast the various design of x and c MOOCs, based on a set of principles where networked learning and mastery learning is leveraged, especially when an institutional education model is based.  This could be demonstrated and applied by taking into consideration the payoff and expected return with each probability (i.e. un-bundling of each of the present services of typical MOOCs services as described here) and re-bundling them with values and benefits for each cohort of learners and educators.

Second, to deliver MOOCs based on Game Theory principles which include those elaborated in this Understanding the MOOC Trend.

Third, to assess MOOCs based on a combination of automation and human intervention, where learning analytics and big data are used to provide feedback to both educators and learners on a continuous basis.  This paper on assessment on MOOCs provides an insightful approach to incorporate

Micro approach

This involves strategically designing MOOCs based more on the games with various multimedia and interactive game story, where assessment and learning are built in to engage both professors and learners to co-explore and learn through the education process.  Games could also be used for assessing learners in a personal and adaptive way, though this would involve a total different design from the instructivist approach.  This includes peer-teaching and learning as proposed by Eric Mazur and other educators.  Indeed peer teaching and learning is one of the pedagogy adopted in a connectivist approaches towards learning.

It should be noted that the majority of peer-tutoring programs for students are intended to complement, not substitute for, regular classroom instruction. Tutoring should never be a substitute for professional teaching. An ideal learning atmosphere is as a rich blend of peer and adult instructional strategies.

Cited From: http://www.opencolleges.edu.au/informed/features/peer-teaching/#ixzz2cTBZTEiL

In summary, game theory could be used in the design, delivery and assessment in MOOCs, with an overall improvement in the learning and education experiences of learners.

On Study Skills and Examination

Another useful video by Dr Stephen Chew on study skills and examination.

Examinations could be highly effectively in mass assessing students of Higher Education in an objective way.  Most of us (as educators) have gone through the examination processes in Universities and Colleges.  Indeed examinations could be critical in determining the grades of students in a College or University degree, and such practice might not change in the foreseeable future.

Examinations are still useful for undergraduate and graduate studies up to PhDs, or even professional association admission or accreditation.  So it is important to learn those examination skills, in order to achieve good results and meet the goals.

Examinations are, however summative assessment tool and there are little that the learners could do to change the results of the examination, unless there are feedback to the learners on where they are fallen short of, in terms of their “mistakes” or “wrong answers” so they could correct.  

Whilst examinations are still important tool in assessing students in Higher Education, there is now a trend towards using various combination of formative and summative assessments – authentic or real life assessment tasks, problem based assignments, workplace projects, and workplace based assessment as a more holistic educational tool in the assessment process, apart from the formal examinations.

In my post here, I share the following:

If assessment is so important in formal education, why do people still prefer to adopt the instrumental teaching based principally on mass lecture, tests and examination rather than assessment as an effective pedagogy?  Take a test or examination, and if you could pass it, you are qualified for a pass of the unit.  Isn’t it simple?

Some of us might have watched this video.

So, a lot of students would ask a basic question: Are the lecture materials delivered by the teacher during the lesson to be tested in the tests and  examinations? If not, could we focus just on what is to be tested or examined, and leave the rest to be “learnt” outside the classroom?  This is exactly the type of questions most students are asking in each semester, in a traditional lecture type of education and learning. Is that what the educators are most concerned too?  Teaching the content of examination or test to the students, so students could achieve high marks in the assessments. So, why not teaching to the test?

A test and or examination is a typical assessment tool used in education for decades. That’s where students could demonstrate their competency, and that is how assessment is conducted in most schools.  And if students are learning in online distance education, then they would be expected to submit the standard assignments (say completing a 2,000 words essay or answering a series of questions as required in the problem or project set), attend the examination, and if they pass in both assessment, congratulations!

Doing assessment requires more than the mere completion of the written assignments.  An excellent example of assignments as shown here requires the preparation and collection of evidences, and through an exploration and research process in the assessment, the learners would be able to demonstrate the competency required.  Also learners could identify their own learning needs and gaps in the learning process, when working through the assignments.  With the feedback from peers and or facilitators, the learner could also identify what would need to do to improve his or her learning.  These will all involve sensemaking (giving meaning to experience) and metacognition (cognition about cognition or knowing about knowing).

What is most important when conversing online?

Jenny Mackness provides an excellent review on what she has learnt from the course OldGlobeMOOC “Update-on-oldglobemooc-and-peer-assessment.”

I would like to share some of her points and responses to my questions here.

She says:

The OldGlobeMOOC is a great experience in terms of the diversity of participants. Unfortunately the younger participants, in their teens, who signed up, seem to have fallen out of the discussion forums. This does not mean that they are no longer participating through observation and reading – it’s difficult to know. But I have wondered how an 11 year old might review the assignment of an academic Professor, or how an academic Professor might respond to a learner with special needs, or a very young participant, or someone whose first language is not English, and so on. The assignment submission is anonymous. Do these differences have implications for the equity of the peer review process?

I ask:

Thanks Jenny for your insightful responses. It is interesting to have peer assessment on such topics. I am wondering if the peer assessment by peers are assessing based on whether the stories are resonating to their personal perceptions. Assessment may also relate to the degree of relatedness to ageing. After reading through those peer reviews, I have a few questions in mind. 1. Since the experience relates to the writer’s story and opinions, would the assessment relate more on how the story and opinions on ageing (emotional response) or the story structure and written expression (clarity and coherence)? 2. How to ensure a fair assessment under such blind assessment arrangement? As you mentioned, it may likely that a novice (11 old teenager) assessing an academic professor, or a veteran or geek. 3. How would one appeal to any “inappropriate assessment or review”? Are there any appeal mechanism or procedures in place? Should there be one, as you mentioned that you have been penalized though you have submitted all five reviews on time. I ask these for the sake of discourse, not as a challenge to the authority, or a “complaint”. I think it is important to ensure any assessment be viewed as open, transparent, and are based on the honest feedback, rather than hasty response. Though there might be good reasons why some people just give others a “2″, I just wonder if this would be “acceptable” in a peer assessment system. What do you think would be a better way to ensure a fairer assessment with xMOOC? John

Jenny elaborates:

Hi John – thanks for all these questions. I don’t think OldGlobe is typical of xMOOCs. In fact Sarah Kagan has called it a cMOOC and it does have a lot of the characteristics of a cMOOC – it is being run mostly on the Coursera platform, but there is also a Facebook group and Twitter, but I’m not aware of other bloggers. It is open in the sense that access is open, the assessment is open and there is very little course content, e.g. there are weekly videos, but no readings, and discussion can follow any path the participants would like. The course has great diversity of participation and discussion, and participants do have a lot of autonomy if they are not bothered about assessment. So a lot more like a cMOOC than an xMOOC. And judging from comments in the forums, the assessment in OldGlobe has been different.

The nature of the assessment questions does lead to storytelling and for me this is a strength because it means that anyone from any background can engage with the assessment – but as I have described it does lead to some difficulties with the peer review system. I think it’s possible for reviewers to take a number of approaches to the peer assessment – but this is how peer reviewers are asked to respond:

Please type your 100-250 word peer assessment below.

What do you think about this participant’s portfolio item choice to answer this question of the week?
How does this participant’s perspective differ from your point of view?
How is your point of view similar?

So in OldGlobe the criteria for peer assessment are quite loose and non-academic. So I don’t think it’s possible to think in terms of fairness, as we normally understand it in academic assessment, but rather in terms of empathy, kindness and respect. Reviewers are told:

An assignment only receives a zero if it is incomplete or did not follow the guidelines set for this week. Don’t be afraid to be generous!

And yes it is possible to appeal, by putting a question in the Help Forum – but participants have to have a genuine cause for appeal. It’s no good saying ‘I didn’t have time to do the peer reviews this week’ – hence the peer review I got which said

peer 2 → I’m headed for an airplane so don’t have time to review, and I won’t be back until after evaluation time ends so I’m just giving everyone a 2.

Based on what Jenny says above, the course does have a lot of characteristics of a cMOOC.  Also the nature of assessment questions does lead to story telling.  Indeed, one of the most attractive nature of online conversation is story telling and sharing, where bloggers share their anecdotes or learning scenarios, and reflect on those experiences that they have learnt.

The assessment criteria of this MOOC thus focuses on the reflection of similarities and differences in views and perceptions, and possibility the resonance or dissonance that one experiences in life upon reading the writings, especially when the author and reader relates to their perception or feelings of old age.  This could also be a powerful lesson for any one to imagine what it means to undergo the various stages of old age, physically, mentally and spiritually.

This sounds quite an interesting way to assessment for the participants too, as it could lead both the learner and the reviewer (who could also be a blogger or another learner) to assess based on his/her affections (feelings, emotions) towards what the learner think about old age, and thus provide a point of view which is distinct from the learner.

Would this assessment be designed to measure certain attitudes, and to a certain degree the emotional awareness, control and responses, and empathy- which are related to emotional and social intelligence?  See my posts here, here, and here.  This seems not to be explicit in the assessment, though I haven’t enrolled into the course, and thus not be aware of how the assessment is related to emotional intelligence.

Another dimension that I reckon such assessment would lead to is clarity and brevity in expressing one’s thoughts in writing (as there is a 250 words limit).  It seems that this is based on writing in English, and that grammar, use and choice of words, and structure of the writing would need to be taken into consideration in the assessment.  Would this be more about assessment on written English, based on the context and content?  How would the length of writing (i.e. 100 -250 words) affect the overall “quality” of the writing?  Are there any penalty if the writing exceeds the limit?  May be not, with such form of assessment.

Finally I could sense that such assessment may help the author and the reviewer to understand and apply the following basic principles in blogging and online conversation, based on writings:

1. Be concise and clear in writing

2. Be empathetic – understand others’ points of view, and empathize

3. Be kind, generous and sincere in comments and feedback – don’t patronize

4. Be open to others’ views and comments, even if we don’t agree with our readers’ points of views

5.  Be patient and listen to the reader’s comments and feedback

6. Be thankful to our commentators and readers

7. Be ourselves – in stating our views

8. Be supportive to each others’ learning, through such sharing of views and experiences

9. Be tolerant even if others are harsh on us, though at times, we need to be assertive

10. Be responsive to others, and be responsible for our own work

Does it also mean that sometimes it is the MacGuffin (as suggested by Stephen Downes) that sets off a good online conversation?  It is the emotional attenuation that would help people relate to our story, so they could also share their stories with us.  

What else have I missed?

Again, I think Jenny has been an master (and exemplary) blogger who role models how to write comprehensive blog posts and how to respond to comments in blogging and assessment.

You would find lots of master pieces in her blog posts.  I would encourage you to read her blog posts, where you would surely find lots of words of wisdom.

Thanks Jenny for her wonderful insights.