#PLENK2010 Evaluation of PLE/N VLE and the course

There is an interesting discussion on formative assessment in the PLENK forum.

There are a few notes here:

Niklas: In a post I wrote about the validation of knowledge is a key issue
“The most interesting question of PLE/PLN is maybe not about learning!” (http://learning-research.blogspot.com/2010/09/most-interesting-question-of-plepln-is.html
it is not only the summative assessment which is interesting in PLE / PLN, but also how the formative assessment takes place.

Ken responded: I’d like to venture a question in response to your questions:
“Who assesses knowledge?” “When is it assessed?”

What do you think about discussion in the forum?  In particular about who is talking to whom, responding with regards to the syntax, semantics and reasoning (what we have discussed in CritLit course).  Do you find any “traces” and patterns of discourse there?

Here is If you build it, they will come, an excellent discussion on the use of assessment on discussion threads in forum.

Here is a copy of the If you build it, they will come. It mentions about using rating scale in indicating student participation.

The rating scale ranges from 1 to 4 with 1 being indicative of student participation which is less than acceptable for the development of a progressive learning community. A score of 4 in any category represents the attainment of the highest standard of participation and reflects a bonafide contribution to the learning community.

California State University at Hayward EDUI 6707

Assessing Effectiveness of Student Participation in Online Discussions

Here I would suggest to analyse the forum discussion based on a rubrics: Consider using a score of 1-10 on each criteria as shown below to analyse each forum post.  A total score of 100 could be used as the benchmark for self and collective assessment.
Then one could aggregate the scores for all forum postings and derive a score comparing the different forums.
1. Communication (1-10):
(a) content of the post or thread – relatedness to the topic, quality (logical reasoning, links to research artifacts), value added to the argument, sensitivity to others’ concerns, questions and responses, emotions etc.
(b) listening pattern – active listening, reflective listening, use of “emoticons” etc.,
(c) questionnaire style – open, closed, summary, re-focussing, reflection, inquiry, challenging, critical etc.
2. Critical thinking (1-10):
(d) reflection-To what extent do the forum poster or responders reflect on what they think and say based on personal experience or anecdotes?
3. Collective intelligence (1-10):
(e) collective & collaborative inquiry & intelligence – What are the gains in intelligence emerged out of the conversation? Any significant new or novel ideas and insights uttered?  Are there diversified views & perspectives behind each inquiry, and followed through collectively?  Any signs of “Group Think”?  Any signs of “logical reasoning” in the threads? Which ideas are amplified? Which are dampened? What are the follow-up actions – further research, further inquiry or research, if any? Will this lead to the further growth of the networks?
4. Moderation (1-10)
(f) Who are leading the discussion? Who are the moderators? Who are the active responders?  How is moderation held?
5. Integration and links to other blogs, tools or networks & strategic actions and synthesis –  (1-10)
(g) Do the discussion lead to further links to experts, blog posts, tools, or network discussion? What further strategic actions would this discussion lead to?  This also relates back to the collective intelligence.
6. Creativity (1-10)
(h) Are there any creative or novel ideas, information and or knowledge emerging out of the conversation?  To what extent would the network members (in and out) rate the creativity of the discussed ideas.
7. Connectedness and connectivity – Connection with others (1-10)
(i) To what extent are the forum posters and respondents feel and sensed connected with others?
8. Openness (1-10)
(j) To what extent do forum posters and respondents feel open to voice their views and opinions?
9. Autonomy (1-10)
(k) To what extent do forum posters and respondents feel the sense of autonomy in sharing and learning?
10. Over all value of learning in the conversation (1-10)
(l) What is the overall value of learning emerged through the conversation, as perceived and judged by the forum posters and respondents?
Similarly, the above criteria could be used for blogs (WP, Twitters (microblogs), and other discussion (like FB in blogs etc.). Here we could derive a score of the blogs and blog posts and an overall score for the blog posts in the blogging community.  A total score of 100 could be used as the benchmark for self and collective assessment in the blogging.
1. Communication (1-10):
(a) content of the post or thread – relatedness to the topic, quality (logical reasoning, links to research artifacts), value added to the argument, sensitivity to others’ concerns, questions and responses, emotions etc.
(b) listening pattern – active listening, reflective listening, use of “emoticons” etc.,
(c) questionnaire style – open, closed, summary, re-focussing, reflection, inquiry, challenging, critical etc.
2. Critical thinking (1-10):
(d) reflection-To what extent do blog poster or responder reflect on what they think and say based on personal experience or anecdotes?
3. Collective intelligence (1-10):
(e) collective & collaborative inquiry & intelligence – What are the gains in intelligence emerged out of the conversation? Any significant new or novel ideas and insights uttered?  Are there diversified views & perspectives behind each inquiry, and followed through collectively?  Any signs of “Group Think”?  Any signs of “logical reasoning” in the blog posts and threads? Which ideas are amplified? Which are dampened? What are the follow-up actions – further research, further inquiry or research, if any? Will this lead to the further growth of the networks?
4. Moderation (1-10)
(f) Who are leading the discussion in the blog posts? Who are the moderators? Who are the active responders?  How is moderation held?
5. Integration and links to other blogs, tools or networks & strategic actions and synthesis –  (1-10)
(g) Do the discussion lead to further links to experts, blog posts, forum posts, tools, or network discussion? What further strategic actions would this discussion lead to?  This also relates back to the collective intelligence.
6. Creativity (1-10)
(h) Are there any creative or novel ideas, information and or knowledge emerging out of the conversation in blogs?  To what extent would the network members (bloggers) (in and out) rate the creativity of the discussed ideas.
7. Connectedness and connectivity – Connection with others (1-10)
(i) To what extent are the blog posters and respondents feel and sensed connected with others?
8. Openness (1-10)
(j) To what extent do blog posters and respondents feel open to voice their views and opinions?
9. Autonomy (1-10)
(k) To what extent do blog posters and respondents feel the sense of autonomy in sharing and learning?
10. Over all value of learning in the conversation (1-10)
(l) What is the overall value of learning emerged through the conversation, as perceived and judged by the blog posters and respondents?
This would then allow a comparison of the scores derived from forum (VLE) to blogs (PLE/N)
In summary, it could be interesting to use some assessment tools to evaluate the blogs (PLE) and forum posting (VLE), in an objective and collective manner. These would address the “quality of the discourse” in blogs and forum with different perspectives.   This would avoid having emotional or personal judgment (prejudiced on personal favourites or discrimination due to difference of views from “ours” or someone else), especially in the forums and blogs.
The above are just my first thoughts.  I know there has been lots of rubrics developed for forums and LMS.  There are many tools used for evaluating blogs, and I would need to do some more research on it.  However, with the PLENK2010, such tools need to be customised.
I know Lisa Lane has used some tools for evaluating the forum postings.  I think most of those tools are restricted to “formal classes” and don’t deal much with openness and autonomy.
I don’t think it’s wise to impose any such tools on any blog and forum posters, as this is just my best wishes tool based on “brainstorming”, and could scare away anyone interested in forum discussion in particular.  It may be important to develop some “objective and collective” tools to evaluate the forum discussion and compare them with the blogs using similar criteria, so as to understand better about how people might have learnt better with those affordances.  So, this could a collective activity that may be of interest if posted on FB, blogs or forum.
I think this is also part of my suggested research into the Design and Delivery of MOOC, in particular, on how the course is evaluated.
What are your views on formative assessment in MOOC?
How would you evaluate your learning in PLE in PLENK2010?

5 thoughts on “#PLENK2010 Evaluation of PLE/N VLE and the course

  1. Pingback: Записная книжка Владимира Алексеева : #PLENK2010. The neXt Web

  2. Just to include some basic information relating to semiotics – in conversation/discourse over blog/forum (source: wikipedia) for reference.
    In linguistics, semiotics, also called semiotic studies or semiology, is the study of sign processes (semiosis), or signification and communication, signs and symbols. It is usually divided into the three following branches:

    Semantics: Relation between signs and the things to which they refer; their denotata
    Syntactics: Relations among signs in formal structures
    Pragmatics: Relation between signs and their effects on the people who use them

  3. Pingback: CCK11 Connectivism and Assessment | Suifaijohnmak's Weblog

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s