ICALT 2015 paper accepted

Hualien city, by Luis Jou García ©2010, CC BY-NC-SA 2.0

Hualien city, by Luis Jou García ©2010, cc by-nc-sa 2.0

I’ve just had the good news that a paper based on my summer project,  Can you tell if they’re learning?, (co-written with my supervisors) has been accepted as a short paper at ICALT2015. The conference takes place at the start of July in Hualien City, Taiwan – about 10,000 km from home. Here’s the abstract:

The proliferation of Web-based objects designed for learning, makes finding and evaluating online resources a considerable hurdle to overcome. While established Learning Analytics methods use Web interaction data as proxies for learner engagement, there is uncertainty regarding the appropriateness of these measures. In this paper we propose a method for evaluating pedagogical activity in Web-based comments using a pedagogical framework, and present a preliminary study using this approach that assigns a Pedagogical Value (PV) to each comment. This has value as it categorises discussion in terms of focus on pedagogical activity rather than interaction on the Web. Using the DiAL-e Framework we code a selection of comments associated with learning objects within a Massive Open Online Course and test the resulting Pedagogical Value against established Language Analysis and Learning Analytics methods. Results show that PV is distinct from typical interactional measures; there are negative or insignificant correlations with established Learning Analytics methods, but strong correlations with linguistic indicators of pedagogical activity. This suggests that adopting pedagogical frameworks may produce more accurate measures of pedagogical activity than interaction analysis, and that linguistic rather than interaction analysis has the potential to automatically identify learning behaviour.

Obviously I’m looking forward to interacting with key academics in the field of technology in education, but I also hear that the surfing’s quite good. What’s not to like?


MSc Summer Project Poster

CanYouTell_PotraitA1FINALMy MSc Dissertation Poster/Tim O’Riordan ©2014/Creative Commons by-nc-nd License.

In my last post (several months ago) I set out a loose plan for my summer dissertation project. Now the dissertation has been submitted and I’ve produced this poster to sum up it’s 14,000 words – or at least its key points. It had it’s first outings at the Barry Wellman Distinguished Lecture event last week, and at the Institute for Learning, Innovation and Development (ILiAD) Conference, and a Public Engagement and Outreach Workshop earlier this week – all of which were held at the University of Southampton.

What are my key findings?

Essentially, after analysing comments associated with learning objects on a Massive Open Online Course (MOOC) in an attempt to identify ‘attention to learning’, I discovered that the coding model I applied to comments was good at identifying where there was not much learning activity going on, and slightly less impressive at identifying attention to learning. While there is a lot of work to do on this area, results so far have been encouraging, and I’m looking at how the usefulness (or otherwise) of my approach can be tested further.

Why is this important?

Making education and training more widely available is vital for human development and the Web has a significant part to play in delivering these opportunities. Running a successful online learning programme (e.g. a MOOC) should involve managing a great deal of learner interaction – answering questions, making suggestions, and generally guiding learners along their paths. But coping effectively with high levels of engagement is time intensive and involves the attention of highly qualified (and expensive) teachers and educational technologists. My hope is that through my research an automated means of showing how well and to what extend learners are attending to learning can be developed that will make a useful contribution to managing online teaching and learning.

Exams, Dissertation Topic and Authorship Issues

University of Southampton libraryUniversity of Southampton library/Jessie Hey © 2006/CC BY 2.0

Phew! Examinations and essay writing are now over for my MSc in Web Science course and I’m now preparing to start my summer dissertation project. Two supervisors have agreed to oversee the progress of my work, the main research question of which is: can comments related to Web-based learning objects be used to good effect in the evaluation of these objects?

This is essentially a learning analytics research project and my initial plan is to collect a suitably large dataset from FutureLearn MOOC’s and analyse this data using social network analysis and sentiment analysis tools. I aim to examine the learning objects and the topics discussed, the language used and what the sentiment polarity is towards these topics and objects. The objective is to identify characteristics of distinct participant roles, look for similarities in language and evaluate these in terms of established pedagogical frameworks (e.g. Dial-e, DialogPLUS) to support relevant schema.org descriptions (e.g. educationalFramework).

I’m in the process of working out exactly what I’m going to do and how I’ll do it. Because it involves ‘scraping’ web sites for people’s opinions, there are ethical issues as well as theorectical and practical hurdles to be overcome. However, I believe that this type of data can be profitably used to assist with learning object evaluation and think it will be useful to find out if there’s any evidence to back this opinion up.

The ‘authorship issues’ referred to in the title of this post aren’t strictly related to my studies as they have arisen from my previous employment as an Advisor at Jisc Digital Media. However, having just discovered that articles I wrote for my old employer are now being attributed to another person, I have been pondering the ‘web sciencey’ issues raised by this – like trust, provenance, the use of metadata and the nature of authorship in the digital age. By way of an example of what’s happened, here’s a link to an article I wrote about Khan Academy videos in March 2011 as preserved on the Internet Archive’s Wayback Machine (correctly attributed) – and here it is currently published on the Jisc Digital Media Blog (incorrectly attributed).

Joe Chernov discusses the issue of byline re-attribution in his post: Creators vs Corporations: Who Owns Company Content? All very interesting, and a topic that I will return to in a future post.