Thursday, January 12, 2017

The art of designing meaningful assessments

Assessment is an inextricable part of learning, and it can be something we do 'in the moment' ("hmmm, that was OK but I need to do X next time"), or it could be a high-stakes formal assessment designed and administered by an official organisation.

However, there is a real art to all types of assessment. For instance, with the in the moment example, the key here would then to decide what the next steps would be, when they would happen, and whether we need the input of anyone else. With more formal assessments the art is ensuring that the assessment is meaningful by ensuring the needs of the learners align with, and are given at least equal (maybe greater) importance than the needs of the assessing organisation and wider stakeholders.

First, a quick question - why do we assess?  We assess to provide information (i.e. quantitative evidence) that helps us make informed decisions about ourselves, individuals or programmes, to find out, for example how we are progressing, and what we need to develop more.

So, what is assessment? John Dewey suggests that “Education is a social process; education is growth; education is not preparation for life but is life itself”. I would like to invite you to swap out the word education in the quote, and replace it with ‘assessment’. It then starts to provide an insight into meaningful assessment that provides a bit more depth than ‘making informed decisions’.  For example, what does the act of assessment mean for students? A challenge, a game, fear, whakama (embarrassment), grief, anxiety, a lifechanger, a promotion, a sense of pride...or loss. The list goes on - and something that can be forgotten in the act of designing - and doing - assessments is the ‘human aspect’. It’s not just about the grade.

With these points in mind, I have set about drafting an eight-category checklist to help with the design of meaningful assessments, that will also help you avoid some of the possible pitfalls along the way. It is a draft and I would love any input or suggestions on how you feel it might be improved - what have I missed out? What isn't reading well? Please jump into the comments below and let me know.


You might also want to have a quick look at the presentation that complements the checklist.



Image: Assessment. CC (BY SA) licensed image by JPhotoStyle.com: http://jphotostyle.com/handwriting/a/assessment.html

Thursday, December 1, 2016

What is a good Moodle course?

Moodle has been used for over 10 years, and yet 'What comprises a good Moodle course'? is still a hot topic.

Some top tips from Yong Liu include:

1) Mobile learning is key, and gamification is built in as much as possible (although it can be heavy on the budget), plus integration of social media. It is important to automate as many of the processes as possible, while also personalising the learning. However, using an animation or the latest technology, may remove the focus from the learner.

2) It is important to offer opportunities for students to link exiting knowledge with new knowledge. Multimedia can help people learn by helping them select organise and integrate information and understanding (Mayer, 2016). We can only take in visual and aural input, but not two aural inputs at the same time. We can also only take in a limited amount of information at one time. To help alleviate the stress we need to remove redundant and gratuitous graphics, place text near graphics, and explain graphics with audio instead of text if possible.

3) A good Moodle course should simplify complex content, by, for example, segmenting content into small chunks. Information should be precise and exactly what they need, without additional information. Activities also have collaboration, peer teaching - activities where the students do the work themselves. We foster generative processes (Mayer, 2016) by letting the student 'pull' knowledge and selecting what they want when they need it. Use conversational tone and pedagogical agents.

What tips could you add to this list? What have you found works a treat for your learners in Moodle?


Image: Design. CC ( BY SA ) licensed Flickr image by Miquel Lopez: https://flic.kr/p/5oQgCo

Wednesday, October 12, 2016

Changing customer experiences through Moodle

At New Zealand Language Centre the students tend to be English language learners, where there are over 25 nationalities represented at the school.
Pete Jones shares that, he sees his 'customers' in relation as the students, the academic staff, and the administrators. Prior to implementing the Moodle project all the diagnostic, placement tests were completed as paper based and had to be manually marked, within very tight time frames. Now students complete most of their placement test online, and the reading, grammar, and vocabulary tests are now automatically graded.
Moodle was a catalyst for NZCL to review and update their placement test. The existing test had many of the gap fill type questions, where the responses are sometimes not cut and dried. It also encourages students to learn the language in a decontextualised way. The new test has an embedded cloze type test that offered a contextualised scenario, where the organisation could be more confident in the responses that were being given in the test. It has been well worth the investment.
Previously, student would write their contact details onto a form, and then someone had to decipher the handwriting and add the details to a database. 
One of the next steps is to develop a theme for Moodle, that would also complement the new web site.
To sum up, the Centre has met their original objective. Over 70% of students said they preferred this system "it is more convenient than the paper test". For those who didn't prefer it the reason given was previous familiarity with paper based tests. Some of the unintended consequences including having to look at the whole orientation of new students, which has meant providing more time to welcome the students - and this has resulted in a longer timeframe for the placement tests removing a lot of stress for the academic staff. In addition, some of the staff have signed up to the Learn Moodle MOOC.

Image: HP Mini. CC ( BY NC ) Flickr image by Tuesday Digital: https://flic.kr/p/6tg3kw

Wednesday, October 5, 2016

Creating meaningful assessments...

What does a meaningful assessment look and feel like? How can Moodle be used to enhance a learner's assessment experience? In this session I covered a few of the key factors to consider when designing assessments, as well as some of the activity types in Moodle that can be used. I dipped into a couple of examples to illustrate what can work well ... and what might not. By the end of the session the idea was that people would have some additional ideas to take away and use when creating assessments in Moodle, along with considerations that would help ensure that the assessments work for your learners and for you.

The lived experience of sing Moodle for online exams

At Nelson Marlborough Institute of Technology (on the South Island of NZ) all of the exams are paper based. In the Bachelor of Nursing (Year 1 course), there are 2 multi choice exams requiring 90% pass rate. They get the questions the week beforehand and work in a study group to debate the responses, and then come in to respond to the same questions.

Printing costs were massive, with exams being 6 or more pages in length, and had to be printed twice. The exams were also all hand graded. So they turned to Moodle.

All of the exam questions were uploaded into a Moodle forum to download or print them off themselves. The students were still able to get into their study groups to debate. In preparation to do the exam on a screen 'spot tests' were administered throughout the semester to familiarise students. There was also discussion around etiquette for once the results were released because the feedback was not released until after they had finished.

Some of the benefits were:
  • questions were randomised
  • cheating was minimised by the format of the exam (questions provided in advance, and randomised questions, so there isn't such an advantage to going through and highlighting the exams)
  • automatic marking
  • results released immediately
  • printing and paper costs reduced
  • it was easy!
There were some challenges including:
  • PC lab room availability
  • increase in number of invigilators
  • iPads were not so user friendly (the Moodle page kept freezing, and wouldn't change the page and created a lot of anxiety. Students were persistent and carried on, and everyone passed)
  • WiFi connections (60 students were all accessing the same thing at the same time)
In terms of the future, at NMIT there is a focus on moving more exams online, and ideally for all the students to use their own devices and sit the exam in one room.

Moodle at Northtec: Past, present and future

Before using Moodle at Northtec, in 2000, they ran a pilot in WebCT using applied writing. The Campus License from WebCT put the cost beyond Northtec's means. They then transitioned across other Learning Management Systems before settling on Moodle.

In 2003 the Flexible Learning team created the Bachelor of Nursing, and built on the 'we just did it' champions. In 2008 'Cyclone Vasi' hit, with a passion for students and learning - and a wide range of ideas. Her managerial skills brought a structure and a methodology with eLearning. In 2009 the 1st instructional designer joined Northtec, and helped identify key design principles which were essential for course design. At the time they had three Moodle instances running. One of them is the 'archive site', which provides a copy of existing courses, which means that Northtec don't have to have sophisticated disaster recovery processes and backup.

By 2012, Northtec was trying to provide some rigour and structure into the quality control of courses. On the 'live server' (NorthNet) they have a 'development category' with restrictions (for example, teachers can't enrol students into courses, nor can they change the name, or shift the course into another category. They used a student focus group to get the look and feel right. The students were looking for something a bit more dynamic.

Northtec are about to move to 3.1, and Moodle is still hosted off site. There has been some feedback around the lack of communication around version changes, so they have made a big effort to address this.

For some of the students (in particular Nursing) Northtec have developed a version of Moodle that installs off a USB. They can install the version onto their hard disk, and they can get to all of their course materials while 'off the grid'.

Plans for the future are dependent on the ability to upskill staff. How do you put a good online course together? Standardisation of courses is important, although they only have one instructional designer. There is a planning process to work through before releasing to students. The keys are staying flexible, and adapting as needed.


Tuesday, October 4, 2016

Is your preference thinking, feeling, intuiting or sensing?

Some people find it really useful to be more aware about the ways they think or react, and there are a range of approaches and tools available that can help us do this. Carl Jung, in the 1920s, for instance, suggested that we process and react to inputs by thinking, feeling, intuiting, or sensing:
  • Thinkers tend to collect and consciously analyse data.
  • Feelers are open to emotions and their consequences.
  • Intuitors don’t find detail useful, and act on on their decisions, although they may not be aware of how they reached the decision.
  • Sensors tend to be kinaesthetic and rely on their senses to guide their reactions and actions.
Marchant (2014) identifies that everyone uses all four. However, many of us “favour one way over the other three, sometimes markedly so” (Para. 15), and it’s being cognisant of whether we favour one over another that can be helpful.
Given these factors, I will now focus on the intuitor-type of person In a coaching context; in particular I’ll identify aspects that might help me coach this person, and what I might need to keep an eye open for.
If I am working with a person who strongly identifies as an intuitor type I would expect that they are likely to:
  • be focussed on potential and future possibilities (although these can seem unrealistic)
  • look for patterns and relationships
  • be (apparently) impulsive because they seem to react rather than taking time to consider ‘the facts’
  • have a wide range of ideas
  • not be keen on detail or data
  • enjoy using their imagination
  • enjoy being creative and inventive
  • like to work with conceptual ideas and information
  • be idealistic
  • be focussed on bigger picture rather than processes and guidelines
I would try to shape coaching sessions to help ensure that there are plenty of opportunities for my coachee to talk through their ideas (including philosophic underpinnings and principles). During these descriptions I would gently encourage them to unpack a bit more detail, but in a way that signals I am curious about, and value, their insights. Supportive questions might be around encouraging the coachee to consider how other people in their context  respond positively to their ideas, and, if appropriate, I would also ask questions about other (online) communities and places the coachee could share, discuss, and develop ideas further.
One of the things I would need to keep an eye on is that the coachee doesn’t end up going through similar cycles and not recognising them as such - especially if they are making the same mistake each time. For example, a coachee might describe a series of situations where they receive the feedback ‘we don’t know what you are working on most of the time - or why’. It could even have led to misunderstandings to the point where they have been pulled in for a meeting and questioned about what they are spending their time doing. Usually, it’s a case that the coachee has been so completely focussed on a task or project that they haven’t taken the time to communicate their progress to anyone else, and have seen the feedback as a minor annoyance - until, to their surprise, serious questions start to be asked about their performance.
In situations like this one, I would encourage my coachee through a deductive process:
  1. to consider the implications of their complete focus (i.e. the bigger picture),
  2. and how the positive progress they are making (i.e. the details),
  3. could be (creatively) communicated with any stakeholders who needed to know so that these stakeholders can / will continue to support and fund the piece of work on which the coachee is focussing,
  4. and finally, encourage the coachee to identify ways they can share with co-workers, managers, family and friends their “preferred intuitive information gathering preference” (Edward, n.d., Para. 5).

The central thing here is to support my coachee to clearly identify the ‘why’, so that they have a clear reason and motivation to carry through with a necessary action (possibly in a creative way). In this example, it would be a strategy that could also ensure that they keep their role! In turn, by opening up awareness and lines of communication, ideally it would lead to increased support and recognition of the value of the work the coachee is doing.

References

Marchant, J. (2014). Thinker, feeler, knower, sensor? Retrieved from http://www.emotionalintelligenceatwork.com/resources/thinker-feeler-knower-sensor/.
Southern Institute of Technology (a). (n.d.) Transformational Coaching and its outcomes (Module C) [Lecture notes]. Retrieved from CBC106 (NET).