I have been thinking about assessment and where we need to head with this for a while. This is the second draft and I would appreciate comments, suggestions etc. Many thanks to Nick and Brett for their comments.

Assessment, in the form of examinations, has be a constant feature of education since the 18^th Century. Some researchers indicate that the first printed examinations at school started in 1830 with the Sixth form at Harrow.[1, 2, 3]. For over 150 years, examinations have been our principal assessment tool. But do they reflect 21^st century teaching and learning. They have a place but is it the flag ship of assessment? More and more we are seeing a shift away from the examination. So what is a 21^st century assessment?

21^st Century Assessments should be focused on both the learning process and the assessment outcome.

So why are these component in this formula?

Rich Real World Tasks.

Our students have the worlds at their fingertips. They are exposed to vast volumes of information in a variety of modes. For our tasks to be relevant to them, for our students to engage we need to make the tasks rich. The tasks need to be relevant to them. This relevance takes the form of not only the purpose and objectives of the task but also the medium in which the task is undertaken. While to many teachers a written report is a suitable medium, to our media savvy students it is boring. Allowing the students to develop their tasks in rich mediums will engage them and an engaged student will learn. Rich Media like, video podcast, enhanced podcasts, cartoons and comics are more likely to engage higher order thinking processes as they plan, design and create their solutions.

Allowing are students to have input into the development of the assessment is a key step towards students ownership and engagement. Why should our students not have involvement in their own learning?

It is not only the process that is important but also the end product. We need to value not only the outcomes of our learning but also the process of reaching those outcomes.

“I have not failed, I’ve just found 10,000 ways that won’t work”

Quote attributed to Thomas Edison http://www.wikiquotes.org

Higher Order Thinking.

This is not an “anti-examination” rant. Examinations do have a place, but there is a caveat with this comment. Most examinations only test lower order thinking skills. If we were to look at the keywords commonly found in examinations they are are lower order thinking skills in Bloom’s Taxonomy or Art Costa three story intellect.

• list (Remembering)
• state (Remembering)
• identify (Remembering)
• name (Remembering)
• describe (Remembering)
• comment (Understanding)
• discuss (Understanding)
• explain (Understanding)
• exemplify (Understanding)
• compare (Analyse)
• analyse (Analyse)
• evaluate (Evaluate)

It is possible to structure an examination to test higher order thinking skills but there are definite limitations within this.

21^st Century assessments must be inclusive of higher order thinking. Inherent in higher order thinking is the lower order elements of recall and understanding. To be able to analyse, evaluate or create we must be able to understand, remember and apply.

Collaboration.

Ask a student at university if collaboration is part of their tertiary education or ask an employer if they value collaboration and the answer will be yes. Our students spend their days and nights collaborating and communicating. They are fluent in a variety of mediums. Whether the medium is collaborating via instant messaging, txt from their cell phones, chatting on embedded chatrooms on their facebook or bebo pages, twittering or working collaboratively on google documents; our students collaborate.

Collaboration is not a 21^st century skill it is a 21^st century essential.

In a report to UNESCO “The four pillars of Education, Learning: The Treasure within” Collaboration and communication is identified within each of the four pillars.

• Learning to know
• Learning to do
• Learning to live together
• Learning to be

(http://www.unesco.org/delors/fourpil.htm)

Ian Jukes, David Warlick, Marc Prensky and many others emphasise the importance of collaboration.

The question of equitable workload is one aspect raised as a draw back for collaborative projects and assessments. This can be overcome by personal and peer assessment. In my experience students are brutally honest in their appraisal of their own performance and that of their peers. For this reason, peer assessment is crucial in the 21^st Century assessment.

Timely and appropriate feedback.

The impact of timely and appropriate feedback is huge. Without feedback, assessment is NOT a learning activity, it is a compliance task.

To maximise learning regular detailed feedback is needed. This is not just end point feedback, which is often the review of the test at the end of the unit or what you could have done to make your project better. Rather this is regular feedback and reflection as students undertake the various tasks that lead to their learning endpoint.

Feedback is essential to refine the learning process, to maintain focus, to provide assistance & direction, to shape and adapt intra-personal and interpersonal processes and actions.

Our feedback must be:

• Timely – The end of the task is too late, we must provide feedback often and in detail
• Appropriate and reflective – The feedback must reflect the students ability, maturity and age. It must be understandable
• Honest & Supportive – Feedback can be devastating, our role as teachers is to nurture and shape. We must provide feedback that is honest and supportive in a manner and mode that does not ostracise the recipient.
• Focused on learning and linked to the purpose of the task
• Enabling – receiving feedback with out the opportunity to act upon it is frustrating, limiting and counter productive. Students must be able to learn from and apply this feedback

When suitable and appropriate, once a safe and open class or group environment has been established, peer feedback is a powerful learning tool. Obviously clear guidelines need to be established.

Appropriate, timely feedback is one of the most powerful learning tools available to educators.

Transparent Schema

How often do we provide our students with the rubrics or assessment schemes for their learning tasks and assessments? Do we involved them in developing these schemes and rubrics?

Crucial to students successfully achieving is transparency, they must know the end points they have to reach, they must clearly understand not only the objectives and goals of the task or task but the criterion they will be judged against.

It is an interesting and worthwhile practice to involve your students in developing rubrics or marking schemes for their tasks. This becomes part of the learning process, they are undertaking a process of goal setting crucial in planning a higher order thinking skill. The students involvement in developing these criteria will lead to greater ownership by the students of the task and increased engagement

Self & Peer assessment

As previously noted, students are often brutally honest in their assessment of not only their own performance but the performance of their peers.

Peer assessment supports students and teachers alike. The students insights and observations are valued. They often have a better grasp on group dynamics and relationships than the teacher.

Peer assessment stresses and reinforces the importance of collaboration. Peer assessment can be, depending on the group, linked to feedback as well.

Encouraging reflection and from this self assessment adds a powerful dimension. Reflective practice is a powerful tool for teachers and students alike. Reflecting via a blog or a journal on the days learning allows the person:

• to consider their actions,
• reflect on decisions,
• revise and solidify concepts developing ownership and
• consider/plan future processes and actions.

Reflective practice is something we should encourage in our students and ourselves, after all we are life time learners.

So in conclusion;

If we take all of these factors

• rich real world tasks,
• using higher order thinking,
• incorporating Collaboration,
• with timely and appropriate feedback,
• assessed against transparent schema and
• self & peer assessment,

link these to assessments that are focused on the learning processes and outcomes rather than compliance, we will have 21^st Century Assessment.

This is a really neat find.

http://www.dnalc.org/ddnalc/resources/animations.html

The site contains resources and animations for Both Mac and PC using flash and links through to other useful sites.

In my last blog on assessment I looked at some of the key words we use in assessment and the fact that these are often Lower Order Thinking skills (LOTS) rather than the Higher Order skills (HOTS).

So I went on to the NZQA (New Zealand Qualifications Authority) web site and downloaded a paper from last years examination round. This is  a level 2 chemistry paper. Have a look at the keywords (highlighted below). I have matched these to Bloom’s taxonomy

• drawing (understanding)
• naming (remembering)
• explain (understanding)
• state (remembering)
• Discuss (understanding)
• identify (remembering)
• describe (understanding)

These are lower level thinking skills. Our assessments need to look at higher order thinking skills rather than low level recall and understanding.

Can we test or examine higher level thinking skills in an examination setting?

And the award for prefect timing goes too…. drum roll plasq for there brilliantly timed release of comictouch

A single image comic creation tool for the iPhone and touch. Great work and very nice timing. This great bit of marketing is only let down by the pricing of the plans for the iPhone here in New Zealand.

I have never been a huge fan of examinations. In fact, many current forms of assessment leave me cold. We are required to assess our students; to measure their progress, to track their learning, to identify weakness and perhaps to measure our own performance as educators.

But do they do our students justice?

We know that in the knowledge economy of the 21st Century, Higher order thinking skills are valued and are indeed essential. But, do our assessments reflect higher or lower order thinking?

If we look at Bloom’s Digital Taxonomy or perhaps its simplified cousin the Art Costa’s Three Story Intellect, many of the verbs we associate with examination and test questions reflect lower order thinking skills.

Bloom’s Digital Taxonomy		Art Costa’s three story intellect
Lower Order Thinking
Remembering	Recognising, Listing, Describing, Identifying, Retrieving, naming, Locating, Finding, bullet pointing, highlighting, bookmarking, social networking, social bookmarking, favouriting/local bookmarking, searching & googling.	Gathering information Count, Define, Match, Observe, Select, Describe, Identify, List, Observe, Name & Recite.
Understanding	Interpreting, Summarising, inferring, paraphrasing, classifying, comparing, explaining, exemplifying, advanced searches, Boolean searches, blog journaling, twittering, catergorising, tagging, commenting, annotating & subscribing.	Processing information Analyse, Categorise, Compare/Contrast, Explain, Infer, Make, Analogies, Sequence, Synthesize & Sort.
Applying	Implementing, carrying out, using, executing, running, loading, playing, operating, hacking, uploading, sharing & editing.
Analysing	Comparing, organising, deconstructing, attributing, outlining, finding, structuring, integrating, mashing, linking, validating, reverse engineering & cracking.
Evaluating	Checking, hypothesising, critiquing, experimenting, judging, testing, detecting, monitoring, blog commenting/critiquing, reviewing, posting, moderating, collaborating, networking, refactoring & testing.	Applying Information. Apply a principle, Evaluate, Forecast, Hypothesize, Imagine, Assess, Predict, Speculate, Judge, if/then, Idealize & Generalise.
Creating	Designing, constructing, planning, producing, inventing, devising, making, programming, filming, animating, blogging video blogging, mixing, re-mixing, wiki-ing, publishing, videocasting, podcasting, directing & broadcasting
Higher Order Thinking

What are the key verbs we use in our examinations.

• list (Remembering)
• state (Remembering)
• identify (Remembering)
• name (Remembering)
• describe (Remembering)
• comment (Understanding)
• discuss (Understanding)
• explain (Understanding)
• exemplify (Understanding)
• compare (Analyse)
• analyse (Analyse)
• evaluate (Evaluate)

These are predominently lower order thinking skills.

Does a one, two or three hour examination, sat in isolation from information sources and reference give a student adequate time to develop analysis, evaluate and be creative? To reflect critically on a year or mores work? Is it realistic to expect a student to complete these in close book format, when in the “real world” they would have a plethera of resources at their finger tips?

Examinations are a poor tool to measure higher order thinking.

Fortunately, examinations are not the only form of assessment we use with our students, but do the other types of assessment actually reflect Higher Order Thinking. Do the assessment tasks reflect the higher order goals and objectives or are they more of the same. Do we put suitable emphasis on:

• developing a hypothesis (Evaluating)
• experimenting (Evaluating)
• planning (Creating)
• designing (Creating)
• judging and evaluating (Evaluating)
• producing and making (Creating)
• critiquing, reviewing and testing (Evaluating)
• refining (Creating)
• mixing and remixing (Creating)

These can not be tested adequately in an examination or test

One of the things I really like about conferences is talk directly to the vendors. We use open text’s First Class and they had a very good presence at the trade show. First class is used by 4 of the top 10 school districts in the US. Some servers are really huge, I know of one licienced to 250,000 users. Talk about scalability.

Anyway, the First Class users meeting was illuminating and very interesting. The changes are exciting and very cool. The web interface that was demonstrated (its a version 10 interface) was easy to use, easy on the eye and very very functional.

Here are the notes on the presentation.

Double click to see at full size

Key elements:

• indexing
• 64bit processor
• AJAX support including wikis (wikis are now very important)
• New web interface - pretty radical shift, but very cool!
• Active sync and iphone support
• versioning
• External user support
• Nov 08 for 10 release
• May 09 for 11

I must thank Jim, Rahul, Greg, Mark, Tim and all the first class team for their brilliant hospitality and for answering my many questions. I came away feeling very satisfied and reassured.

This session was a little on the edge. The three presenters (two real and one virtual via skype) were pushing the boundaries of their presentations - they had a wiki they shared, google documents and forms, presenting ucing google presenter and skyping into the presentation to video conference. The bandwidth and wireless was flaky so the audio was rough in places. But it was very interesting. The find was the integration between google forms and google spreadsheet. Then linking the output and data - this could be images like a graph to web pages. Quite neat and definitely worth following up. The other great thing with google forms and spreadsheets is not limited to 100 respondents like the free version of surveymonkey. Really cool. Here is the mind map of the session:

Here is the mind map:

Ian Jukes is a researcher and  presenter I have wantde to meet. His paper on Digitial Children is one I have read and enjoyed. It even warrants a page on my wiki. So I must admit I was looking forwarded to Ian session today….

….and I was not disappointed.

At about a million miles per hour, Ian presented a brilliant and witty session on Understanding Digital learners - learning in a digital Landscape.

The session was based around 7 key changes that are required to move into the 21st Century and to teach the digital natives.

• Teachers need time to catch up - Essentially taechers need to be given time to play and learn
• Moving beyond 20th C Literacies to 21st C Fluencies
• Changing our instructional approach
• Allowing the students to collaborate
• Allow students to access information natively - using their tools of choice and recreation.
• Let students create real time, real problem digital products/solutions - real rich problems
• reassess evaluation and assessment

ian didn’t say throw out all of our old practice rather keep those aspects that are good, but remove the rest.

Here is the mind map:

More to follow..

Well day one went well. The open source lab was interesting - I attended a session using Blender a 3D image and animation tool. A good session, a little to big for really good learning but it was a good taster and one I will follow up on.

I continue to be impressed with the diversity of open source products available. Wandering around the trade show, I wonder how many of the companies pedaling their wares are being seriously effected by the open source revolution. For most of the products on sale there is an alternative open source, community developed and often supported product. What is the major difference between open source and proprietory software, often I feel it is only the level of structured support available to the user.

The other session was David Warlick. David is an enjoyable and insightful speaker. I try to go to his sessions as they are fun and interesting.

The remainder of the time was spent on the trade show. This was as useful as the other sessions. Once you get past the sales pitch and start asking the interesting questions its amazing what you hear and see. These contacts are as useful as the sessions.

A Good day, roll on day 2.

This is an interesting product - a 3D immersive mathematics game. Kids play in 1st person either individual or in teams. They have instructional modules followed by the games themselves.

Can be competitive between schools and districts. Neat

C-Map Notes

URL: http://www.tabuladigita.com/

Neat game interface, a fun approach to maths and adjustable into other subjects. Would be bandwdith intensive. Cost on application