How to review progress

From Regional Knowledge Resource Kit

Return to Stage 5 - implementation
Return to "Developing a Regional Knowledge Strategy" Quick Links Page

Background

The implementation of a knowledge strategy rarely results in predictable outcomes. Consequently it's important to evaluate progress at regular intervals. The purpose of this evaluation is threefold:

• understand what's happened and what could be done to improve how it's going
• generate new ideas
• develop a common understanding among the decision makers on the merit or worth of the efforts

This last objective is difficult to measure because in a complex and interconnected workplace it's difficult to ascertain all the cause and effect relationships that results in what we see as outcomes. For example, you might introduce After Action Reviews (AAR) in your region. This intervention is designed to create knowledge through personal and group reflection but once the practice is in place is it the AARs or something else creating new knowledge? This knowledge, the argument goes, should create new behaviours. But is it the knowledge gained from the AAR, or something else, creating the behaviours? Finally these new behaviours should impact organisational outcomes. Again, are the new behaviours creating the impact or something else? There are two many causal links in this complex system to know for sure. In these situations the best strategy is to capture stories of change and use them to persuade people of the learning initiative’s effectiveness.

For this reason we use the Most Significant Change technique to monitor the impact of the knowledge strategy.

Facilitation level

Intermediate level facilitation skills are required.

Objectives

As mentioned above, the objective of this activity are twofold:

• understand what's happened and what could be done to improve how it's going
• generate new ideas
• develop a common understanding among the decision makers on the merit or worth of the efforts

Materials

• A room for cafe style seating. Have between 4-10 people per table.
• Each table will require a facilitator
• Each table will need a white board or wall (2m wide) with flip chart paper on it

Time

Most Significant Change has two parts: 1) collecting the significant change stories and 2) selecting the most significant change story per table

Part 1 allow 60 minutes Part 2 allow 90 minutes

The set aside 60 minutes to debrief and document the actions that will be undertaken.

Directions

Part 1 - Collect Stories of Significant Change

Have a stack of story capture sheets ready.

Then ask the group to think back over the time since the knowledge strategy has been implemented or since the last monitoring session and ask the question:

"Since the implementation of the knowledge strategy (or since the last monitoring session), in your opinion what are some of the significant changes you've noticed in the Region in how you use information and knowledge?"

Ask people to recall specific instances or anecdotes. This part of the monitoring process is the same as running an anecdote circle.

When a story is shared of a significant change, ask the storyteller why they thought it was significant. "Why was that change significant for you?"

As the stories emerge keep a track of them and then at the end of the session ask the people who told a story to fill in the story sheet.

Part 2 - Select Most Significant Change Stories

The selection process lies at the heart of Most Significant Change (MSC). Without selection you are not doing MSC.

Remember that the aim of the selection process is to generate conversation, discussion, debate so that participants learn about the impact the knowledge strategy is really having, and about one another—-what they see as important and what as a group they see as important.

Try to work with five or six stories per table.

Use a whiteboard with five columns: Story Title, Facts, Opinions, Voting, Why Selected.

1. Ask each participant to take turns reading a story. Remind them that the selection process works best if the stories are heard because we are dealing with oral stories.

2. After each story is read ask the participants about the essence of the story, key words, facts they heard. The facilitator writes this in the Facts column. Eliciting facts help everyone really hear and remember the story. Try and avoid eliciting opinions at this stage but if opinions are forthcoming, move to the next column and ask for reactions and opinions, both positive and negative, about the story. But don’t compare to the other stories. You might divide the Opinions column into +ve and -ve.

Read the next story.

Repeat steps 1 and 2 until all stories are done.

When all the stories are read, and facts and opinions have been elicited, the facilitator should then read back all the notes in the Facts and Opinions columns. This gives participants some thinking time before voting starts.

The next step is to pose the question for the selection process.

“Which story illustrates the most significant change in how information and knowledge is used in your region?"

Then, with a show of hands, ask people to vote. Write up the number of votes for each story in the Voting column.

Then start with the lowest vote (hear the minority voice first because they might have a radically different perspective the others should hear) and ask why these people chose these stories. Record why they chose these stories. Documenting the reason for selection is the most important part of the process and should be done carefully and fully. Get all the reasons why people made their votes.

Once all the reasons why people have voted the way they have has been drawn out, the facilitator encourages debate and dialogue among the participants about which story is most significant.

Then re-vote.

After the re-vote, negotiate which stories will be chosen. Ideally a single story is selected but this is not always possible. And because the aim is to generate discussion, it’s not vitally important how many stories are selected. Here it is important to gain agreement from the group, even if it is to agree to disagree, which stories will be selected. If someone disagrees with a story that is chosen by the majority, document that a person disagrees strongly. Include any caveats with each story and document clearly why the story was selected.

After the main selection process set aside some time for reflection and ask the following types of questions:

   * What was not in the stories that you thought might have been there?
   * What are some of the provocative questions you could ask? For example, why such pathetic stories?
   * What actions should occur? What should be dropped, kept or created?
   * Are there any lessons for practice?

The selection process might take 45 minutes or more depending on the length of conversation generated.

Return to Stage 5 - implementation
Return to "Developing a Regional Knowledge Strategy" Quick Links Page