How to review progress

From Regional Knowledge Resource Kit

Return to Stage 5 - Implementation
Return to "Developing an Information and Knowledge Strategy" Quick Links Page

1 Background
2 Facilitation level
3 Objectives
4 Materials
5 Time
6 Directions
- 6.1 Part 1 - Collect Stories of Significant Change (approx. 1 hour)
- 6.2 Part 2 - Select Most Significant Change Stories
7 Next steps

Background

The implementation of an Information and Knowledge Strategy often results in unpredicted outcomes. Consequently it's important to evaluate progress at regular intervals. The purpose of this evaluation is threefold:

understand what's happened and what could be done to improve how it's going
generate new ideas
develop a common understanding among the decision makers on the merit or worth of the efforts

This last objective is difficult to measure because in a complex and interconnected workplace it's difficult to ascertain all the cause and effect relationships that results in what we see as outcomes. For example, you might introduce After Action Reviews (AAR) in your organisation. This intervention is designed to create knowledge through personal and group reflection but once the practice is in place is it the AARs or something else creating new knowledge? This knowledge, the argument goes, should create new behaviours. But is it the knowledge gained from the AAR, or something else, creating the behaviours? Finally these new behaviours should impact organisational outcomes. Again, are the new behaviours creating the impact or something else? There are two many causal links in this complex system to know for sure. In these situations the best strategy is to capture stories of change and use them to persuade people of the learning initiative’s effectiveness.

For this reason we use the Most Significant Change technique to monitor the impact of the knowledge strategy.

[edit]

Facilitation level

Intermediate level facilitation skills are required.

[edit]

Objectives

As mentioned above, the objective of this activity are threefold:

understand what's happened and what could be done to improve how it's going
generate new ideas
develop a common understanding among the decision makers on the merit or worth of the efforts

[edit]

Materials

A room for cafe style seating. Have between 4-10 people per table.
Each table will require a facilitator.
Each table will need a white board or wall (2m wide) with flip chart paper on it.
Copies of your Information and Knowledge Strategy for each person participating in the workshop (don't hand these out at the start - wait until the appropriate point as shown in the "Directions" below).

[edit]

Time

The Most Significant Change workshop has two parts:

Collecting the significant change stories - allow 1 hour.
Selecting the most significant change story per table - allow 90 minutes.

You will then need a further 3 hours to plan and document the actions that will be undertaken for the next three to six months, as shown in How to decide new action items (or if time is short, there is an option in the "Variations" section of How to decide new action items that can shorten it to 1 hour).

[edit]

Directions

[edit]

Part 1 - Collect Stories of Significant Change (approx. 1 hour)

Have a stack of story capture sheets ready.

Then ask the group to think back over the time since the Information and Knowledge Strategy has been implemented or since the last review workshop and ask the question:

"Since the implementation of the Information and Knowledge Strategy (or since the last review workshop), in your opinion what are some of the significant changes you've noticed in the organisation in how you use information and knowledge?"

Ask people to recall specific instances or anecdotes. This part of the monitoring process is the same as running an anecdote circle.

When a story is shared of a significant change, ask the storyteller why they thought it was significant. "Why was that change significant for you?"

As the stories emerge keep a track of them and then at the end of the session ask the people who told a story to fill in the story sheet.

[edit]

Part 2 - Select Most Significant Change Stories

[edit]

Story selection (approx. 45 minutes)

The selection process lies at the heart of Most Significant Change (MSC). Without selection you are not doing MSC.

Remember that the aim of the selection process is to generate conversation, discussion, debate so that participants learn about the impact the knowledge strategy is really having, and about one another—-what they see as important and what as a group they see as important.

Try to work with five or six stories per table.

Use a whiteboard with five columns: Story Title, Facts, Opinions, Voting, Why Selected.

Ask each participant to take turns reading a story. Remind them that the selection process works best if the stories are heard because we are dealing with oral stories.
After each story is read ask the participants about the essence of the story, key words, facts they heard. The facilitator writes this in the Facts column. Eliciting facts help everyone really hear and remember the story. Try and avoid eliciting opinions at this stage but if opinions are forthcoming, move to the next column and ask for reactions and opinions, both positive and negative, about the story. But don’t compare to the other stories. You might divide the Opinions column into +ve and -ve.

Then read the next story. Repeat steps 1 and 2 until all stories are done.

When all the stories are read, and facts and opinions have been elicited, the facilitator should then read back all the notes in the Facts and Opinions columns. This gives participants some thinking time before voting starts.

The next step is to pose the question for the selection process.

“Which story illustrates the most significant change in how information and knowledge is used in the organisation?"
Then, with a show of hands, ask people to vote. Write up the number of votes for each story in the Voting column.
Then start with the lowest vote (hear the minority voice first because they might have a radically different perspective the others should hear) and ask why these people chose these stories. Record why they chose these stories. Documenting the reason for selection is the most important part of the process and should be done carefully and fully. Get all the reasons why people made their votes.
Once all the reasons why people have voted the way they have has been drawn out, the facilitator encourages debate and dialogue among the participants about which story is most significant.
Then re-vote.
After the re-vote, negotiate which stories will be chosen. Ideally a single story is selected but this is not always possible. And because the aim is to generate discussion, it’s not vitally important how many stories are selected. Here it is important to gain agreement from the group, even if it is to agree to disagree, which stories will be selected. If someone disagrees with a story that is chosen by the majority, document that a person disagrees strongly. Include any caveats with each story and document clearly why the story was selected.

[edit]

Table discussion (approx. 30 minutes)

After the main selection process set aside some time for reflection. Hand copies of your Information and Knowledge Strategy to each person and draw their attention to:

The list of actions that was compiled in Stage 4 - Sensemaking & action planning.
The projects and small actions that have been the priority for the past three to six months (that is, since the Information and Knowledge Strategy was first implemented or since the last review workshop).

Ask the table group the following questions:

What was not in the stories that you thought might have been there?
Are there any lessons for practice in the stories?
What actions should now occur? What actions in the Information and Knowledge Strategy should be dropped, kept or created?

The table documents which actions in the Information and Knowledge Strategy they think should be dropped, kept or created.

[edit]