Apollo 13 – at PinkAsia2012 at Kuala Lumpur and Singapore

This year from the 16^th till the 20^th of July GamingWorks delivered 2 Apollo 13 simulations to the Asian customers. Two sessions were delivered, one in Kuala Lumpur and one in Singapore.

From both sessions we took the Lessons Learned and capture them in this report

It’s important to prioritize all incidents related to the Business Impact. 

In the first 2 rounds of the simulation the only way the team classified their incidents was High, Medium, Low. But we observe few important observations.

 

1. There was no link to the Business Impact.
2. There was no different process to be observed when an incident got a High or a Low
3. There was no clear escalation procedures defined and agreed.

 

As an outcome of the reflection the teams discover:

 

1. If the Crew’s life is in danger, we classify the call as a HIGH, if the Mission is in danger, we classify as MEDIUM and all others are LOW.
2. If we classify HIGH, we escalate directly to our Flight Director, when we classify as MEDIUM we escalate to the Incident Manager and CapCom will manage the LOW.

 

This works out well for the teams!

 

In order to demonstrate the value to the Business, IT must use a metrics.

The teams did not use the Tool for registration and monitoring in the first 2 rounds. After the round 2 reflection the team discovered the following:

The Flight Director did not share the KPI’s from the Service Strategy Document with the Incident Manager. Because of this the Incident Manager was unable to manage the calls and we could not report the current performance of the team related to the agreement with the customer.

The team improved their performance:

1. The Flight Director informed the team about the Key Performance Indicators.
2. The team developed their own Monitoring Tool (on the flip chart) to manage the incidents.
3. The team used the Dashboard (metrics) to support the Continual Service Improvement Cycle.

They used the KPI’s to manage their Incident Management process. In the round 4 the Incident Manager was able to report the 2 KPI’s to the Flight Director, directly from the tool.

 

Follow the processes is crucial to assure a constant delivery of services according to the agreement

It sounds basic, but following processes are experienced a crucial. The teams discovered that the Service Transition phase was not well performed. It seems that the processes were clear and well transferred to the teams, but during the second round, there was no clear process flow visible. It seems that it looked easy on the board, and it sounds clear but using it in the life environment of Mission Control Center is different.

Other observation was that the chaos started after ‘NOT FOLLOWING’ the process. And the reasons of not following the process were:

1. Pressure on the solution period (Crew needed answer quick)
2. Pressure by Flight Director (he forces to by pass the process)
3. Not finding a solution within the team
4. Lack of ownership
5. No escalation procedures
6. Unclear roles and responsibilities
7. Not knowing the right priorities

The teams worked hard on these root-causes and improved their performance dramatically.

 

Define clear ownership for all the work we do in an IT department.

Who owns an Incident? Who owns a Problem Record? Who owns the Known Errors? And what does the word ‘own’ means.

These were all important discoveries during the session. There were a lot of discussions and improvements.

Outcomes the teams found:

1. The Incident Manager owns the incidents and is accountable to solve them within the agreed service levels. This doesn’t mean he must solve them by himself. CapCom (Service Desk), Specialists or Supplier are responsible to solve the incidents.
2. The Problem Manager owns the Problem Records and the Known Errors. He makes them available to other roles in the team like CapCom (Service Desk). Owning means:
   1. Keep them up to date
   2. Make sure that Problems are solved
   3. And analyzed in order to define Request for Changes

Clear ownership improved the performance a lot.

Always consider the option of leveraging vendor expertise to reduce downtime. Don’t shy away from this

This behavior is seen all over the world. It seems that we don’t like to say:

“Sorry, but I cannot solve this incident within the agreed time frame. If cannot find or I do not have the specific knowledge to solve the call!” and escalate the call to supplier, to the Incident Manager, Manager Specialists or any other role who can speed up the solving process.

Is it our culture? Knowledge is power? Are we afraid of not knowing something?

 

Even after the reflection of round 3 the teams agreed on the fact that this increase the performance if we would escalate quicker as soon as we discover we could not solve the call.

 

Lessons Learned Kuala Lumpur 18^th of July

 

1	Review the metrics we have, how can it help to improve our service. Need for metrics	3
2	Prioritization of our Incidents	5
3	Review our monitoring system	2
4	Assure that processes are always in place	2
5	Assure that all roles and responsibilities are clear and in place
6	Ownership of incidents and others	2
7	Single Point of Contact, for all communication to users	2
8	Identifying critical incidents
9	How effectively use the tool, right tool and dashboard	2
10	Eliminating the non value add service
11	On time escalations and the way it helps, clear escalation path	2
12	Keep the stakeholders updated
13	Enforcement of tool and process ensures control
14	Always consider the option of leveraging vendor expertise to reduce downtime. Don’t shy away from this	2
15	Link People and Process
16	Continual Improvement
17	The CSI process from round to round, as seen in the final round things went really better
18	Learning through simulations gives a better understanding of how the process interacts, roles applied and improvement put to action.
19	Cross function understanding
20	Good tool for team building and interaction and integration
21	The value of control
22	The collaboration between silo’s
23	Different processes for different prioritization levels
24	The value of KPI’s, during day to day work, we lose track of the KPI’s. Value of KPI’s	2
25	We must make sure all roles understand each other’s roles, their activities and their problems.
26	Training is crucial
27	Follow the processes is crucial to assure a constant delivery of services according to the agreement	4
28	Clear roles and responsibilities must be fully understood and accepted.
29	SD should focus on 70% of incidents solve
30	Escalate before it’s too late

 

Take away for your organization Singapore 20^th of July

 

1	Process design needs quality time and teams involvement
2	Communication within teams needs to be improved	2
3	Documentation should be clear
4	To let the team know the business objectives and show them the point a view from the business
5	Clarity in roles and related supporting tools to enhance the process
6	Communication aspect as an ongoing medium for delivering services and to stay all on the some track	2
7	Processes, roles and responsibilities need to be clear defined	5
8	Test the processes and the workflow to ensure it works
9	Accountability, Responsibility communication
10	Team Work	2
11	Understanding the strategy and design processes according to the requirements
12	Document individual roles and responsibilities so processes and tasks can be extended smoothly
13	Cross functional collaboration is inheritable and is critical to service management
14	Process can be a great enabler and or barrier for Problem Management
15	SPOC among teams greatly improve communications effectiveness
16	Proactive monitoring
17	Knowledge for timely decisions
18	SD is a real SPOC, for success and for failure they represent the company
19	Develop tracking tools, even if its manual
20	Control chaos even if everybody was trying to help
21	Empowerment to team members
22	Clear priorities and strategy from business owner and director to design the processes better	2
23	Common understanding of the prioritization across all the process and line managers.
24	Accuracy of the database is crucial for capturing and transferring knowledge
25	Listen to the customer
26	Training and whiteboard discussions are a must

 

 

 

Share this article