Data for Design: Using Data to Improve the Virtual Leadership Development Program

The LMG Leadership, Management and Governance Results Model (PDF file) posits that improved team-centered leadership, management, and governance lead to both improvements in the health system and to improvements in the health status of populations accessing services in the health system. MSH’s leadership development programs (LDPs) give teams located anywhere in the world but working in the same health facility or organization the opportunity to learn and apply these skills to achieve measurable results that address real workplace challenges. We wanted to know  how often teams achieve their measurable result. What are the reasons that some teams succeed while others do not? And how can we improve the content and delivery of our programs so that more teams reach their organizational development and service delivery goals over time? Answering these questions requires investigation.

In 2013, the LMG Project conducted two assessment activities: The External LeaderNet Instructional Design and Technology Audit, and The Internal Analysis of the Virtual Leadership Development Program (VLDP). The findings from these two activities were used to redesign the VLDP and the LeaderNet website, and to develop a global database for collecting data on MSH’s leadership development programs.

Instructional Design and Technology Audit

The recommendations from the audits of the VLDP and LeaderNet gave us direction towards enhancing participants’ learning experience and knowledge sharing. The main recommendation was to build a new site using an open-source social media platform. The proposed vision for the updated LeaderNet was a user-friendly environment where members could engage in multiple learning options (facilitated programs, seminars, self-paced programs) and stay engaged in a community (communities of practice, blogs) focused on leadership, management, and governance improvements for health. A key recommendation aimed at increasing the percentage of teams who achieve their measurable results was to restructure the program into two phases: Phase I) build teams’ leading, managing, and governing practices skills and abilities so they can complete the Challenge Model and develop an action plan designed to achieve their measurable result; and Phase II) Action plan implementation for committed teams with approved plans.

Analysis of the Virtual Leadership Development Program

MSH designed a study to examine the success rate of VLDP teams achieving their measurable result since 2007, and to examine enablers for and barriers to success. Success was determined by comparing the desired measurable result developed during the program to the results reported at six-month follow-up. Past VLDP participants were also interviewed to explore enabling factors for and barriers to success. Key findings that had implications for program revision and restructuring included: 1) Teams need to be systematically coached to develop measurable results that are specific (focused on a feasible goal) and measurable (successful achievement is clearly defined); 2) Teams need more time to complete their action plans; and 3) Provide short-term follow-up of teams’ measurable result achievements, and longer-term follow-up of teams’ use of learned skills over time to continuously improve the VLDP to deliver higher rates of success.

How Is LMG Using This Data?

These findings have informed updates to the VLDP program, the LeaderNet redesign, and the development of a global database for collecting program data. Some of the recent changes to the VLDP program include:

  • There are now two distinct phases to the VLDP: Phase I is planning and Phase II is implementation.
  • VLDP content has been refocused around core leadership, management, and governance practices and the challenge model.
  • Content includes improved monitoring and evaluation (M&E) resources and tools for guiding facilitators and participants to develop clear and measurable results, and for monitoring progress towards achieving those results.

In addition, the LMG Project is currently working with the LeaderNet team to build a web-based database to collect, track, and monitor the results of all leadership development programs. The use of standardized indicators has been introduced to improve the capture of results that demonstrate links between improvements in individual and organizational leading, managing, and governing practices, and improvements in health service delivery and health system performance outcomes. Using standardized indicators makes it possible to aggregate results across program offerings in different countries to show impact on health service delivery and health system performance by health area, and by WHO health system building blocks.

Looking Ahead

The overarching goals of our evaluation activities are to learn whether our LMG Leading, Managing, and Governing for Results Model holds true, and to ensure that our programs are designed to be effective. Improving our short- and longer-term follow-up of teams’ success in achieving measurable results, and aggregating the results using standardized indicators, will help us to demonstrate if our program is effective. 

Also in this edition:

Building Local Capacity One Indicator at a Time by Donal Harbick and Dr. Paul Waibale

Performance Management Plan Development Experience and Lessons Learned by Dr. Ataullah Saeedzai

Capturing In-Country Specifics and Inter-Country Diversity in a Global PMP: Approaches and lessons by Meghan Guida and Dr. Reshma Trasi