An Improvement Effectiveness Review is a structured follow-up after implementation to verify that the change produced the intended result and continues to work.
Implementation answers:
Did we install the change?
Effectiveness answers:
Did the change improve the process?
Those are different questions.
Compare with the original baseline
Process Baseline establishes the starting condition before improvement.
The review should compare:
- baseline;
- target;
- current result.
Use the same operational definition whenever practical.
A before-and-after comparison is weak if the measurement rules changed.
Verify the intended result
Examples include:
- reduced cycle time;
- lower defect rate;
- higher availability;
- improved lead time;
- lower cost.
Benefits Realization helps confirm whether expected operational or financial benefits were actually achieved.
Do not close a project based only on task completion.
Check recurrence
If the project addressed a recurring problem, review whether the problem returned.
A temporary improvement may look successful for several days and then drift back.
The follow-up timing should match the natural recurrence pattern.
Check for side effects
A local improvement can create problems elsewhere.
Ask:
- Did WIP increase?
- Did quality worsen downstream?
- Did maintenance burden increase?
- Did another team absorb the work?
Process Interface Management helps identify effects at cross-process handoffs.
Confirm standardization
Improvement Sustainment Plan defines how ownership, standards, measures, audits, and reaction rules will hold the gain.
The effectiveness review should confirm those controls actually exist and are being used.
Verify behavior, not just documentation
A standard may be updated while actual work follows the old method.
Process Confirmation can verify whether the process is being performed as intended.
Observation is often more useful than checking whether a document was approved.
Decide what happens next
Possible decisions include:
- effective and sustained;
- effective but not yet sustained;
- partially effective;
- ineffective;
- unintended negative effect.
The review should trigger the next action.
Capture learning
If the result differed from expectation, record:
- what was learned;
- what assumption was wrong;
- what should change in future projects.
Lessons Learned System helps make that knowledge reusable.
Common mistakes
Reviewing immediately before the process has enough exposure, comparing with a weak baseline, checking only the primary metric, ignoring recurrence, accepting updated documents as proof of sustainment, and declaring success because all project tasks were completed are common mistakes.
Practical sequence
- confirm the original target.
- compare with the baseline.
- verify the current result.
- review recurrence.
- check side effects.
- confirm standardization.
- observe actual behavior.
- classify effectiveness.
- assign follow-up actions.
- capture learning.
The practical lesson
An Improvement Effectiveness Review protects the organization from confusing implementation with improvement.
The change is successful only when the process result improves and the gain can be sustained.
Related application
This topic also connects with Problem Recurrence Review. Use that method when the improvement requires the related operating or management discipline.