Date of event: June 2008
Emails generated from buyers via the website were stored in the email server for 8 weeks and sent out at one go after that.
REA’s operation team upgraded the email server which was not tested.
Why the cause could happen?
Operations team back then was independent from other technical teams, especially the QA team.
Server upgrades, etc done by the operations team were not visible to the other team or business.
What we learned by it?
Operations team should engage with developers and QA team. Work done Operations team are expected to be undergo the QA team.
As result of this, the QA team was expected to have the system administration side of skills set.