About this interactive
The lesson follows one outage of an application with hundreds of thousands of users.
Before it is declared: things work, and dashboards show the usual stats. Then signs of malfunction appear. An email alert goes out that nobody reads. A more serious alert texts and calls the on-call technician, who checks the dashboard, tells customer support he is looking into something, and asks whether it is a true positive and how urgent it is.
Declared: the system goes offline and the technician declares an incident. That sets off notifications: customer support, an incident response manager to coordinate, database administrators and developers. Support tells callers the team knows and has no estimate yet. The team traces the problem from the application to the database server.
Restored, cause unknown: a database restart brings everything back. Everyone is told it seems fixed but the cause is not confirmed, the team watches the systems, and support updates customers.
Follow-up: the cause turns out to be application code calling a stored procedure wrongly. A postmortem changes the process (alert earlier, bring the whole team in earlier), schedules the code fix through development, and an after-action report records it all.
(The video says the restart contained and eradicated the issue. It restored service by ending the runaway process; the faulty code that caused it was still in place until the fix.)
About TechKnowSurge
TechKnowSurge builds IT and cybersecurity professionals through hands-on, concept-first training built around real understanding — not memorization. Free interactive tools, structured programs, and 25+ years of real-world experience, all in one place.
Explore free tools and programs →