LTVplus

Major Incident Management for MSPs: How to Run a P1 at 3 a.m. Without Waking the Whole Team

MSP support team working on a P1 incident at 3am

Key takeaways

  • A major incident gets its own faster process, with named roles and a fixed update rhythm, separate from the normal ticket queue.
  • A priority matrix (impact × urgency) turns "should I wake someone?" into a rule the overnight responder can follow.
  • For MSPs, one of the highest-impact P1s is a shared-tool outage that hits many clients at once, so one parent ticket should link a child ticket for every affected client.
  • One trained responder with authority to declare, pre-approved containment steps, and a ready-made first update can start a P1 before anyone else is woken.

Many MSPs already have priority levels in their PSA. P1 through P4, each tied to an SLA timer. What fewer MSPs have is a P1 process the person on shift overnight can run alone. Without one, a single bad alert turns into a group call, and five engineers lose sleep over a problem two of them could have handled.

This guide covers major incident management for MSPs: what counts as a P1, who does what, the step-by-step process, and how to staff the night so only the right people get woken.

Still deciding how to cover nights in the first place? Start with our guide to MSP after-hours support. For everyday ticket escalation between tiers, see how to structure your MSP help desk.

What is major incident management?

Major incident management is the process for handling incidents that seriously disrupt a client's business, such as a full office outage or ransomware. It runs faster than standard incident handling and uses named roles, a dedicated communication channel, and a fixed client update schedule until service is restored.

ITIL describes major incidents as the ones that seriously interrupt business activities and need a more urgent fix than a normal incident. In practice, the incident leaves the regular queue and gets its own team, led by one person.

For an MSP, there's an extra layer because a single root cause can hit several clients at once.

Major incident vs. standard incident

Standard incidentMajor incident
ImpactOne user or one functionA client's business, or several clients
Who works itWhoever picks up the ticketNamed roles led by an incident commander
CommunicationTicket notes and repliesA dedicated bridge or channel, plus scheduled client updates
PaceWithin the SLA queueImmediate, worked until service is restored
AfterwardTicket closedPost-incident review with action items

What counts as a P1 incident?

A P1 incident (also called a SEV1 incident) stops or seriously threatens a client's business, with no workaround. Everything below that line follows your normal escalation path and SLA timers.

Incident severity levels

A common setup is four incident severity levels. Here's how they map to real situations:

PriorityWhat it meansMSP exampleWho gets involved
P1 / SEV1Business stopped or seriously threatened for one or more clientsClient office fully offline; ransomware on a server; an RMM, PSA, or Microsoft 365 tenant issue hitting several clientsOn-shift responder, incident commander, technical lead, client contact
P2 / SEV2Major function degraded, workaround existsEmail down for one departmentOn-shift responder, Tier 2
P3 / SEV3One user or a minor function affectedOne laptop can't reach the VPNTier 1 queue
P4 / SEV4Request or low-impact issueNew software installStandard queue

(We left response times out on purpose. Those belong in your client contracts, and they vary by plan. If you're standardizing them, our guide to MSP SLA management covers how to tier them.)

One more note on ransomware:

  • A ransomware P1 is also a security incident. Pull in your security lead or SOC early and follow the client's incident response plan for evidence, legal, and insurer steps.
  • Take containment or recovery steps only within the responder's approved authority, and preserve evidence before doing anything that could change it.
  • NIST's updated incident response guidance, SP 800-61r3, is a good reference if you're building that plan. Our guide to MSP security operations covers the team side.

What a P1 is NOT

Priority inflation is common. Non-urgent issues get tagged P1, everyone gets paged, and over time people tune alerts out, including the real ones. These can usually wait until morning:

  • One server fails in a redundant cluster, and clients don't notice
  • A VIP's printer stops working

The VIP example matters for MSPs. Client pressure is one of the easiest ways for priorities to creep up. Rate each incident by its impact on the business, whoever called it in.

How to build an incident priority matrix

An incident priority matrix sets the P-level from two inputs:

  • Impact: How many users, clients, or revenue-critical systems are affected?
  • Urgency: How fast is it getting worse, and how soon will the business feel it?
Impact ↓ / Urgency →HighMediumLow
HighP1P2P3
MediumP2P3P3
LowP3P4P4

To make it usable at 3 a.m., turn the matrix into a few plain questions the responder can answer in under a minute.

  • Impact: Is a client's business stopped? Is more than one client affected? Is client data at risk? A yes to any of these means high impact.
  • Urgency: Is it getting worse? Will the business feel it within the hour?

Then read the priority off the matrix. Multi-client outages and suspected compromises are rarely low urgency, so they usually land at P1 or P2.

Who does what during a major incident?

Clear incident management roles and responsibilities are what keep a P1 from turning into ten people asking the same question in a group chat. Google's SRE workbook builds its model around an incident commander, a communications lead, and an operations lead. PagerDuty's incident response docs add a scribe. For an MSP, these four roles cover most P1s:

  1. Incident commander (or major incident manager): Declares the incident, sets the severity, assigns work, and owns decisions and the clock. They coordinate and leave the hands-on fixing to the technical lead.
  2. Technical lead: Works the fix with the engineers. This group should be the only one making changes to client systems during the incident.
  3. Communications lead: Sends client and internal updates on schedule and fields incoming questions, so the technical lead isn't interrupted every ten minutes.
  4. Scribe: Keeps a timeline of what was seen, decided, and done. Your post-incident review starts from this timeline.

MSPs also need to decide who talks to the client's decision-maker. That might be the account manager, a vCIO, or the communications lead. Name the person in advance so the client hears from one voice.

At 3 a.m., one person may hold two roles until help arrives. That's fine. Just split the incident commander and technical lead roles first, as soon as a second person joins.

Expert tip: When the incident commander starts troubleshooting, nobody is watching the clock or the client. If you catch yourself typing commands while running the incident, hand the keyboard to someone else.

Step-by-step: the major incident management process

The major incident management process follows the same arc as your regular incident management process. The difference is speed, and how much is decided before anything breaks.

Step 1. Detect and declare

  • An alert fires, a client calls, or a technician spots something wrong. The responder checks the priority matrix and declares a P1 in the PSA.
  • Aim for monitoring to catch it before the client does. In New Relic's 2025 Observability Forecast, 41% of IT and engineering leaders said they still learn about service interruptions through customer complaints, incident tickets, or manual checks.

Step 2. Assemble the right people

  • Page the roles the matrix calls for.
  • Open the bridge call or incident channel so everyone works in one place.

Step 3. Contain and communicate

  • Stop the spread first with the actions your responder is approved to take. Then send the client their first update.
  • Put the next update time in the first message ("Next update in 30 minutes") and keep that rhythm even when there's no news.
  • Match the exact interval to the client's SLA.

Step 4. Resolve and verify

  • Restore service first with a failover, rollback, or workaround. Then chase the root cause.
  • Before calling it fixed, check with the client that service is actually back. A cleared alert doesn't always mean users can work.

Step 5. Hand off and close

  • Brief the day shift with a short summary.
  • Send the client a closing update that explains what happened and what was done to fix it.

Step 6. Run a post-incident review

  • Keep it blameless and hold it within a few days, while details are fresh. Give every action item an owner and a due date, and open a problem record so the root cause gets a permanent fix.
  • Reviews help keep the same P1 from coming back, which matters when teams are already stretched. In the same New Relic report, engineers said they spend 33% of their time fighting fires or addressing disruptions.

Need overnight coverage that follows your incident playbook? LTVplus provides dedicated technicians and NOC engineers who work inside your PSA and follow your escalation rules. Book a free consultation.

Tips to run a P1 at 3 a.m. without waking everyone

A good process still falls apart if the person on shift can't act on it. These practices decide who's awake and what they're allowed to do on their own.

  • Give the overnight responder authority to declare. Training isn't enough. Write down that they can call a P1 without asking permission and that the incident commander can downgrade it later.
  • Escalate one person at a time. Page one primary responder. If they don't acknowledge within the window you set, page the secondary automatically. Wake the incident commander only when the incident needs coordination or client communication. Build the chain by role ("on-call network engineer"), not by name, so it survives rotation changes.
  • Write runbooks for a tired brain. Keep one for each critical system, covering how to confirm the outage, how to restart services safely, where the backups are, and who to call next.
  • Pre-approve containment actions. List what the responder can do alone such as isolating an endpoint, disabling a compromised account, or failing over to a secondary firewall. Keep snapshot restores off the list for suspected security incidents, since they can wipe evidence or bring back an infected system.
  • Keep the engineer off the phone. Route client calls to the incident commander or communications lead, so the person fixing the problem can keep fixing it.
  • Use automation for the paging. Alert correlation and PSA rules can group related alerts and page the right role. A trained person still decides what the alert means and what to tell the client.
  • Consider follow-the-sun coverage. If your overnight work is handled by a team in another time zone, 3 a.m. for your clients is a regular working afternoon for them. Nobody has to be woken to start the response. An internal night shift or an on-call rotation can work too. Our guide to choosing a NOC partner covers what to look for if you go the outside route.

Major incident management best practices for MSPs

These cover the parts an internal IT team never has to think about:

  • Use one parent ticket per incident. Link a child ticket for each affected client. Billing, SLA reporting, and client updates stay clean even when one outage touches ten accounts.
  • Segment client communication. A breach at one client isn't news for the others. A shared-tool outage, on the other hand, gets one consistent message to every affected client.
  • Keep a contact sheet for each client in your documentation tool. Include after-hours contacts and who can approve downtime. Our guide to MSP documentation best practices covers where this fits.
  • Have vendor escalation paths ready. Store support numbers and contract IDs for Microsoft, your ISPs, and key software vendors where the overnight responder can find them.
  • Track time to declare and time to first client update. These two numbers show whether your process works, and both are mostly in your control. Treat them as internal measures. For SLA compliance itself, go by the client contract's definitions and your ticket records. Our guide to MSP SLA management covers how to track it.

Build the P1 playbook before you need it

Most of the work happens before the incident. Once your overnight responder can make the P1 call and start the response alone, your senior engineers only get woken when the matrix says they're needed.

LTVplus is the managed technical support partner MSPs trust to handle their most important clients. If you'd like to test the fit first, our 3-month pilot comes with no long-term commitment.

Book a free consultation or learn more about managed technical support for MSPs.

Frequently asked questions

What is major incident management in an MSP?

Major incident management is the process an MSP follows to handle incidents that seriously disrupt a client's business. It uses a dedicated response team, defined roles, faster escalation, and scheduled client updates to restore services as quickly as possible.

What qualifies as a P1 incident for an MSP?

A P1 incident is an issue that stops or seriously threatens a client's business with no workaround. Think a full office outage, ransomware on a critical server, or an RMM, PSA, or Microsoft 365 problem hitting several clients at once.

How often should MSPs update clients during a major incident?

Follow the interval in each client's SLA. Put the next update time in your first message, and keep to it even when there's nothing new to report.

Who should be woken up for a P1 after hours?

Start with one primary responder. Page the secondary only if the first doesn't acknowledge in time, and wake the incident commander when the incident needs coordination or client communication.

Can an outsourced team handle P1s overnight?

Yes, if they work inside your PSA, follow your runbooks, and have written authority to declare and contain. A team in another time zone can start the response during their normal working hours.