Discussions

Apocalypse Now!

Tags:
+
0 Votes
Locked

Apocalypse Now!

alxcsby
Had something in the way of a fright this weekend, my three mobile users were all bunched in a single hotel room and decided to do their work from the hotel room.
No problem, VPN works wonderfully. Somehow or other, one of the computers came back as thoroughly infected with malware as a common street ***** with syphilis (to paraphrase Schopenhauer).
I got it all fixed up, but realized there was a possibility of serious problems. For a network which relies heavily on mobile users, the possibility of an infected computer with a self-replicating virus being plugged back into the LAN is pretty unnerving.
I'm curious if anyone has any suggestions on how to recover from a total network infection. Containment is already being implemented, but this is a "what if" scenario.
Imagine you walk in on a Monday morning to a thriving culture of viruses on both your networks (LAN, mail, FTP...the works). What would you do?
  • +
    0 Votes
    CharlieSpencer

    I'm not the company security specialist, but I'd start by disconnecting or shutting down all switches, routers, hubs, WAPs, and other communications infrastructure to prevent further spreading.

    +
    0 Votes
    Jaqui

    with no windows boxes I never have to have that nightmare. :)

    Palmetto is right though, stop the spread by pulling the plug then clean everything before brining it back online.
    [ other option, use the backup from friday to restore each machine after cleaning it. ]

    +
    0 Votes
    alxcsby

    I wish I could free my network from the burden of Windows. Unfortunately, I cannot (for several reasons).
    My thinking on it was what you mentioned there at the end-
    1. Cut SMTP to prevent spread to clients via e-mail.
    2. Nuke everything and use my daily backup (what is this friday only backup? Must be a non-Windows convenience)
    3. Process all mail in queue to find any infections.
    4. Bring it all back online.
    ---
    That's my thoughts on it anyhow.

    +
    0 Votes
    neilb@uk

    Well, you did ask "what do you do on a Monday morning if..."

    Had you asked "what do you do on a Thursday morning if...", Jaqui would no doubt have pointed you at the Wednesday backup.

    +
    0 Votes
    CharlieSpencer

    Isn't one of the signs Jaqui using the words, "Palmetto is right"? :-)

    Hope you're not getting too much rain, pal.

    +
    0 Votes
    jdclyde

    First, have your users run as a limited user instead of Admin.

    Second, deny SMTP to or from anything but your mail server.

    Have a baseline of your network usage, especially during off hours. If you get an infected system, it will suck up bandwidth day and night (provided it is turned on) so will give you a clue.

    Watch the managed AV logs. Both for finding infection attempts AND for update failures. Virus have a cool tendency to turn off your AV, so that is another sign of trouble.

    Do you run a firewall on the pc's? You might want to, and then allow only specific traffic to access the LAN.

    +
    0 Votes

    RE:

    alxcsby

    I should stress this is a hypothetical situation, and not actually going on, I'd be a lot more panicked, I'm just bored.
    ---
    I like the baseline idea, hadn't thought of that.

    +
    0 Votes
    jdclyde

    this is all something you HAVE to do ALL the time to watch for the infections.

    Welcome to the world of a Net Admin. B-)

    +
    0 Votes
    alxcsby

    I am doing everything except the baseline, which strikes me as a little excessive for normal operations. It's a well-taken care of system, but I thought it'd be fun to see if there were any gonzo solutions for "the big one".

    +
    0 Votes
    jdclyde

    but "when".

    it is a lot easier to clean one system than to have something running on your network for a few days before you find it.

    Good luck with that.

    +
    0 Votes
    deepsand

    If your system is as large as it sounds, it's large enough to warrant being 100% backed-up every day.

    +
    0 Votes

    Aye

    alxcsby

    And ours is. In two locations.

  • +
    0 Votes
    CharlieSpencer

    I'm not the company security specialist, but I'd start by disconnecting or shutting down all switches, routers, hubs, WAPs, and other communications infrastructure to prevent further spreading.

    +
    0 Votes
    Jaqui

    with no windows boxes I never have to have that nightmare. :)

    Palmetto is right though, stop the spread by pulling the plug then clean everything before brining it back online.
    [ other option, use the backup from friday to restore each machine after cleaning it. ]

    +
    0 Votes
    alxcsby

    I wish I could free my network from the burden of Windows. Unfortunately, I cannot (for several reasons).
    My thinking on it was what you mentioned there at the end-
    1. Cut SMTP to prevent spread to clients via e-mail.
    2. Nuke everything and use my daily backup (what is this friday only backup? Must be a non-Windows convenience)
    3. Process all mail in queue to find any infections.
    4. Bring it all back online.
    ---
    That's my thoughts on it anyhow.

    +
    0 Votes
    neilb@uk

    Well, you did ask "what do you do on a Monday morning if..."

    Had you asked "what do you do on a Thursday morning if...", Jaqui would no doubt have pointed you at the Wednesday backup.

    +
    0 Votes
    CharlieSpencer

    Isn't one of the signs Jaqui using the words, "Palmetto is right"? :-)

    Hope you're not getting too much rain, pal.

    +
    0 Votes
    jdclyde

    First, have your users run as a limited user instead of Admin.

    Second, deny SMTP to or from anything but your mail server.

    Have a baseline of your network usage, especially during off hours. If you get an infected system, it will suck up bandwidth day and night (provided it is turned on) so will give you a clue.

    Watch the managed AV logs. Both for finding infection attempts AND for update failures. Virus have a cool tendency to turn off your AV, so that is another sign of trouble.

    Do you run a firewall on the pc's? You might want to, and then allow only specific traffic to access the LAN.

    +
    0 Votes

    RE:

    alxcsby

    I should stress this is a hypothetical situation, and not actually going on, I'd be a lot more panicked, I'm just bored.
    ---
    I like the baseline idea, hadn't thought of that.

    +
    0 Votes
    jdclyde

    this is all something you HAVE to do ALL the time to watch for the infections.

    Welcome to the world of a Net Admin. B-)

    +
    0 Votes
    alxcsby

    I am doing everything except the baseline, which strikes me as a little excessive for normal operations. It's a well-taken care of system, but I thought it'd be fun to see if there were any gonzo solutions for "the big one".

    +
    0 Votes
    jdclyde

    but "when".

    it is a lot easier to clean one system than to have something running on your network for a few days before you find it.

    Good luck with that.

    +
    0 Votes
    deepsand

    If your system is as large as it sounds, it's large enough to warrant being 100% backed-up every day.

    +
    0 Votes

    Aye

    alxcsby

    And ours is. In two locations.