Testing configuration Changes

By winthrop.polk ·
What is the best way to test configuration changes in a "non-production environment" (meaning we can't just use the system for a couple of days to see what happens).

Configurations could include removing/adding software, diabling services, firewall changes, port redirection, changing security settings, implementing local user policies/groups, installing virus protection, etc.

Once all this is done to a devices that has had no security for years, how can I test it and make sure no loss of functionality has occured? Again, the test has to be in a non-production environment.


This conversation is currently closed to new comments.

Thread display: Collapse - | Expand +

All Answers

Collapse -

Not sure what you mean

by SKDTech In reply to Testing configuration Cha ...

If you are talking a test environment then you would want to create a lab/bench where you can mess with things while isolated from your main environment to see if it is going to break anything before you put it into production. You can use virtual machines for testing software installations and patches but to test hardware you need to physically duplicate the machines you are wanting to test.

Also you would want to document some typical procedures that you could run in the test environment to check planned system changes for potential issues.

Collapse -

Not Really a test bench candidate

by winthrop.polk In reply to Not sure what you mean

What we will be doing is first changing configurations on a secondary, then switching operation to the secondary and changing configurations on the primary. It is impractical to setup a test bench for this, we are taking 300 computers to be hardened without a domain controller in less than 3 months.

What I am trying to figure out is what constitutes a test. I realize we may need one type of test for each type of change. For example, using the same test for new patches and software uninstalls is probably not appropriate.

If I know exactly what the computer is supposed to be doing, and I harden the **** out of it so that it really can't do much else, what is the best formal and documentable method of testing to ensure the computer still does what it is supposed to be doing? Are there any guides for "testing configuration changes"?

Collapse -

you could find....

by ---TK--- In reply to Testing configuration Cha ...

A beta group of 10-20 people that are willing to be a Guinea pig. Or maybe one person from each department, test it on them, and then if all goes well roll it out to the rest of the environment, groups of people at a time. So that you don't get pounded with calls.

The firewall changes and port redirections are going to be the biggies. I would test those out on the weekend, make the changes and test it out on your self.

Also wait a little bit more people with more experience than I will post up... this is a good question...

Collapse -

Good tips, but.....

by winthrop.polk In reply to you could find....

I appreciate it Mr IT guy, but these changes will occur in a power plant with 150 computers and no domain controller.

Each computer serves a very specific function (e.g. historian, process controller, operator interfaces, etc...).

I mentioned in a previous post that we cannot simply use the computer and see if it works. We have to have a testing plan and implement it.

What I had envisions, which may not be accurate, is a testing procedure (perhaps implemented via a custom designed testing program)that can:
e.g. - for closing/blocking ports
1. monitor traffic coming from and to the active main software or the device as a whole.
2. veryify that it is functioning properly.

e.g. - uninstalling software
1. verify that "this doesn't adversely affect the main functionality of the system".

e.g. - installing firewall
1. verify that no needed ports are blocked.

Again, this is in a power plant. I cannot shut down the plant if I want to keep my job. We can not implement changes nor test changes in a adhoc fassion. Everything must be planned. We cannot simply use the device for a couple of days and say "thats our test"; if the test is a failure in the production environment, chances are high that the plant will be shut down and I will be fired.


Collapse -

Somewhat general question, but...

by robo_dev In reply to Testing configuration Cha ...

the best way is to:

a) build a test environment that mirrors your production environment

b) develop and execute test cases to validate that things work as intended.

Is your concern more about building a test environment or doing the testing?

Collapse -

Great! Your response is the most applicable

by winthrop.polk In reply to Somewhat general question ...

A real testing environment is out this year, but is in the works for next.

Our testing environment will consist of the reduncancy of the working system and a laptop(i.e. (1) isolate the back up (2) do your changes on the backup (3) test the backup (4) switch operation to backup (5)isolate primary, make changes, test, put primary back into service.). Done.

The only part I am having problems with is the testing (3 above). I need a formal testing procedure, gobys, software, or something......anything..... that will give me some kind of basis in developing my testing plans.

Related Discussions

Related Forums