In my previous blog "Proof of Concept (POC) vs Proof of Value (POV)", March 11, 2019, I highlighted the differences between a POC and a POV. While the outputs are different, they can both be accomplished similarly. As such, we will examine some of the things you should look for in a Proof of Concept.
For simplicity, we can break down a POC into 4 phases:
Once you “sign up” for a POC , you should receive :
- A summary of the requirements and a pre-install checklist
- POC Sample Evaluation Criteria to help you with Product Evaluations and defining Use Cases.
The initial step you’ll take in the Preparation Phase is to define the objectives. What exactly are you looking to accomplish with the POC? This will help hone and establish what the Success Criteria should be. Examples may look like:
- Provide visibility and create an audit trail of engineering actions on industrial controllers
- Detect attacks on the industrial networks resulting from successful hacking operations, malware, ransomware, or trojans
- Identify Indicators of Compromise (IOCs)
- Identify Indicators of Attack (IOAs)
- Develop a complete Asset Inventory of everything on the industrial control network
- Gain deep level insight into the industrial network
- Validate that the correct version, firmware, and patches are on the industrial controllers
After identifying our Success Criteria, it is time to scope the project. Making sure the project is properly scoped will ensure you have enough resources to accomplish the goals defined in the success criteria, as well as prevent open-ended product evaluations.
At this stage, you’re well equipped to determine if any Operational Risks are present. Putting them in writing is a great safety measure to ensure the message is heard and agreed upon. Consider things like planned maintenance periods, human risks, environmental dangers, existing work tickets, scheduling or resource constraints, etc.
By combing the Success Criteria and Scope, it is possible to now determine which part of the industrial enterprise to focus our efforts on. This information comes in particularly handy during the POC kickoff call.
The Deployment Phase starts when the vendor engineer arrives at your site and begins working with you to rack and stack the equipment. The vendor engineer will configure the equipment, connect it to the network and check it for operational readiness along with any needed tuning.
The engineer should work with you to navigate the product, provide you with some basic training, and steer towards accomplishing the Success Criteria that was previously defined. Once the goals have been accomplished, a summary report is drafted and delivered to you during the POC read-out meeting. During this meeting, you should see the results of the solution in achieving the goals you defined, any risks that were identified, and any findings that you may want to investigate.
A clearly defined initial scoping, as well as the preparation, tuning, execution of the project can help an efficiently run a POC and determine the suitability of the product or solution you are evaluating to help the visibility and security you are seeking to achieve in your OT environment.