In early January, residents of Charleston, W. Va., found that their tap water had a licorice smell and a strong aftertaste, and it resulted in a number of people reporting a variety of ailments. A coal-washing chemical from a nearby plant had leaked from a 35,000-gallon storage tank into the Elk River, and residents were warned not to drink, cook with, or bathe in tap water.  Weeks after the bans were lifted, there was still a deep distrust of tap water, and one health department official reported that 95 percent of residents he had spoken to were still not using it.

“This was an accident that highlights the need for greater information-sharing and notification,” says Kevin Morley, Security & Preparedness Program Manager for the American Water Works Association. “There is the opportunity to observe risks that may be outside the domain of your control, but the key to any response action is rapid notification and closing gaps in the system of laws and regulations that, taken together, should protect our water supplies.”

“We have a moral obligation to our stakeholders and customers,” says Scott Starkey, Security Manager for the Birmingham Water Works Board. “You can live without power or transit for a while, but you can’t live without water.”

Utilities form the backbone of most modern infrastructures in the U.S., and security for these services has been in highlighted focus since 9/11. Risks range from terrorism to natural disasters to sporting marksmen with rifles, who happen to shoot at transformers and isolated substations.

When a copper theft can cost the enterprise upward of $25,000 in repairs and a stray bullet from an amateur marksman in the desert could threaten a blackout, security leaders are looking in all directions to find support and additional eyes to use for security awareness and intelligence within the Utilities sector – from the C-Suite to employees to neighbors.

Grassroots Security

Often, a strong security program is not built by just following compliance. Instead, Starkey is working on a grassroots security approach by adopting the “See Something, Say Something” campaign from the Department of Homeland Security, Starkey is working to get buy-in from more employees.

“We hold meetings in each department, and it’s always popular if you bring breakfast,” he says. “We want to establish the mindset of ‘Security is everybody’s business.’

“Security by its nature is kind of intrusive, so we try to remind employees that we aren’t necessarily watching you; we’re watching for safety,” he adds.

For David Jolley, Director of Police & Emergency Management for the Tennessee Valley Authority, grassroots security is a key part of his copper theft prevention program.

The TVA operates in seven Southeastern states, providing service to nine-million people from various fuel sources: nuclear, coal, gas, wind, solar and hydroelectric. Jolley, who is responsible for physical security at every non-nuclear site, wanted to mitigate the risk of an operations and service interruption due to copper theft, as well as the average $15,000-25,000 in damage costs for every $2,000 theft. His system is a multi-layered approach:

• Federally-commissioned investigators at the TVA have a high success rate for catching and prosecuting copper theft, so although TVA works closely with state and local law enforcement, Jolley doesn’t have to rely solely on local law enforcement for investigative support.
• Employee education comes in the form of a security awareness campaign, including videos that feature the “Chance” character and the tagline: “Don’t Chance It, Report It.” This also boosts awareness for employees in the field, who are encouraged to report anything they deem suspicious.
• Local law enforcement education helps officers to understand the importance of substations – that an outage could blackout an entire town for a prolonged period. Understanding the consequences of a substation attack or outage leads officers to pay more attention to them on their patrols.
• Neighborhood policing is a key of Jolley’s grassroots movement. Because many substations are located in remote areas, Jolley relies on residents of those areas to report any suspicious activity. TVA Police representatives were sent door-to-door at rural houses near substations, providing refrigerator magnets with an information tipline number. “We found that we have a lot of great neighbors,” says Jolley. “You have a few that will call every time they notice anything, but some of those tips have helped us disrupt burglaries.”

The combination of all of these factors has led copper theft to drop tremendously at TVA substations, Jolley says. “The loss of copper is not the main cost for us – we’re looking at the replacement and repair costs for however they got into the substation (cutting through fencing, destroying other substation parts to get to copper), as well as several hours of labor. As a cost avoidance through this program, we save, conservatively, $200,000-400,000 per year.” Other concerns included the economic impact from the disruption of operations and the risk that thieves would accidentally electrocute themselves.

Another enterprise working on bottom-up buy-in and cooperation is Iberdrola USA (IUSA), a $30-billion energy services company with holdings in 24 states across the United States. Keri Glitch, Vice President of Corporate Security, works to have security staff attend business team meetings, including providing PowerPoint slides to the business containing short security awareness tips. Security personnel are also encouraged to give 30-second security tips in meetings, to reinforce awareness of the security department’s mission, as well as boost employee alertness for anything amiss, or to prevent them from clicking on phishing emails.

She is also instituting a “business security liaison” program, in which a non-security employee would be an additional point of contact for reporting incidents or suspicions, and that liaison would report back into the security department. This approach, inspired by similar programs in the banking industry, allows employees to report to people they know well.

Metrics for Awareness

Glitch is also working to get top-down buy-in for many initiatives, and to do so, she needs viable metrics to present to IUSA executives across the country. The security charter statement for the IUSA corporate security group was approved by the CEOs of the company’s three major operating groups in the United States, so each of them are aware of Glitch’s department’s mission and scope.

“I have to be able to show that I add value – I have to justify the people in my group and the dollars that we spend,” Glitch says. “We have a roadmap – a five-year strategy – and we bring everything back to people, processes and technology. We’re able to show that this is our current roadmap, this is where we have current vulnerabilities or risks, and by taking a risk-based approach we can prioritize what we’re working on in the next three to five years. We always try to tie our metrics back to this roadmap strategy.”

“There really is an appetite and a need for this organization,” she says. “As we’ve formed this group and brought in resources to set up the new (security) organization, the business has been extremely receptive to what we’re attempting to do and partnering with us. Employees are coming forward and bringing out their concerns on multiple levels, which shows that we have support from senior leadership to move forward.”

Bruce Barnes, the Manager of Infrastructure Protection, Corporate Security, for NV Energy, is working to increase his department’s visibility to the C-Suite. NV Energy is responsible for electric power for the entire state of Nevada, including the Las Vegas Strip. Without reliable energy, the state’s main center of tourism could be shut down. Barnes has a layered approach to his security program as well, working to tier substations by value, paying special attention to “business essential” substations, including those that provide service to larger casinos and resorts. 

“We have been working diligently for years on substation security, and I feel we’re in decent shape,” he says. “But there is the question of how to protect your substations from an adversary wanting to take down the power grid.”

On April 16, 2013, at least one intruder entered the PG&E Metcalf power station near San Jose, Calif., cutting fiber-optic cables in the area and around the substation. This knocked out some local 911 services, landline service to the substation and cellphone service in the area. Then, more than 100 rounds were fired from a high-powered rifle at several nearby transformers. Due to a cooling oil leak, the transformers overheated and shut down. While there were no major power outages, Mark Johnson, a former vice president for transmission operations at PG&E, said, “My personal view is that this was a dress rehearsal” for future attacks.

To keep abreast of these threats, Barnes hosts training events with federal and local law enforcement officials at NV Energy facilities that help to strengthen relationships, improve information sharing and help build more accurate risk assessments. He has also utilized a team of company leaders to form a Corporate Security Advisory Council, which works to identify issues and enhance security across the company in a proactive manner.

To reinforce the concept of security’s value to the enterprise, he sends monthly reports to the business’s leaders of value-added activities performed by the security department. He tracks these incidents monthly so leaders can be aware of how security is improving the enterprise, as well as what risks the department is working to mitigate at the time.

Barnes also ensures security’s visibility at certain events, including the ribbon-cutting for the new 231-mile-long One Nevada Transmission Line, which put security front and center before many local and national figures.

In Birmingham, Starkey is working with video verification via surveillance prior to alerting law enforcement of an intrusion. This helps to reduce any false alarm charges from first responders, and it also aids the Birmingham Water Works’ credit rating, which improves based on the enterprise’s security stance.

Also, by enhancing the security technology on site, Starkey has seen an annual $60,000-70,000 reduction in guarding costs.

“Security is a non-revenue-generating expense,” says Starkey. “But if I can reduce the expense without reducing our coverage, that helps show how we’re looking out for the business’s bottom line. We reduce our liability for negligent security cases, which is something that many corporations are actively looking to do.”

According to TVA’s Jolley, the addition of a wide range of security technology can help boost security initiatives. For example, his department is working with surveillance, infrared cameras, video analytics for alarm verification and assessment, virtual perimeters, card readers, automated gates and contract security officers.

“The goal here, which is very different from my law enforcement history, is to deter before damage. We might not have caught the bad guy, but he didn’t cost us money,” he says.

“It’s hard to prove what you deter,” continues Jolley. “It’s easy to show the reduction in copper theft numbers, or the reduction in the number of employee-related incidents, however, other measurements are not that easy. Metrics are necessary, though, in order to show value to the company. Measurements of security system function and performance testing can assure (company leaders) that systems and equipment are reliable or showing how the number of security incidents drops after increased employee education – these are examples of metrics that can show senior leadership they are spending their money well.”

“It’s all about the dollars and return on investment – this is the way businesses are going. If you haven’t been asked about metrics yet, you will be questioned about them in the future. You need metrics and data to help illustrate security’s story and how you support the business plan,” he says.

Partnerships for Improved Service and Security

Hurricane Sandy caused more than $87 billion in property damage and left 7 million people without power. Backup was sent in from other energy companies’ locations, and even other enterprises lent generators, gasoline or manpower to get the battered East Coast back on its feet.

“Post Hurricane Katrina, there was a great movement toward the establishment of National Mutual Aid Networks,” says Morley. “These partnerships can provide access to generators or other resources in a crisis. That could include getting power restored faster for critical lifeline services, or building in redundancies between water and power.”

“Within the Utilities sector, we have emphasized an All Hazards approach, which seeks to balance the allocation of limited resources,” Morley adds. “Terrorism is a relatively low-probability threat relative to natural disasters – hurricanes, tornadoes, earthquakes – for any given utility. That means leaders should prioritize resources to prepare for those worst reasonable case incidents. This does not diminish the need to remain vigilant for malevolent acts, but there are a lot of things you do to be prepared for a hurricane that enable you to prepare for any incident by focusing on the potential consequence vs. the cause. This direction of preparation is a better value proposition.”

At IUSA, mutual aid groups are desirable for more than just emergency recovery. Glitch works to build these partnerships through some marketing and sales tactics, including cold calling other utilities or local resources to build connections for information-sharing and aid requests.

“We’ve found that it’s incredibly important to have these relationships – with our regulators, with our lawmakers, but also within different utilities,” Glitch says. “Once we have a foot in to the organization, we start to have conversations at the right level. We’re all looking for best practice ideas; we’re all looking to secure our networks and our assets for the better protection of our customers. We all want to share that information; what’s stopping us is just getting the right attention at the right level.”

The other major partnership for IUSA is within its own walls – as the head of a newly unified security department, Glitch made it a goal to build bridges between the three pillars of utilities security: Physical, Cyber and NERC Compliance. She shares fundamental training courses between the three departments, so a physical security analyst could learn the basics of cybersecurity, or vice versa, which helps to increase awareness for a more secure enterprise.

“You have to understand internal security before you turn to external factors,” says Glitch.

According to North American Electric Reliability Corporation’s Critical Infrastructure Protection (NERC CIP) Standards Version 5, energy companies will have to classify their cyber assets as Low, Medium or High Impact assets, which means that all cyber assets in the bulk electric system must have some level of protection. NERC CIP v5 starts to move from managing compliance risk to managing security risk, including cybersecurity risk, and the shift in mindset suggests a move from a zero-defect mentality to a model of continuous improvement.

“Physical security systems already work on cyber platforms – the silos are already broken down by the technology. But the next step is getting the people to speak to each other and work together. It’s a slow process, but it gets better all the time,” Glitch adds.

According to Barnes, “Our cybersecurity is only as strong as its weakest link – employees. We’re providing training for our employees, and physical and cyber security teams are working very closely now. We can’t have effective cybersecurity without effective physical security. It’s a team effort.”

Meeting New Federal Physical and Cyber Security Mandates with Front-End Engineering and Design

By Jim Fererro

Underscoring the reality that the U.S. faces both cyber and physical threats potentially affecting everything from electrical power to water supplies to people’s lives, Presidential Executive Order (EO)13636 and companion Policy Directive (PPD) were issued in February 2013.

The former focuses on improving critical infrastructure cybersecurity, and the PPD focuses on critical infrastructure security and resilience. According to the Department of Homeland Security, “these policies reinforce the need for holistic thinking about security and risk management,” and EO/PPD “implementation will drive action” toward strengthening and securing national infrastructure.

As a result, many companies are looking at their current security plans and formulating what capital projects will be necessary to meet the framework requirements coming this year.

Follow-up on Key Recommendations

A proven process for firmly defining the costs and time schedules for any capital project is the Front-End Engineering & Design (FEED) study. FEED studies not only provide a detailed survey of existing infrastructure but also identify best available technological solutions and offer design recommendations.

A completed FEED study provides exactly what management should have to align their organizations with EO/PPD directives. The study becomes the functional design basis for a security plan, which is both scalable and expandable for both physical and cyber security. Additionally, this comprehensive study can feature a detailed execution plan and AFE quality budget.

Drawing Up a FEED Game Plan

Realistically, most private critical infrastructure operations do not have adequate information on existing hardware, related equipment and infrastructure necessary to meet Framework requirements; most is either missing or incomplete. Necessary documentation may not exist because of new acquisition or over time certain things have been built anew, added to or maybe an entire facility now must blended; and, of course, key people may have come and gone. Even in a less than ideal way, however, this information must still be pulled together because, one way or the other, it can have an impact on cost savings and the ultimate success of a critical infrastructure Framework implementation.

In broad overview, the FEED study is the work process that is the precursor for actually putting in place a robust plan to protect (a) an organization’s site as well as (b) the operation’s cybersecurity which everyone needs to meet for its critical infrastructure. Comparably, a baseline understanding must be developed about the current infrastructure before a new facility is built.

A FEED survey will identify which aspects of a facility’s physical security, communications and cyber infrastructure should be evaluated. That may include access control, network design, communications, utility connections and uninterruptible power supply (UPS). In addition, the study may cover what role cameras would play in a security plan and how the images would be used, i.e. different types of analytics in the cameras, what equipment and hardware is necessary, can the organization accomplish all this with in-house resources and personnel or will outside specialists have to be hired, and how can an infrastructure be built that addresses and ties together all the cybersecurity requirements?

Key Benefits of Survey Results

When “all is said and done” in preparing the FEED survey, the logical question is: “What will the survey do for the organization’s physical and cyber security?”

First of all, deliverables and actionable information will be presented and/or opportunities highlighted for cost savings and timeline success. The latter is especially important because, in large capital projects such as for physical and cyber security, the biggest driver of cost savings is compressing a timeline. On one hand, stretching timelines out typically drives cost overruns. In contrast, developing a clear-cut execution plan, with Project Management well laid-out, is the best way to get timelines compressed. In other words, organizations are advised to approach and follow through on this new security project precisely as if it were literally the capital project that it will be.

Next, it is critical to determine the best new technologies for this physical/cybersecurity project. For example, evaluate different types of firewalls, different types of cameras and video analytics, and decide how to monitor access and physical security at a facility and on its grounds, along with cybersecurity, all from a common control room.

In the ongoing objective of keeping the plan organization-specific and not generic, the plan should also define which components of the EO/PPD recommendations apply to the organization’s facility and its daily operations. Finally, a detailed Project Execution Plan emerges with resource requirements, timelines and budget estimates. These components are all necessary in order to provide the best guarantee for a successful project brought in on-time and under-budget.

Taking It from Here

Who should take action on this national security issue? Any CEOs and top security management responsible for a critical infrastructure in the context of this Executive Order and Policy Directive should. Want more information? Additional reference material abounds at sites such as the U.S. Department of Transportation (www.dot.gov), Department of Homeland Security (www.dhs.gov) and the Federal Executive Branch (www.usa.gov).

Not only is FEED a proven path toward project success on large capital projects in the real world, it is the ideal tool for identifying the federally-mandated physical and cyber security requirements in 2014 with a concrete, well-defined and sound plan. A FEED study defines the scope, the project budget and a reasonable timeline. With critical infrastructure and lives at stake, it removes the guesswork in providing the best possible security.

About the Author: Jim Fererro is Co-Founder & Senior VP at Houston-based GlobaLogix.