This is the third article in a three-part series about considering best practices for deploying IP security systems. The discussion is about the wider scope of best practices for deploying physical security technology on enterprise networks. Such practices are needed because many security devices and systems were designed on the assumption that the equipment would be deployed on a completely independent security network rather than in an enterprise network environment.
The authors have formed the Bp.IP Initiative to advance best practices for deploying IP-based security systems in enterprise network environments, including practices that compensate or work around the network-related shortcomings of security products currently available.
Living in a Managed World
An enterprise network environment is a managed network environment. In a managed network:
- The locations and points of attachment of devices on the network are entirely planned and tightly controlled.
- Network traffic requirements are anticipated and provided for (consistent with resource constraints.
- Optimization is given attention.
- Devices are actively monitored and offline conditions are quickly investigated and remedied.
- Computer and network access is controlled.
- Firewalls and other measures are in place to help protect the network against unauthorized internal or external traffic.
For example, in a well-managed enterprise network, a port on a network switch will be shut down automatically if an unacceptable network traffic condition occurs. There is usually a critical data backup scheme, whereby a standard backup solution is put into place and used as appropriate for various classes of data. Backup and emergency power provisions keep critical portions of the network operational (along with critical information systems) during power outages. Redundant network paths are designed and managed so that loss of a single pathway triggers a rerouting of affected network traffic.
The Role of Logs
Another aspect of a managed network environment is the use of logs to capture system and network status and event information, including computer and network security events. Logging is so important to the managing of networks that the National Institute of Standards and Technology (NIST) published specific recommendations in its Special Publication 800-92 - Guide to Computer Security Log Management. Additionally, the Internet Engineering Task Force (IETF) formalized a commonly used protocol for log entry information, the syslog protocol (Syslog is short for system log). Originally syslog was created to provide diagnostic information for network operations trouble-shooting; it is now also used for reporting security event information.
Most enterprise networks contain one or more log management infrastructures (meaning the hardware, software, and data storage used to generate, transmit, store, analyze and dispose of log data). Deploying security systems and products that support syslog reporting means that IT's existing network log management infrastructure can be used to monitor and manage the health and security of security systems on the network. Readers with a technical interest in Syslog should see the overview at: www.ciscopress.com/articles/article.asp?p=426638. In March 2009, the syslog protocol was updated (as RFC 5424) to provide a message format that enables vendor-specific extensions to be provided in a structured way. A number of network cameras from Axis, Cisco and others support syslog messaging.
Requirements for Security Systems and Devices
In a managed network environment, security systems and devices that connect to, or operate on the network must not generate prohibited traffic or in other ways act outside the acceptable parameters defined for network computers and devices.
Additionally, the computer and network security of the security systems and devices must meet or exceed the minimum standards of the network. This prevents situations that have plagued some enterprise IT departments. For example, DVRs that buyers thought were "recording devices" turned out to be Windows-based computers with no anti-virus software and unpatched older versions of the operating system - making excellent sites for virus infections that then spread from the DVRs throughout the enterprise network, disabling the video systems in the process. Download a study from Cisco about its reaction to such an experience with DVRs: http://www.bpforip.com/downloads/Cisco_IT_Case_Study_CCTV.pdf.
Finally, security systems and devices should support network management by incorporating appropriate network management features, such as syslog and simple network management protocol (SNMP), to name just two.
Assessing Systems and Devices
IT evaluators are personnel assigned the responsibility of evaluating systems and devices, for approval to be placed on the enterprise network. Since most currently available security systems and devices were not specifically designed to be installed and operated in a managed network environment, many products would fail such an evaluation unless an appropriate set of deployment recommendations were also developed and provided.
Following ISC West 2010, the authors assessed a number of access control and video surveillance products (some equipment provided directly by vendors, and some currently deployed in field installations). The purpose was to establish what would be needed - be it product configuration tuning, compensating controls or workaround procedures - to enable the products to be soundly deployed in an enterprise network environment. The assessment criteria were selected to cover a reasonable but broad sample of enterprise-class networking features and practices that can be judiciously utilized (or their absence worked around) for physical security solutions. General recommendations were developed that can be fine-tuned based on the specifics of any particular technology being considered for deployment.
The basic assessment criteria the author's applied are listed below. Note that some of the assessment criteria require thinking about the use to which the security system or device will be put, as well as consideration of the network environment in which the products will be deployed.
- Authentication. What does it take to connect to the device? Can you set the password? Can you use a secure password (such as a password longer than 8 characters, and with non-alphabetical characters in it)? Can you use two-factor authentication, Windows authentication, network authentication or other mechanisms?
- Telemetry. (Telemetry as used here means the reporting of status information and critical events over the network.) Does it generate logs visible to external log-collection mechanisms? Can you save the logs? Does it generate events? Can you integrate it with some general-use infrastructure such as syslog or SNMP?
- Network Use. What kind of network usage does it create? Is it a heavy load because it's a high resolution camera at a high frame rate? Is it priority traffic because it is an alarm or a PTZ control command? Should Quality of Service (QoS) be used to guarantee priority? What ports and protocols does it use? What firewall rules or intrusion detection system (IDS) profiles should be applied to support this traffic?
- Network Policy. What kind of data is this and what is its criticality? Is it casual "in-store customer traffic past the cookie display," or is it the door to a corporate data center room that should never be accessed at this time of day? Is it confidential information on who is in the doctor's office? Should the day shift manager really be allowed to access that data from a home laptop on the weekend? What organizational network policies cover this type of data?
- Business Continuity. What's the backup scheme? Is there/should there be redundant power or network access? Is there a disaster recovery (DR) plan to recover the infrastructure from configuration backups? Are there redundant systems that should be tested periodically? Are there single points of failure?
- Data at Rest. What kinds of video data are being stored? What data is stored in the employee access card database? Is there PII (Personally Identifiable Information)? Is there historical security-event or another type of information that might be used some day in an HR action and therefore has to be defensible in a formal corporate review or in a court?
- Infrastructure Defense. Would the system or device be an attractive target for hackers? Are encrypted connections in use? Should VPN's and/or virtual LAN's be used? Are firewall rules needed? What infrastructure services are required to securely support this mission-critical system or component?
- Network Testing. How does the device respond to network scans and penetration testing?
- Updates. What is the vendor's update policy? How are bug fixes provided? Does updating require physically touching the device, or can it be handled completely via the network?
- Hardening. What known computer or network vulnerabilities does the system or device have? What disclosure information has the vendor provided? (See the June/July Convergence Q&A Column titled, "Responsible Disclosure and Physical Security Risk" for details on vulnerability information disclosure.) Does the vendor provide hardening instructions? If not, what controls and measures should be applied?
Summary of Assessment Findings and Recommendations
Basic findings and recommendations are provided in summary form below, in the following categories: Video Server Software (VMS, DVR, NVR, video analytics software and related network storage servers and appliances); Video Cameras (network video cameras or analog cameras with network encoders); Access Control Server Software; and Access Control Equipment (controller panels, intelligent readers).
In the space of this article, it is not feasible to provide a complete examination of each type of finding and related sound deployment practice. This article is intended to introduce evaluation criteria commonly used in evaluating enterprise networked systems and devices, and provide some rationale for applying them to security system computers and devices.
A key recommendation for all networked security systems is to put the computers and devices on a dedicated private subnetwork, which provides logical separation of network traffic running on the same physical cable (see the Wikipedia topic: Subnetwork). This improves network performance and helps make sure the systems and devices have a stable environment to operate in without interference from any other network traffic sources.
Security systems, like any other server-based software systems, have backup and restore requirements, and the typical infrastructure requirements for power, heat dissipation (heat is what shortens the life of many DVR/NVR hard drives), and server software maintenance. The systematic application of maintenance upgrades is a practice taken for granted in the IT networking world - in the physical security world it can require extra effort to obtain version information and to learn about what mechanisms may exist to manage the application of updates to servers and devices in a large system.
Video Server Software
The broad category of "video servers" includes the VMS (Video Management System, which is software on a server), DVR (Digital Video Recorder), NVR (Network Video Recorder), Hybrid NVRs (NVRs that support both analog and network cameras) as well as video analytics software.
Some systems are Windows-based machines, as mentioned earlier in this article, and have specific vulnerabilities, and some are proprietary embedded systems that do not have the vulnerabilities of Microsoft Windows. Many brand-name DVRs and NVRs, although they function as servers, use desktop versions of the operating system (Windows 98, Windows 2000 Professional and Windows XP).
Many (not all) are not capable of running anti-virus and other security software, as their processors are not sufficiently powerful to handle both video recording and any other significant task. Video servers inherit the vulnerabilities of their operating system; thus, they need to be kept current with operating system updates (patches) for the sake of system and network security. Some operating system video components, such as Windows DirectX, should have strict update hygiene applied for cyber-security reasons as well as performance reasons.
VMS software provides the opportunity to use network or operating system features to get the level of strong authentication that should be established, for example, for remote login over the network. Therefore, some applications should be set up using the vendor-supplied features that take advantage of the underlying Windows or network login mechanisms. Where the network infrastructure provides one-time password support, it should be used.
For a high level of physical and logical authentication (such as for protecting server rooms and equipment closets), biometric authentication can be used to fortify access to specific doors, systems and applications.
Many network video cameras used clear text (unencrypted) transmissions by default. Some cameras have difficulty running an encrypted connection at high resolutions and frame rates. In this situation, using a dedicated management subnetwork or Virtual LAN (VLAN) to compensate is recommended. This is to provide the appropriate privacy to this sensitive information.
Most network cameras (and network encoders for use with analog cameras) support HTTPS (Hypertext Transfer Protocol Secure), which is a combination of HTTP and TLS/SSL (Transport Layer Security is the successor to Secure Sockets Layer - for more information see the Wikipedia entry for Transport Layer Security).
While monitoring for video loss is an important video software feature, network cameras should also have network monitoring applied. Some network cameras can have their passwords and IP addresses reset to the default by pressing the reset buttons. The default IP addresses and the camera reset procedures are published on the Web for most major brands.
Simply pressing a reset button to cause an IP address reset may take the camera out of the subnet that it is on. It takes knowledge of the default IP address to find it again (as it is has moved to a different logical network segment). Where a fixed IP addressing scheme is in use, resetting multiple cameras to the same default IP address can take all but one off the network due to the IP address conflict.
In a well-managed network, there is logged network activity resulting from a network camera reset. Log messages including "device offline" and "IP address conflict" would be reported by the network management software. The nature of the messages can provide indications of the type and location of trouble.
Access Control Server Software
Access control servers can be vulnerable to cyber attacks. See the May Convergence Q&A Column titled "The Security World Has Changed" for a description and reference information on the access control system attack that was documented at CarolinaCon, an annual hacker's conference in North Carolina. Vulnerabilities are specific to each access control server, but some vulnerabilities are not brand-specific and apply to many brands.
For example, a number of access control systems require that one specific password be used for the SQL Server database, and all systems use the same database password. Sometimes the passwords are contained in the installer or user manuals downloadable from the vendor's Website. This makes it possible for anyone with basic database knowledge - and any brand's password - to access any access control databases of that brand. Such access can often be accomplished across the network - physical access to the server is not necessary.
Where database passwords can be changed, many integrators leave the default password in place or use the same password for all of their customers, to keep things easy for the service technicians. Passwords should be changed when technicians leave employment, but that does not happen in these situations. Integrators should be asked about their approach to password management, and the response should be considered during integrator selection.
Access Control Equipment
Access control equipment (control panels, system controllers, door controllers, IP readers - various items that provide the hardware-based distributed intelligence) can be vulnerable to outside network interactions, including intentional attacks.
Additionally, some systems introduce proprietary "plug-and-play" traffic or device discovery traffic that can appear to be an attack to some types of network monitoring software. Some systems provide DNS server functionality (to enable automatic IP addressing), and there must only be one such DNS server on any network segment. Using a subnetwork will isolate such network traffic from other systems and devices on the enterprise network.
State of Practice
High-caliber consultants and integrators attend to many of these issues, but often do so without educating clients and customers about the protective measures, and without documenting them sufficiently. Some integrators give little attention to these issues.
Security system designs should include a computer and network security plan, which addresses sound deployment practices appropriate for the specific types and brands of software and hardware being deployed. Where vendors have recommended specific hardening practices or options, those that are being applied should be referenced in the plan. Where no vendor recommendations are available, an IT evaluator should examine the system or device (this can be done during a Proof of Concept test), to determine which network practices and protective measures should be applied and included in the plan.
Editor's Note: Read the full three-part IP Best Practices series in the STE magazine archives at SecurityInfoWatch.com/magazine/ste/archives.
About the Authors
James Connor is founder and CEO of security technology consultancy N2N Secure, a security consulting firm specializing in migration of analog to converged IP-based Physical and Logical security solutions. He is the former Senior Manager of Global Security Systems for Symantec Corp.
Rodney Thayer is an independent network researcher focusing on network attack and defense issues as they relate to business infrastructure. Current security research (exploit development) includes product and infrastructure evaluations, and training/lecturing on computer security topics. Mr. Thayer's background is in engineering, deployment and evaluation of computer and network security solutions. He has participated in the authoring of IETF standards, written product reviews for trade publications, taught at venues like RSA and Black Hat, played Capture The Flag at Defcon (on a winning team), and has consulted for large and small enterprises and Infrastructure Operators.
Ray Bernard, PSP, CHS-III is the principal consultant for Ray Bernard Consulting Services (RBCS), a firm that provides security consulting services. Mr. Bernard has also provided pivotal strategic and technical advice in the security and building automation industries for more than 23 years. He is founder and publisher of The Security Minute 60-second newsletter (www.TheSecurityMinute.com). For more information about Ray Bernard and RBCS go to www.go-rbcs.com or call 949-831-6788. Mr. Bernard is also a member of the Subject Matter Expert Faculty of the Security Executive Council (www.SecurityExecutiveCouncil.com).