Images, operating systems and headaches: The realities of security systems running on PCs

[AUTHOR'S NOTE: This article addresses issues which may soon become common place with IP-based camera systems. Accordingly, the potential problems I have presented could allow the reader to take steps to avoid issues with their software-based camera system in the future or recover more quickly after a computer or software issue has occurred. The solutions that I present should be reviewed by your computer administrator or IT department. Keep in mind that not all of these solutions will always be applicable in your operating system, computer or computer network environment.]

It's Monday evening at 6 p.m., Oct. 3, 2009, and I am sitting at my new computer, a thoroughly robust computer loaded with a very fast processor and a great deal of RAM.

My drooling over this new machine is interrupted by the all too frequent occurrence of a spinning hourglass. We have all been there; we could be working on a critical document, and then for some hidden reason, everything stops, and that is happening now to me. My new Ferrari-like computer has become about as useful as my old Ford Pinto-like computer. As the hourglass spins, I review my options, which includes a hard-re-boot, which is like remotely locking up the brakes on a Ferrari that is traveling 80 miles per hour on a slick road with curves in a mountain pass. Alternatively I could go with the "three-finger-salute" or Ctrl-Alt-Delete and end the process of the offending program which has made my machine useless.

Reviewing my options, I hear the computer crunching away at its problem; it's literally like being next to a helicopter. Waiting haplessly, I am stretched for time, and I must make a decision. I go with the lesser of the two evils and select Ctrl-Alt-Delete. I choose the process, which is "not responding", and click the "end-task" button. Essentially, what I have told the computer to do is to "forget it." Forget about this one of the 100 things you are doing. The hourglass vanishes but my temporary relief is instantly replaced with the "Blue Screen of Death" or as we techies call a "BSOD". The characters on the blue screen almost seem alien, "STOP OXXXF......" followed by some senseless language about an IRQ, DMA or worse a "driver that is not equal". BSOD is not a good thing and is usually accompanied by other undesirable affects. Regardless, my problem has become dramatically worse.

It's 6:45 p.m., and now, I have restarted the computer three times and each time I am frustrated with the ever dominating blue luminance glow and alien language of the BSOD. I have all but forgotten about what I was doing and the lost data. I need to get my computer back and running.

As I work tirelessly at undoing what I have done, I am reminded of the transition of analog to IP cameras. The future of camera monitoring will not be based on closed circuit television and solid-state electronics; it will be controlled predominately through software. I have to wonder, could the same thing happen to a camera surveillance system? My mind flashes back to my last trip to the ASIS International Exposition in Anaheim, Calif., where I toured some of the emerging security technologies being pitched by the attending security technology companies. One particular manufacturer -- who will remain nameless -- asked me to demo their IP equipment. The sales rep told me, "Put it through its paces", and I eagerly agreed to do so. As I stood over the manufacturer's camera operator, I felt like I was in the movie "Swordfish" and was simultaneously the captain of the Starship Enterprise. Move over Spock and Captain James T. Kirk!

Six 42-inch large format high-definition monitors surrounded me and the manufacturer's camera operator. I had approximately 36 network cameras at my disposal, including a number of remote/off-site cameras. All cameras at my disposal have been pre-arranged in a variety of customized multi-views. As the captain of this spaceship-ready system, I provide numerous instructions to the camera operator. One of those instructions was to move a camera's video from a monitor on the right side to a monitor on the left side -- and that's when it happened. As the operator clicked and dragged the video to be relocated, the hourglass symbol appeared. Suddenly videos on the right monitor were replaced with the words "connection loss" and the video on the left side is frozen and each monitor is displaying an hourglass. It was the precursor to the total undoing of the system. The salesperson attempted a recovery and jested, "You broke it!" He followed with another recovery, "Obviously, we have some minor tweaking to do". After a few minutes, the demonstration was over, and my attention had been misdirected to the inner workings of their network video recorder (NVR). We never returned to the demonstration, and I don't know which option they took: the three finger salute or the hard re-boot? It seems irrelevant now, but it created questions that are worth asking: What is the impact to an organization that utilizes video monitoring to supplement staff? What if the camera system doesn't work?

This article is about preparedness and prevention. What I am presenting to you, could literally save your camera monitoring function and endless hours of work for you and your staff.

Computer "Issues":

There are basically three essential root issues to a computer malfunction. These include, the most basic, which is hardware failure that commonly occurs with the hard-drive or the power supply.

Secondly, software incompatibility can occur with the operating system software, an added software application or a combination of the two. A software incompatibility can manifest itself through a security officer's personal video game that they loaded on the security department's computer or it could be an officially installed third-party program. In addition, network environments can cause issues with the accessibility of network drives, computer functionality and this literally can bring the fastest computer to a crawl.

User-error also accounts for a number of problems, such as when a large file is being written to the hard-drive and a user restarts the computer or otherwise disregards the warnings that are presented by the operating system.

One of the biggest issues related to software incompatibility is the developers' addition of newer software features designed to take advantage of ever-increasing processing power within new computers. Manufacturers are taking advantage of computer speed to showcase their products, but are not necessarily remembering the persons with the older computers. Software updates and improvements tax older computers beyond their capacities.

Precautions:

Obviously the severity of my issue with my computer would be nothing compared to the crash of a vital server running the business surveillance system. Given that we are literally being catapulted into an IP technology era, it is necessary to understand the differences from what we potentially have now and what we will have in the future. Understanding the technology that we will be forced to use will better help us prepare for contingencies on issues, such as the one I have experienced. So, what do you do if you have an IP system or you are planning to move into an IP camera system? What are the issues?

Computer Operating System(s)

The operating system is the core of the computer and is what software applications are written to run on top of. The operating system manages hardware peripherals and software and typically creates a "Graphical User Interface" (GUI), which allows us to do multiple things at once, such as word processing and surfing the Internet. As it relates to security, operating systems allow us to run the programs we use to manage security and cameras systems. However, not all operating systems are alike; they come in different flavors. Personally, I feel that Microsoft Windows is less reliable, less secure and generally a less ideal operating system that is plagued with issues such as the BSOD. In reality, Windows is a surprisingly fragile operating system, but most users don't realize this because they never work their way down into the inner workings of the system. Although Windows is the predominant operating system for security and surveillance systems, I predict that you will start to see manufacturers begin to write software for other operating systems, such as:

Linux
Linux, which is based on UNIX, is an operating system that is very much different than Windows. Only Microsoft makes Windows, whereas Linux is manufactured by multiple sources. For instance, I have a CD-ROM, which boots a Linux operating system. Linux manufacturers and open source coding, in my opinion, makes for a better product. The development of an open-source operating system works in the way that an informed committee can make a decision by sharing their expertise. Open source code means that the collective of developers is working towards a common goal. Additionally, open-source operating systems potentially means the operating system is available at no cost since one of the hallmarks of open-source is freely distributed code. Another preferable difference between Linux and Windows is that Linux is generally less susceptible to viruses, and that means one less piece of software that needs to be managed and installed.

UNIX/MAC
Love it or hate it, the Apple Macintosh (the "Mac") is here to stay. The strength of Apple is its speed and multi-tasking capabilities. Additionally, the MAC has made a niche for itself in processing photography and digital video. Side-by-side, the Macintosh greatly outperforms a comparably equipped Windows machine. Most intriguing is that I have witnessed a powerful Mac operate multiple virtual operating system environments simultaneously without any issues.

Unlike the PC, the installation of Macintosh software is not managed by a registry\ and therefore is much easier to install or uninstall without leaving file remnants, which will later bog down a system. I also see opportunities for a Mac to be utilized in the surveillance market, especially because these computers are known for their strengths in processing digital video. The Macintosh OS, much like Linux, is also less susceptible to computer viruses. However, the big downside of the Macintosh OS is the price; they are expensive machines. Depending on the configuration, a Mac can be up to two-and-a-half times more expensive than a computer running Windows.

Anti-Virus and Software/Hardware Permissions

No matter what operating system that you have installed or which comes preloaded, when it comes to security, computers need to be locked down both through network, software and hardware controls. Just because attackers aren't writing a great deal of viruses for Mac and Linux operating systems does not mean, if the market-share pendulum shifted from Windows to Linux or Mac, that this would not become a problem. I have worked with clients whose digital video recorders have been the root cause for the introduction of computer viruses into a Windows networked environment. If you are running Windows, precautions must be taken to minimize the introduction of viruses into operating systems dedicated for security use. In addition, although, it may seem inefficient, I strongly recommend that computers dedicated to security be placed on stand-alone networks, which do not allow general connectivity to the Internet for random surfing. Input peripherals, such as floppy disks, USB drives, CD-ROM, DVD-ROM, and Blu-ray DVD should be physically/logically disconnected and removed from computers dedicated to a security function. Although convenient, the peripherals allow the introduction of rogue software from other sources that would not normally be introduced.

Finally, in addition to controlling hardware, software permissions need to be instituted for all users. Software permissions allow or disallow a user from completing a specific task. In the future you will see an increased integration of log-in user permissions at both the operating system and application level, which means less passwords and centralized control of your systems.

Certification

It's very important that applications (computer programs, such as Microsoft Word) are written and certified for a particular operating system. Failure to select a digital video software application that is "certified" could cause real issues. Let's take Windows for an example. We all have been sitting at our Windows computers when that little bubble pops up, to tell us that critical updates are available. (The frequency of these bubbles should support my initial described impression of Windows.) When Microsoft makes changes to their operating system, it can create a conflict with an application that runs on the operating system. Application manufacturers that are Microsoft Certified are given advance notice of the changes that will be made in future operating system updates, so these developers can react prior to the update. Unfortunately, if the application developer is not paying the dues to make their software Microsoft Certified, then you could be left without an operable software application on your next update. Make no mistake that corporate IT departments will mandate operating system updates with or without your permission, so there is good reason behind the certification.

Dedicated Computers

Computers dedicated for a security function should be strictly controlled. Specifically, a computer running digital video should be dedicated to digital video. Avoid the temptation to install other software programs on a single computer for efficiency. Just as operating systems can have a conflict, additional application software could dramatically slow a computer and can create a conflict with the previously installed software. In addition to the updates that are made to the operating system, security application software should be updated when required by the manufacturer; failure to make these updates could be problematic for future changes in the operating system or the computer hardware.

Computer Environment

You would not want to work in a dusty, dingy, hot environment, so what makes you think a computer will want to work there? Out of all of the technical surveys I conduct, you know what I write about 90 percent of the time? Dust! This is a big concern; dust causes problems, it insulates equipment, it restricts airflow by reducing the amount of airflow a fan can produce and more importantly it can create an opportunity for a ground fault or short. In the most literal term, a ground fault or short is an unexpected path of electricity. This, as you can imagine, is not good thing, and will most likely trash your whole computer. There's no BSOD; there's nothing. The computer will just turn off, and you will be left with a blinking cursor at best. The best option is to place the computer in an environmentally controlled location. Cleaning and maintenance will minimize the potential for an unnecessary failure.

Computer Maintenance

There are two type of maintenance that needs to be regularly conducted, one is hard-drive maintenance and the second type of maintenance is physical maintenance, such as dusting. I also regularly eliminate the computer's temporary cache and cookies. I also clean the computer 'registry'. The registry within Windows is essentially the traffic cop for the computer; it tells the computer what has been installed and how everything fits together. However, when you uninstall programs, remnants or links to those programs may still reside on the computer. The program is no longer functional, but still may draw processing functionality from the computer. A clean registry helps this.

Disk maintenance is also a must for non-RAID hard drives. Drives are used to store and access data, and are logically formatted. However, after time, data can become fragmented or disorganized on the drive in a way that may make the computer work harder to find files or programs. Disk maintenance in the form of "defragging" is highly recommended. In addition to defragging, I recommend you utilize software to identify a hard-drive failure prior to its occurrence. Over time, as the moving parts on hard-drive wear, there are subtle changes in the performance of the hard drive. There is software, usually free, which will help you identify a potential drive failure before it occurs. These maintenance tasks aren't always the easiest, so it's best to reach out to your IT department for direction and assistance in performing these tasks.

Recovery

We have discussed precautions to preventing a failure, but unfortunately failures will inevitably occur. Organizations that rely heavily on surveillance systems need to develop continuity planning policies and procedures which address the loss of the surveillance system either in part or wholly as a result of a network or computer failure. The organization, as part of this continuity planning, should establish when back-up computers are required and how many spare computers could be needed. This will all be affected by the manufacturer and their software capabilities, or their lack thereof.

Ideally, an organization will identify how they will deal with a failure. The response will differ depending on the value and importance of the video system. One solution might be the use of a "hot" back-up computer; this is a back-up computer that is constantly running. The hot back-up takes over upon the failure of the primary computer. This computer is fully equipped and from a software perspective, it is a complete mirror copy of the primary computer.

Outside of a hot-back-up, organizations may rely on "cold" back-up computers or disk images. The data on a cold back-up computer or image may not be as current, but could be used in the interim, while the primary computer is repaired. One type of a cold back-up is the "imaging" of a computer. Imaging requires specialized software and takes a "snapshot or "image" of how the computer was configured at a specific time. This image is complete; it contains everything on the computer when the image was taken. This allows someone to restore a computer very quickly with everything that was on the computer at the time of the image.

The above options are preferred because of expense, but there are drawbacks. Imaged computers don't always work. Secondly, and more importantly, the image of a computer is much like a finger-print; it is unique. Therefore, if you have a hardware failure other than a hard drive, the image will not work, because all of the internal settings that are present on that image are related to the computer's native hardware. If you changed a hardware peripheral or motherboard, the image would have the incorrect information for that equipment to work properly.

Yet another opportunity is to install a fault-tolerant computer, which is basically like a hot back-up computer. In this case, one computer is fully redundant including a redundant motherboard, video card, RAM, etc. You can actually pull a computer board out of a running fault-tolerant computer while watching a video without the video stuttering. The downside, as you can imagine, is the cost, which depending on the configuration can be upwards of $35,000.

Conclusion

So if you're still with me, it's now 8:45 p.m., and I have exhausted every option that I can think of to fix my computer. I have tweaked references, deleted files, scoured the Internet for solutions and adjusted the most advanced settings within Windows. The only time the computer will re-start is if I boot into safe mode. Safe mode, however, disables Windows features and functions, and it's very similar to the BSOD, except I get to see the Windows GUI.

My face cringes as I think about the effort and time it will take to do a fresh install. First I need to load the operating system, then the specific drivers. Once that's complete I need to re-install all of my computers application software and so on. Since I am on a Windows machine, everything is managed by the registry file, so installing will take time. Autocad, a drawing program that I use for laying out security systems, will itself take an hour to install. There's no way to finish in time.

Then it dawns on me. I had made an image of my hard drive with everything pre-configured, including all of my software and most of my data right after I bought this computer. "Awesome," I think to myself, and I scramble to find the disc. I insert it, and by 9:35 p.m., I am essentially up and running -- with the exception of some data loss -- just like I was three hours prior to what I am now calling the "incident".

Completely satisfied with my ingenuity, knowledge, insightfulness and ability to save what would have taken a whole day of time to fix; I open the program that caused me so much havoc previously. Reassuring myself, I think, "Surely, I will not have the same problem again." Yet as the program opens, I am once again greeted with that wonderful hourglass symbol that plagued me three hours before. My fingers come off the keyboard, as not to upset "her". I pause for a moment and really contemplate what I had just done. This time, I think to myself, I will just wait it out, and use the time to outline an article on software, operating systems and the impact on IP systems.

Sean Ahrens, CSC, CPPAbout the Author: Sean Ahrens, CPP, CSC is a project manager for Security Consulting and Design Services with Schirmer Engineering and has over 18 years of experience in the security industry, 12 of which has been as a practicing security consultant. Ahrens volunteers his time on U.L., SIA and the ASIS International Commercial Real Estate Council (CREC) and is responsible for providing security threat and risk analysis, contingency planning, loss prevention, and force protection design and planning for private, public, governmental and state organizations. He can be reached at (847) 953-7761 or via e-mail at sean_ahrens@schirmereng.com.

Loading