Airship DVR / NVR Video & Video Management Blog

Product Review: AVG Free Anti-Virus at Home

The best FREE Anti-Virus available:

AVG Free:

http://free.avg.com/download?prd=afe

Many of the Airship folks use this on Windows at home. For professional or office use, you have to pay for it, and it's worth it. For the 'leave me alone' user, it just runs and updates all by itself without bothering you until there's a major version update or a found virus. Small pop-ups by the task bar remind that it's doing it's job, but otherwise it stays out of the way. For the Virus Vigilante, there's plenty of toys to play with in your crusade against digital demons.

ps- we aren't paid to advertise, we just think it's a good product.

by Gryphon MacThoy

Best Marketing is Fixing What Went Wrong

Waiting for a business in downtown Seattle to open, I stopped by one of the ubiquitous Starbucks here. I ordered a drop coffee and a spinach wrap.

A) drip is empty, would you like an Americano?
"Oh I suppose so."

B) oops we forgot your wrap, just another minute.
"Ok thanks!"

Ok, so stuff was a little slow and rather than covering up, the nice lady apologized. Cool, I got breakfast, no big deal. Really it all took only 5 minutes anyway.

I got done with my business and saw a friend on the street. I have good news to share and he wants a warm drink (too cold for April here right now!) so we go to the same Starbucks.

Guess what? Two drinks for free! See the picture...

Aurora said, "This is on me guys. I really need to make things right for you."

Things aren't always perfect. How you treat your customer at that moment is the best marketing you can create.

By Gryphon MacThoy

Airship 4.5 New Feature - Weekly Review

Or open it in full screen at YouTube:
http://www.youtube.com/watch?v=3tEt6-cTXQs

Airship Enterprise Management Server 2.0

WAY ahead of Marketing's pace, our dev and qa team are finishing up Airship Enterprise Management Server 2.0.

The new Manager 2.0 will being shipping in new Airship EMS hardware by May 2009.

Existing customers will be able to upgrade to Airship EMS 2.0 if they are within their 1 year free updates period for FREE! Pricing isn't yet set for older customers, but we expect the decision to upgrade to not be impacted by the price, if you know what I mean...

If you'd like more info, just email sales@airshipdvr.com to get a zip file of screenshots, like this one:

by Gryphon MacThoy

Delphi Resource File Troubles

One of the requirements for our exported media files is that they must have the option to be exported purely as a portable executable file.

In order for certain things to be packed into the executable file, I wanted to add resources to the .exe using kernel32's UpdateResource() function.

The process is simple:

handle := BeginUpdateResource(pchar(fileName), false);
UpdateResource(handle,
                         RT_RCData,
                         pchar(key), 
                         MAKELANGID(LANG_NEUTRAL, SUBLANG_NEUTRAL),
                        data,
                        size);
EndUpdateResource(handle, false);

The file I was adding resources to changed in size, so I thought it worked, however the code I was using to load the resource kept on failing.

After much of debugging, I finally realized that for some reason FindResource() doesn't work for resources that contain lower case letters. Odd since its also a kernel32 call...

-John Su

Choosing the right type of help file

Right now, I'm finishing up a new application. That means it's time to write help docs. It's always been hard for me to decide which format of help file. I talked to Gryph and did some research online. Here's a list of the formats I considered, and some pros and cons:

Microsoft Compiled Help Files

Pros:

• Very Powerful for the User - Can access any page in the help document from one place. Built-in search. Built-in navigation (links, forward, back).
• Comes with most Windows installations.
• Very quick to write

Cons:

• Not available on non-Windows installations.

Rich-Text Format

Pros:

• Very quick to write
• An editor that reads RTF comes with almost every OS.

Cons:

• No links.
• No navigation, table of contents, etc. This is a big problem for anything besides a small help file, as the user must scroll through a lot of information.

Microsoft Word Document

Pros:

• Very quick to write.
• Has some useful navigation tools: page links, easy to print.
• Available on many operating systems.

Cons:

• Many installations will not have an application that reads .doc format already installed.

HTML

Pros:

• Can be very quick to write.
• Pretty Powerful - Navigation and links are available without too much work. With more time, you can set up some complex, helpful features (table of contents, video).
• Almost any installation will be able to read this without having to install any new apps.
• Easy to print.

Cons:

• Many default web browsers are big, bulky applications that take a while to open up.

So that's what I found. I ended up going with HTML because it's easy to print, and you can view it on any machine, and it's pretty powerful, and Google Docs makes it easy to write. Just create a doc, with images and links, and you can export it as HTML. Here's a screenshot:

What do you prefer to use (as a user or doc writer)?

     --AaronO

I/O Performance Calculations

Recent posts reminded us about something Yanda Ma, our VP of Dev, wrote a while back. For your consideration:

Disk Performance

Design a DVR system is a tricky subject.  The pitfall most commonly encountered is overestimating the disk performance of the system.  Since we are using hardware compression, the DVRs tend not to be CPU bound but IO bound.  It is easy to fall into the trap of scaling based on specifications of disk drives and raid controllers.  Everyone puts out their flashy numbers of a few hundred MB/s but the reality is far from it.

Under the best of conditions, you may be able to achieve somewhere near the specified datarates on sustained sequential access, but like any other real world application, very little of the IO is sequential access.

To break it down with some back of the envelope analysis, we can break down the disk utilization into three major components.  The time taken for the actual data to be written, which is directly related to the throughput of the raid controller and drives.  The time taken to seek to the positions of each video file in order to do the writing, directly related to how often you write (various buffers come into play) and the time taken to seek to the positions of various index or metadata files in order to update them.  (these files tend to be trivially small compared to the video so the time to write them is effectively ignored in this analysis).

Putting it altogether:

Constants:

TP - Throughput in MB/s

ST - Seek time in seconds

BS - Buffer size in MB/s set in setup.ini (0.256)

FR - Frame rate (30)

II - I Frame interval (100)

BR - MB/s per channel (0.25 for 4cif normal)

Variables:

C - # of channels possible

Formula:

C = 1 / (BR/TP + BR/BS*ST + ST*FR/II)

Now you can solve for C and get a theoretical number of channels which can be supported with a particular hardware configurations but there's still a lot of other things which need consideration.  (fluctuations in the instaneous bandwidth of the stream, os level overhead, minimum disk idle time required etc).  In practice the number of channels you'll end up seeing is probably about half of what the math indicates. 

Whatever end result of the analysis, it is clear that the spec throughput of the hardware you have only plays a secondary role in the scaling of your DVR.

-yanda

----------------

OK, everyone clear on that? Remember to divide by 2!

Point is - this stuff is difficult to answer in yes/no terms. And sometimes, seemingly unimportant details make a big difference.

"Oh. You have 50 users with dial-up internet who each want to see 16 video streams at once..."

by Gryphon MacThoy

Design a DVR system is a tricky subject.  The pitfall most commonly encountered is overestimating the disk performance of the system.  Since we are using hardware compression, the DVRs tend not to be CPU bound but IO bound.  It is easy to fall into the trap of scaling based on specifications of disk drives and raid controllers.  Everyone puts out their flashy numbers of a few hundred MB/s but the reality is far from it.

Under the best of conditions, you may be able to achieve somewhere near the specified datarates on sustained sequential access, but like any other real world application, very little of the IO is sequential access.

To break it down with some back of the envelope analysis, we can break down the disk utilization into three major components.  The time taken for the actual data to be written, which is directly related to the throughput of the raid controller and drives.  The time taken to seek to the positions of each video file in order to do the writing, directly related to how often you write (various buffers come into play) and the time taken to seek to the positions of various index or metadata files in order to update them.  (these files tend to be trivially small compared to the video so the time to write them is effectively ignored in this analysis).

Putting it altogether:

Constants:

TP - Throughput in MB/s

ST - Seek time in seconds

BS - Buffer size in MB/s set in setup.ini (0.256)

FR - Frame rate (30)

II - I Frame interval (100)

BR - MB/s per channel (0.25 for 4cif normal)

Variables:

C - # of channels possible

Formula:

C = 1 / (BR/TP + BR/BS*ST + ST*FR/II)

Now you can solve for C and get a theoretical number of channels which can be supported with a particular hardware configurations but there's still a lot of other things which need consideration.  (fluctuations in the instaneous bandwidth of the stream, os level overhead, minimum disk idle time required etc).  In practice the number of channels you'll end up seeing is probably about half of what the math indicates. 

Whatever end result of the analysis, it is clear that the spec throughput of the hardware you have only plays a secondary role in the scaling of your DVR.

-yanda

----------------

OK, everyone clear on that? Remember to divide by 2!

Point is - this stuff is difficult to answer in yes/no terms. And sometimes, seemingly unimportant details make a big difference.

"Oh. You have 50 users with dial-up internet who each want to see 16 video streams at once..."

by Gryphon MacThoy

Do IP Cameras Require More Powerful Servers Than Analog?

Jon Honovich wrote in response to the previous blog post:

How does using megapixel cameras impact the total number of cameras
that an individual recorder can record?  The general answer is that
less cameras can be supported. Some manufacturers say this is an issue
with CPU utlilitization (when doing motion detection on the recorder),
others say this is an issue of limited i/o throughput.

Any input on this? I think it would make an interesting blog post. :)

Best,

John

--
IP Video Market Info
http://ipvideomarket.info

I am guessing that the short question everyone wants an answer to is this:
Does an IP-Only solution cost more than an Analog-Only solution? The answer is almost always YES, but there are a LOT of assumptions in that YES. It doesn't HAVE TO COST MORE.

To answer Jon's questions directly, let me give the short answers, followed by lengthy dissertation. (I know, I know, no one has time to read all this... but I'm writing it anyway.)

The maximum number of IP Cameras supported by a single Airship NVR is 32 at 640x480 at 30 fps.
The maximum number of Analog cameras supported by a single Airship DVR is 48 at 2CIF (704x240) at 30 fps OR 24 at 4CIF (704x480) at 30 fps.

There you have it - with current tech, Airship only goes up to 32 IP Cameras, as opposed to 48 Analog cameras. (WOW - there are A LOT of assumptions in that short answer... like the 2CIF vs. 4CIF detail)

The Airship NVR with 32 channels costs more than the Airship DVR with 32 channels, at that.

"Why?"
Because the Analog Encoders INSIDE the Airship DVR are more cost effective and provide more performance impact than most people would normally guess. Also, analog cameras are cheaper than quality-equal IP Cameras. Usually...

It's true that (for Airship) it takes more CPU, RAM, and GPU (video) resources to handle IP Cameras than Analog cameras require. But it doesn't necessarily affect costs of recording equipment. Let's write down some assumptions, to make the comparison fair:

A. Assume all video is 4CIF/30, both Analog and IP Cameras.
B. Assume identical video feeds.
C. Assume network bandwidth is NOT an issue at any point.

JUST GET THE VIDEO
In Airship, the same components that turn Analog video into digital video inside the Airship Server also render that video for display to the Server's VGA monitor. The CPU utilization for this is so trivial, that we literally don't even have to think about it. The dedicated H.264 Encoder Card does ALL the heavy work of encoding the video, rendering it for display, and sending it to the harddrives. Yes, the bits flow over the PCI bus, but that's just traffic, not computation.

To render the same video from an IP Camera, we have to decode the video for display, as well as send it to the harddrives for storage. That extra step of decoding the video must be handled by the CPU and GPU of the Server - there's no dedicated IP Camera Decoder in the Server.

To compare:

Analog

1. Airship Dedicated H.264 Encoder handles all video encoding, display, and storage operation.
2. CPU has a trivial load to direct traffic. (3% CPU utilization on a full operational 32 channel system is common.)
3. GPU has a trivial load to display video.

IP Camera

1. CPU responsible for IP Camera video decoding.
2. GPU helps with some of the video decoding/display.

"But, Gryph, how much CPU does an IP Camera use?"
Dear customer, that's a hard answer to give in a single sentence. Which IP Camera? You see, different IP Cameras use different video encoding, which vary in how hard they are to Decode by the Server.

ALSO - With Airship Servers, we turn the live view OFF on the Server by default because doing so frees up the Server CPU for more important work - like Motion Detection, hosting remote clients, and I/O operations. You CAN turn the live view of the IP Cameras on in a Stand-Alone Server operation, but we restrict the number of concurrent live views according to the Server's abilities. You know, it's almost like we think about it ahead of time... Anyway, we build the Airship Workstations to REALLY SHOW the video.

STORAGE
Now, let's talk about I/O - storing the Analog video to the hard drive vs. storing the IP Camera video to the hard drive.

Analog
In Airship, the dedicated H.264 Encoder results in a visually beautiful, yet small file size. This reduces the amount of I/O resources required. Other products vary, of course, but our 4CIF/30 video example seldom goes over 2Mbps. Compare that to MPEG4 or MPEG2 where comparable quality reaches up to 8Mbps.

IP Camera
This question depends on one important factor: What is the video stream bitrate?
IP Cameras come in many flavors of video encoding: MJPEG, MPEG4, H.264, even ol' wavelet (eek!) each of which will render the video in a "CERTAIN" bitrate.
BIG ASSUMPTION: IF the video stream bitrate is the SAME as the Analog example above (2Mbps), then the I/O performance should be the SAME.
HOWEVER: The efficiency of the NETWORK CHIPSET will have a big impact on overall efficiency of getting that IP Camera stream to the I/O subsystem in the first place.

Even if your server has a Gigabit Ethernet port, THEY ARE NOT ALL THE SAME. (Herein lies some of the magic sauce for Airship Servers, so I won't go into great detail... plus we assumed that network performance wasn't an issue above.) The best advice I can give is TESTING, TESTING, TESTING.

BOIL IT DOWN
DVR: The reason Analog Encoding seems 'cheaper' processing-wise is due to the use of Dedicated Encoding and Decoding hardware. This dedicated hardware ISN'T FREE. It's also not terribly expensive!

NVR: Building a Server capable of handling the IP Camera video we mentioned above is roughly cost-equivalent to the Analog solution. For high-channel counts, the NVR can cost more because the upgraded CPU and GPU actually cost more than the Dedicated Encoder.

Hybrid NVR/DVR: Hey, why not have it all? Costs a little more than just a DVR or just an NVR, but not as much as you'd guess. Stay flexible, impress your techie buddies at home...

In other words, Poh-Tay-Toh, Poh-Tah-Toh - IF THE VIDEO QUALITY IS EQUAL AND THE CHANNEL COUNT IS 16 OR UNDER.

What we haven't covered here:

Megapixel cameras - They Cost More - Money & Tech. Storing the larger video costs more. It takes more CPU and GPU to render the video. Higher network bandwidth requirements. Fewer channels per server, absolutely. (Wow, I really want to write in detail about this. Lemme know if you wanna hear it...)

Airship Workstations - They cost *differently*. An Airship Workstation can use Dedicated Decoders to render the Airship H.264 Encoded video - these are useless for IP Cameras. So, it takes CPU and GPU resources on the Workstation to render that video - just like on the Server. But those Dedicated Decoders cost something - they are not free. OVERALL, Workstations are more cost effective to build for large numbers of IP Cameras than they are for large numbers of Airship H.264 Encoded streams. BUT YOU CAN'T SKIMP ON COMPONENTS - And we don't, when you buy them from us. (See previous blogs about this.)

OK, that was a load of info - this is a loaded question with a load of details leading to loaded answers. My final word on this for now:
 
If ANYONE tries to give you a one sentence answer to this stuff without gathering detailed requirements first, they are MISSING SOMETHING. Or hiding something.

PS - A Sales Plug: Airship DOES ANALOG AND IP AT THE SAME TIME. Choose the solution that works for you today, knowing that you can choose to go a different route in the future and stick with the same Airship Enterprise Surveillance Suite.

by Gryphon MacThoy

How Airship Stores IP Camera Video

Customer Support Question:

Do you have some documentation on how you handle megapixel IP camera data?  I am considering adding a 1.3 megapixel for license plate capture and wondering how you handle the increase in image size / reduced frame rate.

I currently have a DVR with 4 channels @520tvl being captured.  I have 1TB hard drive storage, considering pulling one camera away from the analog inputs and reserving it for the IP camera.

Answer:

We have information available about the exact models of IP Cameras we have testing with on our downloads page here
http://www.airshipdvr.com/support/downloads.php
See the release notes for Airship 4.5.

Airship Server handles the video stream in exactly the same fashion as an analog input; we store it in the same way on the available harddrive volumes and we do not transcode the video stream. All we do is wrap it in a container that allows our Client and other playback software to read the video natively. That container adds an insignificant amount of data to the recorded files.

The best way to find out about storage calculations is to ask the IP Camera maker, and then run your own real-world tests to determine the best balance between video quality and storage costs. Even within the same maker, cameras differ wildly in compression efficiency and abilities.

I hope this helps you choose the best IP Cameras for your applications; don't hesitate to seek us out for more info.

Thank you,

Gryphon MacThoy
Airship Industries
Bellevue, WA
www.airshipdvr.com
877.462.4250

WGP 2009

WGP was expectedly low attendance this year. Most attendees didn't seem to have much interest in the vendors. Those who did had problems and were looking for solutions. We will see who we can help. :)

By Gryphon MacThoy