Recent posts reminded us about something Yanda Ma, our VP of Dev, wrote a while back. For your consideration:
Disk Performance
Design a DVR system is a tricky subject. The pitfall most commonly encountered is overestimating the disk performance of the system. Since we are using hardware compression, the DVRs tend not to be CPU bound but IO bound. It is easy to fall into the trap of scaling based on specifications of disk drives and raid controllers. Everyone puts out their flashy numbers of a few hundred MB/s but the reality is far from it.
Under the best of conditions, you may be able to achieve somewhere near the specified datarates on sustained sequential access, but like any other real world application, very little of the IO is sequential access.
To break it down with some back of the envelope analysis, we can break down the disk utilization into three major components. The time taken for the actual data to be written, which is directly related to the throughput of the raid controller and drives. The time taken to seek to the positions of each video file in order to do the writing, directly related to how often you write (various buffers come into play) and the time taken to seek to the positions of various index or metadata files in order to update them. (these files tend to be trivially small compared to the video so the time to write them is effectively ignored in this analysis).
Putting it altogether:
Constants:
TP - Throughput in MB/s
ST - Seek time in seconds
BS - Buffer size in MB/s set in setup.ini (0.256)
FR - Frame rate (30)
II - I Frame interval (100)
BR - MB/s per channel (0.25 for 4cif normal)
Variables:
C - # of channels possible
Formula:
C = 1 / (BR/TP + BR/BS*ST + ST*FR/II)
Now you can solve for C and get a theoretical number of channels which can be supported with a particular hardware configurations but there's still a lot of other things which need consideration. (fluctuations in the instaneous bandwidth of the stream, os level overhead, minimum disk idle time required etc). In practice the number of channels you'll end up seeing is probably about half of what the math indicates.
Whatever end result of the analysis, it is clear that the spec throughput of the hardware you have only plays a secondary role in the scaling of your DVR.
-yanda
----------------
OK, everyone clear on that? Remember to divide by 2!
Point is - this stuff is difficult to answer in yes/no terms. And sometimes, seemingly unimportant details make a big difference.
"Oh. You have 50 users with dial-up internet who each want to see 16 video streams at once..."
by Gryphon MacThoy
Comments