Are you familiar with Insteon at all? Wondering how you would compare that platform to the Domoticz based system you built out?
nayr's home automation build out...
- Thread starter nayr
- Start date
Haha thats cool i told my mother a month ago that i wanted to do such a thing.
Make a cabinet in the wall or at the wall with a lid , that beneath my doorbell that must ring my phone when pressed.
If i get a connection with the package delivery man i will tell him to place the package in the cabinet and i press something to unlock the magnetic lock.
He an sign it on my behalf and everything will be recorded of course.
Btw , for your screen, is it maybe an idea to hide it behind a shine trough mirror ?
Then you do not have any monitor visible but a mirror on the wall.
Just when the screen turns on you will have a image visible trough the mirror.
Going to place such a thing in my bathroom, buy a plastic mirror that shines trough at the side where the most light is.
Keep it coming, enjoying to read
@vincenttor, yeah this was cheap and easy.. Ive found a really nice metal one that I can install through the brickwork and have packages securely dropped into a cabinet in my garage.. it would be perfect, but the damn thing costs a fortune and is going to require alot of masonry work to install so perhaps sometime in the future.. we work from home so most of time I meet the delivery guy at the door, but when we are on vacation or what ever they start piling up and we have to ask neighbors to hold onto them for me.. this will work better for those situations.
I dont need to go through that much effort to hide the monitor, tho that is a neat idea.. You cant see it unless I invite you inside, but I have no audio recording laws to worry about but the mute on door open was more so Cops/Code Enforcement/Scammers dont realize what they say I am recording for my own protection.. There's going to be many instances where someone comes to the door and we dont answer and someone just watches and listens from inside the mudroom until they leave.
This monitor/pi is going to work out brilliantly for my security system, the RPI video player (omxplayer) supports alpha transparencies and video layering.. I found a nice 60s countdown video, set it to 50% transparency and crop to full screen and then play it and it overlays very nicely over the security feeds and plays an audible countdown.. its going to be trick!
I dont need to go through that much effort to hide the monitor, tho that is a neat idea.. You cant see it unless I invite you inside, but I have no audio recording laws to worry about but the mute on door open was more so Cops/Code Enforcement/Scammers dont realize what they say I am recording for my own protection.. There's going to be many instances where someone comes to the door and we dont answer and someone just watches and listens from inside the mudroom until they leave.
This monitor/pi is going to work out brilliantly for my security system, the RPI video player (omxplayer) supports alpha transparencies and video layering.. I found a nice 60s countdown video, set it to 50% transparency and crop to full screen and then play it and it overlays very nicely over the security feeds and plays an audible countdown.. its going to be trick!
Last edited by a moderator:
expanding my security system with a big fat bonus check.. as a result I have load of Dahua equipment coming my way.. all of which will get my standard IPCT review treatment w/sample videos.
The Trailer/Trash cam has needed an upgrade for a while, its my weakest quality cam. Its new home will be in the garage watching the catfood feeder and cat door, so next time a cat goes missing we have an easier to access audit log.. when these cams are in place I will have 7 deployed and hopefully will be done with outside for a while.. indoors I may add a few more cameras in my server room and garage.
The new trailer cam has AlarmIO and Audio, which is much needed at this location. The current cam is a 3.6mm and its being changed for a 6mm, I wanted a bit more zoom for a couple reasons, one my trailer has reflectors on the back and when trash is on the curb the camera is all messed up from the reflectors.. there are actually reflectors on the bars too but you can see in the attached pix i have covered them with pipe insulation, to block IR.. and lastly the wide angle put my trailer tongue and alot of the yard are too far away with the trailer sucking most of the image.. this made it harder than nessicary to ID people, the'd have to get pretty close first. This new zoom wont see so much of the side yard as it will the front yard, but the FOV should be wide enough nobody can sneak past the camera to get to the trailer.. The reasons I choose this camera were the LXIR, 4MP, Audio/AlarmIO, and lastly Smart Detection features for a reasonable price.. I have an idea to use the facial detection or line cross and hook it up to a strobe light, so any time the camera thinks a human is over there it'll hit them with a strobe for a few moments to say 'hey fucker, your on camera!'
The LPR Camera is going to be a very fun project hidden in a faux rock.. its specs are much like the trailer cam except it has interchangeable lenses, I got it a nice 10x varifocal that should do the job quite well.
and lastly Ive been wanting a matching Dahua NVR for a while, so I pulled the trigger on it... Ive been completely happy with my FTP NVR using Dahua's software to play back as needed, but I need more storage space with these new cameras and my CuBox only has 1 eSata port, an external enclosure with sata expander was most of the cost of the this NVR.. so a no brainer.. I am really looking forward to being able to play back video directly in my desktop, tablet, and mobile apps, and playing with the video notifications.. I'll be upping it to 8TB of storage soon and get a nice cheap SSD for the CuBox.
Items will start pouring in in the next couple weeks, likely starting with the NVR and ending with the ALPR project.. they are all coming off AliExpress, so whom knows.
- Dahua 4MP Box w/6-60mm Lens - IPC-HF5421E - ALPR
- Dahua 4MP Box w/6mm Lens - IPC-HFW4421D-AS - Upgrading my Trailer/Trash Cam
- Dahua NVR NVR2416-4k
The Trailer/Trash cam has needed an upgrade for a while, its my weakest quality cam. Its new home will be in the garage watching the catfood feeder and cat door, so next time a cat goes missing we have an easier to access audit log.. when these cams are in place I will have 7 deployed and hopefully will be done with outside for a while.. indoors I may add a few more cameras in my server room and garage.
The new trailer cam has AlarmIO and Audio, which is much needed at this location. The current cam is a 3.6mm and its being changed for a 6mm, I wanted a bit more zoom for a couple reasons, one my trailer has reflectors on the back and when trash is on the curb the camera is all messed up from the reflectors.. there are actually reflectors on the bars too but you can see in the attached pix i have covered them with pipe insulation, to block IR.. and lastly the wide angle put my trailer tongue and alot of the yard are too far away with the trailer sucking most of the image.. this made it harder than nessicary to ID people, the'd have to get pretty close first. This new zoom wont see so much of the side yard as it will the front yard, but the FOV should be wide enough nobody can sneak past the camera to get to the trailer.. The reasons I choose this camera were the LXIR, 4MP, Audio/AlarmIO, and lastly Smart Detection features for a reasonable price.. I have an idea to use the facial detection or line cross and hook it up to a strobe light, so any time the camera thinks a human is over there it'll hit them with a strobe for a few moments to say 'hey fucker, your on camera!'
The LPR Camera is going to be a very fun project hidden in a faux rock.. its specs are much like the trailer cam except it has interchangeable lenses, I got it a nice 10x varifocal that should do the job quite well.
and lastly Ive been wanting a matching Dahua NVR for a while, so I pulled the trigger on it... Ive been completely happy with my FTP NVR using Dahua's software to play back as needed, but I need more storage space with these new cameras and my CuBox only has 1 eSata port, an external enclosure with sata expander was most of the cost of the this NVR.. so a no brainer.. I am really looking forward to being able to play back video directly in my desktop, tablet, and mobile apps, and playing with the video notifications.. I'll be upping it to 8TB of storage soon and get a nice cheap SSD for the CuBox.
Items will start pouring in in the next couple weeks, likely starting with the NVR and ending with the ALPR project.. they are all coming off AliExpress, so whom knows.
Attachments
Face detection may work for you but I had issues with line crossing detection on a camera that faced the street. By day it would see the shadows of aircraft and clouds as a line crossing, and by night car headlights would set it off. It worked great when someone walked across it but the false alarm rate was annoyingly high for me.
What illuminator did you decide on?
have not chosen an illuminator yet, its going to depend on the final install location.. I have a few to play with and I know where to find some powerful ones should the need arise..
I figured line cross would be prone to false alarms, I am going to use it mainly on the ALPR camera and for very large objects... if it goes off for no reason its of no consequence, as long as it dont bog down the OpenALPR process with too much work.. I hope it consistently captures vehicles in roughly the same location going either direction.. I do have a plan B if it dont work.
but yeah for turning on the strobe, facial detection is what I want.. dont want to be waking up the neighbors because a bunny decided to hide under my trailer.. if it works well enough I am considering replacing my backdoor camera with a 4MP Dahua Wedge with 2.8mm lens, yeah it will be very wide angle but it'll have a full view of my yard and can tell my PTZ to zoom in when a human is back there, negating the disadvantage of the wide angle.. use it as a kinda motion sensor for the black face PTZ's backside.. right now my 3.6mm 4300S only monitors the patio and mebe half of the yard, I am tired of hunting for my son with the ptz when he is just behind something blocking the PTZ's view.. like his jungle gym. Using the API, Events via Motion give coordinates of activity, and those can be synced with the PTZ's visual coordinates using some fancy programming on my part.
The 2 cameras are still in the mail, I messed up re-ordering my LPR camera and selected slow boat shipping sigh.. within a week or they should be in the USA at least.. both mailed out about a week ago.. in the meantime the lens for the ALPR Camera sits here waiting.
I got the Dahua 4k 16ch NVR and its up and running, ordered a 6TB purple for it.. combined with the 4TB purple I already had its loaded with 10TB on the Spindle and its doing a great job. Wish I had done this sooner.. its going to take a while to fill it up.. I got 13 days with my 4 cams 24/7 on the 4TB, wanting to get 21 days minimum but I am adding 2 more cams that will only be recording on trigger. (Catdoor Cam and ALPR).. should have it and then some.. we travel frequently and taking over 2 weeks vacation is normal.. over 3 weeks not so much.
Took a risk and got the retail 6TB Purple from Amazon Warehouse, description said flawless packaging and slight blemish on product.. figured with the 4Tb purple worst case I RMA it through WD and it'd just balance out the ~$60 savings.. well it turned out to be a little smudge on the label, drive tested perfectly fine after a full write over.. nothing critical gets stored on it, so rock on.
I figured line cross would be prone to false alarms, I am going to use it mainly on the ALPR camera and for very large objects... if it goes off for no reason its of no consequence, as long as it dont bog down the OpenALPR process with too much work.. I hope it consistently captures vehicles in roughly the same location going either direction.. I do have a plan B if it dont work.
but yeah for turning on the strobe, facial detection is what I want.. dont want to be waking up the neighbors because a bunny decided to hide under my trailer.. if it works well enough I am considering replacing my backdoor camera with a 4MP Dahua Wedge with 2.8mm lens, yeah it will be very wide angle but it'll have a full view of my yard and can tell my PTZ to zoom in when a human is back there, negating the disadvantage of the wide angle.. use it as a kinda motion sensor for the black face PTZ's backside.. right now my 3.6mm 4300S only monitors the patio and mebe half of the yard, I am tired of hunting for my son with the ptz when he is just behind something blocking the PTZ's view.. like his jungle gym. Using the API, Events via Motion give coordinates of activity, and those can be synced with the PTZ's visual coordinates using some fancy programming on my part.
The 2 cameras are still in the mail, I messed up re-ordering my LPR camera and selected slow boat shipping sigh.. within a week or they should be in the USA at least.. both mailed out about a week ago.. in the meantime the lens for the ALPR Camera sits here waiting.
I got the Dahua 4k 16ch NVR and its up and running, ordered a 6TB purple for it.. combined with the 4TB purple I already had its loaded with 10TB on the Spindle and its doing a great job. Wish I had done this sooner.. its going to take a while to fill it up.. I got 13 days with my 4 cams 24/7 on the 4TB, wanting to get 21 days minimum but I am adding 2 more cams that will only be recording on trigger. (Catdoor Cam and ALPR).. should have it and then some.. we travel frequently and taking over 2 weeks vacation is normal.. over 3 weeks not so much.
Took a risk and got the retail 6TB Purple from Amazon Warehouse, description said flawless packaging and slight blemish on product.. figured with the 4Tb purple worst case I RMA it through WD and it'd just balance out the ~$60 savings.. well it turned out to be a little smudge on the label, drive tested perfectly fine after a full write over.. nothing critical gets stored on it, so rock on.
Last edited by a moderator:
I quite surprised you are not running a RAID setup. You clearly go to a lot of trouble and expense with your home security setup so it seems counter intuitive that you would risk losing everything to a drive failure.
1. all the data gets overwritten eventully so whats the point? none of its is important until an event happens, and then its just important I have it long enough to take it off and put it on my fileserver or cloud with real redundancy.
2. raid is dead, has been for at least 7 years now.. any array >10TB has an extremely high likely hood of failure due to excessive disk sizes and rebuild times.. full backups and restoring from those backups are quicker.. I can restore 10TB in 24H with a Gigabit network, maby a few days from a slow backup.. your talking weeks for a 10TB raid, who the fuck is going to take there NVR down for days or weeks?
if I have a drive failure I pull out a disk and run on diminished capacity until it is replaced.. just a few mins of downtime and I am back on the road..
uptime is king in my universe, I only have mirrors and archive disks in play anymore and I run lots of very critical cloud services for millions of users dealing with audio/video so the amount of data being shuffled about is at a scale hard to grasp. Periodic ZFS Snapshots and cloning changes to a network archive disk is much more manageable if your willing to potentially loose a small amount of data to preserve high availability.. and if your not then use a mirror and still snapshot it to an archive disk because corruption will spread instantly in a raid. If the server goes down it has hot spares that take over and I can quickly clone the last snapshot for a full restoration and put the server back into the hotspare pool before another outage occurs or it gets too far behind sync with the cluster.
2. raid is dead, has been for at least 7 years now.. any array >10TB has an extremely high likely hood of failure due to excessive disk sizes and rebuild times.. full backups and restoring from those backups are quicker.. I can restore 10TB in 24H with a Gigabit network, maby a few days from a slow backup.. your talking weeks for a 10TB raid, who the fuck is going to take there NVR down for days or weeks?
if I have a drive failure I pull out a disk and run on diminished capacity until it is replaced.. just a few mins of downtime and I am back on the road..
uptime is king in my universe, I only have mirrors and archive disks in play anymore and I run lots of very critical cloud services for millions of users dealing with audio/video so the amount of data being shuffled about is at a scale hard to grasp. Periodic ZFS Snapshots and cloning changes to a network archive disk is much more manageable if your willing to potentially loose a small amount of data to preserve high availability.. and if your not then use a mirror and still snapshot it to an archive disk because corruption will spread instantly in a raid. If the server goes down it has hot spares that take over and I can quickly clone the last snapshot for a full restoration and put the server back into the hotspare pool before another outage occurs or it gets too far behind sync with the cluster.
Last edited by a moderator:
For sure, RAID 5 is a false sense of security since it uses only one parity disk. When one disk fails, the probability of finding one or more bad sectors on the remaining disks while the RAID rebuilds is high, and data will be lost from every bad sector. I had to deal with this when I lost a disk in my RAID 5 at home. As I was copying files off the degraded RAID, a few files failed to read. Fortunately they were just movies and entirely replaceable.
Then around the same time my dad's 4-disk RAID 5 in his office failed. One disk failed completely which normally wouldn't have been such a big deal, but at the time another disk was marked as offline due to some glitch and nobody knew about it to bring it back online (thanks, crappy intel raid software with no alerts). So the Intel raid controller marked the entire array as failed because 2 of the 4 disks were down. It absolutely refused to rebuild the array despite 3 of the disks still working to this day. So that one had to be restored from a backup.
So anyway, my FreeNAS has four 6TB reds in a RAID-Z2, which I built with the understanding that the second parity disk greatly diminishes the chance of data loss. I should only lose data if two disks fail at the same time and the remaining two disks have a problem (either a 3rd disk failure or just a bad sector). What I don't know is what FreeNAS would do if I had two disk failures and then a bad sector was encountered. Ideally it would skip the bad sector and restore what it could, but I don't know and hopefully I will never have to find out the hard way.
Speaking of FreeNAS, I am not very impressed by its "reliability". I mean, it works fine for the most part, but my crashplan jail has lost its mounted storage directory two or three times in the year and a half I have had it online, and every time I have to dig in there and re-add it and set up the backups again it is like pulling teeth.
the plugin jails are junk if your running that, just make a vanilla jail and set it up your self.. works much better, they do funky isht with the 'plugin jails' that makes them miserable to maintain your self.
raidz2 (raid6) should okay for a few more years, but it too will fall the way raid5.. https://storagemojo.com/2010/02/27/does-raid-6-stops-working-in-2019/
at home I have 4 6TB HGST Deskstars in a FreeNAS RaidZ2, but the really important stuff is snapshotted nightly and cloned to an archive disk that keeps a few weeks of weekly snapshots of important files and timemachine backups.. so I need 3 disks to fail to be boned, but then I too push encrypted backups to crashplan and will just buy a disk from them with my image on it.
raidz2 (raid6) should okay for a few more years, but it too will fall the way raid5.. https://storagemojo.com/2010/02/27/does-raid-6-stops-working-in-2019/
at home I have 4 6TB HGST Deskstars in a FreeNAS RaidZ2, but the really important stuff is snapshotted nightly and cloned to an archive disk that keeps a few weeks of weekly snapshots of important files and timemachine backups.. so I need 3 disks to fail to be boned, but then I too push encrypted backups to crashplan and will just buy a disk from them with my image on it.
Interesting. Hopefully by the time I need more space my upload speed will be high enough to seriously just back up everything to cloud storage so that people like you can worry about keeping it in good shape 
I'd like to figure out snapshotting and archiving but my worry is that through my own inexperience I would end up filling the disks and having the system not recover gracefully
I'd like to figure out snapshotting and archiving but my worry is that through my own inexperience I would end up filling the disks and having the system not recover gracefully
Wow . I just stumbled across this HA thread, and I'm truly impressed by your incorporation of hardware and software.
I'm currently using homeseer and like you I'm griping about how every single added functionality costs money. I bought homeseer before knowing better. I'm looking into Domoticz now.
Your idea of using a webapp secured by a cert for HA is brilliant, and I'd like to implement something similar. I currently have a VPN server secured using certs, and I have all ports locked down except for for the VPN server. It's one extra step and some added latency for me to access my home automation via the VPN connection. It would be nice if I just pull up the webserver and have instant access.
I'm still a little leery about putting a webserver out there that's attached to my home network. Do you suggest putting the webserver in a DMZ for security purposes? If so, will that present challenges when communicating with the rest of the devices on the network?
I've dabbled with Rails and PHP so I'm little familiar with web apps, but by no means even close to being an expert.
I guess having my VPN server out there poses the same risks, but for some reason I'm just a bit more leery with a webserver.
Any thoughts/suggestions for network security if I were to implement a webapp similar to yours?
I'm currently using homeseer and like you I'm griping about how every single added functionality costs money. I bought homeseer before knowing better. I'm looking into Domoticz now.
Your idea of using a webapp secured by a cert for HA is brilliant, and I'd like to implement something similar. I currently have a VPN server secured using certs, and I have all ports locked down except for for the VPN server. It's one extra step and some added latency for me to access my home automation via the VPN connection. It would be nice if I just pull up the webserver and have instant access.
I'm still a little leery about putting a webserver out there that's attached to my home network. Do you suggest putting the webserver in a DMZ for security purposes? If so, will that present challenges when communicating with the rest of the devices on the network?
I've dabbled with Rails and PHP so I'm little familiar with web apps, but by no means even close to being an expert.
I guess having my VPN server out there poses the same risks, but for some reason I'm just a bit more leery with a webserver.
Any thoughts/suggestions for network security if I were to implement a webapp similar to yours?
lucky for you, I wrote the docs for it and the code to do it: https://www.domoticz.com/wiki/Secure_Nginx_Proxy_Setup
all these IoT devices are not to be trusted, I dont even trust domoticz.. I do trust Nginx however and its trivial to set it up to automatically update security without breaking anything in the process.. I have them all on a locked down vlan with very minimal access outside its network.. all points of entry into that network are guarded by a VPN, or an x509 authenticating proxy.. cant really get any better secured than this.
protect your services with a hardened external authentication system, vpn, x509, or both.. and the services are guarded from anyone without explicit access.. With external policies only trusted users can access the services, thus you have rendered most any security issues with that service moot.. as long as it cant bypass your policies.
if you already have x509 VPN and looking for an x509 Proxy, dont forget to setup an x509 WiFi.. its the only WiFi that offers any decent security and your certs will work for all of them.
US Department of Defense uses physical x509 crypo cards for 2 factor authentication on everything government (CAC Cards), same with most any high security networks.. its good stuff if you do it right, Ive written software to read those cards and perform authentication based upon the x509 cert: http://www.cac.mil/common-access-card/
all these IoT devices are not to be trusted, I dont even trust domoticz.. I do trust Nginx however and its trivial to set it up to automatically update security without breaking anything in the process.. I have them all on a locked down vlan with very minimal access outside its network.. all points of entry into that network are guarded by a VPN, or an x509 authenticating proxy.. cant really get any better secured than this.
protect your services with a hardened external authentication system, vpn, x509, or both.. and the services are guarded from anyone without explicit access.. With external policies only trusted users can access the services, thus you have rendered most any security issues with that service moot.. as long as it cant bypass your policies.
if you already have x509 VPN and looking for an x509 Proxy, dont forget to setup an x509 WiFi.. its the only WiFi that offers any decent security and your certs will work for all of them.
US Department of Defense uses physical x509 crypo cards for 2 factor authentication on everything government (CAC Cards), same with most any high security networks.. its good stuff if you do it right, Ive written software to read those cards and perform authentication based upon the x509 cert: http://www.cac.mil/common-access-card/
Last edited by a moderator:
Thanks for the in depth reply. I will definitely digest this and take the time to implement your suggestions.
I'm also leery of IoT devices. I have a Nest thermostat at home, but I have that on a wireless network that is totally segregated from my IP Cams/home security and anything of importance. I got a true multiport router and the wireless is on it's own broadcast domain and has no access to my IP Cams broadcast domain.
Thanks again nayr!!
It is interesting to see where different people come from on this issue. One of the big reasons I go with raid is that I want to ensure that it is there when an event happens. Almost always there is a time delay between something happening, and you becoming aware of it, and I couldn't think of anything worse than losing that footage in the interim because of a drive failure
I did a bit of googling on the issue after reading this. I wasn't aware of the rebuild time issue because it hadn't been my experience. My array is in RAID1 which, as I discovered this afternoon, doesn't need to recalculate parity, and rebuilding a 3TB drive happened overnight whilst the NVR was running. I didn't have to take it offline at all.
Being a PC based NVR losing the drive containing the operating system means losing the NVR. Serious downtime. RAID1 keeps things ticking over if I have a failure.
I like to make sure the NVR is up at all times too, hence the RAID1 which seems to be working well for me at the level I use it at. Thanks for the info on the rebuild times. My NVR is getting on and an upgrade is looming in the near future and I was considering moving to RAID6 for increased capacity, but after my reading this afternoon I think I will go to RAID10 instead. Thanks again for the heads up on that one.
if Murphy is that out to get me that bad I dont think the video footage would have helped anything.. all my IPCameras have onboard SD Storage with 64GB cards that record incase the NVR goes down..
a Mirror is fine for your OS drive so you can keep up durring a disk failure, but its still got its own issues.. yeah your protected against a disk failure but file corruption is just as common and in a mirror the bad bits propagate and there is no method of recovery.. RAID != Backup, and Ive lost so much data on a raid I have not considered it sufficient protection by its self.. like ever.
If your going to waste all that spindle space on video redundancy in a mirrored raid, without an external backup system in place that can archive that big raid then your just wasting your time.. if your going to do data protection half assed, then dont do it at all... save your money if your not getting real protection anyhow.. If the data is important enough to call for a raid, then its important enough for a proper backup system.. and a proper backup system has offsite storage capabilities.. the raid is just one layer of many layers of data protection.. just like security cameras are just one layer in security, my house is not suddenly vulnerable if I have a disk failure.. there is alot more protecting me than the NVR.
Mirror Raid all alone at this point is little more than a tool to maintain uptime and provide great read speeds, not your data's saftey and integrity.. if your driven by data integrity then you need to take more steps than slapping it on a raid 10 and crossing your fingers... doing it right is hard, and IMHO none of it is worthile for a video NVR that could potentially one day have some very important data on it, but most of the time contains 10TB of junk.
Raid 10 great is for insanely high performance disk access without quadrupling your failure rates, not keeping data safe.. I cant think of a single reason why any NVR should be running a raid10
In an NVR id rather save the stream twice to 2 individual disks than save it once to a mirror.. that way when disk, memory or raid card goes pear shaped and corrupts the filesystem the other recordings will still be fine.. or even better just run 2 NVR's independently.. backup NVR could just be a simple FTP copy of the primary.
ps: if your running software raid w/out ECC memory your playing Russian roulette... you running on ECC in your PC-NVR? All its going to take is one stray cosmic ray and all that redundancy is corrupted.
a Mirror is fine for your OS drive so you can keep up durring a disk failure, but its still got its own issues.. yeah your protected against a disk failure but file corruption is just as common and in a mirror the bad bits propagate and there is no method of recovery.. RAID != Backup, and Ive lost so much data on a raid I have not considered it sufficient protection by its self.. like ever.
If your going to waste all that spindle space on video redundancy in a mirrored raid, without an external backup system in place that can archive that big raid then your just wasting your time.. if your going to do data protection half assed, then dont do it at all... save your money if your not getting real protection anyhow.. If the data is important enough to call for a raid, then its important enough for a proper backup system.. and a proper backup system has offsite storage capabilities.. the raid is just one layer of many layers of data protection.. just like security cameras are just one layer in security, my house is not suddenly vulnerable if I have a disk failure.. there is alot more protecting me than the NVR.
Mirror Raid all alone at this point is little more than a tool to maintain uptime and provide great read speeds, not your data's saftey and integrity.. if your driven by data integrity then you need to take more steps than slapping it on a raid 10 and crossing your fingers... doing it right is hard, and IMHO none of it is worthile for a video NVR that could potentially one day have some very important data on it, but most of the time contains 10TB of junk.
Raid 10 great is for insanely high performance disk access without quadrupling your failure rates, not keeping data safe.. I cant think of a single reason why any NVR should be running a raid10
In an NVR id rather save the stream twice to 2 individual disks than save it once to a mirror.. that way when disk, memory or raid card goes pear shaped and corrupts the filesystem the other recordings will still be fine.. or even better just run 2 NVR's independently.. backup NVR could just be a simple FTP copy of the primary.
ps: if your running software raid w/out ECC memory your playing Russian roulette... you running on ECC in your PC-NVR? All its going to take is one stray cosmic ray and all that redundancy is corrupted.
Attachments
Last edited by a moderator:
If there is one thing Ive learned it's that Murphy is a bastard.
Thanks again for your input on this one. I'm learning heaps. By the bad bits propagating I'm assuming that you mean the mirror will just mirror the corrupted file on both disks if for whatever reason, one copy of it corrupts.
Off site storage is not a realistic proposition for home applications in Australia at present. We don't have the upstream speeds to keep up with the data for the most part. 2Mbps is my upstream limit. The National Broadband Network (NBN), which is still being rolled out, has the capability to give greater speeds upstream, depending on how much you are willing to pay for the privilege, but upstream is metered and trying to back up a terrabyte a month to the cloud is prohibitively expensive here.
I kind of assumed you had an alarm, and no doubt physical security as well.
Well uptime remains extremely important to me. I don't want the NVR stopping at all if I can avoid it. For that reason I will be maintaining Raid in some form, be that RAID1 or RAID10, across my operating system and storage drives, but your input has caused me to think about what other steps I may want to take to ensure data integrity.
First question I am asking myself is do I want to do anything more at all? As you point out you don't bother with any form of backup or redundancy in your setup. RAID still appears to be the best way of achieving the uptime goals that I have but because the data is overwritten fairly regularly, do I need to be too concerned about long term data integrity?
If I do decide that I want that additional integrity do I rely on on-board SD cards like yourself? From a financial point of view SD cards don't make a lot of sense to me. Good SD cards are not cheap locally and you can get far more space on a hard drive for far less money.
This could be an option. Let the the cameras themselves record to a NAS/FTP as a backup, or just buy a stand alone 1U NVR and throw a couple of drives in it, which seems to be the cheapest option. That could limit my camera choices though.
Overkill? I was going to go with RAID6 till you gave me that info about the rebuild times following a failure. My reading so far tells me that RAID1 and 10 do not suffer from that issue, and RAID10, whilst not true two drive redundancy, there being a one in three chance of the wrong second drive failing during rebuild, is better than RAID1, and the capacity outcome in a 4 drive array is the same.
Cant say I am, nor could I if I wanted to at the moment, my motherboard doesn't support it.
Thanks again for the food for thought.
Yep raids propagate corruption across your drives.. lost my very first job in the datacenter (17 years old, fresh meat) when the raid controller on the local school districts's web server took a shit and corrupted the entire array.
Boss had given me a used tape autoloader he bought off ebay and told me to back it up, well because it was a used ebay one it did not work, 3rd day fighting the tape loader trying to get it to work is when the data decided to go Poof! back then the only way to backup 200GB of data was on tape, because that took a ton of disks to get to that point.. Ive been weary of raids my entire career because of this experience.
Offsite backup dont have to be done via network connectivity, it could be as simple yanking a hotswap disk out, replacing it with a spare, then taking that archive disk home with you and locking it in your safe until next month when you bring home another disk and take that one back to be re-written.. Sneakernet has practically unlimited throughput, just very high latency... as they say, its hard to beat the raw bandwidth of a station wagon full of harddrives barreling across the country @ 70mph..
If it took more than a $2 Halloween a mask to defeat my cameras, then I'd consider paying 2-4x more for storing that data.. but cameras are simply not that important to my security.. I dont run a prison.
Boss had given me a used tape autoloader he bought off ebay and told me to back it up, well because it was a used ebay one it did not work, 3rd day fighting the tape loader trying to get it to work is when the data decided to go Poof! back then the only way to backup 200GB of data was on tape, because that took a ton of disks to get to that point.. Ive been weary of raids my entire career because of this experience.
Offsite backup dont have to be done via network connectivity, it could be as simple yanking a hotswap disk out, replacing it with a spare, then taking that archive disk home with you and locking it in your safe until next month when you bring home another disk and take that one back to be re-written.. Sneakernet has practically unlimited throughput, just very high latency... as they say, its hard to beat the raw bandwidth of a station wagon full of harddrives barreling across the country @ 70mph..
If it took more than a $2 Halloween a mask to defeat my cameras, then I'd consider paying 2-4x more for storing that data.. but cameras are simply not that important to my security.. I dont run a prison.
Last edited by a moderator: