Help needed with the installation of camera system (NAS+proxy)

[Zak]

Hi,

I think I have managed to create too complex setup for my experience. I really would like to hear some advice. I have asked around a lot about this and no proper solutions yet.

In my summer house I have at the moment 6 Hikvision 8MP 2085FWD-I cameras. The estate is in the middle of no-where by a forest lake and the internet connection speed there is using 4G/LTE and is very slow (like 20-60mbps with powerful amplified directional antennas on roof. The weather and time of the day effect heavily on the speed). Nearest link is 10km from the site and some small hills inbetween.

The problem is complicated because of FTP issue with Hikvision. This would be much more easier problem if cameras could record videos with FTP and simply synchronize/manage single video files but according to Hikvision customer service FTP protocol (!?) of Hikvision cameras support only picture transfer not videos (which in my mind is astonishing! It is like I could save to my local HDD only Word documents and not Excel files!).

Now to the setup:
1) 6x 2085FWD all recording 4k/20fps with the best picture quality. All with wired/PoE connections. Working fine.
2) Summer house basic infra concerning this is basically switch and firewall connected using 4G to my town house with similar infra.
3) Planned a Windows computer as proxy for transferring/mirroring the data to main server to my house in town
4) In my town house just Qnap NAS server with a dedicated pool/volume for cameras. Working fine. The data is in town because there have been many cases when uninvited visitors have stolen the local NVR too. I want the data to be safe.
5) Now I watch the live stream from cameras over the internet using RTSP stream (VLC player). With Android/iOS devices I use Hik's 4200. Working fine but connecting to to camera(s) takes a lot of time and connection fail many times before stream start to work. Maybe bandwide issue, dunno.

The problem is that the very slow internet connection in my summer house needs record proxy? (= 1st hand NVR = WIndows computer maybe) which would mirror data with the server in town. In town the internet speed is 1G so it should be fine. I would therefore really like to use that to town mirrored copy for all records. Summer house internet speed is not fast enough even to transfer one RTSP stream smoothly and I would want to prepare all cameras recording simultaneously to proxy and all the time mirror the records to town server max speed available.

In practice file transfer is always only to one direction: Summer house -> Town house. All the record viewings should be done using town resources.

Couple of questions:
1) In your opinion what would be the optimal setup for this to keep the data safe in town and use only data in town with the fact of throttled internet connection to summer house?
2) I think I need that local (summerhouse) proxy because of throttled connection. What would be the best way to synchronize the files (Qnap has it own apps for this but is this the best way to do it?). Is it even possible to keep two mirrored/synchronized copies of Hik camera data.
3) Is it possible to select from the software which dataset is used. (Meaning that like 4200 can live stream cameras but use mirrored dataset as main files and not that proxy). Maybe I will move data from proxy to town and then the camera files are not identical. In proxy there is only not yet moved files which are to be processed.
4) I realized that the files the cameras make to server are not the size of the actual recordings but all are the same size (about 256M) even the actual recording is 3-5secs long and would take 20M of space. FTP would solve the issue but Hik not supporting as earlier said. How can I save the disk space and size of recordings to copy/move files faster?
5) How can I setup record viewing so that it use the mirrored copy in town and not the cameras.

Any good ideas how to make this system work well?

I think I have asked this before but now the system have to be setup because autumn is coming and it seems that unwanted visitors drive around to search for catch in the dark.

Long story but I would appreciate some practical advices on this or even someone could help me during the process using PM. Thanks for help.

 

[tech_junkie]

On a Hikvision NVR, make sure to check zero channel encoding. Then use platform access, then use hik-connect for the phone, and IVMs on a remote computer. The zero channel encoding is for low bandwidth. And I know of many that is set up this way on 256k DSL connections that 128K is their upload speed.

There must be something awry because 4GLTE is 20 to 30 times faster than the speed of a typical cable internet. Cell phone boosters can cause issues like this, so does a noisy amplified antenna. If your internet setup was installed by a cell company, I would have them check it out. Because it sounds like its not working correctly.

 

[Zak]

On a Hikvision NVR, make sure to check zero channel encoding. Then use platform access, then use hik-connect for the phone, and IVMs on a remote computer. The zero channel encoding is for low bandwidth. And I know of many that is set up this way on 256k DSL connections that 128K is their upload speed.

There must be something awry because 4GLTE is 20 to 30 times faster than the speed of a typical cable internet. Cell phone boosters can cause issues like this, so does a noisy amplified antenna. If your internet setup was installed by a cell company, I would have them check it out. Because it sounds like its not working correctly.

Thank you very much for your answer.

First of all there is no specific NVR device (if you mean NVR as a separate dedicated recording device). The cameras record at the moment directly to HDD (shared folders in Qnap). Working but the issue is the throttled bandwide. If the line would be faster there would be no problems and no need of send proxy. Bandwide is the bottleneck here. Ordering of fibre/copper line to my cottage would cost around 50-100k USD. That's a LOT.

Secondly, Hik-connect is a cloud service. Not sure if the video itself flow thru the cloud servers but H-C is at least a kind of DDNS and needs a hole bored to firewall. I do not trust any cloud service which have access from/to devices in my LAN no matter if they are in DMZ or in isolated networks. There is no safe cloud services. There is only 2 kinds of cloud services: Those which have been hacked and those which will be hacked. My system is completely personal firewall-to-firewall site-to-site VPN tunnel. Thus I can prevent totally WAN connections to my cameras from anywhere but my own VPN. Thus for example security holes in firmware and outdated firmware is not a problem. Therefore Hik-connect is not a real option for me.

The bandwide demand for 4K/20fps with all the bells and whistles is around 15Mbps per camera. 6x15M=90M is A LOT for 4G with bad connection.

4G LTE speed depends on 1) the quality of the signal, 2) strength of the signal, 3) other sources of interference, 4) signal reflections and 5) cap of the ordered LTE type by user from NSP. In my case 200M is the cap. But in these conditions the hills and distance weaken the signal so much that error correction algorithms and signal spectrum cannot transfer more than that 20-60M also fluctuating too much for constant and reliable stream of cameras.

Typical household wired internet line here is about 200-400M (copper) and more with fibre.

 

[wittaj]

Most here run 15FPS or less. It is shutter speed that is more important than FPS. FPS doesn't get you the clean freeze frame capture, it is shutter speed.

You will save a lot of bandwidth dropping the FPS. I will take a higher shutter speed and less FPS over default settings and more FPS every day of the week.

I capture plates with my plate camera at 8FPS because it runs at a 1/2,000 shutter.


Watch these, for most of us, it isn't annoying until below 10FPS

 

[Zak]

Most here run 15FPS or less. It is shutter speed that is more important than FPS. FPS doesn't get you the clean freeze frame capture, it is shutter speed.

You will save a lot of bandwidth dropping the FPS. I will take a higher shutter speed and less FPS over default settings and more FPS every day of the week.

I capture plates with my plate camera at 8FPS because it runs at a 1/2,000 shutter.


Watch these, for most of us, it isn't annoying until below 10FPS

Thanks for the proposition. It is true that fps naturally cause high need of bandwidth. Resolution is the worst bandwidth hog and fps is the second worst.

Still, there are couple of "buts" in your suggestion:
1) Firstly higher shutter speed means darker picture. I would need better lights. All the cameras are outside in most in very dark conditions in the evening/night.
2) Secondly I really like smooth movement. I like to watch videos and not slideshows. I think that in my case I use the cameras also for video purposes and not only as evidence of burglary visit. I like to watch all the natural life visiting in my property like deers, foxes, rabbits, badgers, and all the other small creatures. Smooth movement is important to me.

I would take 4K/60fps if they would be available at reasonable price. Naturally 4K and high fps need a lot processing power and not just bandwidth.

Still if I would lower the fps to 10fps it would not change the big picture so much that all the data would fit to the same pipe.

There are 2 ways to approach this problem:
1) To reduce the need for bandwide to fit all to present LTE pipe like ftp, resolution, quality, ... (meaning traditional approach that the cameras are surveillance only)
2) Maintain full abilities with smart proxies and get all the bells and whistles. (kinda of more or less surveillance and entertainment)
I prefer 2 option if possible.

 

[tech_junkie]

The bandwide demand for 4K/20fps with all the bells and whistles is around 15Mbps per camera. 6x15M=90M is A LOT for 4G with bad connection.

The other function of the NVR is to facilitate the remote viewing as the host video compression server. So the up data is only relevant to what is viewed. Zero channel encoding takes the multi camera views and compiles it to one stream so when you see 8 camera channels simultaneously, the up traffic is one stream @ 128K instead of 8 streams at 15M/cam. Besides that, an NVR does hardware compression on the cam's connection so its not that much bandwidth being tied up on the LAN either. So It drops its usage to about 4M/channel wich increases the record time on hard drives compared to just using a generic NVR system or just its SD card.

For someone who doesn't want to use platform access, which is just a point to point negotiation for the app to connect to the NVR. You just use the IVMS software for the computer and Hik-connect on the phone using manual configuration with the LAN Address via wifi or VPN.

Btw, there isn't much to hack on a platform access server. Because nothing gets stored there, because the phone app session is serviced at the NVR. Since the incoming connection is most likely to be a dynamic address, it would be useless to store it in a DB in the first place.

 

[tech_junkie]

I would take 4K/60fps if they would be available at reasonable price. Naturally 4K and high fps need a lot processing power and not just bandwidth.

60 FPS cams are interlace video, which means odd and even lines are transmitted on consecutive fallowing frames where a non-interlace video displays all of the lines per frame. 60FPS interlace is the equivilent to 30 fps non-interlaced. Security cams are non-interlace video.

 

[Zak]

The other function of the NVR is to facilitate the remote viewing as the host video compression server. So the up data is only relevant to what is viewed. Zero channel encoding takes the multi camera views and compiles it to one stream so when you see 8 camera channels simultaneously, the up traffic is one stream @ 128K instead of 8 streams at 15M/cam. Besides that, an NVR does hardware compression on the cam's connection so its not that much bandwidth being tied up on the LAN either. So It drops its usage to about 4M/channel wich increases the record time on hard drives compared to just using a generic NVR system or just its SD card.

Interesting. NVR tech is new to me because I have always preferred client/server because of flexibility of resources.

So you are saying that basically I can squeeze 90M separate streams to one 128K stream by using this zero channel transmit? That would be interesting. Probably it can be reached in following conditions: a) there is no movement and basically the transmit is to keep alive still picture, 2) resolution is very low, 3) all streams are combined as one and recompressed (=re-encoded), 4) it is only for live video watching, not for full video recording.

Or if you mean that fact that in the screen of NVR all the secondary/third streams are combined as one picture (tiles) with very low resolution each tile and that picture is transmitted with 128K? If so there are couple of buts: A) Main stream still have to be transmitted thru the network and therefore the basic issue does not change and B) making of that tile view needs great amount of resources itself. In my case send proxy would need practically no processing power at all. No positive effect to the main issue.

In LAN the size of transmitted data is not so important to me after the data has been transferred to my town system. It is a full 10G network and therefore it does not get filled up fast. NAS is also 10G with fibre optic wiring to switch and there is a large SSD cache. So intra and speed is not an issue, but only when I get the data squashed thru the 4G pipe.

For someone who doesn't want to use platform access, which is just a point to point negotiation for the app to connect to the NVR. You just use the IVMS software for the computer and Hik-connect on the phone using manual configuration with the LAN Address via wifi or VPN.

Btw, there isn't much to hack on a platform access server. Because nothing gets stored there, because the phone app session is serviced at the NVR. Since the incoming connection is most likely to be a dynamic address, it would be useless to store it in a DB in the first place.

You must mean a case where camera and NVR are in the same LAN, right? But Hik-connection is about connecting camera using internet to NVR. Then at least DNS have to be got thru internet if you don't have static public IP. I have dynamic IP. Working fine with DDNS. But anyway at least H-C must be a service where NVR and camera know each others address = DNS or is a service which transfer the data without saving it from camera to NVR (kinda proxy). If someone hack that system you basically have access to all the streams flowing between the systems. DNS data itself is not so crucial because if hacked you only know which IP to stress (like with DoS and worse).

60 FPS cams are interlace video, which means odd and even lines are transmitted on consecutive fallowing frames where a non-interlace video displays all of the lines per frame. 60FPS interlace is the equivilent to 30 fps non-interlaced. Security cams are non-interlace video.

There are two kinds of 60fps; 60p and 60i (same with all resolutions like for example 1080p and 1080i). Only latter is interlaced. I was talking about 60p cameras. Interlacing is fooling oneself. Normally when honest manufacturers talk about fps they talk about progressive (p), not about interlaced (i).
--------------------

Still, anyone have any ideas how to solve this technology issue described in the first post?

Terms:
1) 4G connection cannot get faster. It is as fast as it can get. For 5G the distance is way too long. And no other substituting tech is available for reasonable costs.
2) I want to use 4K/20fps (=camera max) for all cameras. Secondary stream naturally can be lower but recorded videos must be max video quality.
3) Investing into NVR does not change the problem because the problem is not LAN or recording device or camera but it is pool WAN between cottage and town house.

So anyone have any idea how to create send proxy to cottage where in the first hand all recordings are saved and then immediately "slowly" using this 4G transmitted to main server in town. Town server is where I watch all the recordings. Proxy need to be a pool to gather files to be transmitted.

Only if Hikvision would support FTP video this would be extremely trivial case. But because video recording using FTP is disabled (?) it is now much more challenging issue.

I have one ancient Dahua 3200S which is able to record FTP videos and it is perfect for me. I had no idea that Hikvision have crippled normal Linux FTP protocol (!) Do you know if new modern Dahuas still have this ability? Because if they have I think I'm gonna throw these shitty Hikvisions to the lake and shop some Dahuas.

 

[tech_junkie]

on typical mobile streams 320 x 240 @400Kbps is the norm. The NVR is the trans-coding server.
NVR records the main stream as well as the other steams and event styles. I usually set up main stream and motion event main stream to record so I don't miss anything.

Only some models have the "Picture Capture" feature. But I noticed over the years some a lot of them are removing it.

Looks like the changed hik-connect and moved that direct NVR connect into an app called HikCentral. Which you could use via wifi there or over your VPN.

But since you have your own vpn bridge, you wouldn't really need any software with an NVR, You would just log into it by web browser. Then you can access your cameras without having to stream all those cameras across. If you want to redundant remote storage, that could be set up as well over that VPN

 

[jack7]

Hi,

I think I have managed to create too complex setup for my experience. I really would like to hear some advice. I have asked around a lot about this and no proper solutions yet.

In my summer house I have at the moment 6 Hikvision 8MP 2085FWD-I cameras. The estate is in the middle of no-where by a forest lake and the internet connection speed there is using 4G/LTE and is very slow (like 20-60mbps with powerful amplified directional antennas on roof. The weather and time of the day effect heavily on the speed). Nearest link is 10km from the site and some small hills inbetween.

The problem is complicated because of FTP issue with Hikvision. This would be much more easier problem if cameras could record videos with FTP and simply synchronize/manage single video files but according to Hikvision customer service FTP protocol (!?) of Hikvision cameras support only picture transfer not videos (which in my mind is astonishing! It is like I could save to my local HDD only Word documents and not Excel files!).

Now to the setup:
1) 6x 2085FWD all recording 4k/20fps with the best picture quality. All with wired/PoE connections. Working fine.
2) Summer house basic infra concerning this is basically switch and firewall connected using 4G to my town house with similar infra.
3) Planned a Windows computer as proxy for transferring/mirroring the data to main server to my house in town
4) In my town house just Qnap NAS server with a dedicated pool/volume for cameras. Working fine. The data is in town because there have been many cases when uninvited visitors have stolen the local NVR too. I want the data to be safe.
5) Now I watch the live stream from cameras over the internet using RTSP stream (VLC player). With Android/iOS devices I use Hik's 4200. Working fine but connecting to to camera(s) takes a lot of time and connection fail many times before stream start to work. Maybe bandwide issue, dunno.

The problem is that the very slow internet connection in my summer house needs record proxy? (= 1st hand NVR = WIndows computer maybe) which would mirror data with the server in town. In town the internet speed is 1G so it should be fine. I would therefore really like to use that to town mirrored copy for all records. Summer house internet speed is not fast enough even to transfer one RTSP stream smoothly and I would want to prepare all cameras recording simultaneously to proxy and all the time mirror the records to town server max speed available.

In practice file transfer is always only to one direction: Summer house -> Town house. All the record viewings should be done using town resources.

Couple of questions:
1) In your opinion what would be the optimal setup for this to keep the data safe in town and use only data in town with the fact of throttled internet connection to summer house?
2) I think I need that local (summerhouse) proxy because of throttled connection. What would be the best way to synchronize the files (Qnap has it own apps for this but is this the best way to do it?). Is it even possible to keep two mirrored/synchronized copies of Hik camera data.
3) Is it possible to select from the software which dataset is used. (Meaning that like 4200 can live stream cameras but use mirrored dataset as main files and not that proxy). Maybe I will move data from proxy to town and then the camera files are not identical. In proxy there is only not yet moved files which are to be processed.
4) I realized that the files the cameras make to server are not the size of the actual recordings but all are the same size (about 256M) even the actual recording is 3-5secs long and would take 20M of space. FTP would solve the issue but Hik not supporting as earlier said. How can I save the disk space and size of recordings to copy/move files faster?
5) How can I setup record viewing so that it use the mirrored copy in town and not the cameras.

Any good ideas how to make this system work well?

I think I have asked this before but now the system have to be setup because autumn is coming and it seems that unwanted visitors drive around to search for catch in the dark.

Long story but I would appreciate some practical advices on this or even someone could help me during the process using PM. Thanks for help.

Blue Iris software on a PC at your summer house should do what you want.
Here's a start on how to do it. See page 129 at

https://blueirissoftware.com/BlueIris.PDF

I haven't done it.

Also, if not using it, try using H.265 video compression to reduce bandwidth.