Hi there,
I work for a business who has been using the blue iris software for several years now, and at this point we are considering switching software. If we are able to fix the issues we have (or if its the software, then if an update will fix them) we will be happy to continue to pay for blue iris. Infact, because we rely on blue iris for the security of our business so much, we wouldn't mind paying more for the software.
I realize this forum isn't the official blue iris forum, nor are the developers for it here, but I am out of ideas on how to get this software to stop crashing. Im hoping someone here can help point out why this is happening.
What works and doesn't crash:
-the webserver software that blueiris runs for users to log in and view the cameras that are assigned to them and their group. this software is stable, although I am often worried about how secure it is for web facing servers
-the recordings are often fine, but what clips get recorded to what HDD and in what folder is not very intuitive, but for the most part, this part of the software works
-the AI integration works. when we had codeproject.ai integrtated it would work fantastic
-encoding to different formats works as long as there is enough cpu and gpu resources, but this is almost never done
-windows itself and other windows services are stable and do not cause issues as far as we can tell
-almost everything else, works except for the crashing
What crashes and when it crashes:
When we remote into blue iris or when we are infront of the actual computer and we bring up the blue iris admin interface, this is generally when it crashes. A few years ago, when we first started adding cameras it worked great. We knew from the start of this project that we would need a decently powerful server for the amount of feeds and how much we wanted to record. Here are the specs of our server:
Supermicro Server
RAM: DDR3 ECC buffered 1333Mhz, 128GB total
CPU: intel xeon E5-2687W @3.10 Ghz x2 (two CPUs, 64GB of ram per CPU)
GPU: nvidia Quadra k1200 x2 - used for encoding sometimes if we need them, but they remain unused otherwise. the CPU does most of the work
HDDs:
C: drive - intel D3-S4610 series 800GB SSD - used for the OS and the instal of blue iris only. (originally we were using windows 10 pro, but changed to windows server 2019. performance was better but it still crashed.)
D drive through to K drives - Seagate Seahawk 16TB drives
L: drive - NVMe Samsung 1tb drive (this was added wen we realized that the database was slowing down the performance of the computer, we though by adding the blueiris database to a super fast NVMe drive that it would stop it from crashing and make things faster. It made blueiris start up faster, and do clip lookups faster, but it did not help the crashing)
We thought it might have been the NIC not giving it enough bandwidth for the amount it needed, so we tried setting up Link Aggregation with the 4 port server NIC card that was installed in it, but we noticed that it was dropping packets, so we just use 1x regular 1gbps NIC. We have tried several different types of NICs, PCI, PCI-e and even usb3 ones and it makes no difference.
Other services running on the computer: None. We did have CodeProject.AI running on it for the ALPR, but we removed that when trying to figure out what makes it crash and have not yet put it back on. It is running on a fully updated version of windows and an up to date version of blue iris.
Almost never does the CPU usage get anywhere near 100%.. or even near 60%. The same goes for the RAM.
We have thoroughly tested the hardware. We did 2 full passes over a full week to test out all the ram the last time we upgraded it, and as far as we can tell theres nothing wrong with the motherboard or CPUs.
We monitor the HDDs constantly, looking for any errors or signs of bad sectors or the like. Occasionally one of the spinning Seagate drives die, but thats to be expected. Whenever they die we replace them with Seagate Skyhawk drives of the same size or larger.
The server has dual power supplies and a battery backup with a feature that is supposed to give it clean power all the time. It is in a server room with other servers with air conditioning so we know its not overheating, plus we watch the IMPI sensors for that sort of thing. Infact we often go through the event logs looking for hardware issues and other reasons this might occur with no results.
How we have blue iris set up:
As far as I know there really arnt any settings that can make the whole thing unstable.
We have a total of 35 cameras currently set up, mostly axis cameras. We have them all to record straight to disk without re-encoding. We have set all their framerates low, some of them at 10, some lower. The cameras themselves all have sub streaming set up at the lowest bandwidth settings possible with the default settings on the main stream.
It crashes whenever we click anywhere in the main GUI of the blue iris admin control panel, weather it is to zoom out of a camera to view all of the cameras, or when we go to right click and to add a new camera, or when we try to double click or right click on a n existing camera to change settings it will first hang and turn gray. at this point, if it does this and stays this way for any longer than 3 seconds it will hang for sure.
When blueiris hangs we have to kill it from the task manager in order to re-launch it.
About half the time when we do that it will work for a few clicks to add or edit a camera, but the other half the time we need to kill it again and re-start the service.
No additional windows server have been added. For backups we make a weekly backup of the system drive and of the blueiris database. We occasionally make backups of the settings on blueiris as well. What im saying is that its not backups that are causing it to crash, the backups only take about a half an hour once a week to complete. the recordings are not important enough for us to retain them. Retention of the videos, by the way, is set to 23 days maximum before simply getting deleted. By choosing this short retention period it looks like the huge amount of storage we invested into this machines isn't at all being used.
We require adding more cameras but at this point its very difficult too. We have contacted support for this in the past and they always come out with an update that sometimes works for a bit then breaks, or simply does not work for our issue at all. We are going to attempt to contact them again for advice, but i was hoping maybe someone here can give me some ideas.
It has gotten to the point where we are now installing microSD cards in all of the cameras because whenever the crashing happens it screws up the recordings to disk. Sometimes if we close blueiris from the task manager and then restart the service and dont launch the UI... THATs when it can remain stable for a long time,, but often we require logging into the admin panel on the computer
Does anyone else experience this? Is there a solution to this? If so, what is it? better hardware? A more powerful GPU? Different software?
Thank you for any help
I work for a business who has been using the blue iris software for several years now, and at this point we are considering switching software. If we are able to fix the issues we have (or if its the software, then if an update will fix them) we will be happy to continue to pay for blue iris. Infact, because we rely on blue iris for the security of our business so much, we wouldn't mind paying more for the software.
I realize this forum isn't the official blue iris forum, nor are the developers for it here, but I am out of ideas on how to get this software to stop crashing. Im hoping someone here can help point out why this is happening.
What works and doesn't crash:
-the webserver software that blueiris runs for users to log in and view the cameras that are assigned to them and their group. this software is stable, although I am often worried about how secure it is for web facing servers
-the recordings are often fine, but what clips get recorded to what HDD and in what folder is not very intuitive, but for the most part, this part of the software works
-the AI integration works. when we had codeproject.ai integrtated it would work fantastic
-encoding to different formats works as long as there is enough cpu and gpu resources, but this is almost never done
-windows itself and other windows services are stable and do not cause issues as far as we can tell
-almost everything else, works except for the crashing
What crashes and when it crashes:
When we remote into blue iris or when we are infront of the actual computer and we bring up the blue iris admin interface, this is generally when it crashes. A few years ago, when we first started adding cameras it worked great. We knew from the start of this project that we would need a decently powerful server for the amount of feeds and how much we wanted to record. Here are the specs of our server:
Supermicro Server
RAM: DDR3 ECC buffered 1333Mhz, 128GB total
CPU: intel xeon E5-2687W @3.10 Ghz x2 (two CPUs, 64GB of ram per CPU)
GPU: nvidia Quadra k1200 x2 - used for encoding sometimes if we need them, but they remain unused otherwise. the CPU does most of the work
HDDs:
C: drive - intel D3-S4610 series 800GB SSD - used for the OS and the instal of blue iris only. (originally we were using windows 10 pro, but changed to windows server 2019. performance was better but it still crashed.)
D drive through to K drives - Seagate Seahawk 16TB drives
L: drive - NVMe Samsung 1tb drive (this was added wen we realized that the database was slowing down the performance of the computer, we though by adding the blueiris database to a super fast NVMe drive that it would stop it from crashing and make things faster. It made blueiris start up faster, and do clip lookups faster, but it did not help the crashing)
We thought it might have been the NIC not giving it enough bandwidth for the amount it needed, so we tried setting up Link Aggregation with the 4 port server NIC card that was installed in it, but we noticed that it was dropping packets, so we just use 1x regular 1gbps NIC. We have tried several different types of NICs, PCI, PCI-e and even usb3 ones and it makes no difference.
Other services running on the computer: None. We did have CodeProject.AI running on it for the ALPR, but we removed that when trying to figure out what makes it crash and have not yet put it back on. It is running on a fully updated version of windows and an up to date version of blue iris.
Almost never does the CPU usage get anywhere near 100%.. or even near 60%. The same goes for the RAM.
We have thoroughly tested the hardware. We did 2 full passes over a full week to test out all the ram the last time we upgraded it, and as far as we can tell theres nothing wrong with the motherboard or CPUs.
We monitor the HDDs constantly, looking for any errors or signs of bad sectors or the like. Occasionally one of the spinning Seagate drives die, but thats to be expected. Whenever they die we replace them with Seagate Skyhawk drives of the same size or larger.
The server has dual power supplies and a battery backup with a feature that is supposed to give it clean power all the time. It is in a server room with other servers with air conditioning so we know its not overheating, plus we watch the IMPI sensors for that sort of thing. Infact we often go through the event logs looking for hardware issues and other reasons this might occur with no results.
How we have blue iris set up:
As far as I know there really arnt any settings that can make the whole thing unstable.
We have a total of 35 cameras currently set up, mostly axis cameras. We have them all to record straight to disk without re-encoding. We have set all their framerates low, some of them at 10, some lower. The cameras themselves all have sub streaming set up at the lowest bandwidth settings possible with the default settings on the main stream.
It crashes whenever we click anywhere in the main GUI of the blue iris admin control panel, weather it is to zoom out of a camera to view all of the cameras, or when we go to right click and to add a new camera, or when we try to double click or right click on a n existing camera to change settings it will first hang and turn gray. at this point, if it does this and stays this way for any longer than 3 seconds it will hang for sure.
When blueiris hangs we have to kill it from the task manager in order to re-launch it.
About half the time when we do that it will work for a few clicks to add or edit a camera, but the other half the time we need to kill it again and re-start the service.
No additional windows server have been added. For backups we make a weekly backup of the system drive and of the blueiris database. We occasionally make backups of the settings on blueiris as well. What im saying is that its not backups that are causing it to crash, the backups only take about a half an hour once a week to complete. the recordings are not important enough for us to retain them. Retention of the videos, by the way, is set to 23 days maximum before simply getting deleted. By choosing this short retention period it looks like the huge amount of storage we invested into this machines isn't at all being used.
We require adding more cameras but at this point its very difficult too. We have contacted support for this in the past and they always come out with an update that sometimes works for a bit then breaks, or simply does not work for our issue at all. We are going to attempt to contact them again for advice, but i was hoping maybe someone here can give me some ideas.
It has gotten to the point where we are now installing microSD cards in all of the cameras because whenever the crashing happens it screws up the recordings to disk. Sometimes if we close blueiris from the task manager and then restart the service and dont launch the UI... THATs when it can remain stable for a long time,, but often we require logging into the admin panel on the computer
Does anyone else experience this? Is there a solution to this? If so, what is it? better hardware? A more powerful GPU? Different software?
Thank you for any help
