AXIS P3346-VE and P3364-VE has jumpy frames even though measuring network performance looks good

[CrazyFin]

Hi all

I have a 3-year ongoing issue with my AXIS P3346-VE and P3364-VE cameras that I have mounted outdoors around my house.

When vehicles are passing by the video playback from the recorded video (as well as the live stream) has some jumpy frames / "jumpiness". It looks sometimes like the jumps / short freezes (like 100-500 milliseconds) comes when the cameras are sending an I-frame.

Here are some shared links to my Google Drive where you can download some samples and look on the "jumpiness":
AXIS P3346-VE - Jumpy frames - Terrace.20170206_110000.avi
Jumpy frames - 20 fps - I-frame 20 - Comp 30 - VBR - 2048x1536 capture - 2480x1536 stream - Jumpiness back! - Terrace.20170412_170000_1.avi

My hardware setup is the following:

Cameras:
3 of the AXIS P3346-VE 3MP cameras with latest firmware 5.50.3.5 running @ 20 fps
1 of the AXIS P3364-VE 1MP camera with latest firmware 6.10.1.3 running @ 25 fps

(I have 2 more of the P3346-VE:s to be mounted and connected in the coming spring as well as a Vivotek FD9381-EHTV 5MP camera that supports H.265 encoding which BlueIris now supports as well)

PC is dedicated for my video surveillance:
Motherboard is Asus X99-A (X99-A | Motherboards | ASUS Global)
CPU Intel Core i7 5820K / Haswell-E/EP
Specification Intel(R) Core(TM) i7-5820K CPU @ 3.30GHz
Number of cores 6 (max 8)
Number of threads 12 (max 16)
RAM is 16GB

Graphics card is Asustek NVIDIA GeForce GTX 770 with 2GB graphics RAM.

Intel CT Gigabit network adapter (the one on the motherboard has been disabled by me in the BIOS) with the latest driver.

All this running on Windows 10 PRO 64-bit OS together with (as of today) latest BlueIris x64 version.

My PC indicates a load of 10-15% with all 4 cameras running at full FPS and my resource monitor looks like the dedicated surveillance PC is running at idle.
I have also run the software "DPC Latency checker" (DPC Latency Checker) to see if I have any IRQ / response time issues but it all looks good with an average DPC latency of approx 32 microseconds and the absolute maximum I saw under a 5 minute testing period was 362 microseconds.

Note that my problems with the jumpiness are NOT related to BlueIris or settings within BlueIris since I have EXACTLY the same problem with many other software platforms I have tested (even AXIS' own NVR software AXIS Companion and the more professional AXIS Camera Station). Other platforms I have tested are Milestone, Luxriot, Avigilon and so on. They all show the same stuttering / jumpiness as I see with BlueIris.

I have done extensive research, testing of both software parameters, camera settings, testing of hardware involved such as measuring network performance with both software JPERF and actual network measurement tools like the Fluke Cable IQ Qualification Tester (Cable IQ Qualification Tester - Network Cable Tester | Fluke Networks)
NONE of the measurements indicates networking issues.

I have actually connected a laptop at the cable ends where the cameras are normally connected and then I have run JPERF between that laptop all the way through the involved switch (Zyxel GS1900-8HP) and to my dedicated surveillance PC. The JPERF measurements shows excellent results with NO packet losses and speeds of up to 400Mbits/s.
I have even tested to play 1080p HD video files through the same network cables and switch and there is NO glitch/jumpiness/freezes whatsoever. (I had the laptop in the camera cable end playing up a video from my dedicated surveillance PC.)

I have set GOV to 20 (i.e. one I-frame per 20 frames) for the P3346-VE cameras running @ 20 fps and GOV = 25 for the P3364-VE camera running @ 25 fps. I have tested with many different GOV settings between 1 and up to 120 but no difference.

I have tested to switch between VBR (variable bit rate) and CBR (constant bit rate, which for AXIS is not TRUE constant bit rate but more like a ceiling of the bit rate: FAQ: What is the difference between Variable Bit Rate and Constant Bit Rate? | Axis Communications). Neither CBR or VBR gets rid of the jumpiness.

When I look at the camera log I see NO lost packets in the camera's own debug logs.
As you can see there are NO lost packets or errors on the transmitted packets. I think this log shows the packets since the previous restart of the camera (10 days):

----- Network statistics -----

Inter- |Receive |Transmit
face |bytes packets errs drop fifo frame compressed multicast|bytes packets errs drop fifo colls carrier compressed
lo: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
eth0: 2500775728 252227755 13 0 0 13 0 677459 842642917 430564708 0 0 0 0 0 0
sit0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

These problems are for ALL my 4 connected cameras.

Finally I contacted Ken at BlueIris support and when he investigated some BVR-files that I sent to him I got some really interesting observations from him.
Below is a log output from the actual frames and packets coming from the AXIS cameras into BlueIris

BVR.cpp(746) : atlTraceGeneral - FFMPEG decode 50 8/9 1 used 40410/40410 12; 2048x1536 b=0 0
BVR.cpp(746) : atlTraceGeneral - FFMPEG decode 50 9/10 1 used 38821/38821 12; 2048x1536 b=0 0
BVR.cpp(746) : atlTraceGeneral - FFMPEG decode 50 0/1 1 used 493678/493678 12; 2048x1536 b=0 1
BVR.cpp(746) : atlTraceGeneral - FFMPEG decode 50 1/2 1 used 34162/34162 12; 2048x1536 b=0 0
BVR.cpp(746) : atlTraceGeneral - FFMPEG decode 50 2/3 1 used 38819/38819 12; 2048x1536 b=0 0
BVR.cpp(746) : atlTraceGeneral - FFMPEG decode 50 3/4 1 used 40175/40175 12; 2048x1536 b=0 0
BVR.cpp(746) : atlTraceGeneral - FFMPEG decode 50 4/5 1 used 40682/40682 12; 2048x1536 b=0 0
BVR.cpp(746) : atlTraceGeneral - FFMPEG decode 50 5/6 1 used 40498/40498 12; 2048x1536 b=0 0
BVR.cpp(746) : atlTraceGeneral - FFMPEG decode 100 6/7 1 used 47902/47902 12; 2048x1536 b=0 0
BVR.cpp(746) : atlTraceGeneral - FFMPEG decode 50 7/8 1 used 41071/41071 12; 2048x1536 b=0 0
BVR.cpp(746) : atlTraceGeneral - FFMPEG decode 50 8/9 1 used 39751/39751 12; 2048x1536 b=0 0
BVR.cpp(746) : atlTraceGeneral - FFMPEG decode 50 9/10 1 used 39023/39023 12; 2048x1536 b=0 0
BVR.cpp(746) : atlTraceGeneral - FFMPEG decode 50 0/1 1 used 493911/493911 12; 2048x1536 b=0 1
BVR.cpp(746) : atlTraceGeneral - FFMPEG decode 50 1/2 1 used 34346/34346 12; 2048x1536 b=0 0
BVR.cpp(746) : atlTraceGeneral - FFMPEG decode 50 2/3 1 used 38734/38734 12; 2048x1536 b=0 0
BVR.cpp(746) : atlTraceGeneral - FFMPEG decode 50 3/4 1 used 40758/40758 12; 2048x1536 b=0 0
BVR.cpp(746) : atlTraceGeneral - FFMPEG decode 450 0/1 1 used 493835/493835 12; 2048x1536 b=0 1
BVR.cpp(746) : atlTraceGeneral - FFMPEG decode 50 1/2 1 used 34756/34756 12; 2048x1536 b=0 0
BVR.cpp(746) : atlTraceGeneral - FFMPEG decode 50 2/3 1 used 38810/38810 12; 2048x1536 b=0 0
BVR.cpp(746) : atlTraceGeneral - FFMPEG decode 150 3/4 1 used 55366/55366 12; 2048x1536 b=0 0
BVR.cpp(746) : atlTraceGeneral - FFMPEG decode 50 4/5 1 used 43049/43049 12; 2048x1536 b=0 0
BVR.cpp(746) : atlTraceGeneral - FFMPEG decode 50 5/6 1 used 40959/40959 12; 2048x1536 b=0 0
BVR.cpp(746) : atlTraceGeneral - FFMPEG decode 50 6/7 1 used 40396/40396 12; 2048x1536 b=0 0
BVR.cpp(746) : atlTraceGeneral - FFMPEG decode 150 7/8 1 used 51766/51766 12; 2048x1536 b=0 0
BVR.cpp(746) : atlTraceGeneral - FFMPEG decode 200 0/1 1 used 493701/493701 12; 2048x1536 b=0 1
BVR.cpp(746) : atlTraceGeneral - FFMPEG decode 50 1/2 1 used 34694/34694 12; 2048x1536 b=0 0
BVR.cpp(746) : atlTraceGeneral - FFMPEG decode 49 2/3 1 used 38824/38824 12; 2048x1536 b=0 0
BVR.cpp(746) : atlTraceGeneral - FFMPEG decode 450 0/1 1 used 494446/494446 12; 2048x1536 b=0 1
BVR.cpp(746) : atlTraceGeneral - FFMPEG decode 500 0/1 1 used 493330/493330 12; 2048x1536 b=0 1
BVR.cpp(746) : atlTraceGeneral - FFMPEG decode 700 0/1 1 used 494509/494509 12; 2048x1536 b=0 1
BVR.cpp(746) : atlTraceGeneral - FFMPEG decode 50 1/2 1 used 35046/35046 12; 2048x1536 b=0 0
BVR.cpp(746) : atlTraceGeneral - FFMPEG decode 50 2/3 1 used 39252/39252 12; 2048x1536 b=0 0
BVR.cpp(746) : atlTraceGeneral - FFMPEG decode 699 0/1 1 used 494857/494857 12; 2048x1536 b=0 1
BVR.cpp(746) : atlTraceGeneral - FFMPEG decode 100 1/2 1 used 47381/47381 12; 2048x1536 b=0 0
BVR.cpp(746) : atlTraceGeneral - FFMPEG decode 50 2/3 1 used 42377/42377 12; 2048x1536 b=0 0
BVR.cpp(746) : atlTraceGeneral - FFMPEG decode 100 3/4 1 used 50138/50138 12; 2048x1536 b=0 0
BVR.cpp(746) : atlTraceGeneral - FFMPEG decode 50 4/5 1 used 42644/42644 12; 2048x1536 b=0 0
BVR.cpp(746) : atlTraceGeneral - FFMPEG decode 450 0/1 1 used 495498/495498 12; 2048x1536 b=0 1
BVR.cpp(746) : atlTraceGeneral - FFMPEG decode 50 1/2 1 used 35001/35001 12; 2048x1536 b=0 0
BVR.cpp(746) : atlTraceGeneral - FFMPEG decode 50 2/3 1 used 39323/39323 12; 2048x1536 b=0 0
BVR.cpp(746) : atlTraceGeneral - FFMPEG decode 50 3/4 1 used 41198/41198 12; 2048x1536 b=0 0
BVR.cpp(746) : atlTraceGeneral - FFMPEG decode 50 4/5 1 used 40929/40929 12; 2048x1536 b=0 0
BVR.cpp(746) : atlTraceGeneral - FFMPEG decode 400 0/1 1 used 495413/495413 12; 2048x1536 b=0 1
BVR.cpp(746) : atlTraceGeneral - FFMPEG decode 50 1/2 1 used 34923/34923 12; 2048x1536 b=0 0
BVR.cpp(746) : atlTraceGeneral - FFMPEG decode 50 2/3 1 used 39417/39417 12; 2048x1536 b=0 0
BVR.cpp(746) : atlTraceGeneral - FFMPEG decode 50 3/4 1 used 40514/40514 12; 2048x1536 b=0 0
BVR.cpp(746) : atlTraceGeneral - FFMPEG decode 50 4/5 1 used 40987/40987 12; 2048x1536 b=0 0
BVR.cpp(746) : atlTraceGeneral - FFMPEG decode 150 5/6 1 used 53744/53744 12; 2048x1536 b=0 0
BVR.cpp(746) : atlTraceGeneral - FFMPEG decode 250 0/1 1 used 495890/495890 12; 2048x1536 b=0 1
BVR.cpp(746) : atlTraceGeneral - FFMPEG decode 50 1/2 1 used 34347/34347 12; 2048x1536 b=0 0
BVR.cpp(746) : atlTraceGeneral - FFMPEG decode 50 2/3 1 used 38817/38817 12; 2048x1536 b=0 0

The time distance from the previous frame is right after the word "decode" in this data. You can see most are 50 milliseconds which is equivalent of 20 fps. However, look at the long jumps, some 100, 150, 250, even 500 and 700. That's more than 1/2 sec between frames!

After the log word "used" you can see the frame size in bytes. The ones with 50 ms jumps are only about 40-50kb. The ones causing the jumpiness are much larger, nearly 500kb! So the camera is definitely sending larger frames when there is motion, which makes sense if you are on VBR (variable bit rate). However as I mentioned higher up in my text above I have tested with CBR (constant bit rate) ... in that case I would expect each frame to be about the same size but that does not happen (as my link above to how AXIS works with CBR being a CEILING for the bitrate, not a fixed bitrate).

So it's not the playback in BlueIris causing the "jumpiness". It's the way the video is being received from the camera with these long gaps. This is causing the "jumpiness" in playback.

So the question now is WHY is this happening? Is this a defect of the firmware of AXIS cameras and how they produce the data for each I-frame and/or P-fram or is this a "proof" of something being screwed up in my network anyway even though all my measurements with both software and hardware instruments indicates that my network is capable of Gigabit speeds and the ethernet interfaces of my AXIS cameras are only for 100MBit/s while the actual bitrate used by the cameras is hovering around 2-6Mbit/s.
My "feeling" is that this is bad behaviour of the AXIS P33-series of cameras.

Does any of you have a suggestion what tool I could use to inspect real time frames coming from the camera similar to the way Ken is showing for me in the BlueIris log above?

I found this interesting blog post talking about AXIS doing its own implementation of multicasting of video that only seems to work with IE: AXIS IP Cameras = Fail - System Overlord
I tried to configure multicasting but I was never able to get it to work in BlueIris.
I am currently using RTSP-streams but I have tested with normal HTTP-streams and I have also played with the settings "Use RTP/UDP ports", "Send RTSP keep-alives", "Use RTSP/stream timecode" but no difference with any of the settings. Still the jumpiness as can be seen the packet log above.

I am really hoping to be able to mount and connect my new Vivotek FD9381-EHTV 5MP camera within 4-5 weeks from now and see if I get the same jumpiness with that camera as well.

Any other AXIS camera users out there seeing similar problems?

 

[alastairstevenson]

the ethernet interfaces of my AXIS cameras are only for 100MBit/s

Is this the spec, or an observation? And how?
You have tested the cabling with a laptop perf tester and found it good.
The large camera frames of, say, 500KB, would require a minimum of 50mS with a 100Mbps ethernet link at 100% utilisation if full duplex.
If the link was running at 10Mbps the equivalent would be 500mS.

 

[CrazyFin]

Is this the spec, or an observation? And how?

It is according to the specs, for example page 63 in the user manual: https://www.axis.com/files/manuals/um_p3346ve_54958_en_1312.pdf
I realise now when reading that page that the camera supports both 10MBit/s and 100Mbit/s on its Ethernet port.
Looking at the ports on my Zyxel GS1900-8HP switch where all my 4 cameras and the dedicated video surveillance PC is connected to I can see that the negotiated link speed is 100Mbit/s. I can also see that the utilization (during 2 second updates) is only a few percent when viewing in H.264 mode. I can get it up to approx 30-35% utilization on a port if I view the camera stream via a web browser using MJPEG instead of H.264. Doing that I see approx a bitrate of around 28-30MBit/s.

Here is a screenshot of the port speeds that I can see on my Zyxel GS1900-8HP switch web management page:


As you can see above the negotiated speeds for all my 4 cameras are 100Mbit/s with full duplex.

This is how the port bandwidth utilization looks like when viewing streams in H.264 mode:

As you can see above bandwidth usage per port is VERY little. Most of my 4 cameras are running around 2 to 5Mbit/s in bitrate. (This is when viewing all 4 cameras simultaneously in Blue Iris.)

If I view the camera stream through a web browser and use MJPEG stream instead of H.264 the port bandwidth utlization jumps to 30-35% and bitrate from the camera increases to around 28 to 30Mbit/s:


And looking in the debug log for the cameras themselves I clearly see that the port speed that has been negotiated for the camera is 100Mbit/s:

----- Settings for eth0 -----

Settings for eth0:
Supported ports: [ TP MII ]
Supported link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
Supports auto-negotiation: Yes
Advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
Advertised auto-negotiation: Yes
Speed: 100Mb/s
Duplex: Full
Port: MII
PHYAD: 0
Transceiver: internal
Auto-negotiation: on
Link detected: yes


And under network configuration for the camera I see the following in the log (I have replaced adress info with XX):
----- Network configuration -----

1: lo: <LOOPBACK,UP,10000> mtu 16436 qdisc noqueue
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 brd 127.255.255.255 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,10000> mtu 1500 qdisc pfifo_fast qlen 1000
link/ether xx:xx:xx:xx:xx:xx
inet xxx.xx.xx.xx/24 brd xxx.xxx.xx.xxx scope global eth0
3: sit0: <NOARP> mtu 1480 qdisc noop
link/sit 0.0.0.0 brd 0.0.0.0

The MTU is set to 1500 and I am not sure if this setting could affect the problems I see?

 

[alastairstevenson]

OK, that detailed check certainly lets out that (rather) long shot.
MTU=1500 is standard, it's not likely to be implicated.

 

[nayr]

can you plug the camera directly into your BI server and power it externally? cutting out everything inbetween network wise...

 

[ChengXin]

If you have a spare network card that can be plug into your VMS machine and configure NIC teaming. It should solve your jittery issue. I have worked on other vms with jittery issue, this solve them as frames are received and clear within the timeout period set for the frames.

If you are interested in further discussion, please contact me via Skype @ Cheng.Xin.SG

 

[CrazyFin]

can you plug the camera directly into your BI server and power it externally? cutting out everything inbetween network wise...

I haven´t tried this option actually and I do have a PoE+ adapter that I could use to power the camera and I could also try with powering the camera completely separately without a PoE+ adapter.
However, I would have to wait a month or so before doing this kind of test since outdoor temp is around -5 to -10 degrees here in Sweden where I live. A little bit too cold to start climbing a ladder to connect separate power to my cameras for testing...
And since I have run IPERF/JPERF testing with a laptop connected at the cable end where the camera is and the results from those tests are completely fine with no packet losses at all and network bitrates up to 300-500Mbit/s I am thinking that my network connection between the cameras and the surveillance PC is fine.

I run the IPERF/JPERF server on my dedicated surveillance PC and I run the IPERF/JPERF client on the laptop.

I am not sure if I used proper parameters in IPERF/JPERF for the testing?

I start iperf with the following command:
iperf.exe -s -P 0 -i 2 -p 5001 -f m

and after two minutes I get the following end results (I have omitted the log printouts from iperf on every 2 seconds):
[ ID] Interval Transfer Bandwidth
[320] 0.0-120.0 sec 4642 MBytes 325 Mbits/sec
[ ID] Interval Transfer Bandwidth
[348] 0.0-120.0 sec 4999 MBytes 349 Mbits/sec
Done.

Another iperf command I have tested:
iperf.exe -s -u -P 0 -i 2 -p 5001 -w 64.0K -f m
(which should set iperf to test in UDP mode?)

and after two minutes I get the following end results (I have omitted the log printouts from iperf on every 2 seconds):
[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams
[140] 0.0-120.0 sec 735 MBytes 51.3 Mbits/sec 0.465 ms 0/523964 (0%)

What command line parameters should I use with iperf to simulate a surveillance camera video stream?

If you have a spare network card that can be plug into your VMS machine and configure NIC teaming. It should solve your jittery issue. I have worked on other vms with jittery issue, this solve them as frames are received and clear within the timeout period set for the frames.

This is an interesting option! I do have an identical spare network card (Intel Gigabit CT Desktop Adapter) and I will try that out during the coming weekend.
I have never used or configured NIC teaming so I am a little bit unsure on how to set that up but I´ll do some Google re-search.
However, an initial impression is that NIC teaming is not well supported in Windows 10 (PRO) that I am using according to many articles on the web. Look at this one for example: Teaming with Intel® Advanced Network Services
The article is dated 23 Jan 2017 and states:
"Intel ANS and Teaming is currently non functional in Windows 10. Creating Intel ANS teams and VLANs on Windows® 10 is currently not supported. As a result, when created, teams and VLANs do not pass traffic. We expect that ANS will be supported on Windows 10 client in a future release."

 

[ChengXin]

Oops, I have usually used it with windows 7, windows server 2008/2012 OS which works quite well. You will need to take a risk to test it out windows 10 use another harddisk to install windows 7 to test out.

 

[CrazyFin]

Alright, after many weeks of more testing and discussions with AXIS tech support as well as cross-compiling an iPerf version suited for the AXIS P3346 and P3364 cameras I have found out the following:

1. When running iPerf in server mode ON the camera itself and NOT pulling a video stream while measuring with an iPerf client running on the surveillance PC I get approx 75MBit/s which should be considered as excellent (cameras has 100MBit Ethernet port). I tested also with UDP mode on iPerf and no jitter or lost packets on speeds up to 75MBit/s. I tested this on all my 4 cameras so we should be able to conclude that there is NO issues with my network cabling, the switch nor the surveillance PC.

2. I have tested with anything from 15 fps to 25 fps (max 20 fps for the P3346-series and max 25 fps for the P3364-series) as well as increasing the compression from default 30 up to 50 and even 60. Increasing compression has helped a little bit to reduce the jumpiness but not all. I have kept the GOV-setting (I-frame rate) at the same number as the fps so with 20 fps I had GOV = 20, i.e. one I-frame each second. I did test with GOV = 40 and GOV = 10 but no difference.

BUT this is interesting!
If I keep the CAPTURE mode of the camera at its max resolution, which is 2048x1536 @ 20fps for the P3346-VE cameras I have BUT put the VIDEO stream to for example 1280x960 then the jumpiness is GONE?!
Look at this video which has capture mode 2048x1536 @ 20fps but video stream 1280x960 @ 20fps:
Terrace.20170418_080000_1 - 20 fps - comp 30 - I-fram 20 - 1280x960 stream - 2048x1536 capture.avi

Compare that with this video where both capture and video streams are at 2048x1536 @ 20fps and you clearly see the jumpiness is back:
Terrace.20170418_090000_1 - 20 fps - comp 30 - I-fram 20 - 20480x1536 stream - 2048x1536 capture.avi

The problem now is that I am not able to test what would happen if I record/capture at 2048x1536 @ 20fps BUT view only the stream at 1280x960 @ fps since (as far as I understand) Blue Iris is not capable of having separate video and capture streams. I really think though that it wont help and this is a design flaw in the AXIS cameras causing the jumpiness in the recorded video as soon as the capture mode is at the highest available resolution and fps. (This is NOT a Blue Iris problem since I have this problem no matter what surveillance software I test with.)

I think I´ll revisit Luxriot and their new free so called Luxriot EVO Complimentary version which is a free version and see if I can have a recording stream separated from the viewing video stream. I think Milestone has this option so I might test Milestone (again) as well.

 

[Kawboy12R]

Try latest firmware, reset to default settings, and don't run ACC scripts in the camera. I had issues like yours on a 3364VE a few years ago and I think resolved it by turning a script/feature off in camera. I had done a number of things including updates/tinkering but IIRC it was purging an ACC "feature" that did it. Might've been edge storage on motion?

 

[mobbarley]

The camera is Artpec-3 and struggles with large (high res) I-frames. From memory using a client with a larger RTP blocksize helps but it is hard to find a VMS that implements it.

 

[mobbarley]

From the VAPIX video streaming API document:

If using unicast in combination with TCP, it is recommended to increase the size of the RTP packets to 64 000 bytes (from the standard 1500 bytes), provided that the client can accept larger packets. Also for unicast streaming over RTP/UDP it might be bene cial to increase the packet size if no packets are dropped. The packet size is changed using the following header field in the SETUP request:

Blocksize - Request a specific media packet size. The packet size should be a positive decimal number measured in octets.

 

[CrazyFin]

Try latest firmware, reset to default settings, and don't run ACC scripts in the camera. I had issues like yours on a 3364VE a few years ago and I think resolved it by turning a script/feature off in camera. I had done a number of things including updates/tinkering but IIRC it was purging an ACC "feature" that did it. Might've been edge storage on motion?

Firmware is the latest on all cameras. I confimed it by re-checking AXIS website.
With ACC scripts, do you mean running own scripts that is available under the Advanced menu of the AXIS P3364-VE and P3346-VE?
I have no add-on programs suchs as motion detection etc running on the cameras. All motion detection is done by my VMS software BlueIris.

The camera is Artpec-3 and struggles with large (high res) I-frames. From memory using a client with a larger RTP blocksize helps but it is hard to find a VMS that implements it.[

From the VAPIX video streaming API document:

If using unicast in combination with TCP, it is recommended to increase the size of the RTP packets to 64 000 bytes (from the standard 1500 bytes), provided that the client can accept larger packets. Also for unicast streaming over RTP/UDP it might be beneficial to increase the packet size if no packets are dropped. The packet size is changed using the following header field in the SETUP request:

Blocksize - Request a specific media packet size. The packet size should be a positive decimal number measured in octets.

That sounds really interesting to try out!
Yepp, the P3346-VE cameras are Artpec-3 which I learned when trying to cross compile iPerf 2 to be able to run iPerf on the cameras directly. With iPerf 2 running in client mode on the camera itself and iPerf 2 in server mode on the dedicated PC where BlueIris is running I could test network speed from the cameras and the cables through the switch (HPE OfficeConnect Switch 1820 24G PoE+ (185W),J9983A) and finally to my dedicated PC. Test results clearly displayed no issues at all and I was getting around 70-75Mbit/sec throughput without packet losses at all. (BlueIris was of course turned off during these tests so that I had no video streams being pulled from the cameras.)

I cant see in BlueIris that I can change the RTP blocksize anywhere in the camera setup page:


You wrote that the RTP packet size could be changed in the header field in the setup request but I think that you missed to include how the setup header should look like?

 

[mobbarley]

Can't really help you there - you might want to do a packet capture to see if a blocksize parameter is being requested, or get in touch with BI support to see if they implement the feature.

 

[mobbarley]

It might be interesting to see if a recent version of Companion or Station send this parameter - and if it even fixes the problem.

 

[CrazyFin]

Can't really help you there - you might want to do a packet capture to see if a blocksize parameter is being requested, or get in touch with BI support to see if they implement the feature.

Yepp, checking with Ken at BlueIris now.
I´ll try to do a packet capture with Wireshark and see if I can see the blocksize parameter being sent.

It might be interesting to see if a recent version of Companion or Station send this parameter - and if it even fixes the problem.

I did a new test 3-4 weeks ago with both Axis Station and Companion but they are experiencing same issues as I have with BlueIris (and other VMS software packages). I have not seen the possibility to change BlockSize parameter in any of those software packages.

I´ll start with Ken @ BlueIris first and see if he can confirm or show me how to change the BlockSize of the video stream using BlueIris.

How about using ffmpeg and trying to capture a direct stream with ffmpeg? Can the blocksize be changed with ffmpeg in someway?
I previously have found these interesting threads about capturing H.264 streams with ffmpeg:
How to dump raw RTSP stream to file?
ZoneMinder Wiki - Wiki - How to stream h264 with ffmpeg from an Axis P3343
RTSP SETUP - The VideoLAN Forums
I couldn't find any info about how to change the blocksize though... :-(

Anyway, I thiiiiink that since the normal camera stream link in BlueIris for a H.264 RTSP stream is for example:
rtsp://192.168.xx.xx:yyyy/axis-media/media.amp?videocodec=h264
I decided to test with
rtsp://192.168.xx.xx:yyyy/axis-media/media.amp?videocodec=h264&blocksize=175000 (64 000 as blocksize in decimal is 175000 in octal and it was in octal format that the blocksize request should be sent in?)
Video stream seems to open up properly in BlueIris but it is evening and dark light here in Sweden now so I need to wait until tomorrow before I can see the results of this change.