BI5 logging - is FTP uploaded logged?

kc8tmv

Young grasshopper
Joined
May 27, 2017
Messages
32
Reaction score
5
Forgive if this is covered somewhere, but could not find the answer on my own.

Now that Weather Underground “seems” to have fixed some of its webcam ftp issues, I was trying to get images to ftp up. For troubleshooting I was hoping I could get some details in the log to help out.

-Steve


Sent from my iPhone using Tapatalk
 

TheDank

Pulling my weight
Joined
Jan 5, 2017
Messages
165
Reaction score
234
Location
SF Bay Area, California
Hey Steve, have you been able to get your images to ftp up to wunderground successfully yet? If not, here's a tip you might try. I saw this on one of the wunderground forums somewhere.

If you try to ftp to webcam.wunderground.com, they have a pool of servers that that name will resolve to. Most of them are broken, which isn't surprising with most of Weather Underground's stuff. But one of the servers, 52.36.136.128, seems to work somewhat reliably. When I set up my weather cam a few months ago, I could never get it to work at all, until I saw that suggestion. I configured BI to ftp directly to that IP address and it worked and has worked ever since.

So if you're having trouble, try that specific IP address instead of the hostname.

Oh, and for troubleshooting, you've tried manually ftp'ing from the command line, right? That's the first thing to try, of course.
 

TheDank

Pulling my weight
Joined
Jan 5, 2017
Messages
165
Reaction score
234
Location
SF Bay Area, California
I saw errors logged, sometimes, when I bothered with a weather cam. I gave up on it since WU is such a mess now.
You should give it a try again, @sebastiantombs, it seems to be pretty stable now, at least the one server I've been using is. I haven't tried switching my config to ftp to the named server, but at least ftp'ing to that specific IP address has been working. (Disclaimer: using an explicit IP address isn't a "best practice", but hey it works for now :cool: )

And big agreements here, WU has been a big hot mess for quite a while! It makes me sad, since it had such potential to be so awesome.
 
Joined
Dec 28, 2019
Messages
2,475
Reaction score
4,442
Location
New Jersey
I hung on as long as I could, but WU is such a mess overall it doesn't deserve the effort to keep a webcam going. I'm stuck with them for reporting conditions, AccuRite station, so the data keeps flowing with the constant interruptions on their end. If WU is an indicator of the quality that IBM has evolved to they will be out of business before too long.
 

kc8tmv

Young grasshopper
Joined
May 27, 2017
Messages
32
Reaction score
5
So, I deleted my old Webcam device from WU (which never worked and could never get logged in via Filezilla).
I created a new one and now it seems I can at least get connected to WU via Filezilla, BUT, I am not seeing what I would expect to see.
I uploaded a jpg there (/ "root") and it said it transferred, but I don't see it, even after a refresh of the directory. All I see is .ftpquota with zero file size. (I will attach screenshot)
If I try to transfer an image that is very large it keeps "TRYING" to upload but fails over and over and over.

On the my WU site I see a test image I put out there yesterday as the webcam picture, but do not see ANY image files via filezilla. Where is this old image coming from?

Anyone have any knowledge on how things work on the WU side?
 

Attachments

Joined
Dec 28, 2019
Messages
2,475
Reaction score
4,442
Location
New Jersey
Anyone have any knowledge on how things work on the WU side?
Poorly, if at all?

Seriously, file name should be image.jpg if I remember correctly. File size needs to be 150KB of less. Use the specific IP of 52.36.136.128 to FTP your images to. If that doesn't work I don't have any other tricks, other than not using WU for a weathercam.
 

TheDank

Pulling my weight
Joined
Jan 5, 2017
Messages
165
Reaction score
234
Location
SF Bay Area, California
www.Ambientweather.net is slowing working to overtake WU.
The owner is actually one of the first people to work on the WU site 20yrs ago.
But, it only works with stations sold by www.AmbientWeather.com
They just added webcams last week.
Yah, if one of the original WU guys does Ambientweather.net now, it might be worth looking at. Also, a lot of people use a MeteoBridge device to get their PWS info uploaded, which I think can upload to the Ambient Weather network. But I think if you didn't buy a device from Ambient Weather, then you have to pay to upload or something (I could be wrong about that, I'm trying to remember something I read a while back).


Anyone have any knowledge on how things work on the WU side?
As Sebastian said, your filename has to be "image.jpg". The ftp server will see that file after a minute or two and move it away to wherever it does the processing (you won't see it there anymore). I don't think they still have the 150KB file size limit though. I don't do any downsizing or anything, I just let BI ftp my full 2MP cam image as is, and it works. My jpg file sizes are around 350KB each. As a test, I just took a snapshot from my weather cam and re-saved it with a much higher jpg quality setting to get it up to about 900KB. I manually uploaded it to WU, and WU saved and processed it into my camera stream.

Here's a theory I have about how to get your camera stream to show up. You can use your favorite ftp client like Filezilla to test the ftp process and make sure that it's working ok, as you're doing. But to get your stream to actually show up, I think you have to set up your automatic ftp from BI (or from the camera or whatever). The point is, it seems like you have to have a bunch of images consistently uploaded before it will start displaying on WU. The first time I did it, like you, I did a bunch of manual tests, and I even saw the image.jpg files disappear from the directory a few minutes after uploading them. But they didn't start displaying on WU until after a couple hours. Maybe it's because the WU image processor waits to make sure you have a consistently reliable upload? Maybe it's because it needs a certain number of images uploaded before it can start making the time lapse? Maybe it's some other dumb reason that only the WU dummies have some reason for. But whatever it is, once you have tested it manually as you have, then turn BI loose on it and let it upload an image every 60 seconds. Then take a look at your WU page after an hour or so (do a full refresh of the page).

If you're interested, here's my weather cam:

Edit: I just tried manually ftp'ing to webcam.wunderground.com rather than the explicit IP address of 52.36.136.128. It did NOT work. I was able to sign in and did a "put image.jpg" which it said was successful. But the file did NOT show up in the directory, and I don't think it was because of the server quickly moving it away. So it looks like that's still broken and you're stuck using the explicit IP address. Incidentally, the IP addresses that webcam.wunderground.com resolves to are:

> webcam.wunderground.com
Server: 192.168.26.1
Address: 192.168.26.1#53

Non-authoritative answer:
Name: webcam.wunderground.com
Address: 52.36.136.128
Name: webcam.wunderground.com
Address: 52.37.52.183
Name: webcam.wunderground.com
Address: 54.148.11.142
The one that I hit that didn't work was 54.148.11.142. So for now, I suggest explicitly using 52.36.136.128.
 
Last edited:

kc8tmv

Young grasshopper
Joined
May 27, 2017
Messages
32
Reaction score
5
Yah, if one of the original WU guys does Ambientweather.net now, it might be worth looking at. Also, a lot of people use a MeteoBridge device to get their PWS info uploaded, which I think can upload to the Ambient Weather network. But I think if you didn't buy a device from Ambient Weather, then you have to pay to upload or something (I could be wrong about that, I'm trying to remember something I read a while back).




As Sebastian said, your filename has to be "image.jpg". The ftp server will see that file after a minute or two and move it away to wherever it does the processing (you won't see it there anymore). I don't think they still have the 150KB file size limit though. I don't do any downsizing or anything, I just let BI ftp my full 2MP cam image as is, and it works. My jpg file sizes are around 350KB each. As a test, I just took a snapshot from my weather cam and re-saved it with a much higher jpg quality setting to get it up to about 900KB. I manually uploaded it to WU, and WU saved and processed it into my camera stream.

Here's a theory I have about how to get your camera stream to show up. You can use your favorite ftp client like Filezilla to test the ftp process and make sure that it's working ok, as you're doing. But to get your stream to actually show up, I think you have to set up your automatic ftp from BI (or from the camera or whatever). The point is, it seems like you have to have a bunch of images consistently uploaded before it will start displaying on WU. The first time I did it, like you, I did a bunch of manual tests, and I even saw the image.jpg files disappear from the directory a few minutes after uploading them. But they didn't start displaying on WU until after a couple hours. Maybe it's because the WU image processor waits to make sure you have a consistently reliable upload? Maybe it's because it needs a certain number of images uploaded before it can start making the time lapse? Maybe it's some other dumb reason that only the WU dummies have some reason for. But whatever it is, once you have tested it manually as you have, then turn BI loose on it and let it upload an image every 60 seconds. Then take a look at your WU page after an hour or so (do a full refresh of the page).

If you're interested, here's my weather cam:

Edit: I just tried manually ftp'ing to webcam.wunderground.com rather than the explicit IP address of 52.36.136.128. It did NOT work. I was able to sign in and did a "put image.jpg" which it said was successful. But the file did NOT show up in the directory, and I don't think it was because of the server quickly moving it away. So it looks like that's still broken and you're stuck using the explicit IP address. Incidentally, the IP addresses that webcam.wunderground.com resolves to are:

> webcam.wunderground.com
Server: 192.168.26.1
Address: 192.168.26.1#53

Non-authoritative answer:
Name: webcam.wunderground.com
Address: 52.36.136.128
Name: webcam.wunderground.com
Address: 52.37.52.183
Name: webcam.wunderground.com
Address: 54.148.11.142



The one that I hit that didn't work was 54.148.11.142. So for now, I suggest explicitly using 52.36.136.128.
What does your BI Camera POST page look like? (if you don't mind sharing) and also the "Archive" page.
After the first couple of successful image uploads, I am seeing a "Post: FTP: 552 Quota exceeded: ..." message in my BI logs.
 

kc8tmv

Young grasshopper
Joined
May 27, 2017
Messages
32
Reaction score
5
Quota exceeded means the image sizes are too large. WU keeps 24 hours worth, I think, and limits the total size based on that 24 hour period. Cut your image sizes down, definitely under 300KB or even lower.
You are getting me in the right direction! What resolution are you using to save the image? You seem to have a different aspect ratio then mine and I like the looks of yours better. Mine is: Webcam Dashboard | Weather Underground
 
Joined
Dec 28, 2019
Messages
2,475
Reaction score
4,442
Location
New Jersey
I'm not sending any images to WU anymore, but when I did I kept the file size just a hair under 200K. You can vary that on the snapshot configuration and check the file size, manually, if you start out by simply saving the snapshots to your hard drive and check the file size until you get it set right. That's how I fine tuned mine when I was uploading.
 
Last edited:

looney2ns

IPCT Contributor
Joined
Sep 25, 2016
Messages
10,145
Reaction score
9,694
Location
Evansville, In. USA
Yah, if one of the original WU guys does Ambientweather.net now, it might be worth looking at. Also, a lot of people use a MeteoBridge device to get their PWS info uploaded, which I think can upload to the Ambient Weather network. But I think if you didn't buy a device from Ambient Weather, then you have to pay to upload or something (I could be wrong about that, I'm trying to remember something I read a while back).




As Sebastian said, your filename has to be "image.jpg". The ftp server will see that file after a minute or two and move it away to wherever it does the processing (you won't see it there anymore). I don't think they still have the 150KB file size limit though. I don't do any downsizing or anything, I just let BI ftp my full 2MP cam image as is, and it works. My jpg file sizes are around 350KB each. As a test, I just took a snapshot from my weather cam and re-saved it with a much higher jpg quality setting to get it up to about 900KB. I manually uploaded it to WU, and WU saved and processed it into my camera stream.

Here's a theory I have about how to get your camera stream to show up. You can use your favorite ftp client like Filezilla to test the ftp process and make sure that it's working ok, as you're doing. But to get your stream to actually show up, I think you have to set up your automatic ftp from BI (or from the camera or whatever). The point is, it seems like you have to have a bunch of images consistently uploaded before it will start displaying on WU. The first time I did it, like you, I did a bunch of manual tests, and I even saw the image.jpg files disappear from the directory a few minutes after uploading them. But they didn't start displaying on WU until after a couple hours. Maybe it's because the WU image processor waits to make sure you have a consistently reliable upload? Maybe it's because it needs a certain number of images uploaded before it can start making the time lapse? Maybe it's some other dumb reason that only the WU dummies have some reason for. But whatever it is, once you have tested it manually as you have, then turn BI loose on it and let it upload an image every 60 seconds. Then take a look at your WU page after an hour or so (do a full refresh of the page).

If you're interested, here's my weather cam:

Edit: I just tried manually ftp'ing to webcam.wunderground.com rather than the explicit IP address of 52.36.136.128. It did NOT work. I was able to sign in and did a "put image.jpg" which it said was successful. But the file did NOT show up in the directory, and I don't think it was because of the server quickly moving it away. So it looks like that's still broken and you're stuck using the explicit IP address. Incidentally, the IP addresses that webcam.wunderground.com resolves to are:

> webcam.wunderground.com
Server: 192.168.26.1
Address: 192.168.26.1#53

Non-authoritative answer:
Name: webcam.wunderground.com
Address: 52.36.136.128
Name: webcam.wunderground.com
Address: 52.37.52.183
Name: webcam.wunderground.com
Address: 54.148.11.142



The one that I hit that didn't work was 54.148.11.142. So for now, I suggest explicitly using 52.36.136.128.
You have to purchase an Ambient.net license for the metobridge to be able to upload to them. No on going charges.
The new stations come with the ability to upload to WU, Ambient, CWOP, etc. directly from the stations console.
Wow, and your timelapse actually works correctly! Last I tried a couple of months ago it was a hot mess all around.
Lots more info at www.wxforum.net
 

TheDank

Pulling my weight
Joined
Jan 5, 2017
Messages
165
Reaction score
234
Location
SF Bay Area, California
What does your BI Camera POST page look like? (if you don't mind sharing) and also the "Archive" page.
After the first couple of successful image uploads, I am seeing a "Post: FTP: 552 Quota exceeded: ..." message in my BI logs.
You are getting me in the right direction! What resolution are you using to save the image? You seem to have a different aspect ratio then mine and I like the looks of yours better. Mine is: Webcam Dashboard | Weather Underground
I'm not sending any images to WU anymore, but when I did I kept the file size just a hair under 200K. You can vary that on the snapshot configuration and check the file size, manually, if you start out by simply saving the snapshots to your hard drive and check the file size until you get it set right. That's how I fine tuned mine when I was uploading.
Hey Steve, that's great it looks like your weather cam is starting to work. I'm attaching my FTP Servers config, my camera's Post config, and my Clips/Archiving config (though I don't think that one plays a part in all this).

I think Sebastian is right about file sizes, even though larger sizes can work, it can exceed your quota. As I mentioned, my file sizes are in the 350KB range, but it sure doesn't hurt to go smaller. You can control this either by lowering your resolution (the "Size:" pull-down on the Post config), or by lowering the "Quality %:" setting. As you see from my screen shot, I use the "default" size, and quality of 60%. My camera is a Dahua 2231, so the default is a 2MP 1920x1080 image. WU downsizes that for the time lapses, but it keeps the high res for the snapshots.

Note that the Filename setting should just be "image", not "image.jpg", because BI automatically appends ".jpg". Since you do have some images appearing, you must have figured this out already also.

I think people in the past have used the "Maintain a ring" setting, but I can't imagine that that would work for WU, so I leave it off and simply queue an image every 60 seconds. The time lapse that WU creates looks like it inserts a frame into the movie every 4 minutes, so me uploading every 60 seconds could be overkill, but it works. I don't know if WU literally inserts a frame every 4 minutes or whether it grabs every 4th image that you upload. I guess the result would be the same in my case. It would be interesting for me to change my frequency to every 120 seconds and see how that affects the time lapse.

The FTP Server config is pretty straightforward. For Login, use the "Camera ID" from WU, not the "Camera Name". And use the "Key" from WU for password. Since you are ftp'ing successfully and have some images appearing, you also know this, but it could be a helpful tip to others reading this in the future.

Also... I think some people in the past have used the "Post files to a folder" rather than "Post files to an FTP server", and then ftp the files from the folder or something like that. I haven't tried that myself it just seems more straightforward to post directly to the server.

clipsArchive.jpgftpServer.jpgpost.jpg
 
Top