I reccived several emails today morning that nodes are offline only on US1 sattelite. looks strange, nodes show 0m last contact.
Received one too for 1 node only. Thought it could be related to 1.121.2 update so rebooted and got ānode back onlineā email soon after. Not sure what the problem was.
3 Likes
2 Likes
Yes may be but I steel getting this emails for different nodes.
1 Like
Are all those nodes on v1.121.2 ?
I had a few nodes that saw ~500kbps ingress starting about the time where the US1 incident is reported. Restarted the node and returned to normal. Some other nodes were fine, all were on 1.120.
1 Like
I got an email for 2 nodes. All other nodes are fine. The 2 nodes arenāt even related to another, because they are in 2 completely different locations
But online score doesnāt seem to be dropping
1 Like
Same experience. Received a US1 offline notification for one of my nodes. Looking at the logs, the node was not receiving any upload/download requests from US1. Remaining satellites working correctly. A restart of the node resolved the issue.
This seems to have affected close to 10% of the network so far:
5 Likes
Iām piling on here. I have the same. I have two nodes running on one server behind the same IP.
Store01 gave me an alert, and the traffic was dribbling in. Running ādocker logs -f store01ā showed very few āputā updates, whereas store02 had them flying in.
A simple restart later, and both nodes are flying again.
Out of interest, is there a way to stop the auto-update in docker? More than once, Iāve had the dreaded forced update cause issues. Iād much rather schedule an update script to run once a week than the system updating when it feels like it and causing downtime because Iām not around to fix it!
EDIT: I just read the incident linked above (Storj DCS Status - US1: Elevated Errors and Degraded Performance). Itās not solved. It says it was resolved Feb 9th at 22.57 UTC. My node alerted as offline on the 10th Feb 05:30 UTC and wasnāt resolved until I restarted the node.
I made restart to nodes that i got email and got online conformation email from US1. So for some reason US1 decided that some nodes are offline. They was on different servers even different places. But lot of other nodes on same servers and places was not effected.
1 Like
restarted just now - and got the āonlineā notification from Storjā¦ - Frustrating this will cause from ādowntimeā from to the SNO fault.
2 Likes
still receiveing ānode offlineā emails from US1. node restart brings them online. 10+% of node fleet affected. different locations , different isp`s ,all nodes v1.121.2
And keep in mind that emails come after 4h of downtime. I didnāt receive any.
1 Like
Same here, node is online but didnāt receive back online email 
Good to know seems offline on US1, no traffic todayš«£
Did the restart hope it will get reconnectedā¦
1 Like
None of my four potatoes received emails, same IP, normal traffic on US1
1 Like
only 6 or 7 of my nodes from 108 got mail, so statistically you maybe just passed.
Same here, got the offline notice at 6:30am (UTC) and got no traffic from us1 all day until just now after I restarted the node. Other satellites were fine.
1 Like
My nodes self recovered through no intervention and I received both sets of emails for each.
Finally received the online Mail as well 
1 Like