# Nodes in the same machine with different ingress

**URL:** <https://forum.storj.io/t/nodes-in-the-same-machine-with-different-ingress/23294>\
**Category:** Node Operators\
**Tags:** multiple-nodes\
**Created:** [July 19, 2023, 12:21pm UTC](https://forum.storj.io/t/nodes-in-the-same-machine-with-different-ingress/23294 "2023-07-19T12:21:50Z")\
**Posts on this page:** 19\
**Page:** 1

<div class="post-metadata">

**Author:** ![humbfig](https://storj.bcdn.literatehosting.com/letter_avatar/humbfig/32/6_449c533f586939b6e5f246632a46421f.png) [@humbfig](https://forum.storj.io/u/humbfig)\
**Post date:** [July 19, 2023, 12:21pm UTC](https://forum.storj.io/t/nodes-in-the-same-machine-with-different-ingress/23294/1 "2023-07-19T12:21:50Z")

</div>

I have 3 nodes running in a single Synology. 2 of them share the same disk, the other in a different disk. Don’t ask why, it’s about legacy.  
Ever since all nodes are vetted, they get the same ingress.  
Since ~1 week ago, one of the nodes (the oldest one) is getting almost 3 times the ingress of each of the other 2.  
Also, the ingress of a single node in another location used to correlate with these 3 nodes. It used to be the same as the 3 nodes combined. It used to make sense. Now it doesn’t correlate in anyway. And both sites have no storj neighbours.  
Can someone explain this?

---

<div class="post-metadata">

**Author:** ![michaln](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/michaln/32/62_2.png) [@michaln](https://forum.storj.io/u/michaln)\
**Post date:** [July 19, 2023, 2:05pm UTC](https://forum.storj.io/t/nodes-in-the-same-machine-with-different-ingress/23294/2 "2023-07-19T14:05:15Z")

</div>

hey @humbfig,  
could you provide some details about your nodes, like:

- ids of affected nodes
- info if ingress is different for all satellites or maybe one specific
- any additional details you think is important

---

<div class="post-metadata">

**Author:** ![michaln](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/michaln/32/62_2.png) [@michaln](https://forum.storj.io/u/michaln)\
**Post date:** [July 19, 2023, 2:24pm UTC](https://forum.storj.io/t/nodes-in-the-same-machine-with-different-ingress/23294/3 "2023-07-19T14:24:52Z")

</div>

Node IDs and other sensitive data can be send by DM or by creating support ticket. Thanks in advance.

---

<div class="post-metadata">

**Author:** ![humbfig](https://storj.bcdn.literatehosting.com/letter_avatar/humbfig/32/6_449c533f586939b6e5f246632a46421f.png) [@humbfig](https://forum.storj.io/u/humbfig)\
**Post date:** [July 19, 2023, 3:33pm UTC](https://forum.storj.io/t/nodes-in-the-same-machine-with-different-ingress/23294/4 "2023-07-19T15:33:24Z")

</div>

Here is a fact-table. This is about ingress only. Apparently, things started to not make sense on the 15th of July and only for the US1 satellite.

Expected behaviour:  
i) Node 4 = Node 1 + Node 2 + Node 3  
ii) Node 1 = Node 2 = Node 3

The “expected” happened since all 4 nodes were vetted.

Apparently, what doesn’t make sense is nodes 2 and 3 getting little download, because if nodes 2 and 3 were getting the same download as node 1 (as they should), then Expected behaviour(i) would be right.  
PS- One could also notice that on EU1, the ingress on the node1 is somewhat higher than in the rest of the “parallel” nodes (2 and 3) since the 15th. Being the only one running tha v1.81.3, it makes you wonder…

Other info: Node 1 is the only one running v1.81.3. All other nodes are running v1.82.1. I can’t say when each node was upgraded.

Node ids will be DMd. I didn’t know that was sensitive information…

 ![Screenshot 2023-07-19 at 16.17.55](https://storj-s3.bcdn.literatehosting.net/original/2X/1/1ceacb91cd64b51b4d26eeb110a88c4244399398.png)

---

<div class="post-metadata">

**Author:** ![digitalfrank](https://storj.bcdn.literatehosting.com/letter_avatar/digitalfrank/32/6_449c533f586939b6e5f246632a46421f.png) [@digitalfrank](https://forum.storj.io/u/digitalfrank)\
**Post date:** [July 19, 2023, 6:08pm UTC](https://forum.storj.io/t/nodes-in-the-same-machine-with-different-ingress/23294/5 "2023-07-19T18:08:52Z")

</div>

also i have this issue. On 22 nodes only the nodes with 1.81.3 have identical ingress, others on 1.82.1 is more down

---

<div class="post-metadata">

**Author:** ![Ambifacient](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/ambifacient/32/7815_2.png) [@Ambifacient](https://forum.storj.io/u/Ambifacient)\
**Post date:** [July 19, 2023, 6:33pm UTC](https://forum.storj.io/t/nodes-in-the-same-machine-with-different-ingress/23294/6 "2023-07-19T18:33:20Z")

</div>

1.82.1 fixed an ingress reporting issue. [storagenode/piecestore: fix ingress graph skewed by larger signed orders · storj/storj@b6026b9 · GitHub](https://github.com/storj/storj/commit/b6026b9)

---

<div class="post-metadata">

**Author:** ![humbfig](https://storj.bcdn.literatehosting.com/letter_avatar/humbfig/32/6_449c533f586939b6e5f246632a46421f.png) [@humbfig](https://forum.storj.io/u/humbfig)\
**Post date:** [July 19, 2023, 7:14pm UTC](https://forum.storj.io/t/nodes-in-the-same-machine-with-different-ingress/23294/7 "2023-07-19T19:14:28Z")

</div>

That would settle it…  
I just have to wait until node1 upgrades to v1.82.1 for the world to start making sense again…

PS- Actually not. Node4 is on v.1.82.1, so, that wouldn’t explain Expected behaviour(i)… if node4 was upgraded to v1.82.1 before the 18th, which I think it was…  
If there is a problem with 1.81.3 reporting higher ingress than it should, then Expected Behaviour should be:  
i) node 2 = node3 (check!)  
ii) node4 = 3 X node2 (or node3) (No check!)

---

<div class="post-metadata">

**Author:** ![snorkel](https://storj.bcdn.literatehosting.com/letter_avatar/snorkel/32/6_449c533f586939b6e5f246632a46421f.png) [@snorkel](https://forum.storj.io/u/snorkel)\
**Post date:** [July 19, 2023, 7:45pm UTC](https://forum.storj.io/t/nodes-in-the-same-machine-with-different-ingress/23294/8 "2023-07-19T19:45:28Z")

</div>

I imagine node 2 and 3 are on the same disk. Is the disk OK? No baddies, no errors etc? Are the db-es OK?

---

<div class="post-metadata">

**Author:** ![Toyoo](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/toyoo/32/1126_2.png) [@Toyoo](https://forum.storj.io/u/Toyoo)\
**Post date:** [July 19, 2023, 7:55pm UTC](https://forum.storj.io/t/nodes-in-the-same-machine-with-different-ingress/23294/9 "2023-07-19T19:55:38Z")

</div>

I also wonder if there might be any performance differences between your nodes.

You can try comparing just the number of upload attempts—satellites consider each piece as equal for node selection, regardless of the piece size. Looking at just the number of times your node was selected as an upload targets reduces variance of the comparison, but most importantly here disables the impact of failed uploads.

You can either parse your node logs for the number of upload attempts, or, if you have debugging enabled, look at the upload\_started\_count counter (side note: it would be nice if this counter was per satellite, and even better if it was a rate meter!).

If these numbers will be equal, then we’ll know the difference comes from failed/canceled uploads.

---

<div class="post-metadata">

**Author:** ![humbfig](https://storj.bcdn.literatehosting.com/letter_avatar/humbfig/32/6_449c533f586939b6e5f246632a46421f.png) [@humbfig](https://forum.storj.io/u/humbfig)\
**Post date:** [July 19, 2023, 7:58pm UTC](https://forum.storj.io/t/nodes-in-the-same-machine-with-different-ingress/23294/10 "2023-07-19T19:58:29Z")

</div>

> [@snorkel](#):
>
> I imagine node 2 and 3 are on the same disk. Is the disk OK? No baddies, no errors etc? Are the db-es OK?

That would explain a lot. But no. Nodes 1 and 2 are on the same disk. Node3 is alone in one disk that serves no other purpose. Not a single bad sector in any of the disks, ever.

---

<div class="post-metadata">

**Author:** ![humbfig](https://storj.bcdn.literatehosting.com/letter_avatar/humbfig/32/6_449c533f586939b6e5f246632a46421f.png) [@humbfig](https://forum.storj.io/u/humbfig)\
**Post date:** [July 19, 2023, 8:10pm UTC](https://forum.storj.io/t/nodes-in-the-same-machine-with-different-ingress/23294/11 "2023-07-19T20:10:40Z")

</div>

Well, you tell me. I see no performance difference. And anyway, before the July 15th, the world made sense…

node1

 ![Screenshot 2023-07-19 at 21.05.03](https://storj-s3.bcdn.literatehosting.net/original/2X/4/45a4e39fabe4169648c056d2bc8029c12fcd718c.png)

node2

 ![node3](https://storj-s3.bcdn.literatehosting.net/original/2X/7/7f98cc952a84c56bf8584416113f585be68d8e2d.png)

node3

 ![Screenshot 2023-07-19 at 21.06.57](https://storj-s3.bcdn.literatehosting.net/original/2X/3/3497d7d44a87059de111346cc156f333273198a7.png)

node4

 ![node4](https://storj-s3.bcdn.literatehosting.net/original/2X/9/9ca0118024913ab23c21ee4ed3252ba71291d991.png)

Also, some of the numbers you point out, I don’t know how to get them. Debug is not enabled.

---

<div class="post-metadata">

**Author:** ![Toyoo](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/toyoo/32/1126_2.png) [@Toyoo](https://forum.storj.io/u/Toyoo)\
**Post date:** [July 19, 2023, 10:20pm UTC](https://forum.storj.io/t/nodes-in-the-same-machine-with-different-ingress/23294/12 "2023-07-19T22:20:56Z")

</div>

Indeed, this looks a bit weird. Can’t help more than that, sorry, but this probably should be useful to Storj engineers.

---

<div class="post-metadata">

**Author:** ![Alexey](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/alexey/32/41_2.png) [@Alexey](https://forum.storj.io/u/Alexey)\
**Post date:** [July 20, 2023, 5:07am UTC](https://forum.storj.io/t/nodes-in-the-same-machine-with-different-ingress/23294/13 "2023-07-20T05:07:22Z")

</div>

> [@humbfig](#):
>
> Apparently, what doesn’t make sense is nodes 2 and 3 getting little download, because if nodes 2 and 3 were getting the same download as node 1 (as they should)

do you mean upload to the nodes (ingress) or download from the nodes (egress)?  
We always use the customer’s point of view, so need clarification.

---

<div class="post-metadata">

**Author:** ![snorkel](https://storj.bcdn.literatehosting.com/letter_avatar/snorkel/32/6_449c533f586939b6e5f246632a46421f.png) [@snorkel](https://forum.storj.io/u/snorkel)\
**Post date:** [July 20, 2023, 7:33am UTC](https://forum.storj.io/t/nodes-in-the-same-machine-with-different-ingress/23294/14 "2023-07-20T07:33:07Z")

</div>

From the first post I get that he reffers to ingress.

---

<div class="post-metadata">

**Author:** ![Alexey](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/alexey/32/41_2.png) [@Alexey](https://forum.storj.io/u/Alexey)\
**Post date:** [July 20, 2023, 7:55am UTC](https://forum.storj.io/t/nodes-in-the-same-machine-with-different-ingress/23294/15 "2023-07-20T07:55:00Z")

</div>

I hope so. Because downloads = egress and egress can vary.

---

<div class="post-metadata">

**Author:** ![snorkel](https://storj.bcdn.literatehosting.com/letter_avatar/snorkel/32/6_449c533f586939b6e5f246632a46421f.png) [@snorkel](https://forum.storj.io/u/snorkel)\
**Post date:** [July 20, 2023, 8:19am UTC](https://forum.storj.io/t/nodes-in-the-same-machine-with-different-ingress/23294/16 "2023-07-20T08:19:28Z")

</div>

I’ll ask the obvious question: are any of them full?  
Guessing that the answear is NO, I’ll stop all of them, rm them, restart the machines, update all the packages and DSM, manual update the storagenode image on all, start the nodes, just to get them on the same version, and the software up-to-date. The last dsm and docker are working perfectly with storj, so no worries.

---

<div class="post-metadata">

**Author:** ![humbfig](https://storj.bcdn.literatehosting.com/letter_avatar/humbfig/32/6_449c533f586939b6e5f246632a46421f.png) [@humbfig](https://forum.storj.io/u/humbfig)\
**Post date:** [July 20, 2023, 9:13am UTC](https://forum.storj.io/t/nodes-in-the-same-machine-with-different-ingress/23294/17 "2023-07-20T09:13:09Z")

</div>

I mean ingress!  
This POV thing is very confusing…

---

<div class="post-metadata">

**Author:** ![humbfig](https://storj.bcdn.literatehosting.com/letter_avatar/humbfig/32/6_449c533f586939b6e5f246632a46421f.png) [@humbfig](https://forum.storj.io/u/humbfig)\
**Post date:** [July 20, 2023, 9:20am UTC](https://forum.storj.io/t/nodes-in-the-same-machine-with-different-ingress/23294/18 "2023-07-20T09:20:09Z")

</div>

None of the nodes are full. DSM is the latest version. Packages, including docker, are all updated. Node1 updated to v1.82.1 last night. So, all nodes on the same version.

PS- I’ll be offline doing IRL stuff until Monday or Tuesday. I’ll post my new ingress statistics when I get back.

---

<div class="post-metadata">

**Author:** ![humbfig](https://storj.bcdn.literatehosting.com/letter_avatar/humbfig/32/6_449c533f586939b6e5f246632a46421f.png) [@humbfig](https://forum.storj.io/u/humbfig)\
**Post date:** [July 27, 2023, 2:33pm UTC](https://forum.storj.io/t/nodes-in-the-same-machine-with-different-ingress/23294/19 "2023-07-27T14:33:24Z")

</div>

Hi!  
Just to say that the world makes sense again.  
In the meanwhile all nodes were the same version (now, not anymore) and numbers started to add up.  
Thank you all for your help.
