Now i just figured out that 2 of my nodes are down from prometheus. In the GUI it tells me its out of bounds.
Systemctrl status prometheus gives me:
Jul 27 19:33:02 prometheus prometheus[910]: level=warn ts=2020-07-27T16:33:02.690Z caller=scrape.go:1170 component=“scrape manager” scrape_pool=storj-bay3 target=http://192.168.1.121:9651/metrics msg="Error on ingesting samples that are>
Jul 27 19:33:02 prometheus prometheus[910]: level=warn ts=2020-07-27T16:33:02.691Z caller=scrape.go:930 component=“scrape manager” scrape_pool=storj-bay3 target=http://192.168.1.121:9651/metrics msg=“append failed” err=“out of bounds”
Jul 27 19:33:02 prometheus prometheus[910]: level=warn ts=2020-07-27T16:33:02.691Z caller=scrape.go:934 component=“scrape manager” scrape_pool=storj-bay3 target=http://192.168.1.121:9651/metrics msg=“append failed” err=“out of bounds”
Jul 27 19:33:02 prometheus prometheus[910]: level=warn ts=2020-07-27T16:33:02.691Z caller=scrape.go:945 component=“scrape manager” scrape_pool=storj-bay3 target=http://192.168.1.121:9651/metrics msg=“appending scrape report failed” err=>
Jul 27 19:33:11 prometheus prometheus[910]: level=warn ts=2020-07-27T16:33:11.554Z caller=scrape.go:1170 component=“scrape manager” scrape_pool=storj-pi4 target=http://192.168.1.120:9651/metrics msg=“Error on ingesting samples that are >
Jul 27 19:33:11 prometheus prometheus[910]: level=warn ts=2020-07-27T16:33:11.554Z caller=scrape.go:945 component=“scrape manager” scrape_pool=storj-pi4 target=http://192.168.1.120:9651/metrics msg=“appending scrape report failed” err=”>
Jul 27 19:33:12 prometheus prometheus[910]: level=warn ts=2020-07-27T16:33:12.178Z caller=scrape.go:1170 component=“scrape manager” scrape_pool=storj-bay6 target=http://192.168.1.122:9651/metrics msg="Error on ingesting samples that are>
Jul 27 19:33:12 prometheus prometheus[910]: level=warn ts=2020-07-27T16:33:12.179Z caller=scrape.go:930 component=“scrape manager” scrape_pool=storj-bay6 target=http://192.168.1.122:9651/metrics msg=“append failed” err=“out of bounds”
Jul 27 19:33:12 prometheus prometheus[910]: level=warn ts=2020-07-27T16:33:12.179Z caller=scrape.go:934 component=“scrape manager” scrape_pool=storj-bay6 target=http://192.168.1.122:9651/metrics msg=“append failed” err=“out of bounds”
Jul 27 19:33:12 prometheus prometheus[910]: level=warn ts=2020-07-27T16:33:12.179Z caller=scrape.go:945 component=“scrape manager” scrape_pool=storj-bay6 target=http://192.168.1.122:9651/metrics msg=“appending scrape report failed” err=>
~