Since my node updated to 1.1.1 on Wednesday, the dashboard shows that there’s been 0 bytes of transfer and the satellite audit checks % have been steadily dropping. The dashboard still shows the node status as online and last contact as 0 minutes ago. Nothing else about the environment has changed since.
Is this just some odd coincidence, or otherwise what should I be looking at to diagnose the issue? The node runs on Linux in a docker container.
If your audit checks are dropping, you should investigate very closely! Please send the latest log lines, copy your logfile and then I’d aadvise to take your node offline until the problem is fixed. There is currently no downtime punishment, so that’s not an issue. But if your node is failing audits, it could be disqualified very soon.
I’ve stopped the node for now. The last 50 lines of the log are below
2020-04-12T17:47:06.220Z INFO piecestore download started {"Piece ID": "EAQEHQZWJFNOOBPXLR7GTZTSDTARZR4LBB5N4HB33RE3I6QXCJ2A", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET"}
2020-04-12T17:47:08.150Z INFO piecestore download started {"Piece ID": "VJG4VL3IVRIQSFDR3VKG3H4SSA4GOQSDKCMSZQIKBPPZMFUUX4VA", "Satellite ID": "118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW", "Action": "GET"}
2020-04-12T17:47:23.555Z INFO piecestore download started {"Piece ID": "KPHX2UQY3AGRN26NKIVBJLWRVWD2HBG26755IXAEB3WS7HSLBIWQ", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET"}
2020-04-12T17:47:41.741Z INFO piecestore download started {"Piece ID": "VJG4VL3IVRIQSFDR3VKG3H4SSA4GOQSDKCMSZQIKBPPZMFUUX4VA", "Satellite ID": "118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW", "Action": "GET"}
2020-04-12T17:47:59.970Z INFO piecestore download started {"Piece ID": "KBNTZDPLFVVXXPIT7J5Z3C6VOFA37MF5Q6CDBPBXCT2KWA4AMSQQ", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET"}
2020-04-12T17:48:06.909Z INFO piecestore download started {"Piece ID": "M2KELQEIOJMITGRUH5FOZZRO62YHDDVTUSSWHXH225PD4TSVRQ2Q", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET"}
2020-04-12T17:48:06.953Z INFO piecestore download started {"Piece ID": "M2KELQEIOJMITGRUH5FOZZRO62YHDDVTUSSWHXH225PD4TSVRQ2Q", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET"}
2020-04-12T17:48:06.955Z INFO piecestore download started {"Piece ID": "M2KELQEIOJMITGRUH5FOZZRO62YHDDVTUSSWHXH225PD4TSVRQ2Q", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET"}
2020-04-12T17:48:07.916Z INFO piecestore download started {"Piece ID": "NCMDJWVMM7Y63OCQTSG5BBMVAOBCUKMI5DCC3TPF5XRCD3LYG4AA", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET"}
2020-04-12T17:48:10.168Z INFO piecestore download started {"Piece ID": "3G2PL466MGF3QZDYDNGUTF5B4UVWBTQREHO4KNHNHXCX7IL7YUAQ", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET"}
2020-04-12T17:48:13.051Z INFO piecestore download started {"Piece ID": "VJG4VL3IVRIQSFDR3VKG3H4SSA4GOQSDKCMSZQIKBPPZMFUUX4VA", "Satellite ID": "118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW", "Action": "GET"}
2020-04-12T17:48:13.441Z INFO piecestore download started {"Piece ID": "FWRCHGLW5KEHZUOK23DZ6LQOXENNMECAZOOK64COHT2CUPJZRQIQ", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET"}
2020-04-12T17:48:18.105Z INFO piecestore download started {"Piece ID": "CESBW6PP7HPDOZ46A3F3DIP3FIKBZA2PWZUK32C4G4B6MZTK74PQ", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET"}
2020-04-12T17:48:24.401Z INFO piecestore download started {"Piece ID": "REHR4ZFIHMCLMMUXLNW74DPBO3EED2ZJMRYXUVKYBG7MTK52SAIA", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET"}
2020-04-12T17:48:24.603Z INFO piecestore download started {"Piece ID": "UMZ37UVDZPIBJNTQZX2J4SIUVMOCGZGCXKDWQT6LBEK4MAU6KXZA", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET"}
2020-04-12T17:48:31.725Z INFO piecestore download started {"Piece ID": "KPAJKZIWIJWZGKBWQ6BCLJUSKM3GSJRS7EMOU3APIPBT3KKX2CGQ", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET"}
2020-04-12T17:48:33.540Z INFO piecestore download started {"Piece ID": "4L4OTUCA4U4LX3WZ56FM6WQ5QEXGMISUBDEUQ5TZZXCTAHYFPCKA", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET"}
2020-04-12T17:49:10.057Z INFO piecestore download started {"Piece ID": "JYE4OAHNHU4L7HXON7BWTYL5N4M2SNO3NQD7MZAEDYMQITEJEHCQ", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET"}
2020-04-12T17:49:34.586Z INFO piecestore download started {"Piece ID": "IIW2SNLOCG2T4LYDMRDCFEMPU76ZB53YJUBPYJODVV53BNBCKCSQ", "Satellite ID": "118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW", "Action": "GET"}
2020-04-12T17:49:43.317Z INFO piecestore download started {"Piece ID": "NJT3UFEURI7HBRK7Y3KZJAR2EZTV3STFIMV6H4ZMBNYGMGTZC4QA", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET"}
2020-04-12T17:49:52.544Z INFO piecestore download started {"Piece ID": "RA7VTADYACOBI5JS772NNOHWI6T76AJHHII3MMVJL5R42R7JSIEA", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET"}
2020-04-12T17:49:53.772Z INFO piecestore download started {"Piece ID": "WMBHWGHWPLRERYAZFPHFKZNP6JEPV6OLDNRZYAXU6ZQM7CEX6RUQ", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET"}
2020-04-12T17:50:15.307Z INFO piecestore download started {"Piece ID": "LTLFBBFFCIB45CIZWL3NB2UXAUPAYOM5QTKM3YWP2HO7U2XEPW6A", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET_AUDIT"}
2020-04-12T17:50:19.631Z INFO piecestore download started {"Piece ID": "XLGG73U56DEHRAPVCTV74IE2CC7BFZRZCVKFYRVDKZHYBOTJMSBA", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET"}
2020-04-12T17:50:19.875Z INFO piecestore download started {"Piece ID": "FWRCHGLW5KEHZUOK23DZ6LQOXENNMECAZOOK64COHT2CUPJZRQIQ", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET"}
2020-04-12T17:50:20.507Z INFO piecestore download started {"Piece ID": "W3JEQ2CUZ5RDIZRXBLGVNAC4CHCO35QAJG623ZZPC5CFIURSZW5Q", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET"}
2020-04-12T17:50:34.813Z INFO piecestore download started {"Piece ID": "OLFXK3V34XFVP4GCANIWZQN5VAJFL2UE64AK6HKUEC7OCRL3BFIA", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET"}
2020-04-12T17:50:37.240Z INFO piecestore download started {"Piece ID": "XVTR3H4GJSI2H4ADDQQYYUTKNN6QGXFSOOHY7GWGSCXEBP6U6L7A", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET"}
2020-04-12T17:50:49.619Z INFO piecestore download started {"Piece ID": "P7HPB73JNKYPGEL3M6OR5AB5XL65PQJRYVPZDBA3WHM7F6OGK42Q", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET"}
2020-04-12T17:51:46.680Z INFO piecestore download started {"Piece ID": "D3JHKB6SBZGC4LFCNBYB5AUOSK3RQ5KHLWWICGP37DT67R7CWXFA", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET"}
2020-04-12T17:51:59.910Z INFO piecestore download started {"Piece ID": "LW4AJSPUEQ26NZSKBCIQK77WUHC6V7TAFKQCOPFJ3FK7FAFDXOCQ", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET"}
2020-04-12T17:52:09.580Z INFO piecestore download started {"Piece ID": "FWRCHGLW5KEHZUOK23DZ6LQOXENNMECAZOOK64COHT2CUPJZRQIQ", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET"}
2020-04-12T17:52:13.814Z INFO piecestore download started {"Piece ID": "DPXR2BNC6IYTHN5WGYVA3RMPTO55FISWNQIPVF5HU2BYNOPJMEAA", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET"}
2020-04-12T17:52:24.377Z INFO piecestore download started {"Piece ID": "JT2NSOX6HTR5YQJTLVJJV2V7FRTQETSMYT2E463MHHCXWJB4OGSA", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET"}
2020-04-12T17:52:42.962Z INFO piecestore download started {"Piece ID": "5BU6DVQC7VJ6UPUPB55BE3GNXRSFJ6WF5HIY36KOQNTKYMU26GRA", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET"}
2020-04-12T17:53:20.113Z INFO piecestore download started {"Piece ID": "SV42TJLJKSSB2C74ZAGI4V3NQTEU4FFRPQL3J6GRB5UL63ISPLOA", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET"}
2020-04-12T17:53:35.608Z INFO piecestore download started {"Piece ID": "HT7WEXL5S2UI6ACSYC6TUHSKH6JM3VEDDLZPB7I3MNVPIFF3WSQQ", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET"}
2020-04-12T17:53:52.429Z INFO piecestore download started {"Piece ID": "LPASJ2EFLV47725LWP2AGUSX6EDLLVXZLLZS33FTWAV5QP2BVFRA", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET"}
2020-04-12T17:54:02.720Z INFO piecestore download started {"Piece ID": "IHBHQYK27F5HSAULS4MDSC2N3E4Z4LLRBTDLGNNPEO4FX2XYW2JA", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET"}
2020-04-12T17:54:08.473Z INFO piecestore download started {"Piece ID": "FWRCHGLW5KEHZUOK23DZ6LQOXENNMECAZOOK64COHT2CUPJZRQIQ", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET"}
2020-04-12T17:54:11.995Z INFO piecestore download started {"Piece ID": "W54AOMYRYVSIKTT4BTGIDN27LA37Q5BI6O6VMLGH3GTKC3GQOT7A", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET"}
2020-04-12T17:54:19.936Z INFO piecestore download started {"Piece ID": "SVE6ZTEEIXXFHNBRSSMCHPAVVNLWOKEVYGNYZO4OUHG57JSDHU6Q", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET"}
2020-04-12T17:54:22.332Z INFO piecestore download started {"Piece ID": "ACBTZMMJMXAFBME73DQK5QVW2TOH4SVIQUBGG75AX3HA5YBT2CUA", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET"}
2020-04-12T17:54:37.481Z INFO piecestore download started {"Piece ID": "SY2I5NFAR4ZB7HR3QGYZGH5MSJIKDLAQ2QXTUIYNVDXZUZ4H3Z7Q", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET"}
2020-04-12T17:54:54.323Z INFO piecestore download started {"Piece ID": "E7E7LPAULFQAYOKM5UYIB7YDXT4LGEDBULUFO2X6MEXFBTETJTLQ", "Satellite ID": "118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW", "Action": "GET_AUDIT"}
2020-04-12T17:55:08.031Z INFO piecestore download started {"Piece ID": "BE5GSSUQUZTDEFZPWWSXH47DBDU2QJULATIJGBHN7BLRYVI42MIQ", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET"}
2020-04-12T17:55:08.898Z INFO piecestore download started {"Piece ID": "Q76BBE5TNU3DZLPDZVCWEDOA3L2QZMLO3UPREHW6LP67HGZHKJKQ", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET"}
2020-04-12T17:55:22.299Z INFO piecestore download started {"Piece ID": "RK22MCFHDHF22YYOAFQ2CPSOW65DNGGOAXHYKLAT3HBGSM6YXLMQ", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET"}
2020-04-12T17:55:32.127Z INFO piecestore download started {"Piece ID": "37V3OHBO3SVGSLIEPR3PPQMIKHT2AUYIF3M3JT3EFWXOOWLGLD7A", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET"}
2020-04-12T17:55:46.998Z INFO piecestore download started {"Piece ID": "SZ3SEMURHMYHUTH27TJ4FI5FDU5HLFQ7UF7KMMD5NDQNIC5N4CGQ", "Satellite ID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "Action": "GET"}please send more lines. these are only download started and don’t help
How is your HDD connected ?
The logs since the node updated with the download/upload lines removed is below.
2020-04-08T15:26:31.357Z INFO Configuration loaded from: /app/config/config.yaml
2020-04-08T15:26:31.383Z INFO Operator email: xxx@xxx.xxx
2020-04-08T15:26:31.383Z INFO operator wallet: 0x000000
2020-04-08T15:26:32.381Z INFO version running on version v1.1.1
2020-04-08T15:26:32.500Z INFO db.migration Database Version {"version": 33}
2020-04-08T15:26:34.058Z INFO preflight:localtime start checking local system clock with trusted satellites' system clock.
2020-04-08T15:26:34.911Z INFO preflight:localtime local system clock is in sync with trusted satellites' system clock.
2020-04-08T15:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-08T15:26:34.911Z INFO trust Scheduling next refresh {"after": "4h56m3.009447399s"}
2020-04-08T15:26:34.911Z INFO Node xxxxx started
2020-04-08T15:26:34.911Z INFO Public server started on [::]:28967
2020-04-08T15:26:34.911Z INFO Private server started on 127.0.0.1:7778
2020-04-08T15:26:35.155Z INFO version running on version v1.1.1
2020-04-08T15:30:34.535Z INFO orders.118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW sending {"count": 11}
2020-04-08T15:30:34.535Z INFO orders.1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE sending {"count": 14}
2020-04-08T15:30:34.535Z INFO orders.12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs sending {"count": 25}
2020-04-08T15:30:34.704Z INFO orders.12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs finished
2020-04-08T15:30:34.752Z INFO orders.118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW finished
2020-04-08T15:30:35.021Z INFO orders.1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE finished
2020-04-08T16:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-08T16:26:39.781Z INFO orders.1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE sending {"count": 61}
2020-04-08T16:26:39.781Z INFO orders.118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW sending {"count": 19}
2020-04-08T16:26:40.122Z INFO orders.118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW finished
2020-04-08T16:26:40.630Z INFO orders.1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE finished
2020-04-08T17:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-08T17:29:00.308Z INFO orders.1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE sending {"count": 198}
2020-04-08T17:29:00.308Z INFO orders.118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW sending {"count": 116}
2020-04-08T17:29:00.308Z INFO orders.121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6 sending {"count": 267}
2020-04-08T17:29:00.671Z INFO orders.118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW finished
2020-04-08T17:29:01.339Z INFO orders.1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE finished
2020-04-08T17:29:02.259Z INFO orders.121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6 finished
2020-04-08T18:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-08T18:29:51.305Z INFO orders.118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW sending {"count": 135}
2020-04-08T18:29:51.305Z INFO orders.121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6 sending {"count": 291}
2020-04-08T18:29:51.305Z INFO orders.1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE sending {"count": 247}
2020-04-08T18:29:51.654Z INFO orders.118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW finished
2020-04-08T18:29:52.431Z INFO orders.1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE finished
2020-04-08T18:29:53.231Z INFO orders.121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6 finished
2020-04-08T19:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-08T19:29:21.713Z INFO orders.121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6 sending {"count": 72}
2020-04-08T19:29:21.713Z INFO orders.118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW sending {"count": 159}
2020-04-08T19:29:21.713Z INFO orders.1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE sending {"count": 83}
2020-04-08T19:29:22.122Z INFO orders.118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW finished
2020-04-08T19:29:22.535Z INFO orders.1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE finished
2020-04-08T19:29:23.179Z INFO orders.121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6 finished
2020-04-08T20:22:38.531Z INFO trust Scheduling next refresh {"after": "5h25m36.98896851s"}
2020-04-08T20:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-08T20:29:46.609Z INFO orders.1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE sending {"count": 92}
2020-04-08T20:29:46.609Z INFO orders.118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW sending {"count": 111}
2020-04-08T20:29:46.996Z INFO orders.118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW finished
2020-04-08T20:29:47.488Z INFO orders.1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE finished
2020-04-08T21:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-08T21:29:24.659Z INFO orders.118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW sending {"count": 87}
2020-04-08T21:29:24.659Z INFO orders.1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE sending {"count": 565}
2020-04-08T21:29:25.912Z INFO orders.1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE finished
2020-04-08T21:29:27.941Z INFO orders.118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW finished
2020-04-08T22:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-08T22:31:19.139Z INFO orders.1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE sending {"count": 2}
2020-04-08T22:31:19.139Z INFO orders.118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW sending {"count": 85}
2020-04-08T22:31:20.183Z INFO orders.1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE finished
2020-04-08T22:31:21.153Z INFO orders.118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW finished
2020-04-08T23:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-09T00:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-09T01:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-09T01:48:16.192Z INFO trust Scheduling next refresh {"after": "3h4m15.536154592s"}
2020-04-09T02:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-09T03:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-09T04:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-09T04:52:32.838Z INFO trust Scheduling next refresh {"after": "7h42m41.433800046s"}
2020-04-09T05:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-09T06:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-09T07:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-09T08:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-09T09:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-09T10:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-09T11:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-09T12:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-09T12:35:14.855Z INFO trust Scheduling next refresh {"after": "3h51m40.360283172s"}
2020-04-09T13:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-09T14:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-09T15:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-09T15:26:35.586Z INFO version running on version v1.1.1
2020-04-09T16:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-09T16:26:56.315Z INFO trust Scheduling next refresh {"after": "8h8m48.718867653s"}
2020-04-09T17:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-09T18:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-09T19:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-09T19:27:23.578Z ERROR nodestats:cache Get stats query failed {"error": "node stats service error: satellitedb: pq: column nodes.free_bandwidth does not exist", "errorVerbose": "node stats service error: satellitedb: pq: column nodes.free_bandwidth does not exist\n\tstorj.io/storj/storagenode/nodestats.(*Service).GetReputationStats:73\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheReputationStats.func1:112\n\tstorj.io/storj/storagenode/nodestats.(*Cache).satelliteLoop:216\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheReputationStats:111\n\tstorj.io/storj/storagenode/nodestats.(*Cache).Run.func1:78\n\tstorj.io/common/sync2.(*Cycle).Run:150\n\tstorj.io/common/sync2.(*Cycle).Start.func1:69\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57"}
2020-04-09T20:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-09T21:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-09T22:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-09T23:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-10T00:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-10T00:35:46.485Z INFO trust Scheduling next refresh {"after": "4h51m24.920809321s"}
2020-04-10T01:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-10T02:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-10T03:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-10T04:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-10T05:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-10T05:27:12.560Z INFO trust Scheduling next refresh {"after": "3h42m13.849309837s"}
2020-04-10T06:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-10T07:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-10T08:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-10T09:09:27.505Z INFO trust Scheduling next refresh {"after": "7h25m8.679109886s"}
2020-04-10T09:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-10T10:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-10T11:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-10T12:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-10T13:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-10T14:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-10T15:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-10T15:26:35.579Z INFO version running on version v1.1.1
2020-04-10T16:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-10T16:34:36.744Z INFO trust Scheduling next refresh {"after": "6h38m33.067782707s"}
2020-04-10T17:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-10T18:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-10T19:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-10T20:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-10T21:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-10T22:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-10T23:13:10.917Z INFO trust Scheduling next refresh {"after": "5h59m19.73822442s"}
2020-04-10T23:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-11T00:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-11T01:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-11T02:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-11T03:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-11T04:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-11T05:12:31.798Z INFO trust Scheduling next refresh {"after": "6h0m57.295734914s"}
2020-04-11T05:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-11T06:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-11T07:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-11T08:26:34.915Z INFO bandwidth Performing bandwidth usage rollups
2020-04-11T09:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-11T10:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-11T11:13:30.273Z INFO trust Scheduling next refresh {"after": "6h23m10.638963537s"}
2020-04-11T11:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-11T12:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-11T13:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-11T14:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-11T15:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-11T15:26:35.550Z INFO version running on version v1.1.1
2020-04-11T15:27:05.814Z ERROR blobscache trashTotal < 0 {"trashTotal": -5171580022}
2020-04-11T16:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-11T17:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-11T17:36:42.032Z INFO trust Scheduling next refresh {"after": "7h53m48.895628664s"}
2020-04-11T18:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-11T19:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-11T20:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-11T21:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-11T22:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-11T23:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-12T00:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-12T01:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-12T01:30:31.518Z INFO trust Scheduling next refresh {"after": "7h18m53.852861372s"}
2020-04-12T02:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-12T03:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-12T04:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-12T05:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-12T06:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-12T07:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-12T08:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-12T08:49:26.581Z INFO trust Scheduling next refresh {"after": "5h46m9.170152354s"}
2020-04-12T09:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-12T10:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-12T11:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-12T12:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-12T13:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-12T14:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-12T14:35:36.330Z INFO trust Scheduling next refresh {"after": "6h55m9.189783758s"}
2020-04-12T15:26:34.911Z INFO bandwidth Performing bandwidth usage rollups
2020-04-12T15:26:35.579Z INFO version running on version v1.1.1
2020-04-12T15:26:35.756Z ERROR blobscache trashTotal < 0 {"trashTotal": -6569984}
2020-04-12T15:27:41.988Z INFO Got a signal from the OS: "terminated"
2020-04-12T15:27:45.645Z INFO Configuration loaded from: /app/config/config.yaml
2020-04-12T15:27:45.654Z INFO Operator email: xxx@xxx.xxx
2020-04-12T15:27:45.654Z INFO operator wallet: 0x000000
2020-04-12T15:27:46.971Z INFO version running on version v1.1.1
2020-04-12T15:27:47.026Z INFO db.migration Database Version {"version": 33}
2020-04-12T15:27:48.549Z INFO preflight:localtime start checking local system clock with trusted satellites' system clock.
2020-04-12T15:27:49.441Z INFO preflight:localtime local system clock is in sync with trusted satellites' system clock.
2020-04-12T15:27:49.441Z INFO bandwidth Performing bandwidth usage rollups
2020-04-12T15:27:49.441Z INFO trust Scheduling next refresh {"after": "3h59m51.794991015s"}
2020-04-12T15:27:49.442Z INFO Node xxxxx started
2020-04-12T15:27:49.442Z INFO Public server started on [::]:28967
2020-04-12T15:27:49.442Z INFO Private server started on 127.0.0.1:7778
2020-04-12T15:27:49.678Z INFO version running on version v1.1.1
2020-04-12T16:27:49.441Z INFO bandwidth Performing bandwidth usage rollups
2020-04-12T17:05:57.073Z INFO Got a signal from the OS: "terminated"
2020-04-12T17:05:58.897Z INFO Configuration loaded from: /app/config/config.yaml
2020-04-12T17:05:58.906Z INFO Operator email: xxx@xxx.xxx
2020-04-12T17:05:58.906Z INFO operator wallet: 0x000000
2020-04-12T17:05:59.880Z INFO version running on version v1.1.1
2020-04-12T17:05:59.960Z INFO db.migration Database Version {"version": 33}
2020-04-12T17:06:01.642Z INFO preflight:localtime start checking local system clock with trusted satellites' system clock.
2020-04-12T17:06:02.526Z INFO preflight:localtime local system clock is in sync with trusted satellites' system clock.
2020-04-12T17:06:02.527Z INFO bandwidth Performing bandwidth usage rollups
2020-04-12T17:06:02.527Z INFO trust Scheduling next refresh {"after": "2h52m9.124565843s"}
2020-04-12T17:06:02.527Z INFO Node xxxxx started
2020-04-12T17:06:02.527Z INFO Public server started on [::]:28967
2020-04-12T17:06:02.527Z INFO Private server started on 127.0.0.1:7778
2020-04-12T17:06:02.763Z INFO version running on version v1.1.1Do you keep removing your container and restarting your node?
I didn’t mean without download/upload lines. those are still relevant. Can’t see anything obviously wrong in that log
No, the container has been running constantly from when it was updated on Wednesday to this evening when I restarted it to see if that updated the dashboard
[quote=“caetel, post:6, topic:5844”]
2020-04-09T19:27:23.578Z ERROR nodestats:cache Get stats query failed {“error”: “node stats service error: satellitedb: pq: column nodes.free_bandwidth does not exist”, “errorVerbose”: “node stats service error: satellitedb: pq: column nodes.free_bandwidth does not exist\n\tstorj.io/storj/storagenode/nodestats.(*Service).GetReputationStats:73\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheReputationStats.func1:112\n\tstorj.io/storj/storagenode/nodestats.(*Cache).satelliteLoop:216\n\tstorj.io/storj/storagenode/nodestats.(*Cache).CacheReputationStats:111\n\tstorj.io/storj/storagenode/nodestats.(*Cache).Run.func1:78\n\tstorj.io/common/sync2.(*Cycle).Run:150\n\tstorj.io/common/sync2.(*Cycle).Start.func1:69\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57”}
[/quote]
Saw this error did your node ever fill to the max space?
that link doesn’t work for me.
Please, answer to this question
Your situation is very similar to storage connected via NFS or SMB.
They are not compatible with storagenode.
They are directly attached devices, mounted to a folder and mapped through docker
Please,
satellites.db
sqlite3 satellites.db
CREATE TABLE satellite_exit_progress (
satellite_id BLOB NOT NULL,
initiated_at TIMESTAMP,
finished_at TIMESTAMP,
starting_disk_usage INTEGER NOT NULL,
bytes_deleted INTEGER NOT NULL,
completion_receipt BLOB,
PRIMARY KEY (satellite_id)
);
CREATE TABLE versions (version int, commited_at text);
CREATE TABLE satellites (
node_id BLOB NOT NULL,
added_at TIMESTAMP NOT NULL,
status INTEGER NOT NULL,
PRIMARY KEY (node_id)
);
.exit
I’m Update 0.35 to 1.1.1 and Dashboard to show me 0 Bandwidth status…
i’m already read https://forum.storj.io/t/no-bandwidth-usage-since-update-to-1-1-1/5844/11
But this solution https://support.storj.io/hc/en-us/articles/360029309111-How-to-fix-a-database-disk-image-is-malformed-
is only CLI mode
Hello @COSIA,
Welcome to the forum!
Yes, there is no GUI mode fix.
I’ll merge your post with existing
I’ve been seeing similar this month, my bandwidth usage hasn’t quite dropped to 0 but very close. Is this something I should be worried about?
Your node is full your not seeing anything because its full, Not alot of egress because there just isnt alot of people downloading from your node and no ingress because its full.
Cheers for the reply. Towards the end of last month I was getting something like 20GB a day egress so to fall to 100s of megabytes is quite a drop, I was concerned my node had been disqualified of something.
Is egress roughly proportional to data stored, if I double my storj capacity and it was filled would I expect to see close to double the egress?
Cheers
