Node stopped working I have no ideea how to fix it

           Good morning guys, bit of an issue with a node I have on a raspberry pi4, it worked very well since January but yesterday it crashed and I still can't figure it out, the logs look like this 

2020-05-01T08:28:02.998Z INFO Configuration loaded from: /app/config/config.yaml
2020-05-01T08:28:03.000Z INFO Operator email: xxxxxxxxxx
2020-05-01T08:28:03.000Z INFO operator wallet: 0x5827d67A7A0958436c9b5908Bf6B589eFb46EC05
2020-05-01T08:28:03.758Z INFO version running on version v1.1.1
2020-05-01T08:28:03.861Z INFO db.migration Database Version {“version”: 33}
Error: Error during preflight check for storagenode databases: storage node preflight database error: unable to open database file: input/output error
storj.io/storj/private/dbutil/sqliteutil.QuerySchema.func1:35
storj.io/storj/private/dbutil/sqliteutil.QuerySchema:53
storj.io/storj/storagenode/storagenodedb.(*DB).Preflight:293
main.cmdRun:199
storj.io/private/process.cleanup.func1.2:312
storj.io/private/process.cleanup.func1:330
github.com/spf13/cobra.(*Command).execute:840
github.com/spf13/cobra.(*Command).ExecuteC:945
github.com/spf13/cobra.(*Command).Execute:885
storj.io/private/process.ExecWithCustomConfig:84
storj.io/private/process.ExecCustomDebug:66
main.main:329
runtime.main:203

Any suggestions?

How is your HDD connected to your PC with storagenode?

Please, stop the storagenode and check this disk for errors by OS-related tools for disk checking.

1 Like

Thank you Alexey, very active as allways. I have installed Smartmontools , and the check result is sudo smartctl -s on -a /dev/sda1
smartctl 6.6 2017-11-05 r4594 [armv7l-linux-4.19.97-v7l+] (local build)
Copyright © 2002-17, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Device Model: Samsung SSD 860 EVO 1TB
Serial Number: S3Z9NB0M715366E
LU WWN Device Id: 5 002538 e49782764
Firmware Version: RVT03B6Q
User Capacity: 1,000,204,886,016 bytes [1.00 TB]
Sector Size: 512 bytes logical/physical
Rotation Rate: Solid State Device
Form Factor: 2.5 inches
Device is: Not in smartctl database [for details use: -P showall]
ATA Version is: ACS-4 T13/BSR INCITS 529 revision 5
SATA Version is: SATA 3.2, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is: Fri May 1 10:53:09 2020 BST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF ENABLE/DISABLE COMMANDS SECTION ===
SMART Enabled.

=== START OF READ SMART DATA SECTION ===
SMART Status not supported: Incomplete response, ATA output registers missing
SMART overall-health self-assessment test result: PASSED
Warning: This result is based on an Attribute check.

General SMART Values:
Offline data collection status: (0x00) Offline data collection activity
was never started.
Auto Offline Data Collection: Disabled.
Self-test execution status: ( 0) The previous self-test routine completed
without error or no self-test has ever
been run.
Total time to complete Offline
data collection: ( 0) seconds.
Offline data collection
capabilities: (0x53) SMART execute Offline immediate.
Auto Offline data collection on/off support.
Suspend Offline collection upon new
command.
No Offline surface scan supported.
Self-test supported.
No Conveyance Self-test supported.
Selective Self-test supported.
SMART capabilities: (0x0003) Saves SMART data before entering
power-saving mode.
Supports SMART auto save timer.
Error logging capability: (0x01) Error logging supported.
General Purpose Logging supported.
Short self-test routine
recommended polling time: ( 2) minutes.
Extended self-test routine
recommended polling time: ( 85) minutes.
SCT capabilities: (0x003d) SCT Status supported.
SCT Error Recovery Control supported.
SCT Feature Control supported.
SCT Data Table supported.

SMART Attributes Data Structure revision number: 1
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
5 Reallocated_Sector_Ct 0x0033 100 100 010 Pre-fail Always - 0
9 Power_On_Hours 0x0032 099 099 000 Old_age Always - 2562
12 Power_Cycle_Count 0x0032 099 099 000 Old_age Always - 278
177 Wear_Leveling_Count 0x0013 099 099 000 Pre-fail Always - 3
179 Used_Rsvd_Blk_Cnt_Tot 0x0013 100 100 010 Pre-fail Always - 0
181 Program_Fail_Cnt_Total 0x0032 100 100 010 Old_age Always - 0
182 Erase_Fail_Count_Total 0x0032 100 100 010 Old_age Always - 0
183 Runtime_Bad_Block 0x0013 100 100 010 Pre-fail Always - 0
187 Reported_Uncorrect 0x0032 100 100 000 Old_age Always - 0
190 Airflow_Temperature_Cel 0x0032 068 060 000 Old_age Always - 32
195 Hardware_ECC_Recovered 0x001a 200 200 000 Old_age Always - 0
199 UDMA_CRC_Error_Count 0x003e 100 100 000 Old_age Always - 0
235 Unknown_Attribute 0x0012 099 099 000 Old_age Always - 171
241 Total_LBAs_Written 0x0032 099 099 000 Old_age Always - 4702410344

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
No self-tests have been logged. [To run self-tests, use: smartctl -t]

SMART Selective self-test log data structure revision number 1
SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS
1 0 0 Not_testing
2 0 0 Not_testing
3 0 0 Not_testing
4 0 0 Not_testing
5 0 0 Not_testing
Selective self-test flags (0x0):
After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

But I don’t understand much from it , but it looks ok, do you know of any other tool that I can use , or anything else I can try?

I mean fsck in your case :slight_smile: