"infodb: database is locked" errors after update to v0.15.2

I upgraded a storage node from v0.13.5 to v0.15.2. It was offline for a month.

I noticed a lot of “infodb: database is locked” errors after the upgrade:

019-07-17T03:18:43.309-0400	INFO	db.migration	Latest Version	{"version": 3}
2019-07-17T03:18:43.309-0400	INFO	db.migration.4	Add index on pieceinfo expireation
2019-07-17T03:18:54.236-0400	INFO	db.migration.5	Partial Network Wipe - Tardigrade Satellites
2019-07-17T03:18:55.101-0400	INFO	db.migration.6	Add creation date.
2019-07-17T03:18:55.107-0400	INFO	db.migration.7	Drop certificate table.
2019-07-17T03:31:02.810-0400	INFO	db.migration.8	Drop old used serials and remove pieceinfo_deletion_failed index.
2019-07-17T03:34:01.931-0400	INFO	db.migration.9	Add order limit table.
2019-07-17T03:34:01.932-0400	INFO	db.migration.10	Optimize index usage.
2019-07-17T03:34:25.877-0400	INFO	db.migration.11	Create bandwidth_usage_rollup table.
2019-07-17T03:34:25.878-0400	INFO	db.migration.12	Clear Tables from Alpha data
2019-07-17T03:36:34.940-0400	INFO	db.migration.13	Free Storagenodes from trash data
2019-07-17T03:57:03.567-0400	INFO	vouchers	Checking vouchers
2019-07-17T03:57:03.586-0400	INFO	Node 15Y7KzGXxnjfo3wQDzLBxdkuYeyDc9AMiXpdHxNmsmsuLfyePb started
2019-07-17T03:57:03.586-0400	INFO	Public server started on [::]:28967
2019-07-17T03:57:03.586-0400	INFO	Private server started on 127.0.0.1:7778
2019-07-17T03:57:03.617-0400	INFO	vouchers	Requesting voucher	{"satellite": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs"}
2019-07-17T03:57:03.620-0400	INFO	vouchers	Requesting voucher	{"satellite": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S"}
2019-07-17T03:57:03.623-0400	INFO	vouchers	Requesting voucher	{"satellite": "118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW"}
2019-07-17T03:57:03.634-0400	INFO	vouchers	Requesting voucher	{"satellite": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6"}
2019-07-17T03:57:04.445-0400	INFO	running on version v0.15.2
2019-07-17T03:57:10.915-0400	INFO	piecestore:orderssender.12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S	sending	{"count": 6}
2019-07-17T03:57:10.915-0400	INFO	piecestore:orderssender.12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs	sending	{"count": 7}
2019-07-17T03:57:10.915-0400	INFO	piecestore:orderssender.118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW	sending	{"count": 7062}
2019-07-17T03:57:38.023-0400	INFO	piecestore:monitor	Remaining Bandwidth	{"bytes": 10995116277760}
2019-07-17T03:57:38.023-0400	WARN	piecestore:monitor	Disk space is less than requested. Allocating space	{"bytes": 1729649541120}
2019-07-17T03:57:38.118-0400	ERROR	vouchers	Error requesting voucher	{"satellite": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "error": "database is locked"}
2019-07-17T03:57:40.521-0400	ERROR	vouchers	Error requesting voucher	{"satellite": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "error": "database is locked"}
2019-07-17T03:57:41.336-0400	ERROR	piecestore:orderssender.12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs	failed to archive order as rejected: serial: SUDULPEAJNCIXMXB7LELT5VVHA, infodb: database is locked
2019-07-17T03:57:41.770-0400	ERROR	piecestore:orderssender.12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S	failed to archive order as accepted: serial: BJSNW3UDLVGGPED7XOFGU3Y2R4, infodb: database is locked
2019-07-17T03:57:44.438-0400	ERROR	piecestore:orderssender.118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW	failed to archive order as rejected: serial: 5D3O66MRSNDCPEMOANWGB7S2OY, infodb: database is locked
2019-07-17T03:57:51.494-0400	ERROR	piecestore:orderssender.12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs	failed to archive order as rejected: serial: 7ANQNP6TXRFX5A3I3ZTFKZZE6I, infodb: database is locked
2019-07-17T03:57:51.938-0400	ERROR	piecestore:orderssender.12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S	failed to archive order as accepted: serial: I65G3H2HGBAMFEBWBBK7GDGOC4, infodb: database is locked
2019-07-17T03:57:54.539-0400	ERROR	piecestore:orderssender.118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW	failed to archive order as rejected: serial: Y453CWFA7FBJHEH756WIUSC2LE, infodb: database is locked
2019-07-17T03:58:00.603-0400	ERROR	vouchers	Error requesting voucher	{"satellite": "118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW", "error": "database is locked"}
2019-07-17T03:58:01.595-0400	ERROR	piecestore:orderssender.12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs	failed to archive order as rejected: serial: AGHVZ4UK2FGOVFKIO54SVKXKZM, infodb: database is locked
2019-07-17T03:58:02.026-0400	ERROR	piecestore:orderssender.12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S	failed to archive order as accepted: serial: 7VY5YKCTVJGCLBNRY5YBLIKSFE, infodb: database is locked
2019-07-17T03:58:04.622-0400	ERROR	piecestore:orderssender.118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW	failed to archive order as rejected: serial: ZEFXCMM3GNEPDB2ZHOQ7KXSQXQ, infodb: database is locked
2019-07-17T03:58:04.722-0400	ERROR	vouchers	Error requesting voucher	{"satellite": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "error": "database is locked"}
2019-07-17T03:58:11.680-0400	ERROR	piecestore:orderssender.12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs	failed to archive order as accepted: serial: 2WUFXNBITJGGRHXTBJF6ATUVSY, infodb: database is locked
2019-07-17T03:58:12.110-0400	ERROR	piecestore:orderssender.12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S	failed to archive order as accepted: serial: HBPVHNPHG5GQFIQYY2OPQ4BA3Y, infodb: database is locked
2019-07-17T03:58:14.704-0400	ERROR	piecestore:orderssender.118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW	failed to archive order as rejected: serial: F4JW3HSGWBCJTFZFB6O3RA7JQI, infodb: database is locked
2019-07-17T03:58:21.767-0400	ERROR	piecestore:orderssender.12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs	failed to archive order as accepted: serial: S54XRLKICNDQXJWUO6K5CXCUKU, infodb: database is locked
2019-07-17T03:58:22.196-0400	ERROR	piecestore:orderssender.12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S	failed to archive order as accepted: serial: X7WOTHZ3INBPBBPW53LOLWLCYI, infodb: database is locked
2019-07-17T03:58:24.788-0400	ERROR	piecestore:orderssender.118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW	failed to archive order as rejected: serial: WOGDS4LJGJAOJOKEJG64K4L4IU, infodb: database is locked
2019-07-17T03:58:31.849-0400	ERROR	piecestore:orderssender.12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs	failed to archive order as accepted: serial: MV2ANHEPENBIPFEB6SZ43OTZD4, infodb: database is locked
2019-07-17T03:58:32.283-0400	ERROR	piecestore:orderssender.12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S	failed to archive order as accepted: serial: CNFIEVH7DZAHJFK5W42J4EDJQY, infodb: database is locked
2019-07-17T03:58:32.283-0400	INFO	piecestore:orderssender.12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S	finished
2019-07-17T03:58:32.284-0400	ERROR	piecestore:orderssender.12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S	failed to settle orders	{"error": "order: failed to archive order as accepted: serial: BJSNW3UDLVGGPED7XOFGU3Y2R4, infodb: database is locked; order: failed to archive order as accepted: serial: I65G3H2HGBAMFEBWBBK7GDGOC4, infodb: database is locked; order: failed to archive order as accepted: serial: 7VY5YKCTVJGCLBNRY5YBLIKSFE, infodb: database is locked; order: failed to archive order as accepted: serial: HBPVHNPHG5GQFIQYY2OPQ4BA3Y, infodb: database is locked; order: failed to archive order as accepted: serial: X7WOTHZ3INBPBBPW53LOLWLCYI, infodb: database is locked; order: failed to archive order as accepted: serial: CNFIEVH7DZAHJFK5W42J4EDJQY, infodb: database is locked", "errorVerbose": "group:\n--- order: failed to archive order as accepted: serial: BJSNW3UDLVGGPED7XOFGU3Y2R4, infodb: database is locked\n\tstorj.io/storj/storagenode/orders.(*Sender).settle.func3:191\n\tstorj.io/storj/storagenode/orders.(*Sender).settle:208\n\tstorj.io/storj/storagenode/orders.(*Sender).Settle:142\n\tstorj.io/storj/storagenode/orders.(*Sender).runOnce.func1:127\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57\n--- order: failed to archive order as accepted: serial: I65G3H2HGBAMFEBWBBK7GDGOC4, infodb: database is locked\n\tstorj.io/storj/storagenode/orders.(*Sender).settle.func3:191\n\tstorj.io/storj/storagenode/orders.(*Sender).settle:208\n\tstorj.io/storj/storagenode/orders.(*Sender).Settle:142\n\tstorj.io/storj/storagenode/orders.(*Sender).runOnce.func1:127\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57\n--- order: failed to archive order as accepted: serial: 7VY5YKCTVJGCLBNRY5YBLIKSFE, infodb: database is locked\n\tstorj.io/storj/storagenode/orders.(*Sender).settle.func3:191\n\tstorj.io/storj/storagenode/orders.(*Sender).settle:208\n\tstorj.io/storj/storagenode/orders.(*Sender).Settle:142\n\tstorj.io/storj/storagenode/orders.(*Sender).runOnce.func1:127\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57\n--- order: failed to archive order as accepted: serial: HBPVHNPHG5GQFIQYY2OPQ4BA3Y, infodb: database is locked\n\tstorj.io/storj/storagenode/orders.(*Sender).settle.func3:191\n\tstorj.io/storj/storagenode/orders.(*Sender).settle:208\n\tstorj.io/storj/storagenode/orders.(*Sender).Settle:142\n\tstorj.io/storj/storagenode/orders.(*Sender).runOnce.func1:127\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57\n--- order: failed to archive order as accepted: serial: X7WOTHZ3INBPBBPW53LOLWLCYI, infodb: database is locked\n\tstorj.io/storj/storagenode/orders.(*Sender).settle.func3:191\n\tstorj.io/storj/storagenode/orders.(*Sender).settle:208\n\tstorj.io/storj/storagenode/orders.(*Sender).Settle:142\n\tstorj.io/storj/storagenode/orders.(*Sender).runOnce.func1:127\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57\n--- order: failed to archive order as accepted: serial: CNFIEVH7DZAHJFK5W42J4EDJQY, infodb: database is locked\n\tstorj.io/storj/storagenode/orders.(*Sender).settle.func3:191\n\tstorj.io/storj/storagenode/orders.(*Sender).settle:208\n\tstorj.io/storj/storagenode/orders.(*Sender).Settle:142\n\tstorj.io/storj/storagenode/orders.(*Sender).runOnce.func1:127\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57"}
2019-07-17T03:58:34.871-0400	ERROR	piecestore:orderssender.118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW	failed to archive order as rejected: serial: 323EAZP2F5H3RDDZUIOK5LB26Q, infodb: database is locked
2019-07-17T03:58:41.961-0400	ERROR	piecestore:orderssender.12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs	failed to archive order as accepted: serial: G6QP6VCCXNEFLFJQHJYMQE3TPM, infodb: database is locked
2019-07-17T03:58:41.961-0400	INFO	piecestore:orderssender.12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs	finished
2019-07-17T03:58:41.962-0400	ERROR	piecestore:orderssender.12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs	failed to settle orders	{"error": "order: failed to archive order as rejected: serial: SUDULPEAJNCIXMXB7LELT5VVHA, infodb: database is locked; order: failed to archive order as rejected: serial: 7ANQNP6TXRFX5A3I3ZTFKZZE6I, infodb: database is locked; order: failed to archive order as rejected: serial: AGHVZ4UK2FGOVFKIO54SVKXKZM, infodb: database is locked; order: failed to archive order as accepted: serial: 2WUFXNBITJGGRHXTBJF6ATUVSY, infodb: database is locked; order: failed to archive order as accepted: serial: S54XRLKICNDQXJWUO6K5CXCUKU, infodb: database is locked; order: failed to archive order as accepted: serial: MV2ANHEPENBIPFEB6SZ43OTZD4, infodb: database is locked; order: failed to archive order as accepted: serial: G6QP6VCCXNEFLFJQHJYMQE3TPM, infodb: database is locked", "errorVerbose": "group:\n--- order: failed to archive order as rejected: serial: SUDULPEAJNCIXMXB7LELT5VVHA, infodb: database is locked\n\tstorj.io/storj/storagenode/orders.(*Sender).settle.func3:191\n\tstorj.io/storj/storagenode/orders.(*Sender).settle:213\n\tstorj.io/storj/storagenode/orders.(*Sender).Settle:142\n\tstorj.io/storj/storagenode/orders.(*Sender).runOnce.func1:127\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57\n--- order: failed to archive order as rejected: serial: 7ANQNP6TXRFX5A3I3ZTFKZZE6I, infodb: database is locked\n\tstorj.io/storj/storagenode/orders.(*Sender).settle.func3:191\n\tstorj.io/storj/storagenode/orders.(*Sender).settle:213\n\tstorj.io/storj/storagenode/orders.(*Sender).Settle:142\n\tstorj.io/storj/storagenode/orders.(*Sender).runOnce.func1:127\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57\n--- order: failed to archive order as rejected: serial: AGHVZ4UK2FGOVFKIO54SVKXKZM, infodb: database is locked\n\tstorj.io/storj/storagenode/orders.(*Sender).settle.func3:191\n\tstorj.io/storj/storagenode/orders.(*Sender).settle:213\n\tstorj.io/storj/storagenode/orders.(*Sender).Settle:142\n\tstorj.io/storj/storagenode/orders.(*Sender).runOnce.func1:127\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57\n--- order: failed to archive order as accepted: serial: 2WUFXNBITJGGRHXTBJF6ATUVSY, infodb: database is locked\n\tstorj.io/storj/storagenode/orders.(*Sender).settle.func3:191\n\tstorj.io/storj/storagenode/orders.(*Sender).settle:208\n\tstorj.io/storj/storagenode/orders.(*Sender).Settle:142\n\tstorj.io/storj/storagenode/orders.(*Sender).runOnce.func1:127\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57\n--- order: failed to archive order as accepted: serial: S54XRLKICNDQXJWUO6K5CXCUKU, infodb: database is locked\n\tstorj.io/storj/storagenode/orders.(*Sender).settle.func3:191\n\tstorj.io/storj/storagenode/orders.(*Sender).settle:208\n\tstorj.io/storj/storagenode/orders.(*Sender).Settle:142\n\tstorj.io/storj/storagenode/orders.(*Sender).runOnce.func1:127\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57\n--- order: failed to archive order as accepted: serial: MV2ANHEPENBIPFEB6SZ43OTZD4, infodb: database is locked\n\tstorj.io/storj/storagenode/orders.(*Sender).settle.func3:191\n\tstorj.io/storj/storagenode/orders.(*Sender).settle:208\n\tstorj.io/storj/storagenode/orders.(*Sender).Settle:142\n\tstorj.io/storj/storagenode/orders.(*Sender).runOnce.func1:127\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57\n--- order: failed to archive order as accepted: serial: G6QP6VCCXNEFLFJQHJYMQE3TPM, infodb: database is locked\n\tstorj.io/storj/storagenode/orders.(*Sender).settle.func3:191\n\tstorj.io/storj/storagenode/orders.(*Sender).settle:208\n\tstorj.io/storj/storagenode/orders.(*Sender).Settle:142\n\tstorj.io/storj/storagenode/orders.(*Sender).runOnce.func1:127\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57"}
2019-07-17T03:58:44.957-0400	ERROR	piecestore:orderssender.118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW	failed to archive order as rejected: serial: K47AKAB53RF6VMFBS6YPK6EARE, infodb: database is locked
2019-07-17T03:58:55.033-0400	ERROR	piecestore:orderssender.118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW	failed to archive order as rejected: serial: QGJZGF5GLNFSTPMAGOCHIV4FXI, infodb: database is locked
2019-07-17T03:59:05.174-0400	ERROR	piecestore:orderssender.118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW	failed to archive order as rejected: serial: 5LIDMBKXOFGDFAI425CYGCRSM4, infodb: database is locked
2019-07-17T03:59:49.124-0400	ERROR	piecestore:orderssender.118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW	failed to receive response: rpc error: code = Unavailable desc = transport is closing
2019-07-17T03:59:49.124-0400	ERROR	piecestore:orderssender.118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW	sending agreements returned an error: EOF
2019-07-17T03:59:49.124-0400	INFO	piecestore:orderssender.118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW	finished
2019-07-17T03:59:49.124-0400	ERROR	piecestore:orderssender.118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW	failed to settle orders	{"error": "order: failed to archive order as rejected: serial: 5D3O66MRSNDCPEMOANWGB7S2OY, infodb: database is locked; order: failed to archive order as rejected: serial: Y453CWFA7FBJHEH756WIUSC2LE, infodb: database is locked; order: failed to archive order as rejected: serial: ZEFXCMM3GNEPDB2ZHOQ7KXSQXQ, infodb: database is locked; order: failed to archive order as rejected: serial: F4JW3HSGWBCJTFZFB6O3RA7JQI, infodb: database is locked; order: failed to archive order as rejected: serial: WOGDS4LJGJAOJOKEJG64K4L4IU, infodb: database is locked; order: failed to archive order as rejected: serial: 323EAZP2F5H3RDDZUIOK5LB26Q, infodb: database is locked; order: failed to archive order as rejected: serial: K47AKAB53RF6VMFBS6YPK6EARE, infodb: database is locked; order: failed to archive order as rejected: serial: QGJZGF5GLNFSTPMAGOCHIV4FXI, infodb: database is locked; order: failed to archive order as rejected: serial: 5LIDMBKXOFGDFAI425CYGCRSM4, infodb: database is locked; order: failed to receive response: rpc error: code = Unavailable desc = transport is closing; order: sending agreements returned an error: EOF", "errorVerbose": "group:\n--- order: failed to archive order as rejected: serial: 5D3O66MRSNDCPEMOANWGB7S2OY, infodb: database is locked\n\tstorj.io/storj/storagenode/orders.(*Sender).settle.func3:191\n\tstorj.io/storj/storagenode/orders.(*Sender).settle:213\n\tstorj.io/storj/storagenode/orders.(*Sender).Settle:142\n\tstorj.io/storj/storagenode/orders.(*Sender).runOnce.func1:127\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57\n--- order: failed to archive order as rejected: serial: Y453CWFA7FBJHEH756WIUSC2LE, infodb: database is locked\n\tstorj.io/storj/storagenode/orders.(*Sender).settle.func3:191\n\tstorj.io/storj/storagenode/orders.(*Sender).settle:213\n\tstorj.io/storj/storagenode/orders.(*Sender).Settle:142\n\tstorj.io/storj/storagenode/orders.(*Sender).runOnce.func1:127\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57\n--- order: failed to archive order as rejected: serial: ZEFXCMM3GNEPDB2ZHOQ7KXSQXQ, infodb: database is locked\n\tstorj.io/storj/storagenode/orders.(*Sender).settle.func3:191\n\tstorj.io/storj/storagenode/orders.(*Sender).settle:213\n\tstorj.io/storj/storagenode/orders.(*Sender).Settle:142\n\tstorj.io/storj/storagenode/orders.(*Sender).runOnce.func1:127\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57\n--- order: failed to archive order as rejected: serial: F4JW3HSGWBCJTFZFB6O3RA7JQI, infodb: database is locked\n\tstorj.io/storj/storagenode/orders.(*Sender).settle.func3:191\n\tstorj.io/storj/storagenode/orders.(*Sender).settle:213\n\tstorj.io/storj/storagenode/orders.(*Sender).Settle:142\n\tstorj.io/storj/storagenode/orders.(*Sender).runOnce.func1:127\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57\n--- order: failed to archive order as rejected: serial: WOGDS4LJGJAOJOKEJG64K4L4IU, infodb: database is locked\n\tstorj.io/storj/storagenode/orders.(*Sender).settle.func3:191\n\tstorj.io/storj/storagenode/orders.(*Sender).settle:213\n\tstorj.io/storj/storagenode/orders.(*Sender).Settle:142\n\tstorj.io/storj/storagenode/orders.(*Sender).runOnce.func1:127\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57\n--- order: failed to archive order as rejected: serial: 323EAZP2F5H3RDDZUIOK5LB26Q, infodb: database is locked\n\tstorj.io/storj/storagenode/orders.(*Sender).settle.func3:191\n\tstorj.io/storj/storagenode/orders.(*Sender).settle:213\n\tstorj.io/storj/storagenode/orders.(*Sender).Settle:142\n\tstorj.io/storj/storagenode/orders.(*Sender).runOnce.func1:127\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57\n--- order: failed to archive order as rejected: serial: K47AKAB53RF6VMFBS6YPK6EARE, infodb: database is locked\n\tstorj.io/storj/storagenode/orders.(*Sender).settle.func3:191\n\tstorj.io/storj/storagenode/orders.(*Sender).settle:213\n\tstorj.io/storj/storagenode/orders.(*Sender).Settle:142\n\tstorj.io/storj/storagenode/orders.(*Sender).runOnce.func1:127\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57\n--- order: failed to archive order as rejected: serial: QGJZGF5GLNFSTPMAGOCHIV4FXI, infodb: database is locked\n\tstorj.io/storj/storagenode/orders.(*Sender).settle.func3:191\n\tstorj.io/storj/storagenode/orders.(*Sender).settle:213\n\tstorj.io/storj/storagenode/orders.(*Sender).Settle:142\n\tstorj.io/storj/storagenode/orders.(*Sender).runOnce.func1:127\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57\n--- order: failed to archive order as rejected: serial: 5LIDMBKXOFGDFAI425CYGCRSM4, infodb: database is locked\n\tstorj.io/storj/storagenode/orders.(*Sender).settle.func3:191\n\tstorj.io/storj/storagenode/orders.(*Sender).settle:213\n\tstorj.io/storj/storagenode/orders.(*Sender).Settle:142\n\tstorj.io/storj/storagenode/orders.(*Sender).runOnce.func1:127\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57\n--- order: failed to receive response: rpc error: code = Unavailable desc = transport is closing\n\tstorj.io/storj/storagenode/orders.(*Sender).settle.func3:191\n\tstorj.io/storj/storagenode/orders.(*Sender).settle:200\n\tstorj.io/storj/storagenode/orders.(*Sender).Settle:142\n\tstorj.io/storj/storagenode/orders.(*Sender).runOnce.func1:127\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57\n--- order: sending agreements returned an error: EOF\n\tstorj.io/storj/storagenode/orders.(*Sender).settle.func3:191\n\tstorj.io/storj/storagenode/orders.(*Sender).settle:221\n\tstorj.io/storj/storagenode/orders.(*Sender).Settle:142\n\tstorj.io/storj/storagenode/orders.(*Sender).runOnce.func1:127\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57"}
2019-07-17T04:12:04.203-0400	INFO	running on version v0.15.2
2019-07-17T04:27:04.179-0400	INFO	running on version v0.15.2
2019-07-17T04:42:04.163-0400	INFO	running on version v0.15.2

After restarting the storage node everything seems to be fine:

2019-07-17T04:50:30.167-0400	INFO	running on version v0.15.2
2019-07-17T04:50:30.190-0400	INFO	db.migration	Latest Version	{"version": 13}
2019-07-17T04:50:30.192-0400	INFO	vouchers	Checking vouchers
2019-07-17T04:50:30.194-0400	INFO	Node 15Y7KzGXxnjfo3wQDzLBxdkuYeyDc9AMiXpdHxNmsmsuLfyePb started
2019-07-17T04:50:30.207-0400	INFO	Public server started on [::]:28967
2019-07-17T04:50:30.207-0400	INFO	Private server started on 127.0.0.1:7778
2019-07-17T04:50:30.218-0400	INFO	vouchers	Requesting voucher	{"satellite": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs"}
2019-07-17T04:50:30.221-0400	INFO	vouchers	Requesting voucher	{"satellite": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S"}
2019-07-17T04:50:30.221-0400	INFO	vouchers	Requesting voucher	{"satellite": "118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW"}
2019-07-17T04:50:30.223-0400	INFO	vouchers	Requesting voucher	{"satellite": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6"}
2019-07-17T04:50:30.233-0400	INFO	piecestore:monitor	Remaining Bandwidth	{"bytes": 10995116277760}
2019-07-17T04:50:30.233-0400	WARN	piecestore:monitor	Disk space is less than requested. Allocating space	{"bytes": 1729663594496}
2019-07-17T04:50:30.327-0400	INFO	running on version v0.15.2
2019-07-17T04:50:30.446-0400	INFO	piecestore:orderssender.12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S	sending	{"count": 6}
2019-07-17T04:50:30.460-0400	INFO	piecestore:orderssender.118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW	sending	{"count": 4450}
2019-07-17T04:50:30.463-0400	INFO	piecestore:orderssender.12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs	sending	{"count": 7}
2019-07-17T04:50:36.296-0400	INFO	piecestore:orderssender.12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs	finished
2019-07-17T04:50:38.369-0400	INFO	piecestore:orderssender.12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S	finished
2019-07-17T04:50:40.252-0400	INFO	vouchers	Voucher received	{"satellite": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs"}
2019-07-17T04:50:43.829-0400	INFO	vouchers	Voucher received	{"satellite": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S"}
2019-07-17T04:51:09.991-0400	INFO	piecestore:orderssender.118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW	finished
2019-07-17T04:51:17.456-0400	INFO	vouchers	Voucher received	{"satellite": "118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW"}
2019-07-17T04:51:20.258-0400	INFO	vouchers	Voucher received	{"satellite": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6"}
2019-07-17T05:05:30.769-0400	INFO	running on version v0.15.2

We have seen that sometimes that happens, restarting should usually fix the issue as you could observe.