Release preparation v1.114

Product Discussions STORJLINGS
1

Changelog

General

  • 6706e72 Makefile: generate go1.20 modfile for storagenode
  • 73bfc36 cmd/tools/piecestore-benchmark: bump order expiration from 1h to 24h
  • d7e1dde go.mod,ci: update Go version
  • ccfca98 shared/dbutil/sqliteutil: move sqlite specific WithTx
  • b13de5f Makefile: use docker to run the script
  • 3665b7c shared: import bloomfilter,location,nodetag,processgroup,strictcsv from common
  • 30139a1 cmd/scripts: use specific version for development builds
  • 1c8cfdf Makefile: forgot to add image name
  • dba2d2e go.mod: bump storj.io/common, storj.io/uplink
  • 5d59841 build: (attempt to) fix binary build
  • d21ee8c shared/dbutil/spannerutil: Fix doc comment
  • 8e5c437 scripts/start-spanner-emulator: increase max databases per instance
  • 0f14a96 release v1.114.0-rc

Satellite

  • 81ae547 satellite/console/ā€¦/consoleapi: fix project deletion test flakiness
  • 9e96c7b web/satellite: fix gallery view missing dialogs
  • 83d5601 satellite/repair/repair_test: add test for overridden RS placement
  • 473d58a satellite/metainfo: use new object lock status codes
  • 6f3ef7f satellite/nodeselection: add EU except DE region in tests
  • a2da5e5 web/satellite: update lucide icons dependency
  • 256483d satellite/satellitedb: GetBucketTotals, GetProjectBandwidth Project Accounting Spanner Implementation
  • 4e148b2 satellite/metabase: Adjust PutObject endpoint (governance & legal hold)
  • 804c8a0 satellite/satellitedb: Enable remaining users tests
  • c59b44f web/satellite: fix account setup bug
  • 0090e59 web/satellite: make versions view default for versioned buckets
  • 0efa935 satellite/metainfo: reduce BeginSegment response size
  • 6733306 satellite/satellitedb: enabled storjscanwallets.go spanner test cases
  • e9d783c web/satellite: add new versions delete dialog to file preview
  • 0a300a3 web/satellite: update dependencies
  • 195535c web/satellite: fail the build on typescript errors
  • dcfef6e web/satellite: update ui copy
  • 2284a09 satellite/metainfo: update validateRetention to respect governance mode
  • 8bc3abf web/satellite,satellite/console: Update managed encryption behavior
  • 7595753 web/satellite: Rename ā€œuseTrialCheckā€ composable to ā€œusePreCheckā€
  • 6ed7701 satellite/{metabase,metainfo}: update SetObjectRetention errors
  • f12fd2a satellite/metainfo: change RPC code for suspending a locked bucket
  • 708ff7d satellite/metainfo: Enable valid test
  • 9e8ca2f satellite/metabase: update error when getting OL info of pending object
  • 604f30e satellite/satellitedb: Fix stripe payments TestUpdateGetPackage test time comparison
  • 0f33088 web/satellite: prevent locked object delete in file preview dialog
  • 43fd764 satellite/payments/accountfreeze: Fix time passed into warn email
  • 9e8a091 web/satellite: add OL-related info to create bucket flow
  • acc92a9 satellite/metainfo: revert ā€œreduce BeginSegment response sizeā€ Changes
  • c141578 satellite/metainfo: Adjust PutObject endpoint (governance & legal hold)
  • 3db51f8 satellite/nodeselection: upgrade success tracker interface
  • 026bb42 satellite/{metabase,metainfo}: update SetObjectLegalHold delete marker error
  • 6e269d8 satellite/metabase: replace spanner ARRAY_INCLUDES with BatchUpdate
  • 49acbc8 satellite/metabase: make CommitInlineObject respect governance, legal hold
  • 36ce500 satellite/metainfo: make CommitInlineObject respect legal hold
  • dea5d10 satellite/metabase: make SetObjectRetention respect governance, legal hold
  • 240cfc9 satellite/metainfo: make SetObjectRetention respect governance mode
  • cc7edb4 satellite/metainfo: fix flaky test on Spanner
  • 73187b9 satellite/repair/repairer: emit event for irreparable segments
  • b4b86df satellite/metainfo: use new Object Lock RPC status codes
  • a7c56f0 satellite/nodeselection: add a tracker for number of pieces
  • b260dca satellite/console: fix erroneous trial freeze escalation
  • 54cd235 web/satellite: further improve bulk object deletes
  • 68356ce cmd/satellite: debug logging for satellite repair-segment
  • 9a3c5d2 satellite/metabase: remove test segment commit modes
  • a134dd3 web/satellite: add blur to session expired dialogā€™s overlay
  • f253bcc web/satellite: ui ux improvements
  • fe4dfbd satellite/metabase/rangedloop: extend inline segment definition
  • 818b60f web/satellite: truncate file version ID on small screens
  • 438f24f web/satellite: prevent ā€˜suspend versioningā€™ on bucket with lock enabled
  • 6ff93bd web/satellite: update lock version dialog
  • 68f1bd4 web/satellite: show browser view switch in versions mode
  • 4a35489 web/satellite: donā€™t use bucket force delete if OL is enabled for a bucket
  • ad06cf0 satellite/{console, web}: adjust check DNS error handling
  • 6b5711a web/satellite: change ā€˜fileā€™ to ā€˜objectā€™
  • f14c729 web/satellite: clean up unused vars
  • f17d48c satellite/satellitedb: Fix spanner implementation of QueryStorageNodeUsage
  • 6f2938d web/satellite: fix prompt for passphrase dialog show up (special scenario)
  • 356ed3e web/satellite: updated links handling for versioning and OL dialogs in project settings
  • 1e585b2 satellite/metabase: clear columns on segment override while upload
  • 29f5596 satellite/metabase: use migrate.Migration for spanner

Storagenode

  • 345b83a cmd/storagenode-benchmark: support custom TTL settings for uploaded files
  • ba68d91 storagenode/pieces: read+delete piece expirations from both stores
  • 66057fb storagenode/piece: enable flat expiration store by default
  • baa28f4 storagenode/storagenodedb: donā€™t create useless transactions in migration
  • a0a2f6f storagenode/piecestore: put size into piece expiration store on upload
  • de176d7 storagenode/trust: Update trusted satellites URL
  • 7b4be35 storagenode/monitor: support fully dedicated disks with simplified space calculations
  • 21d58e7 storagenode/pieces: save-state-resume feature for used space filewalker

Uplink

  • a8edb6e cmd/uplink: avoid materializing recursive copy iterator
  • 4626543 cmd/uplink/cmd_ls: donā€™t buffer entire listing into ram just for tab alignment
2 Likes
2

I would like to use this.

Do I set DedicatedDisk=true ?

4 Likes
3

Same here.
What are the other implications? In the commit it is mentioned node dashboard will show fake numbers - I assume in regards to free space?
And does this work with LVM as well, or how does the node determine on which volume the mounted folder resides?
And is there a way, when activated, to get the free space the node thinks the volume has?
Thank you.

4

Iā€™d like to try it too, but can you tell me exactly what role it plays? Thank you for telling me the difference between using it and not using it.

5 
storage2.piece-scan-on-startup: false
storage2.monitor.dedicated-disk: true

Garbage collection, trash cleanup and TTL cleanup will still run a stats call. So the improvement is limited to not having to run used space calculation even when the numbers on the storage node dashboard are wrong. With this flag the used space value doesnā€™t matter and the node will continue to accept uploads as long as there is enough free space available.

3 Likes
6

I personally donā€™t like the filewalker, which makes too much noise. Iā€™m hoping that DedicatedDisk will turn off that

3 Likes
7

storage2.monitor.dedicated-disk does not show up in a fresh storagenode setup config.yaml

8

Thats correct: storagenode/monitor: support fully dedicated disks with simplified spā€¦ Ā· storj/storj@7b4be35 Ā· GitHub

9

Why hide it? The only way people will find out about it is if they see it on the forum.

10

Was the reason the hard drive kept making noises because of filewalker??? I too was bothered by the sound of the hard disk being read, so Iā€™m looking forward to it if it can be quiet!

11

I donā€™t understand that question. The line I linked also contains the information that it is experimental. I would split the storage node community into 2 groups. One group that is experienced enough to run experimental features and fully understands the risk. And the other group that would just turn it on blindly and is unprepared for any side effects. So hidden ist just an additional filter to make sure this feature gets enabled on the nodes that can deal with it. If it turns out that this feature doesnā€™t work you would ask why make it available for everyone. So thats why it is hidden.

(I wouldnā€™t call it hidden. It just forces me to read the code in order to enable it. Feels more like a fully visible feature just with an extra ā€œread the manualā€ requirement to get to it.)

3 Likes
12

I donā€™t understand that reply. Badger is also experimental (with breaking changes that prevent nodes from starting if it gets corrupted), yet itā€™s in the config.yaml.

2 Likes
13

The difference is badger cache is relative low risk.

2 Likes
14

Maybeā€¦

storage2.monitor.dedicated-disk:false
storage2.monitor.dedicated-disk:true
storage2.monitor.dedicated-disk:true-trash-scan-off

The true trash scan off option would just say ā€œtrash included in disk usageā€ or N/A. And not scan trash

:smiley:

15

Whatā€™s the point of trash scan off? When should it be used?
Dosenā€™t this make the reported space wrong?

16

The reported space can be wrong anyway, so I could live with it

17

1.114 suffers from high memory usage (graph shows memory leak to me).

Screenshot_20240928_231732
Screenshot_20240928_2317321305Ɨ375 84.3 KB

Restarting storagenode brings it back to normal and it starts climbing again.

Edit: Confirmed by a second graph (01:30 was a restart to test without pieces.flat-expiration-include-sq-lite):

Screenshot_20240929_102532
Screenshot_20240929_1025321302Ɨ378 85.5 KB

Edit2: at the request of another member: the node takes up 16GB or RAM (yes, one process).

2 Likes
18

3 2 1 ā€¦ Alex denying everything :wink:

1 Like
19

Sorry for that. However, itā€™s not a released version. You may test it for the QA satellite though

P.S.
Do you know why is it called a Release Candidate (-rc), but not a Release and even not a Pre-release?

Disclose:
because itā€™s not considered for a release, until itā€™s fully tested.
So, thanks for a feedback! Itā€™s really helpful! :heart:
meaning, that I shared it with the team :slight_smile:

3 Likes
20

Beta release: we added a new feature/fixed a bug. We think the change needs testing.
Alpha release: we took your suggestions and implemented them, fixing what was broken in the beta release. We are pretty sure itā€™s ok, but it still needs more testing.
Release Candidate: Barring any last minute bugs, this code should be identical to the production release (=a candidate for being released)
Production: we fixed what we could/identified in the previous releases, and we now consider this as OK to release to the public as a stable version.

Edit: see below

1 Like