Release preparation v1.90

Product Discussions STORJLINGS
1

New release candidate is already deployed to QA Satellite

List of changes:

Changelog

General

  • a14a181 Update logo in README.md (#6354)
  • bec981a gc-bf: add live count observer to loop
  • 9d7ef17 private/apigen: Fix code generation for slices & arrays
  • fd83585 go.mod: bump stripe-go from v73 to v75
  • ac86eb3 private/apigen: Fix bug TypeScript optional class members
  • 956109a private/apigen: Rename Endpoint fields
  • 71c9547 private/apigen: Don’t allow certain types for Request/Response
  • 47f3449 private/apigen: Fix TS generator to apply naming conventions
  • 3193ff9 private/apigen: Don’t force casing for API group name/prefix
  • 99c4359 private/apigen: Avoid clashes of types names
  • 05901aa cmd/tools/tag-signer: fail for comma separated tags
  • cfbb5da go.mod: update storj.io/common
  • 386a978 private/apigen: Validate param types
  • 3019471 {Jenkinsfile, Makefile}: update Go to 1.21.3

Satellite

  • 1e3da9f web/satellite/vuetify-poc: add upgrade account dialog
  • f3dbeed web/satellite/vuetify-poc: add upgrade options step
  • a5c1d9a web/satellite/vuetify-poc: add card and token option steps
  • 1d1f881 web/satellite: update wording of limit increase success notification
  • f40954c satellite/admin: return burst limit in API response
  • b3d12a2 web/satellite/vuetify-poc: include project and route name in page title
  • 41799ef web/satellite/vuetify-poc: add pricing plan steps
  • 4737e91 satellite/metabase: simplify main commit object query
  • e5f6be2 web/satellite/vuetify-poc: prepend icons to dashboard card titles
  • ae91fa3 {satellite/console,web/satellite}: change invite text in register page
  • 46ee1c1 satellite/{payment,console} add pending deletion user status
  • c34202f web/satellite/vuetify-poc: localize big numbers
  • 4dbf26e web/satellite/vuetify-poc: shorten project ID in URLs
  • 33fe731 satellite/{payment,console} rename freeze events and add violation event
  • c6436e1 satellite/metabase: rename BeginObjectExactVersion method
  • 5295afb web/satellite: use modal instead of old create project view
  • 25ffa3a satellite/analytics: stop sending normal account activity events directly to Hubspot
  • 4d5335d web/satellite: use project limits for upgrade modal
  • 2f19636 go.mod,satellite/payments/stripe: bump stripe-go from v72 to v73
  • a7c246b web/satellite: reworked delete bucket modal
  • eddfacc web/satellite/vuetify-poc: remove project member feature
  • 2c76c1e satellite/satellitedb: alter bucket_metainfos primary key
  • 487ac17 web/satellite/vuetify-poc: updates for dashboard limit cards
  • 512e063 web/satellite: ui updates (#6366)
  • 8a1bedd satellite/payments/{billing,stripe}: handle pending invoice payments
  • a06735c satellite/gracefulexit: add missing test cases
  • 6304046 satellite/nodeselection: read email + wallet from db to SelectedNode
  • a2acf35 satellite/satellitedb: Remove dbx references to partner_id
  • 95958a3 web/satellite/vuetify-poc: emit success notification on access grant delete
  • 7b50ece satellite/admin: add missing test assert to project geofence tests
  • 27a13ef satellite/console/consoleweb/consoleapi: fix defer in loop
  • 5d28639 web/satellite/vuetify-poc: improve the share dialog
  • 22ad017 web/satellite/vuetify-poc: implement browser onboarding flow
  • e072b37 satellite/console: add endpoint to request project limit increase
  • 65f14ec web/satellite: prevent row action for project owner
  • 7f02b73 cmd/satellite/main: complete/fail pending payments cmd
  • 60bf953 web/satellite/vuetify-poc: add client-side sorting for buckets table
  • 3ca626a web/satellite: send more analytics events for onboarding selection
  • 3216674 satellite/payments/accountfreeze: Increase exclusion threshold
  • 013ddbc web/satellite: fixed balance inconsistency for billing tabs
  • 1c0b23a web/satellite: ensure “s/” prefix in path for linksharing urls
  • d60e0f0 web/satellite: fix locked files calculation when pagination is enabled
  • 0cc0420 satellite/console: add limit increase errors to getUserErrorMessage
  • dd3779a web/satellite/vuetify-poc: invite through all projects dashboard
  • 99ba88a web/satellite/vuetify-poc: allow adding coupons
  • b6b9ccc web/satellite/vuetify-poc: object browser improvements
  • 0db898b web/satellite/vuetify-poc: tiny changes for project dashboard
  • 211659b web/satellite: update and optimize error page (#6391)
  • 6308da2 satellite/{payment,console,analytics} extend freeze functionality for violation freeze
  • de4559d satellite/repair/checker: optimize processing, part 1
  • e3e3037 satellite/repair/checker: optimize processing, part 2
  • 7038dde web/satellite/vuetify-poc: fixed styling of stripe input
  • 4e0f062 web/satellite/vuetify-poc: support PDF previewing
  • f41e117 web/satellite/vuetify-poc: fix object browser pagination inside folders
  • 863c96b web/satellite: update signup pages plus logo (#6409)
  • db3578d satellite: durability rangeloop observer for monitoring risks
  • 3f6dd44 web/satellite: fix object count calculation inside folders
  • 335ebd6 satellite/console: rate limit project invitation endpoint (#6417)

Storagenode

  • a2c162d storagenode/trust: ensure trust pool updates satellite status on Refresh
  • c33475f storagenode/gracefulexit: fix flaky test (hopefully)
  • d432a71 storagenode/piecestore: notify if download piece was restored from trash
  • 9186365 storagenode/pieces: more granular io and hashing statistic
  • db7c6d3 storagenode/orders: archive unsent order for untrusted satellite

Test

  • b28439b cmd/tools/placement-test: cli to test placement configuration
  • bde3e48 docs/testplan: Adding a testplan for GE revamp (#6355)
3 Likes
2

Also can You add info in log about filewalker?
The goal is to know if the filewalker is still going so user can let him finish its work, and optionally to see whats its progress in % if possible.
That would help try different machine setups and understand its effect on speed of a filewalker.

Example: Currently i can’t check that. I want to stop the machine, to add more RAM, but i see the HDD is probably counting files for filewalker(my assumption, coz not much upload, or download, the node is full), if normal filewalker is running (not lazyfilewalker) then i don’t want to interrupt for various reasons, also to not waste its work and the time needed for that. Because i noticed for windows, to count all files in my E:\storj folder, it takes 6 days, for it is over 15 milion filles in there, so the filewalker got to go thru same work i guess.
(This is classic PC build, 1 SSD, 1 HDD, connnected via sata3, all NTFS. Windows 10.
Storagenode Windows GUI is installed on ssd with logs, and files are on HDD.)

V1dv932

4

Do you not see any log entries like below ?

2023-10-18T06:54:04Z    INFO    lazyfilewalker.used-space-filewalker    starting subprocess     {"process": "storagenode", "satelliteID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs"}
2023-10-18T06:54:04Z    INFO    lazyfilewalker.used-space-filewalker    subprocess started      {"process": "storagenode", "satelliteID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs"}
2023-10-18T06:54:04Z    INFO    lazyfilewalker.used-space-filewalker.subprocess Database started        {"process": "storagenode", "satelliteID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "process": "storagenode"}
2023-10-18T06:54:04Z    INFO    lazyfilewalker.used-space-filewalker.subprocess used-space-filewalker started   {"process": "storagenode", "satelliteID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs", "process": "storagenode"}
2023-10-18T06:59:59Z    INFO    lazyfilewalker.used-space-filewalker.subprocess used-space-filewalker completed {"process": "storagenode", "satelliteID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs",
2023-10-18T07:00:01Z    INFO    lazyfilewalker.used-space-filewalker    subprocess finished successfully        {"process": "storagenode", "satelliteID": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs"}
2 Likes
5

Yes, im getting

lazyfilewalker.used-space-filewalker    starting subprocess

long time not saw “completed” tho:

lazyfilewalker.used-space-filewalker.subprocess used-space-filewalker completed

the problem is, for example 13-10-2023 i saw it started in logs, and then, after receiving a shutdown, i saw it stopped with error obviously, bcoz node restart interrupted.
The problem is after that restart there was no “starting subprocess” again, but i guess its ongoing, because of HDD work, but no mention in log it started, but im pretty sure, when “Stop/Shutdown request received” occurs, i will see right away in log that filewalker faild, subbprocess exited with status error, that mention in log i see always after shutdown.
(info never skipped in log, if filewalker was running, but then its too late, interrupted obviously)

So its a bit confusing it may not be precise if the node currently is in filewalker process.

mayby there could be sporadically info, like collector, or orders every 1h or so sometiems i see, about progress of filewalker? like
for exampe something like: filewalker status (running from 13-10-2023)
so i know for sure somethign is running and counting files, and not to interrupt now

6

You may check it on the debug port

7

Thank You, but

under any of those below:
localhost:5999/mon/funcs
localhost:5999/mon/ps
(it got only (…)“storj.io/private/process.root() (elapsed: 57m28.4370041s)”
localhost:5999/mon/stats
localhost:5999/debug/pprof/goroutine?debug=2 (or debug=1 or debug=3)

i can’t see a mention about filewalker process running: (search words: file, space, walk, used)
and i have confirmation from storagenode.log that the process started and its ongoing, sooo yeahh

8

Ok, I asked the team, maybe they have an idea, where is it now.

9

image
image798×953 35.7 KB

Did you add debug port to the config file or the docker run command ?

1 Like
10

i guess so, i added a line in config.yaml:
debug.addr: 127.0.0.1:5999

under mon/func i got
sync2.Copy
process.root
checker.(*Client).all
piecestore.live-request
Only those 4

11

Add it to the docker run command instead of config file.

1 Like