Release preparation v1.112

Product Discussions STORJLINGS
1

New release candidate v1.112 is alredy deployed on QA Satellite

Changelog

General

  • 4e8a495 all: ignore fmt.Fprint errors explicitly
  • f6750a1 all: fix format string
  • e3a1d94 shared/dbutil: rename CheckApplicationNAme → EnsureApplicationName
  • 3e15487 Jenkinsfile: split go cache for different test executions
  • 4840337 Jenkinsfile: fix where go cache for main should be used
  • b2689e9 ci: set abort transaction probability in spanner emulator
  • 811fbd3 ci: add git restore-mtime
  • 629f62f go.mod: bump dependencies (common,uplink)
  • b3aa67d release v1.112.1-rc

Satellite

  • fd1e23f satellite/satellitedb: spannerdb queries to storjscanpayments
  • 81f8427 satellite/durability: placement level durability stat
  • 85ea9c4 satellite/metabase: rename UseObjectLock → ObjectLockEnabledForProject
  • 696620a satellite/metainfo: don’t check upload limits for some projects
  • 7c2f1f3 satellite/metainfo/config: support overriding repair threshold for placement
  • 3ca9ebf web/satellite: update account onboarding
  • 2b22ccd satellite/analytics: don’t export unnecessarily
  • 403e17d satellite/satellitedb: use automatic structs for spanner
  • 35f2f06 satellite/satellitedb: spanner query to storagenodeaccounting
  • 158141c satellite/satellitedb: added spannerdb queries to nodeevents
  • c496f3a satellite/metainfo: fix error handling for GetBucketObjectLockConfiguration endpoint
  • ef5c586 satellite/satellitedb: spanner queries for apikeys
  • 8fbf18f satellite/satellitedb: Add Spanner support for Reverify Queue
  • 6d09055 satellite/metabase: change batching for delete expired objects
  • a37a0dc satellite/metabase: get rid of IterateLoopSegments.AsOfSystemTime
  • 09a6e27 satellite/metabase: establish that “first” db adapter will control node aliases
  • be14229 satellite/metabase: comment uses of metabaseDB.Now()
  • f7088bb satellite/metabase: make GetSegmentByPosition check all adapters
  • 2c68e78 satellite/metabase: fix IterateLoopSegments on Spanner
  • 2e4af60 satellite/metabase: make IterateLoopSegments work over multiple db adapters
  • 9a59c1b satellite/metabase: TestingSetObjectVersion for db adapters
  • a7a9ce8 satellite/metabase: TestingSetPlacementAllSegments for db adapters
  • 15138f1 satellite/metabase: split TestMigrateToLatest to db adapters
  • 8028c6b satellite/metabase: remove ListVerifySegments
  • 49a410f satellite/metabase: remove IterateLoopObjects
  • 9150e41 satellite/metabase: split MigrateToLatest to db adapters
  • 467afec satellite/satellitedb: Spanner support for project accounting
  • 466ad04 satellite/metabase: allow configuration of multiple db adapters
  • 2a56897 satellite/metabase: test flag to set projects which will use spanner
  • f110ee6 satellite/satellitedb: Spanner support for GetProjectDailyUsageByDateRange
  • 91e55d3 satellite/satellitedb: Spanner support for SaveTallies
  • 805d50c satellite/satellitedb: implement DeleteProjectBandwidthBefore for Spanner
  • 399ff97 satellite/satellitedb: add GetProjectSettledBandwidth for Spanner
  • 64aea78 satellite/metainfo/expirationdeletion: delete expired objects concurrently
  • 08a7042 satellite/satellitedb: Implement Users UpdateFailedLoginCountAndExpiration Spanner support
  • 3661332 satellite/satellitedb: Implement Users GetExpiredFreeTrialsAfter Spanner support
  • ec37829 satellite/satellitedb: Implement Users GetExpiresBeforeWithStatus Spanner support
  • 18eaccf satellite/satellitedb: Implement Users GetEmailsForDeletion Spanner support
  • 446cfed satellite/satellitedb: Implement Users GetUnverifiedNeedingReminder Spanner support
  • 9918299 satellite/gc/bloomfilter: add flag to exclude expired pieces
  • 272ef83 satellite/satellitedb: Implement Users UpdateVerificationReminders Spanner support
  • 420da66 satellite/satellitedb: Implement Users DeleteUnverifiedBefore Spanner support
  • ad4c19a satellite/satellitedb: add GetPagedWithInvitationsByProjectID for Spanner
  • 3f4fcc0 satellite/satellitedb: spannerdb query to verifyqueue
  • 32a20d4 web/satellite: add optional app selection to regular new access flow
  • 3b0ea48 satellite/metabase: adjust moving objects for Object Lock
  • 8c48c75 satellite/metabase: test moving with retention
  • 2c75e85 satellite/metainfo: adjust FinishMoveObject endpoint for Object Lock
  • dafa914 satellite/metabase: add more tests for moving locked objects
  • 3e5eecd satellite/metainfo: add TestEndpoint_CopyObjectWithRetention
  • c383e75 satellite/metainfo: add ErrObjectExpiration to ConvertMetabaseErr
  • 2f93b4e web/satellite: rework the approach of receiving object preview and map
  • 5a70e8c satellite/payments: remove all expired credit cards after successful new card addition
  • aa6f807 satellite/metabase: make FinishCopyObject’s errors consistent
  • 858ad84 satellite/satellitedb: enabled nodeapiversion.go spanner test cases and resolved issues in GetProjectSettledBandwidthTotal
  • 45da760 satellite/metabase: wrap Retention.Verifies in copy/move code
  • 64616dd satellite/metainfo: make FinishMoveObject respect Delete permission
  • 0edbaca satellite/satellitedb: Fix potential leak of database connections
  • 22288c8 satellite/metainfo: add TestEndpoint_MoveObjectWithRetention
  • acc0221 satellite/console: add restrictions to account deletion
  • 99e7ea8 satellite/nodeselection: support last_ip node attribute
  • 34a1020 web/satellite: design updates
  • ea8dc3f satellite/console: fix account deletion test flakiness
  • 0021aeb satellite: update separate console API configs
  • b1e3b09 web/satellite: fix version restore error handling
  • 06ff49e web/satellite: frontend account deletion changes
  • e906f1f satellite/{gc,audit}: don’t process expired segments
  • c8a2307 satellitedb: separate out console functionality to consoledb sub-package
  • f21dbc6 web/satellite: fix onboarding upload tooltip not disappearing
  • 171155c satellite/metabase: fix SetObjectRetention race condition
  • aa27976 satellite/nodeselection: support continent filters
  • f451073 satellite/metabase: fix data race when deleting locked objects
  • 4eb4091 satellite/metainfo: refactor simple validation of request fields
  • eda810d satellite/satellitedb: enabled resetpasstokens.go spanner test cases
  • c5e7020 satellite/metainfo: add UUIDsFlag for uuids list

Storagenode

  • 8f1bba6 storagenode/blobstore/filestore: remove some monkit monitoring

Test

  • f9df6a1 testsuite/playwright-ui: Skip upload/download test
  • ca1b014 testsuite/playwright-ui: re-enable object browser upload/download test
2 Likes
2

Thanks for the update! I noticed the flat file piece expiration store code was merged, but I don’t see it in the release notes. Did it make it into version v1.112?

1 Like
3

I don’t think so. It could be included only as an experimental, because we currently doesn’t have a migration from the databases to a plain files. At least I didn’t find it.
This is mean that all collected TTL records will not be used if you would switch to a new storage, only new TTL records would start to be added there.

4

Yes, I was wondering about migration as well. As the code is not included how would we be able to experiment with it? What exactly are the plans for this feature?

5

I think the simplest plan would be to enable it in the config. The best way - is to make it enabled by default, but for that the migration should be implemented.

6

Alright, so if I’m understanding correctly, existing nodes can’t enable this feature without a migration plan—otherwise, all that expired TTL data in the pieces database will just be hanging out, uncollected, right? And since it’s not included in the current release, it seems like we’re in a bit of a holding pattern.

So, what are the next steps? Will we need to work on a migration plan before this feature gets included in a future version? Any idea when we might see it added?

7

yes, and will be moved later to the trash by the retain process and deleted 7 days after. However, it would be up on you.

I have no information.

9

Don’t current BFs ignore TTL data, or am I misunderstanding something?

10

I understand the confusion. I attempted to explain this on another topic as well:

So if the expired piece is not added to the BF it will be removed from you node. Does this answer your question?

2 Likes
12

Not at all.

A TTL piece is uploaded 1 minute before the BF is generated. This piece will expire 40 days from now. You are saying that since it’s not included in the BF that will make its way to my node in 2 days, it will be deleted on the 9th (10th) day.

13

So let me clarify: A TTL piece that’s uploaded before the BF is generated will stick around in your node for 40 days, and during that time, the BF will include it since it hasn’t expired yet. Once those 40 days pass and the piece expires, it will no longer be part of the BF. If your node still holds the expired piece, the GC process will move it to trash, and 7 days later, it’ll be fully deleted. Does that clear things up a bit better?

2 Likes