Release preparation v1.66

Product Discussions STORJLINGS
1

Release v1.66 already deployed on QA Satellite

Changelog

Satellite

  • 6bb3ea3 satellite/metabase: add tests for ListVerifySegments
  • 47f9915 satellite/payments/storjscan: add info level log statement when a user successfully claims a wallet address.
  • fe3aede web/satellite: move mobile Billing nav item into My Account dropdown
  • 05e57ed web/satellite: see token payments history
  • e374356 satellite/audit: optimize loop observer
  • 5dc5f07 satellite/repair/checker: remove monitoring from fast methods
  • 91c3a45 satellite/accounting/nodetally: remove monkit call from observer
  • 6fcc5c5 satellite/metabase/segmentloop: remove unused monkit calls
  • 509bb1a web/satellite: optimized project selection
  • 480715e web/satellite: remove prefix from downloaded access grant contents
  • 4ab8031 cmd/satellite: combine draft invoice generation commands
  • 3d6c3e3 satellite/payments/stripecoinpayments: auto advance $0 invoices
  • 85f9dad satellite/payments/stripecoinpayments/service.go: add date to pay invoices command
  • 7f6f7e1 satellite: optional migration for integration tests
  • 9ddd20b satellite/projectaccounting: fix project usage right after the genesis
  • bbed10d satellite/{accounting/tally,metabase}: remove monkit for fast methods
  • f61507a satellite: include ‘unsafe’ in the new migration options
  • c8506cd satellite/metabase,cmd/tools/segment-verify: simplify interface
  • 4d9c913 satellite/metainfo: use multiple object versions internally
  • 6884b58 web/satellite: limit warning banner for free tier users added
  • ef04eb8 satellite/satellitedb: add columns in projects table
  • d39caf7 cmd/satellite: change invoicing flags to iso8601
  • 903ea38 web/satellite: increment login failed_login_count in sql
  • ff99392 web/satellite: update bucket share modals to match designs
  • db1409e satellite/metabase: use SUBSTRING with objects iterator
  • 11d1e62 satellite/metabase/segmentloop: don’t do rate limiting if disabled
  • 1c9f4d1 satellite/metainfo: more tests for CopyObject
  • 2e6b473 web/satellite: improved modal experience
  • 7736832 web/satellite: add right border to desktop navbar to match designs.
  • e5ac843 satellite/metainfo: delete pieces from nodes on object commit
  • 1887660 web/satellite: update custom HTML styling for registration page
  • 98fed4b {satellite/console,web/satellite}: get project salt from satellite
  • b23b3e0 web/satellite: rewrite VButton with composition API
  • fac638f web/satellite: rewrite VInput with composition API
  • 35f74b7 satellite/metabase: IterateLoopSegments accepts ranges
  • 419fb9c web/satellite: update color and add y-axis on bandwidth graph
  • a52f766 satellite/overlay: add email-sending functionality to overlay service
  • 3acd133 web/satellite: change token payment link to etherscan
  • fa287b8 satellite/metabase: added CollectBucketTallies
  • e014e88 satellite/payments: remove Deposit from Tokens interface
  • 34a8837 web/satellite: remove add storj token deposit code
  • c63a401 satellite/accounting/tally: bring back monkit call
  • d6f8efb web/satellite: color palette added
  • 149b590 satellite/{web, console}: removed old object flow
  • 56128b9 web/satellite: clear payment account details on logout
  • 2ed18a0 satellite/metainfo/piecedeletion: configurable threshold
  • a2ca443 satellite/overlay: send Node Online emails
  • 11df98a satellite: use evenkit instead of evenstat/top endpoint
  • eae71ae satellite/reputation: remove tautology in node status comparison
  • 08052b3 web/satellite: update date format
  • 8c8688c satellite/overlay: return email as part of NodeReputation
  • 4826e4a web/satellite: session timer reset added while uploading files added

Test

  • 14d0b0e testsuite/storjscan: add backfill payments test
  • c921cd5 docs/testplan: Adding a testplan for Token Payment Processor (#4672)
  • c256da5 docs/testplan: Testplan for Session Management (#4800)

Uplink

  • d632f23 cmd/uplink: add eventkit
  • 9c09a8d cmd/uplink: don’t send TODO as the instance id

Multinode

  • d800b51 cmd/multinode: generate identity when not provided

General

  • 4c374a2 cmd/tools/segment-verify: add tiny readme
  • c1817ab cmd/tools/segment-verify: a few fixes
  • ac6bb1e storage: experimental flag to turn off file sync
  • ea4b302 cmd/tools/segment-verify: fix piece id derivation
  • 5f01dad cmd/tools/segment-verify: add total progress indicator
  • 9e50d83 cmd/tools/segment-verify: add tool for summarizing log
  • 8916f2e cmd/tools/segment-verify: allow ignoring specific nodes
  • 142a04f cmd/tools/segment-verify: add connection pool
  • ff22fc7 all: fix deprecated ioutil commands
  • 8b70f96 all: fix nolint directives
  • 5cfd645 private/blockchain: fix Hex to add 0x prefix
  • dd60318 cmd/tools/segment-verify: use resolved ip
  • ea106f8 go.mod: bump storj/private
  • 0cfbd34 docs/blueprints: byte range multipart copy
  • a80a0eb cmd/tools/segment-verify: redial once rather than giving up
  • 22c0b0a cmd/tools/segment-verify: don’t mark node immediately offline
  • 4fd59fb go.mod: bump common
  • 2dccfdf certificate/authorization: add golden test for Group marshaling
  • 52ede93 mod: bump dependencies
  • 58a9c55 mod: bump dependencies
  • bcbf30a ci: make go compatibility check parallel
  • 4be0421 sdatellite/reputation: add overlay service to reputation service
  • 29aac75 build: provides earthfile for nightly build
  • 1837dc4 mod: bump dependencies
  • 1dd7dbe storj: bump storj common package version
4 Likes
2

Not sure if this is the right place to reply, but regarding:

Is it possible that this leads to files lost during a crash or power cut?
Is it possible for SNOs to opt out (i.e. keep the file sync always on)? What’s the default?

3

It’s an implementation of this idea, which makes a lot of sense under some circumstances. Very glad it becomes reality!

4

That was an interesting read. Great analysis!
In the end I think it comes down to the individual’s preference (and trust in their setup), as you and Stefan summarized in the thread.

So that’s why I’m curious if in this implementation it is configurable by the SNO and on or off by default.

5

image
image1067×84 6.88 KB

As you can see it mentions this is an experiment not meant for production. Devs would want to test this before it can negatively affect the network.

2 Likes
6

As long as it can be set to the previous way of working (fsync after write) it’s OK. I would not want to run my node with fsync disabled, that just invites trouble. I doubt that ingress is ever going to be so big that it would make sense disabling fsync.

7

Maybe something to throw in:

fdatasync() is similar to fsync(), but does not flush modified metadata unless that metadata is needed in order to allow a subsequent data retrieval to be correctly handled. For example, changes to st_atime or st_mtime (respectively, time of last access and time of last modification; see stat(2)) do not require flushing because they are not necessary for a subsequent data read to be handled correctly. On the other hand, a change to the file size (st_size, as made by say ftruncate(2)), would require a metadata flush.

The aim of fdatasync() is to reduce disk activity for applications that do not require all metadata to be synchronized with the disk.

This sounds like a middle thing that might help to reduce activity without compromising data consistency. So maybe the toggle should include an option for this setting.

8

You can mount the filesystem with ‘noatime’. Storj files are created, read and deleted, but not modified. Creating/deleting the file will probably write the metadata block anyway.

The exception is access to the databases though.

9

The operations that a node performs before the fsync() is exactly what would trigger a full sync within fdatasync(): ftruncate and, obviously, writing to file. In this specific location in the code fdatasync() is equivalent to fsync().

The code path that is modified in this commit is about creating files, so noatime won’t change anything here.