Release preparation v1.62

New release v1.62 already deployed on QA Satellite
This time there are a lot more changes
List of changes:

Changelog

Satellite

  • c8b72c8 satellite/{payments/billing,satellitedb}: refactor billing DB
  • 5f6892f satellite/console/consoleweb: early rejection in auth middleware
  • 92be1d8 satellite/payments/stripecoinpayments: storjscan invoice generation
  • 799b159 satellite/reputation: offset write times by random, not by satelliteID
  • ad37ea4 satellite/{web, console}: login captcha implemented
  • af6f556 web/satellite: navigation mobile adaptation
  • f562277 satellite/{web,console}: added string length limits for signup partner and promo params
  • ec72adb satellite/console: send email when user’s account gets locked
  • e0c863e web/satellite: update “deposit storj” message to use support link rather than email
  • c13a470 web/satellite: update dashboard chart data on project switch
  • 30727b9 web/satellite: add data delay info for new project dashboard
  • d76acda satellite/{web, console}: removed account locked statuses
  • 27f6fbd web/satellite: refactored modals to use common VModal component
  • e1a85fc web/satellite: added notification when storage limit is exceeded in file browser
  • 81327f6 web/satellite: Fix documentation URL (#5049)
  • 0f3da7f web/satellite: Added token transaction history to new payment methods flow (#4977)
  • c5abc0d satellite/metainfo: Homogenize object operations logs
  • e5877fa web/satellite: fix error with “drop files” box in file browser
  • 3f6bf64 satellite/console: Default NewProjectDashboard to “true”
  • 938efd7 satellite/metabase: Don’t stop on error expired & zombie objects
  • 9e64a87 satellite/metabase: measure delete object batch
  • 4c5466a web/satellite: update projects table component
  • 3afd7bc satellite/console: Reset passwords invalidates all current user sessions.
  • e8b716f satellite/analytics: Added track calls for user behavioral events (#5009)
  • 174d2ec satellite/{payments/billing,satellitedb}: Add retry logic to insert
  • eca1689 satellite/console: add wallet payments API
  • b398035 satellite/console/consoleweb: Log trace reqs with INFO
  • 917925b satellite/metabase: delete migrated zombie objects
  • 6cc2052 satellite: fix segment loop observers metrics
  • a416654 web/satellite: dashboard banners fixed
  • b7e9d04 web/satellite: common table component applied to file browser
  • 0550731 web/satellite: add S3 credential creation to quickstart dropdown
  • 4418216 Web/satellite: Create Access Flow Modal Refactor (#5015)
  • f507de6 satellite: in-memory ‘top’-like counter for project/partner
  • cff8158 satellite/console: add audit logs for failed logins
  • 8b338fb web/satellite: show guide on bucket page
  • c3e7b14 satellitedb: add salt to projects table
  • 4be219c Web/satellite: Add responsiveness to onboarding flow (#5065)
  • c552c34 web/satellite: make modals responsive
  • 2abe709 satellite/metabase: add metabase.GetObjectLatestVersion back
  • d910074 web/satellite Billing screen bug fixes (#5050)
  • 0ec28ca web/satellite: add token funds modal
  • 26bed35 web/satellite: update access grants table
  • ac24150 satellitedb: add wallet address index to storjscan_wallet table
  • 7299e9f satellite/console: Add missing monkit calls
  • f596f72 satellite/payments/storjscan: add billing update to chore
  • 8eec61d satellitedb: add timestamp index to billing table
  • ffd35ec web/satellite: move billing nav item into My Account dropdown
  • eb54774 web/satellite: fix mobile signup page
  • c82424c satellite/admin: add Burst field to Update Project Limits params
  • 286f68a web/satellite: fix page content bottom padding
  • 9dcef77 web/satellite: fix for billing screen
  • 4183726 web/satellite: improve access grants empty state
  • 99dee2d web/satellite: fix bucket page guide positioning
  • 0dcc0a9 satellite/reputation: reconfigure lambda and alpha
  • c59d0d4 satellitedb: add signup_captcha column to users table
  • 874e28e web/satellite: fix inactive buttons on billing page

Storagenode

  • cf92220 {satellite,storagenode}/gracefulexit: simplify limiter usage
  • e9692c5 storagenode/gracefulexit: remove unused interface
  • 4fdb81c storagenode/pieces: allow to configure initial piece scan (#5024)
  • 9a539c4 storagenode/storageusage: add interval_end_time, rename interval_start to timestamp
  • 25f8f67 storagenode/nodestats: retrieve storage usage starting from last day of previous month

Test

  • 53db7a8 testsuite/storjscan: add testsuite to test storjscan integration
  • a6ad86d testsuite/storjscan: add client claim wallet test
  • 379711c testsuite/storjscan: add client payments request test
  • 0036ec9 testsuite/storjscan: add chore test
  • 1f06387 private/apigen,cmd/apigentest: add tests for generated API code

Uplink

  • 04c0e5d cmd/uplink: fix access setup flags setup

General

  • 5a2e348 build: bump timeout for premerge/public
  • 731fecd private/apigen: rework request parameter handling
  • 37a4edb all: reformat comments as required by gofmt 1.19
  • 0698b1e go.mod: bump storj/private version
  • 96b88e2 build: fix conditional run of web builds
6 Likes

Great. And now, how do you activate / deactivate? Docker command or change config file?

And what possible problems can there be with a node that has the filewalker turned off?

Thanks

2 Likes

Are there any updates in this version, that somehow breaks fetching (some) data with curl ?

root@backup001:/root# curl -s 127.0.0.1:52000/api/sno/  (works)
{
  "nodeID": "1ko9N9LXiWsY9hcF85eeug2Y6gkxbEppWzNFAFa6n4ZuXrhMgR",
  "wallet": "0x165A1Da6F4d3c0555a9E1f94288A7e17BD14f7d1",
  "walletFeatures": [
    "zksync"
  ],
  "satellites": [
    {
      "id": "12tRQrMTWUWwzwGh18i7Fqs67kmdhH9t6aToeiwbo5mfS2rUmo",
      "url": "us2.storj.io:7777",
      "disqualified": null,
      "suspended": null,
      "currentStorageUsed": 5004544
    },
    {
      "id": "1wFTAgs9DP5RSnCqKV1eLf6N9wtk4EAtmN5DpSxcs8EjT69tGE",
      "url": "saltlake.tardigrade.io:7777",
      "disqualified": null,
      "suspended": null,
      "currentStorageUsed": 222742528
    },
    {
      "id": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6",
      "url": "ap1.storj.io:7777",
      "disqualified": null,
      "suspended": null,
      "currentStorageUsed": 286775808
    },
    {
      "id": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S",
      "url": "us1.storj.io:7777",
      "disqualified": null,
      "suspended": null,
      "currentStorageUsed": 1941721344
    },
    {
      "id": "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs",
      "url": "eu1.storj.io:7777",
      "disqualified": null,
      "suspended": null,
      "currentStorageUsed": 913410560
    },
    {
      "id": "12rfG3sh9NCWiX3ivPjq2HtdLmbqCrvHVEzJubnzFzosMuawymB",
      "url": "europe-north-1.tardigrade.io:7777",
      "disqualified": null,
      "suspended": null,
      "currentStorageUsed": 181933056
    }
  ],
  "diskSpace": {
    "used": 3559492608,
    "available": 1000000000,
    "trash": 94163200,
    "overused": 2653655808
  },
  "bandwidth": {
    "used": 716437248,
    "available": 0
  },
  "lastPinged": "2022-08-19T20:03:38.159757415+02:00",
  "version": "1.62.0-rc",
  "allowedVersion": "1.24.0",
  "upToDate": true,
  "startedAt": "2022-08-19T19:55:43.196547175+02:00",
  "configuredPort": "50000",
  "quicEnabled": true
}
root@backup001:/root# curl -s 127.0.0.1:52000/api/sno/satellite/12tRQrMTWUWwzwGh18i7Fqs67kmdhH9t6aToeiwbo5mfS2rUmo | jq
{
  "error": "consoleapi storagenode: console: sql: Scan error on column index 1, name \"24 * (at_rest_total / COALESCE((CAST(strftime('%s', interval_end_time) AS NUMERIC) - CAST(strftime('%s', LAG(interval_end_time) OVER (PARTITION BY satellite_id ORDER BY interval_end_time)) AS NUMERIC)) / 3600, 24))\": converting NULL to float64 is unsupported"
}

And on a side note - @BrightSilence earnings.py script also seems to be having issues, since the node db was upgraded from v53 to v54

root@backup001:/tmp/222# python ./earnings.py /tmp/222
Traceback (most recent call last):
  File "./earnings.py", line 323, in <module>
    for data in con.execute(query):
sqlite3.OperationalError: no such column: interval_start

Th3van.dk

2 Likes

Thanks for the heads up. I will implement a fix for this when it goes live on production nodes.

4 Likes

That might be a bug on the storage node side. I have reported it to @clement. Let’s see if we can fix this before the new storage node release goes out. To me, it looks like the root cause is that Stefans test satellite is empty and the storage node dashboard has issues displaying that on the storage node dashboard. If I select the QA satellite it looks all fine (besides the known issue with the last 2 days). Both satellites are running the latest version.

Could probably be fixed by using either NullFloat64 here (I noticed database/sql is already imported)

Or adding a fallback to 0 using COALESCE around at_rest_total here

I’d do it myself, but I don’t have a go dev environment set up. And I don’t like making untested PR’s (though the COALESCE is a pretty safe bet).

5 Likes

This is already great contribution. Thank you very much. @clement what are your thoughts?

1 Like

Great suggestion @BrightSilence. I’m adding new test cases to test for such instances and try out your suggestion as a fix.
Though I’m thinking it should already return an empty result set if there is no data for the stefanlite, and this error should not be thrown in the first place.

3 Likes

@BrightSilence the storage node rollout for v1.62.3 should start shortly. The binaries are already available on github. It looks like the storage node release still breaks your script.

It turns out the null values are already wrong on the satellite side. There is a fix available for the satellite but testing the fix would have delayed the satellite release too long. So we fixed just the storage node side to display a used space graph like last release. It is still lacking behind 2 days. Thats why I call it a quick and dirty fix.

At the end it is a risk vs benefit question. How much risk are we willing to accept in order to get the storage node dashboard working. This time we have choosen a low risk approach even if it is only a partial fix.

4 Likes

Thanks for the heads up, I am aware of this.
This is due to changed column names on the node end. It’s unrelated to the error here discussed here. I already have a change ready and waiting to publish that fixes it (it helps to have a testnet node that updates early). But it would break the script on older node versions, so I’m waiting until rollout begins to publish it.

I’m also working on making some changes to accommodate the new audit scoring threshold. Not sure if that will be in the same version, but it might be. I don’t think I need to do much for this.

3 Likes

There should be a table that tells you about the DB version. Shouldn’t that allow you to just use the old query for any node that isn’t updated and the new query for the nodes that have been updated? The value you need for this decision should in in the info.db. SELECT max(version) FROM versions;

1 Like

Yeah, that’s possible, but the script currently works with a single big query. It would require a bit more rework than I’m willing to do for maybe a few weeks of overlap.

I prefer to just see it as an extra incentive to update your node and keep the script more simple. It’s too much code as it is already, haha.

1 Like

Ready for testing :smiley:

1 Like

Yea I am also waiting to download your new version :smiley:

1 Like