Node with less than 500gb

[andrijajepro]

I have disk that has less than 500gb (electricity cost is less than what earnings are).
How to add that to my docker command (so i can run)?

[pasatmalo]

The minimum required space to run a node is 500GB + 50GB for overhead (Step 1. Understand Prerequisites - Storj Docs).

Because of this, if you attempt to run a node with an allocated space lower than 500GB the node will refuse to start.

[andrijajepro]

I know it’s possible (saw it on some post), but I’m not sure how to run

[pasatmalo]

You can find the guide on how to run a node in docker here:

Storage Node - Storj Docs

Regarding the post you mention where someone was able to run a node with less than 500GB of allocated space, I am not familiar with it, therefore I cannot help you with that.

Just be warned that nodes of these sizes as far as I know are not supported.

[andrew2.hart]

storage2.monitor.minimum-disk-space option in the config.yaml

[andrijajepro]

How can I add it in docker command?

[nerdatwork]

Welcome to the forum @andrijajepro !

@andrijajepro

I know it’s possible

As per current Supplier Terms & Conditions

If you run a node that’s less than 500 GB then you are in breach of the supplier terms and conditions.

[Roxor]

You must have cheap power! For me if a disk isn’t at least filled with 2TB of Storj data… it isn’t paying for it’s power/internet/enclosure costs. Really with TB/TiB conversion, filesystem overhead, and some spare space for the node I need HDDs 4TB or larger

[JWvdV]

@pasatmalo

Because of this, if you attempt to run a node with an allocated space lower than 500GB the node will refuse to start.

That’s not true.

@andrijajepro

How to add that to my docker command (so i can run)?

Something like:

docker run -d --restart unless-stopped --stop-timeout 300 \
                -p 28967:28967/tcp \
                -p 28967:28967/udp \
                -p 14002:14002 \
                -e WALLET="$sWallet" \
                -e EMAIL="$sEmail" \
                -e ADDRESS="$sAddress" \
                -e STORAGE="10GB" \
                --user $(id -u):$(id -g) \
                --mount type=bind,source="$sIDFolder",destination=/app/identity \
                --mount type=bind,source="$sNodeMnt",destination=/app/config \
                --name "storagenode" storjlabs/storagenode:latest \
                --storage2.monitor.minimum-disk-space="1MiB"

@nerdatwork

If you run a node that’s less than 500 GB then you are in breach of the supplier terms and conditions.

Depends, since all nodes behind one IP are seen as one, I don’t see that much of a problem with also running some nodes with less than 500GB. It’s the same as having some of those drives in a RAID0 config, which just increases the failure rate of the while node. I think this is a better solution in that case.

[andrijajepro]

@JWvdV

docker run -d --restart unless-stopped --stop-timeout 300 \
                -p 28967:28967/tcp \
                -p 28967:28967/udp \
                -p 14002:14002 \
                -e WALLET="$sWallet" \
                -e EMAIL="$sEmail" \
                -e ADDRESS="$sAddress" \
                -e STORAGE="10GB" \
                --user $(id -u):$(id -g) \
                --mount type=bind,source="$sIDFolder",destination=/app/identity \
                --mount type=bind,source="$sNodeMnt",destination=/app/config \
                --name "storagenode" storjlabs/storagenode:latest \
                --storage2.monitor.minimum-disk-space="1MiB"

It’s restarting, refusing to connect

[JWvdV]

Show the log?
Because at my place if I adapt a node or start one with this code, it does work without a glitch:

[andrijajepro]

It shows something like this:
2024-05-22 19:22:54,747 INFO exited: storagenode (exit status 1; not expected)
2024-05-22 19:22:57,752 INFO spawned: ‘storagenode’ with pid 37
2024-05-22 19:22:57,756 INFO spawned: ‘storagenode-updater’ with pid 38
Error: While parsing config: yaml: line 10: did not find expected key
Usage:

What does that mean?

[JWvdV]

Show your full code please?
And also a larger portion of your logs.
And also your first ~20 lines of the config.yaml-file
It seems to be that your config.yaml isn’t parseable or cannot be found.

[donald.m.motsinger]

@andrijajepro

Error: While parsing config: yaml: line 10: did not find expected key

That message couldn’t be clearer. Look in line 10 of your config.yaml. What’s written there?

[andrijajepro]

Config yaml few first lines:

how frequently bandwidth usage cache should be synced with the db

bandwidth.interval: 1h0m0s

how frequently expired pieces are collected

collector.interval: 1h0m0s

use color in user interface

color: false

server address of the api gateway and frontend app

console.address: 0.0.0.0:14002

path to static resources

console.static-dir: “”

the public address of the node, useful for nodes behind NAT

contact.external-address: “”

how frequently the node contact chore should run

contact.interval: 1h0m0s

protobuf serialized signed node tags in hex (base64) format

contact.tags: “”

Maximum Database Connection Lifetime, -1ns means the stdlib default

db.conn_max_lifetime: 30m0s

Maximum Amount of Idle Database connections, -1 means the stdlib default

db.max_idle_conns: 1

Maximum Amount of Open Database connections, -1 means the stdlib default

db.max_open_conns: 5

address to listen on for debug endpoints

debug.addr: 127.0.0.1:0

If set, a path to write a process trace SVG to

debug.trace-out: “”

Logs (just repeating):

2024-05-22 19:22:57,802 INFO exited: storagenode-updater (exit status 1; not expected)
Error: While parsing config: yaml: line 10: did not find expected key
2024-05-22 19:22:57,807 INFO gave up: storagenode-updater entered FATAL state, too many start retries too quickly
Usage:
storagenode run [flags]

Flags:
–bandwidth.interval duration how frequently bandwidth usage cache should be synced with the db (default 1h0m0s)
–collector.interval duration how frequently expired pieces are collected (default 1h0m0s)
–console.address string server address of the api gateway and frontend app (default “127.0.0.1:14002”)
–console.static-dir string path to static resources
–contact.external-address string the public address of the node, useful for nodes behind NAT
–contact.interval duration how frequently the node contact chore should run (default 1h0m0s)
–contact.tags signedtags protobuf serialized signed node tags in hex (base64) format
–debug.addr string address to listen on for debug endpoints (default “127.0.0.1:0”)
–edit-conf open config in default editor
–filestore.force-sync if true, force disk synchronization and atomic writes
–filestore.write-buffer-size memory.Size in-memory buffer for uploads (default 128.0 KiB)
–forget-satellite.chore-interval duration how often to run the chore to check for satellites for the node to forget (default 1m0s)
–forget-satellite.num-workers int number of workers to handle forget satellite (default 1)
–graceful-exit.chore-interval duration how often to run the chore to check for satellites for the node to exit. (default 1m0s)
–graceful-exit.min-bytes-per-second memory.Size the minimum acceptable bytes that an exiting node can transfer per second to the new node (default 5.00 KB)
–graceful-exit.min-download-timeout duration the minimum duration for downloading a piece from storage nodes before timing out (default 2m0s)
–graceful-exit.num-concurrent-transfers int number of concurrent transfers per graceful exit worker (default 5)
–graceful-exit.num-workers int number of workers to handle satellite exits (default 4)
–healthcheck.details Enable additional details about the satellite connections via the HTTP healthcheck.
–healthcheck.enabled Provide health endpoint (including suspension/audit failures) on main public port, but HTTP protocol. (default true)
-h, --help help for run
–identity.cert-path string path to the certificate chain for this identity (default “identity/identity.cert”)
–identity.key-path string path to the private key for this identity (default “identity/identity.key”)
–nodestats.max-sleep duration maximum duration to wait before requesting data (default 5m0s)
–nodestats.reputation-sync duration how often to sync reputation (default 4h0m0s)
–nodestats.storage-sync duration how often to sync storage (default 12h0m0s)
–operator.email string operator email address
–operator.wallet string operator wallet address
–operator.wallet-features wallet-features operator wallet features (default )
–pieces.delete-to-trash move pieces to trash upon deletion. Warning: if set to false, you risk disqualification for failed audits if a satellite database is restored from backup. (default true)
–pieces.enable-lazy-filewalker run garbage collection and used-space calculation filewalkers as a separate subprocess with lower IO priority (default true)
–pieces.write-prealloc-size memory.Size deprecated (default 4.0 MiB)
–preflight.database-check whether or not preflight check for database is enabled. (default true)
–preflight.local-time-check whether or not preflight check for local system clock is enabled on the satellite side. When disabling this feature, your storagenode may not setup correctly. (default true)
–retain.cache-path string path to the cache directory for retain requests. (default “config/retain”)
–retain.concurrency int how many concurrent retain requests can be processed at the same time. (default 5)
–retain.max-time-skew duration allows for small differences in the satellite and storagenode clocks (default 72h0m0s)
–retain.status storj.Status allows configuration to enable, disable, or test retain requests from the satellite. Options: (disabled/enabled/debug) (default enabled)
–server.address string public address to listen on (default “:7777”)
–server.debouncing-enabled whether to debounce incoming messages (default true)
–server.extensions.revocation if true, client leaves may contain the most recent certificate revocation for the current certificate (default true)
–server.extensions.whitelist-signed-leaf if true, client leaves must contain a valid “signed certificate extension” (NB: verified against certs in the peer ca whitelist; i.e. if true, a whitelist must be provided)
–server.peer-ca-whitelist-path string path to the CA cert whitelist (peer identities must be signed by one these to be verified). this will override the default peer whitelist
–server.peer-id-versions string identity version(s) the server will be allowed to talk to (default “latest”)
–server.private-address string private address to listen on (default “127.0.0.1:7778”)
–server.revocation-dburl string url for revocation database (e.g. bolt://some.db OR redis://127.0.0.1:6378?db=2&password=abc123) (default “bolt://config/revocations.db”)
–server.tcp-fast-open enable support for tcp fast open (default true)
–server.tcp-fast-open-queue int the size of the tcp fast open queue (default 256)
–server.use-peer-ca-whitelist if true, uses peer ca whitelist checking (default true)
–storage.allocated-bandwidth memory.Size total allocated bandwidth in bytes (deprecated) (default 0 B)
–storage.allocated-disk-space memory.Size total allocated disk space in bytes (default 1.00 TB)
–storage.k-bucket-refresh-interval duration how frequently Kademlia bucket should be refreshed with node stats (default 1h0m0s)
–storage.path string path to store data in (default “config/storage”)
–storage.whitelisted-satellites storj.NodeURLs a comma-separated list of approved satellite node urls (unused)
–storage2.cache-sync-interval duration how often the space used cache is synced to persistent storage (default 1h0m0s)
–storage2.database-dir string directory to store databases. if empty, uses data path
–storage2.delete-queue-size int size of the piece delete queue (default 10000)
–storage2.delete-workers int how many piece delete workers (default 1)
–storage2.exists-check-workers int how many workers to use to check if satellite pieces exists (default 5)
–storage2.expiration-grace-period duration how soon before expiration date should things be considered expired (default 48h0m0s)
–storage2.max-concurrent-requests int how many concurrent requests are allowed, before uploads are rejected. 0 represents unlimited.
–storage2.max-used-serials-size memory.Size amount of memory allowed for used serials store - once surpassed, serials will be dropped at random (default 1.00 MB)
–storage2.min-upload-speed memory.Size a client upload speed should not be lower than MinUploadSpeed in bytes-per-second (E.g: 1Mb), otherwise, it will be flagged as slow-connection and potentially be closed (default 0 B)
–storage2.min-upload-speed-congestion-threshold float if the portion defined by the total number of alive connection per MaxConcurrentRequest reaches this threshold, a slow upload client will no longer be monitored and flagged (default 0.8)
–storage2.min-upload-speed-grace-duration duration if MinUploadSpeed is configured, after a period of time after the client initiated the upload, the server will flag unusually slow upload client (default 10s)
–storage2.monitor.interval duration how frequently Kademlia bucket should be refreshed with node stats (default 1h0m0s)
–storage2.monitor.minimum-bandwidth memory.Size how much bandwidth a node at minimum has to advertise (deprecated) (default 0 B)
–storage2.monitor.minimum-disk-space memory.Size how much disk space a node at minimum has to advertise (default 500.00 GB)
–storage2.monitor.verify-dir-readable-interval duration how frequently to verify the location and readability of the storage directory (default 1m0s)
–storage2.monitor.verify-dir-readable-timeout duration how long to wait for a storage directory readability verification to complete (default 1m0s)
–storage2.monitor.verify-dir-warn-only if the storage directory verification check fails, log a warning instead of killing the node
–storage2.monitor.verify-dir-writable-interval duration how frequently to verify writability of storage directory (default 5m0s)
–storage2.monitor.verify-dir-writable-timeout duration how long to wait for a storage directory writability verification to complete (default 1m0s)
–storage2.order-limit-grace-period duration how long after OrderLimit creation date are OrderLimits no longer accepted (default 1h0m0s)
–storage2.orders.archive-ttl duration length of time to archive orders before deletion (default 168h0m0s)
–storage2.orders.cleanup-interval duration duration between archive cleanups (default 5m0s)
–storage2.orders.max-sleep duration maximum duration to wait before trying to send orders (default 30s)
–storage2.orders.path string path to store order limit files in (default “config/orders”)
–storage2.orders.sender-dial-timeout duration timeout for dialing satellite during sending orders (default 1m0s)
–storage2.orders.sender-interval duration duration between sending (default 1h0m0s)
–storage2.orders.sender-timeout duration timeout for sending (default 1h0m0s)
–storage2.piece-scan-on-startup if set to true, all pieces disk usage is recalculated on startup (default true)
–storage2.retain-time-buffer duration allows for small differences in the satellite and storagenode clocks (default 48h0m0s)
–storage2.stream-operation-timeout duration how long to spend waiting for a stream operation before canceling (default 30m0s)
–storage2.trust.cache-path string file path where trust lists should be cached (default “config/trust-cache.json”)
–storage2.trust.exclusions trust-exclusions list of trust exclusions
–storage2.trust.refresh-interval duration how often the trust pool should be refreshed (default 6h0m0s)
–storage2.trust.sources trust-sources list of trust sources (default https://www.storj.io/dcs-satellites)
–version.check-interval duration Interval to check the version (default 15m0s)
–version.request-timeout duration Request timeout for version checks (default 1m0s)
–version.run-mode run-mode Define the run mode for the version checker. Options (once,periodic,disable) (default periodic)
–version.server-address string server address to check its version against (default “https://version.storj.io”)

Global Flags:
–color use color in user interface
–config-dir string main directory for storagenode configuration (default “config”)
–db.conn_max_lifetime duration Maximum Database Connection Lifetime, -1ns means the stdlib default (default 30m0s)
–db.max_idle_conns int Maximum Amount of Idle Database connections, -1 means the stdlib default (default 1)
–db.max_open_conns int Maximum Amount of Open Database connections, -1 means the stdlib default (default 5)
–debug.trace-out string If set, a path to write a process trace SVG to
–defaults string determines which set of configuration defaults to use. can either be ‘dev’ or ‘release’ (default “release”)
–identity-dir string main directory for storagenode identity credentials (default “identity”)
–log.caller if true, log function filename and line number
–log.custom-level string custom level overrides for specific loggers in the format NAME1=ERROR,NAME2=WARN,… Only level increment is supported, and only for selected loggers!
–log.development if true, set logging to development mode
–log.encoding string configures log encoding. can either be ‘console’, ‘json’, ‘pretty’, or ‘gcloudlogging’.
–log.level Level the minimum log level to log (default info)
–log.output string can be stdout, stderr, or a filename (default “stderr”)
–log.stack if true, log stack traces
–metrics.addr string address(es) to send telemetry to (comma-separated) (default “collectora.storj.io:9000”)
–metrics.app string application name for telemetry identification. Ignored for certain applications. (default “storagenode”)
–metrics.app-suffix string application suffix. Ignored for certain applications. (default “-release”)
–metrics.event-addr string address(es) to send telemetry to (comma-separated IP:port or complex BQ definition, like bigquery:app=…,project=…,dataset=…, depends on the config/usage) (default “eventkitd.datasci.storj.io:9002”)
–metrics.event-queue int size of the internal eventkit queue for UDP sending (default 10000)
–metrics.instance-prefix string instance id prefix
–metrics.interval duration how frequently to send up telemetry. Ignored for certain applications. (default 1m0s)
–monkit.hw.oomlog string path to log for oom notices (default “/var/log/kern.log”)
–tracing.agent-addr string address for jaeger agent (default “agent.tracing.datasci.storj.io:5775”)
–tracing.app string application name for tracing identification (default “storagenode”)
–tracing.app-suffix string application suffix (default “-release”)
–tracing.buffer-size int buffer size for collector batch packet size
–tracing.enabled whether tracing collector is enabled (default true)
–tracing.interval duration how frequently to flush traces to tracing agent (default 0s)
–tracing.queue-size int buffer size for collector queue size
–tracing.sample float how frequent to sample traces

2024-05-22 19:22:57,812 WARN received SIGQUIT indicating exit request
2024-05-22 19:22:57,813 INFO waiting for storagenode, processes-exit-eventlistener to die
2024-05-22 19:22:57,814 INFO stopped: storagenode (exit status 1)
2024-05-22 19:22:57,815 INFO stopped: processes-exit-eventlistener (terminated by SIGTERM)

[JWvdV]

Please, put your stuff betwoon quotes… 3x ` at the beginning and the end (on a new line).

And again, please show your full code (docker run statement).

[Roxor]

It will be counting the non-zero/non-comment lines. Go into your confg.yaml directory and run this:

grep . config.yaml | grep -v "^#"

That should show the 10th line with the problem.

[andrijajepro]

’
console.address: 0.0.0.0:14002
contact.external-address: “”
healthcheck.details: false
healthcheck.enabled: true
identity.cert-path: identity/identity.cert
identity.key-path: identity/identity.key
log.level: info
operator.email: “”
operator.wallet: “”
operator.wallet-features: “”
server.address: :28967
server.private-address: 127.0.0.1:7778
storage.allocated-bandwidth: 0 B
storage.allocated-disk-space: 2.00 TB
storage2.monitor.minimum-disk-space: 5.00 GB
version.server-address: https://version.storj.io
’
This config.yaml doesnt match the docker run command

[andrijajepro]

I run docker this way:
'docker run -d --restart unless-stopped --stop-timeout 300 -p 28967:28967/tcp -p 28967:28967/udp -p 14002:14002 -e WALLET=“0x582e0Fcxxxxxxxxxxxxx” -e EMAIL=“(mymail)@gmail.com” -e ADDRESS=“89.213.xxx.xxx:28967” -e STORAGE=“10GB” --mount type=bind,source=/storj/identity/storagenode,destination=/app/identity --mount type=bind,source=/storj/data,destination=/app/config --name “storagenodereal” storjlabs/storagenode:latest --storage2.monitor.minimum-disk-space=“1MiB” \ –operator.wallet-features=zksync
’

[JWvdV]

@andrijajepro

Error: While parsing config: yaml: line 10: did not find expected key

This is the problem.

But again, you should show us your code on the docker statment.
Besides, not a single `, but triple!

You should end all lines of your docker statement with
See my docker statement.