Nightly / Intermittent Edge Mirror Nodes for Idle Consumer Hardware

Hi everyone,
I wanted to throw a concept out there for discussion regarding network architecture and how we might tap into a massive, underutilized pool of edge capacity: intermittent or nightly mirror nodes.
While traditional Storj nodes require high reliability, 24/7 uptime, and public IP stability to handle primary data storage and strict audit schedules, a vast amount of consumer hardware sits plugged in, connected to wired LAN/unmetered internet, and completely idle every single night.
The Problem / Opportunity
Millions of users have secondary devices, old Android hardware, or low-power setups combined with large auxiliary storage (like high-capacity SD cards or spare SSDs) that they plug in overnight while charging. Right now, this capacity is entirely locked out of decentralized storage networks because the architecture demands continuous availability.
How an Intermittent Mirror Model Could Work (Concept):
Secondary / Cold Data Redundancy: Instead of holding primary shards that require strict 24/7 audits, these nodes could operate as localized read-only or secondary cached mirrors specifically for cold or less-frequent data.
Defined Charging & Connection Windows: Nodes would automatically activate only when specific conditions are met—such as being connected to power, wired Ethernet, and running during preset local off-peak hours (e.g., 11 PM to 6 AM).
Adjusted Reputation / Audit Metrics: The network protocol could treat these as “transient” or “edge-cached” mirrors with relaxed uptime metrics, preventing them from being penalized for going offline during daytime hours when users take their devices off the charger.
Why It Might Be Useful:
Edge Proximity: Increases geographic distribution and localized caching for regional requests during peak nighttime hours.
Low-Barrier Participation: Lowers the barrier to entry for hobbyists who want to repurpose spare hardware without running power-hungry dedicated servers 24/7.
I’d love to hear thoughts from the core team and community developers: Is there any architectural room in decentralized storage for intermittent, scheduled, or edge-cached mirror tiers, or do the requirements of audit cycles and uptime make this fundamentally incompatible?
Looking forward to your feedback!

Hello @Mason36,
Welcome back!

We do not use replication (“mirroring”); instead, we employ Reed-Solomon erasure codes. You can view the current parameters for these Reed-Solomon codes here:

Implementing your proposal would require increasing the data expansion factor, which would effectively mean higher payouts to Storage Node Operators (SNOs) for the same volume of user data. We cannot afford this; our objective is to reduce costs, not increase them. The only viable approach is for the client—if they require more nodes for their data—to cover the additional costs themselves. To date, no client has expressed a desire to do so.
The alternative is to implement a special pricing, which would use these nodes as a backup solution (something like an Active Archive, which we have had before), but with a reduced SLA, where nodes availability is not guaranteed in some time periods. If there are many cross-time zoned nodes, this actually might work.

We do not have a caching feature. Most clients utilize either an S3-compatible interface or our Linkshare services. The caching you suggest would only be feasible if the majority of clients used the libuplink library; however, there is currently no version of libuplink that can run in a web browser. Consequently, this would change nothing for clients, as libuplink currently operates exclusively on the backend. If the web server (frontend) is not distributed, then access to the data will not be distributed either—only the actual storage of the data remains distributed. This is usually where we recommend to use a CDN, either before your site, or before the S3 bucket or before the Linksharing service.

Our S3 Gateway-MT and Linkshare services—much like everything else within the Storj network—are also distributed, although their total number is significantly lower than that of the storage nodes.
Therefore, your proposal implies that every node would need to run its own instance of Gateway-MT. This would necessitate a level of trust, as the nodes would then be required to handle the clients’ secret keys.
As of today, there is no solution that allows for the secure operation of Gateway-MT within a Byzantine network (a network where participants may act maliciously or unpredictably).

As for the storagenode binary itself, you could technically run it on a modern smartphone. However, there is an issue with port forwarding: modern mobile operating systems do not provide this functionality (except for mobile versions of Linux or Windows), and most mobile network operators do not provide a public IPv4 address.

I’m thinking the same, AWS Glacier Deep Archive retrieval time is 12-48h and charge arm and leg, so it might work.

But I’m worry how SNO market think of this option? AWS Deep Glacier pricing at $0.00099/GB, so SNO will get ~$0.25/TB? Wonder if there is a market for this…

This is entirely feasible, given the vast number of potential devices. The only requirements are port forwarding, a public IPv4 address, and the use of libuplink running within the client-side browser.
While this poses no obstacle for the BitTorrent protocol, our network architecture is designed differently: in version v3 (the v3 network), a full-fledged P2P architecture has been eliminated. Instead, we have implemented a system of “satellites” that reduces coordination to zero, while delivering exceptionally high data upload and download speeds. In version v2, there were significant delays—sometimes reaching several seconds—while the client searched for nodes to exchange data; this stands in stark contrast to current performance, which is measured in milliseconds.

One of the benefits of paying for Cloud Storage is that the files are available 24x7. If someone isn’t going to be online to give me my files whenever I ask… I’m not going to pay them to hold those files.

An Intermittent Mirror Model could be made to technically work. But it’s value would be so low it may not be possible to charge for it. Glacier has to be cheap for people to be willing to deal with its recovery delays but you know Amazon has your data. In the suggested architecture you don’t know if the nodes will ever come back… how much is that worth?

Cool to think about though…

The cost comparison could be a trade off for 24/7 uptime but I was talking about was a mirror, and not a 24/7 uptime server. The node serves millions of data in pieces of chunks. If one node mirror server would go offline on other part of the world the other node mirror backs it up. This mirror serves as a back up, not the main node server.

If the devices only get paid if they’re online often enough… well… that’s what we have right now, isn’t it? Offline too long and you’re suspended, then DQ’d. Every SNO is already “intermittent”?

A device that’s not online… isn’t a mirror. Or… I could claim I have a mirror of the entire Internet.. so pay me… but… my “mirror” is just temporarily offline now but I may turn it back on later. Trust me bro! :money_mouth_face:

The Repair system is already busy enough dealing with nodes that try to stay online.

The trick here is how to get something reliable-enough on a further reduced timescale.

Maybe intentionally think of it as actually tiered storage. The upload still goes to the main network like it currently does, though perhaps the node selection algorithm might target slower/more empty nodes or something, just to spread demand more widely. Potentially increase the RS params to break it into more pieces, as a further hedge against the reliability loss of nodes that stay offline too long, and the increased overhead of doing a repair operation.

Current storage nodes hold the initial upload, but the satellites keep track of “meant for intermittent” pieces, and whenever an intermittent node is online, it can request a few of those pieces, which it goes and fetches from the node that initially received the upload, after which the storage node drops the pieces. The uploads to the storage nodes can be done with a TTL, so that the space really only is used for bounded periods.

The same thing works in reverse for retrieval. A client that wants to retrieve data stored on the intermittent layer makes requests, which the satellites receive and broker for all pieces. As intermittent nodes periodically come online, the satellite will hand out upload requests if any of its data has been requested (until enough pieces + some margin), the intermittent node re-uploads those requested bits to a storage node. The data re-uploaded to the live storagenodes also has a TTL on it, and will be removed again after a while. The satellite would notify the requester of the locations of uploaded blocks as they become available.

Just like glacier, it’d be slow to get your data back. Pricing-wise, retrieving data would probably need to be fairly expensive. Because of the extra overhead, intermittent nodes would also earn less for their storage and the use of their bandwidth.

Seems like a cool idea. But, Storj keeps telling us there’s more supply for the existing network that is needed. How many times cheaper than the existing payout rates would this sort of feature have to be to attract demand?

Glacier still gives a timeframe for when your data will be ready. If things were set up as you describe… customers would have no idea when they’d get their data back? Intermittent-nodes are online, sometimes, depending on how they feel?

How much would you pay to store a backup… that you could restore… maybe sometime in the future? If you’re lucky. Like asking Santa for it, nicely :christmas_tree:

Nodes that aren’t online… have no reason to be paid. They aren’t providing a valuable service. The primary thing cloud storage has to do… is hand you back your data when you ask for it.

It’s not that bad. Assume a piece that has, let say, 29 chunks on the regular storage, and the rest (46-29=17) chunks on storage that is known to be available once a day. In most cases it’s still available almost immediately (even if slow, as we can’t really leverage races). But once in a while one of the nodes fails, so we need to wait for one (or maybe two) of the nodes that hold redundancy data to show up.

The thing is, though… that would mean saving only on 17/46 nodes, which is not much. This sounds like instead of paying 7 USD/TB/m for hot storage, paying 5 USD/TB/m for cold storage that is available with an SLA of a day.

So, not really a great value proposal.

The nodes we’re talking about are online, just not all the time. Perhaps the way to balance it is to make them earn their pay primarily when data is retrieved. So, the benefit to the operator is only achievable if the node is actually providing benefits to the network.

Also, of course, there’s no reason to skip reputation here. It’s just a different definition. A node might not get valuable data stored on it unless it is available, say, 8 hours/day for several months. Or, to match the idea here better, if it does not return requested (non-customer data during reputation development) data within a certain number of hours.

Yes, it’s a different model of storage, but that doesn’t mean it can’t be modeled, and produce a useful system with expected durability.

I’m thinking of 1h every 2 days is still good enough, as long as they online, sat can still pick up data and transfer back to normal storagenode. But what kind of device is that? a washer machine? refrigerator? We are getting close to middle out compression :laughing:.

The starting topic suggests using smartphones that are only accessible at night or while charging.

In addition to Glacier, don’t forget about Google Archive class. I have a backup customer storing 2.6TB there for about $3.95/mo (it should be $3.20/mo but there are extra fees for early deletes). That works out to about $1.52/TB. There is no retrieval delay with any of Google’s storage classes.

There are some gotchas. For example, 365 vs glaciers 180 minimum billable duration. It’s fine for archival storage, but for backup may not ideal. Depends on amount of turnover: you end up paying for at least a year worth of storage for every object.

It’s tempting to use archival storage tiers for backup, but they don’t always end up cheaper.

To add to the discussion on cost and hardware, we’re overlooking a massive pool of idle devices: smartphones paired with power banks. Many people charge their phones by day and cycle high-capacity power banks at night, meaning millions of devices are sitting powered on and idle overnight.

​Of course, the concern with mobile/Wi-Fi nodes is always network stability. What if Storj introduced pricing tiers based on ping and jitter? If the network rewards SNOs who maintain low latency and stable connections, it creates an economic incentive for operators to invest in proper gear—like a simple Type-C to Ethernet cable setup. Suddenly, those nightly idle phones become reliable, ultra-low-latency edge mirrors that can easily compete with the retrieval speeds of traditional cloud archives."

It’s happening naturally - more fast nodes receives more data and will handle more egress.

We will not incentive to do so, it’s contrary - we always do not recommend to invest to anything with purpose only for Storj. Without a proper network connection these nodes will not be available for the customers anyway.

We don’t use replication, we don’t need mirrors. The number of pieces for the same data can be increased only by increasing Reed-Solomon erasure coding numbers. We already have 54/29 expansion factor, so we paying to operators for storing 1.86 TB of each 1 TB from the customer. Increasing this number meaning, that somebody need to pay for that.

Well then, I think that’s it!
And btw, the USB to Ethernet costs starts from around $10. That’s not a heavy investment.
Thank you for your time.

Still the same problem with a port forwarding and public IP, this adapter didn’t help much.
But seems here is the app which can do that: GitHub - elixsr/FwdPortForwardingApp: Port Forwarding App for Android · GitHub

In that case you need to setup a node on smartphone and install this app to forward a port, also add feature on your tariff plan to have a public IP, and you almost done.

Update:
Checked on my Xiaomi Pad 5. The compiled storagenode binary for Linux arm64 cannot be run directly in the Terminal, you need to install Linux. For example - using UserLAnd, then install at least Alpine to run a node. Also make changes via developer’s options to do not allow Android to kill background processes. Then the port forwarding app will not be needed. You can configure it directly in that Linux.

Found docker for Android: ThothTerm • ThothDock | F-Droid - Free and Open Source Android App Repository