Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

What's missing from this system(and P2P systems in general), is automatic pinning - i.e. the system automatically choses which files to pin on which user, to ensure every file is available on the net.

It's a bit of tricky problem, because people may not like unknown, random files on their pc, and probably, because such algorithm is distributed and will need to resist attacks.

But i think that the payoff is big: what Bitorrent did to popular files, such an algorithm could do to rare files.



Ipfs has something called “bitswap” which solves some of what you’re describing but is lacking in areas such as long term file availability. That’s where filecoin sort of finds sort fills in the gap.

https://docs.ipfs.io/concepts/bitswap/


See the comment above about Freenet, which does something like this; although according to https://freenetproject.org/pages/documentation.html#content it seems to prioritise popular files rather than rare ones.

IIRC eDonkey prioritised rare files, although I don't know if this was a property of the network or just a common choice in clients.


There is a Freenet "KeepAlive" plugin which can be used to selectively pin files which you want to preserve, even if they are completely unpopular.


We built a simple pinning system for a project I'm involved with: each node queries a blockchain-based database for a set of resources to pin locally. Resource publishers pay blockchain fees to advertise their data for pinning.

We also found it necessary to build a kind of "overlay" to the p2p network (how each peer selects its peers) because the default algorithm produced too sparse a network. The probability of one node ever being connected to another node that holds content it would like to sync approaches zero. The topology for the overlay network is also fetched from said blockchain.


If the pinning came with encryption where the key is not known to the pinned system, I think most would be OK with it. There's no way to know, or demonstrate that anyone could know, the contents of any given file. Furthermore, the file could be broken into encrypted chunks and distributed redundantly to many users. Kind of like BT turned inside out.


https://en.wikipedia.org/wiki/Perfect_Dark_(P2P) is this kind of system. You allocate a slice of your disk for the program to use as a global cache, and random encrypted chunks from the network end up downloaded into it.


This is pretty much exactly how Freenet works.



This doesn’t automatically pin, rather distributes a pin set across a set of trusted nodes.

It’s good stuff nonetheless.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: