Reposting an old comment of mine here just to see if it can get an interesting conversation going:
"I was daydreaming recently after revamping and dialing in my home media server really well (Plex plus all of the *Arr services plus torrents automatically downloading shows and hosting a large movie library)…
Shouldn’t it be possible to create a “virtual media library” that is actually BitTorrent under the hood, powered by people either contributing storage space or content or both, where no one ever hosts any in-tact media and it’s all encrypted enough where each storage provider is basically hosting useless shards of content, and then some client side software can put it all together on demand and serve it up to apps like Plex in a way Plex doesn’t even know it’s not a local file? Assuming everyone has decent bandwidth and there’s enough contributed storage for redundancy to power the torrent network, you could basically have an unlimited and resilient media library…
There’s probably a way to craft an incentive scheme in there for both content submitters and storage providers, probably a decent use case for crypto honestly so it’s all decentralized.
At this point, it seems like content capture and uploading to the torrent world is basically “solved”, and now it’s all about ensuring the network and storage health is there to preserve everything in perpetuity. There’s so many redundant torrents out there, should be possible to merge it all in a giant media library mesh network right?"
Any time you make a new media library hosting platform, it attracts exactly the people who need it most.
The answer is good metadata and categorization, but that requires active moderation.
Moderation is the single most expensive part of the internet, and most of it is being done for free. Big tech like Facebook and Google are making themselves incompatible with traditional moderation, which is why they keep trying (and failing) to automate it.
--
I think the best solution is a web of trust. Instead of using a tracker/site like TPB (and its moderation) to introduce trust, let each user make whatever authoritative claims they want (this torrent is good, that one is corrupted, and this other one has a virus), and let other users back those claims via gpg signature. Instead of forum moderators, you get trust curators.
Power corrupts, and any set of curators will eventually fall to corruption, but the nice thing about having software that lets you explicitly trust people (instead of implicitly trusting the platform) is that the switching costs are low. You don't have to wait for somebody who is both tech savvy enough to build a competing platform and also trustworthy enough to curate the content, you just need the trustworthy part. You just twiddle your trust settings and you've got a fresh start.
I also think that the who-trusts-whom-in-what-category dataset would be incredibly useful. For instance, you can follow the edges until you get a cycle, and that cycle is going to be at the heart of a community of experts.
Or if you're having trouble seeing eye to eye with someone you can consult the trust graph and find a mediator, or if they're not to be found then you'll find thought leaders who exemplify the disagreement.
You'll have to be hygenic about who you trust though, and that's not the lazy path forward. So culturally I think any project of this sort is going to face an uphill battle.
> Facebook and Google are making themselves incompatible with traditional moderation, which is why they keep trying (and failing) to automate it
Its the other way around. They try to automate and fail miserably after spending billions and fallback to manual moderation by farms in asian countries coz its cheaper.
I think an idea of mine is related: index clips from various sources to put together an entire movie. Or index quotes posted to various forums or quote websites to reassemble the original book. No one hosts the full content, all is fair use, and the index is also meaningless by itself. Though I suppose you'd get a similar conviction as The Pirate bay which also hosts no copyright infringing content.
Your idea would be a bit more 'direct' in that the parts are hosted by people actively contributing to have a copyrighted work in full rather than linking to random third parties. A lot more reliable, at least until it catches on and contributors start receiving desist notices.
Can someone tell me what is the appeal of *arr apps? I'm not a hoarder or professional movie/show watcher, why should I set up what is basically a glorified "periodic metadata downloader with a tiles UI"?
I tested radarr and readarr recently. Didn't find out why people use them. Like libgen was better than readarr and I couldn't even select the encoding/file size of a movie title in radarr (there are some predefined profiles and you can define yours too but I expected something waay too simpler to do this. Like if you select fhd, it may download a very large file. You have to manually set the bitrate slider for all profiles, but then you may risk losing some available item if the release is very new and not encoded. I mean the UX was confusing)
And I see people host yet another tiles UI (Plex/Jellyfin) on top of that just to show a folder structure with pretty thumbnails? Each one of these apps is usually written in C#, taking ~300MB of RAM mind you...
What would help me is simply a search box that crawls all available torrents and presents rows with filtering based on size and encoding. I tested prowlarr for this but the UX was poor.
*Arr is automation. Add a TV show, connect some pieces together and your favorite TV show gets downloaded, catagorized and added to your own private Netflix server (Plex).
Sonarr? TV shows. Radarr? Movies. Books. Music. Etc.
"missing something obvious" the UX might be horrible... but once you set it up? you don't have to look at it.
"And I see people host yet another tiles UI (Plex/Jellyfin)" if all you think Plex is a "tiles UI" then there's no wonder you don't understand it... I have hundreds of movies, dozens of shows, audio books, music, etc in Plex and all the metadata gets automatically added and I can automatically play anything on my phone, tv, etc.
Oh... my friends/family has access to my Plex library as well so they can request and watch when they want.
Plex does transcoding and all that fun stuff.
You're missing all the details that go miles beyond a "tiles UI".
>Can someone tell me what is the appeal of *arr apps? I'm not a hoarder or professional movie/show watcher, why should I set up what is basically a glorified "periodic metadata downloader with a tiles UI"?
As streaming gets split up into little fiefdoms it can become difficult to have access to the content you want, furthermore as some content for various reasons is never made available or fully available - for example if you want WKRP in Cincinatti with the original music you will need to pirate it (I would of course like to hear I was wrong about this - other than find some CD somewhere with all that content at a yard sale)
I don’t want to download tv shows or movies when they come out. I don’t even know their schedules. Just add them in *arr and they appear in Plex automagically as they come out. For TV shows specifically this is a huge time saver.
Plus they let less technical family users add and manage media on their own.
Regarding not being able to select encoding, etc - you setup profiles and it automatically chooses the best release for based on the criteria you set. Again, it’s one less thing to think about.
It’s weird to me too. I consume a decent amount of media but if I can’t think “oh yeah I want to watch this particular thing” I just don’t download it and it falls out of my brain. Which I’m fine with. Foundation was like that, and His Dark Materials. I guess I would have finished them if they downloaded automatically but they were kinda lame.
I guess I just consume media more haphazardly than most people. The whole point of my PLEX is only exactly what I’ve requested shows up.
To be clear, the *arr suite of apps don’t just download things randomly. You still only get what you request. You just don’t have to go download a new TV show episode every week or keep an eye on exactly when the new movie hits the shelves.
You just tell the apps ‘follow this tv show/movie/whatever’ and it grabs it when it becomes available.
I personally solve for this by configuring my profiles to download decent sized files for my preferred formats, and then I run FileFlows as well on my media server to automatically transcode everything into the same h265 codec and transcode settings and stuff...it's all just hands free, everything gets spit out perfectly how I want it.
Now that being said, the plex app blows on Apple TV with h265 codecs for some reason, so I'm now using the Infuse app which is ok, but yet another thing. But overall, yeah, I've been able to cancel multiple streaming services (hulu, sling, soon netflix) and have MORE content available than before. saving a grand a year in streaming services ain't bad for a few hours of learning how to set it all up.
> how do you control the quality and encoding of the show?
There are settings for setting the desired quality parameters, and even for transparently replacing the on disk version with a higher quality version becomes available.
The key is Overseerr. It allows non-technical folks to say "I want XYZ show/movie" and it'll just show up as part of the private Netflix (Plex). It's incredibly simple to use, but it depends on the various *Arr apps to do the finding and downloading.
Is it possible to maintain such a project? I thought it was called popcorn-time, but earlier I remember MythTV. Both projects have seemingly been hobbled from general use, the main idea is to watch whatever you want, when you want, and it doesn't have advertising in it. (If the original source recording had it, then it could be removed.) Maybe it's overly dramatic to say I think this should be a human right, but maybe it isn't? I don't need advertising in my life, I think.
> how would there be any money to produce content?
the same way content on youtube gets produced today.
But this argument is a hyperbole. The content gets paid for, and some people will pay - convenience, access etc. The rest can free ride off those people. So far, it has worked.
convenience and access exist because it is not a human right to watch whatever you want when you want it. if there were such a right free content would be convenient and accessible.
Hypothetically, you could setup (maybe with crypto) a rewards system for people who contribute content, add storage to the network, do transcoding, etc, and then allow other people who don't want to do any of that to simply pay for accessing the mesh network. That would open up a flow of funds that the development organization could take a percentage of.
That being said, it's likely just a lawsuit honeypot at that point, so mostly a daydream :)
There was such a thing round 2007-2010. People on forums were opening free email accounts, then a desktop app was splitting files into chunks of predefined size and uploading them as emails. Individually each chunk was an encrypted chuink of 7z or zip archive, so it made no sense on its own, but the client was building a repository of email account and email IDs. This index was then shared with other people on a forum and you could download Shrek.
Someone made a desktop application much like that, it was wonderful, truly wonderful... until I started chatting with the developer about not keeping semi-permanent logs of who [previously] hosted what. I had hoped to solve the puzzle but he started unpublishing his application immediately.
I think simpler things could help more. I would like to see 1) a client that (in stead of just seeding) periodically checks how rare the rare blocks are on a torrent then seed just those blocks. 2) attempt to extend the tit for tad by looking for similar titles and file names (or some other formula) if insufficient seeds are available. So you might have [parts of] foo1 and foo3 and are looking for [parts of] foo2. foo2 is served by someone looking for [parts of] foo3. One could even ask to help seed something they are uninterested in and/or unfamiliar with. It does seem less of a "crime" than getting involved in permanently seeding the motherload.
Copyright holders would still track the IPs that these shards are coming from and sue/threaten them for copyright infringement. 5 seconds of Mulan compressed and sitting in isolation is still illegal to distribute
It's fine to use small snippets, but it specifically has to be for commentary, satire, derivative works, etc. The term you're looking for is 'fair use'
I had a similar idea, at least in terms of resilience. It was, basically, to compress each piece of content just so that the compressed version would be theoretically uncensorable.
Jon Lech Johansen did this with DeCSS back in the day [1], and the compressed version of the program was a prime number, which gave it a sort of "untouchable" quality.
Obviously, doing this for much larger content (i.e. movies and general videos) would be a challenge, and this technique might not be the best choice. Still an interesting concept, though.
What you guys seem to be describing is "Freenet", but that project has been around forever and relatively obscure. These days, unfortunately, I wouldn't touch it with a 10ft pole due to safety concerns.
"Freenet is a peer-to-peer platform for censorship-resistant, anonymous communication. It uses a decentralized distributed data store to keep and deliver information"
From what I remember, it basically takes content and distributes "anonymous" and encrypted chunks of it across various members on the network. Content stays active by being replicated, and replication happens proportionally based on popularity. So content that never gets used basically disappears.
while true, the actual reason it's untouchable is that the companies cannot realistically sue everyone - it's too costly, and they cannot recover the funds.
My reaction exactly, "this sounds a lot like Freenet". Although I wouldn't be ideologically opposed to the idea of getting another stab at the concept of FN, I think it's imperative for people interested in doing that to look at the history and challenges of that project. Even if we could get over the scalability issues (as the technology landscape has undoubtedly changed since then - "free" storage, larger bandwidth, faster processors... - we'd still have to wrangle with the unavoidable truth that nobody wants to risk hosting CP. I think you need to solve that problem, before you look at building such a system.
> Shouldn’t it be possible to create a “virtual media library” that is actually BitTorrent under the hood
We actually had bittorrent-enabled video players in the early days of bittorrent. There seems to be some options active today, too. Google suggested frostwire.com
If I'm following correctly, I believe this is basically what Sia does, although not optimized to be used directly as a media server (or maybe it could?).
Sort of. It would be more like firing up Netflix, but it has every piece of content ever in it. No choosing to download anything, no real notion of a personal library of content (maybe favoriting or something for UX I guess)...just a infinite mesh of content, powered by a distributed network, with economics involved to incentivize everyone to either consume the content or help host the network.
"I was daydreaming recently after revamping and dialing in my home media server really well (Plex plus all of the *Arr services plus torrents automatically downloading shows and hosting a large movie library)… Shouldn’t it be possible to create a “virtual media library” that is actually BitTorrent under the hood, powered by people either contributing storage space or content or both, where no one ever hosts any in-tact media and it’s all encrypted enough where each storage provider is basically hosting useless shards of content, and then some client side software can put it all together on demand and serve it up to apps like Plex in a way Plex doesn’t even know it’s not a local file? Assuming everyone has decent bandwidth and there’s enough contributed storage for redundancy to power the torrent network, you could basically have an unlimited and resilient media library…
There’s probably a way to craft an incentive scheme in there for both content submitters and storage providers, probably a decent use case for crypto honestly so it’s all decentralized.
At this point, it seems like content capture and uploading to the torrent world is basically “solved”, and now it’s all about ensuring the network and storage health is there to preserve everything in perpetuity. There’s so many redundant torrents out there, should be possible to merge it all in a giant media library mesh network right?"
https://news.ycombinator.com/item?id=35213039