The file storage system with deduplication between users

the Beginning of this project gave a very simple idea, which, a year ago wrote pages on habrahabr. It was then that I wondered why all of the services file storage so greedy? Why they earn that several different users store the same file? Why should we pay double, triple, and sometimes in hundreds times more, than really is it worth it? I think it's unfair.

The second problem all paid services file storage – subscription fee. We are forced constantly to pay for storage of our files. If you do not pay, the files will delete. More like extortion than a convenient service, especially in the light of these issledovaniya on the topic of the cost of data storage. So I decided that it was time to create a project that does not have these disadvantages.

So the idea, and what came of it can be read under characato.

MD5-File is a personal file storage system c deduplication. While others earn on storing many equal files from different users, the service offers to share the cost of storage of the file between all users. Thus, the permanent storage of the file can practically be nothing for the user, and the maximum storage cost does not exceed $ 1 per Gigabyte. This is a one-time fee. No monthly fee will be charged.

How it works.

When loading a file, the system checks whether there are other users that use the same file and calculates the cost of storage. For example, if 10 users have the same 10 GB file, in fact, each of them will only use 1 GB of its space. When the number of users increases, the system automatically recalculates the space occupied by the file and free space for all users of the file increases.

I really wish the project didn't become "another file storage system", because in addition to simply upload your files, especially for the long term, it is necessary to have a convenient system of search and systematization. Personally I do not like the system folders, as it does not allows you to simultaneously add a file to a few different groups, so I decided that it was necessary to implement support for tags. That is why the service supports a very flexible search system that allows you to set such rules:
the
    the
  • Paris!wife – finds all files with a tag name or Paris, but without the tag or the name of the wife
  • the
  • Paris!tag:I,name=2012 – finds all files with a tag name or Paris, but without I, which has the text 2012
  • the
  • Paris|tag:wife,ext:pdf — finds all files with a tag name or Paris or wife and an extension pdf
Search the tags can be easily added to your bookmarks and thus go directly to the desired file group that I think is very convenient.

The same system is implemented in webdav, i.e. you can mount a webdav disk and do a link to it on tags, e.g. \md5file\storage\Paris\children \md5file\storage\family\children in any combination. Currently, the webdav service is running in beta-readonly mode at webdav.md5file.com/storage

What's in your future plans?
    the
  • Download files from torrents and HTTP/FTP links
  • the
  • WebDav (full support)
  • the
  • file Sharing between users
  • the
  • Integration with other services file storage
  • the
  • Instant download of the file (if available on server)

Link to the project: MD5-File

P. S. unfortunately, registration through social networks is not yet supported, due to the fact that a password is required for WebDav. In the future we plan to implement such authorization.
Article based on information from habrahabr.ru

Комментарии

Популярные сообщения из этого блога

Fresh hay from the cow, or 3000 icons submitted!

Knowledge base. Part 2. Freebase: make requests to the Google Knowledge Graph

Group edit the resources (documents) using MIGXDB