Dec. 29th, 2010

ocschwar: (Default)
USB thumb drives are now available at 8GB sizes. That is a significant number for the following reason: 8GB is enough to store an entire lifetime's creative work for an individual. By that I mean every last bit of typed prose, poetry, computer code, or musical composition. I mean every last bit of computer aided drafting or graphical design. verything you ever type or click onto a computer should fit.

Even photo editting probably qualifies. If you took every photo you seriously editted, and stored the starting photo and a record of the operations you put it through to get to the finished product, the amount of space you take up would be light enough to fit a lifetime's work in this 8GB archive. Actual photography would not. And forget audio/video editting. But if we just include writing, coding, drafting and graphic design, we encompass a huge set of people for whom their entire professional career's work could fit onto an 8GB hard drive. Authors, lawyers, journalists, researchers, engineers, programmers, and many more.

With that in mind, there ought to be a product that will collect a life's archive for its user, and take advantage of the properties of flash storage, and the current state of network bandwidth to make that archive readily available and easily, cheaply, reliably, and securely backed up.

Flash storage is amazingly reliable for data that are not rewritten. So this product would use a write-once or at most a write-twice paradigm. It would use a revision-tracked directory structure to store a user's work, and archive changes to file data and meta data into diffs, which are then appended to each and every storage volume the archive is backed on to. This product should be able to deal with writing these diffs out-of-order to different drives so that if it is given access to different storage volumes (i.e. different thumb drives) at widely disparate times, it can still make both of them complete archives, and useful for reading at a later time.

The product should also be able to deal with online archiving. While it is designed to take advantage of the properties of flash storage, it should also be able to use online storage services. To that end, it should be able to commit file data and metadata in encripted form, while leaving revision metadata in the clear (so the online server can collect additional diffs as time goes by).

What it should not make a priority is data deletion. It is here to archive an entire life's work, and it is here to serve a user and only that user. So, the user should not have to worry about deleting files to make room. (Oh, how I loved living with a 5 meg Athena quota...) And, it should not worry about deleting a user's juvenalia, since the user and only the user will expose such work to the world. Without data deletion, a write-once paradigm would be easy to implement. In fact, most of hte code for creating this kind of product already exists in the Subversion project.

So, does this already exist? Should I consider trying to write it?


ocschwar: (Default)

April 2017

16171819 202122

Most Popular Tags

Style Credit

Expand Cut Tags

No cut tags
Page generated Oct. 23rd, 2017 04:59 pm
Powered by Dreamwidth Studios