Moderators: richierich, ua900, hOMSaR

 
User avatar
einsteinboricua
Topic Author
Posts: 8242
Joined: Thu Apr 15, 2010 4:11 pm

Personal cloud storage

Fri Jun 26, 2020 8:22 pm

I'm thinking about getting a personal cloud so that I can have a single place of file storage and not be forking cash over to different companies for space that may be unused, not to mention the privacy of the items (not have to worry about hacks getting account information and stealing items).

Does anyone use a personal cloud? Which brand would you recommend? What alternatives are there to cloud?

I'm not keen on portable storage drives. Ever since a mishap with one a few years ago (I lost ALL my Flight Simulator files and data) and another instance where it was stolen in college, I'd rather not go down that route anymore. They're also not configured for Mac/Windows, meaning that if you format it for Windows you need add-ons to use it for Mac.

Would prefer a personal cloud I can set up at home, but if you make a convincing case for an online company, by all means go ahead.
"You haven't seen a tree until you've seen its shadow from the sky."
 
seat64k
Posts: 608
Joined: Fri Mar 14, 2014 11:48 pm

Re: Personal cloud storage

Fri Jun 26, 2020 9:17 pm

This is, by definition, not a cloud - cloud being someone else's computer ;)

What you're looking for is a NAS = Network Attached Storage. There are two problems to consider here:

1) The actual storage
2) How you're going to interact with it.

Problem 1 is easy. Broadly two options:

A: Buy a consumer NAS. QNAP or Synology will both serve you well. They both have a wide variety of models with different features and speficiations. You'll have to read up on what they offer and how they differ. There are a few other brands that I wouldn't touch with a 10ft pole.

B: Roll your own if you're comfortable with building computers. Any old PC with a couple of drives will do. But if you care about data integrity, a server class board with ECC memory is a must. Luckily there are manufacturers catering specifically for this market. Supermicro is my one and only recommendation here. Depending on how many drives you intend to use, the A2SDi-4C-HLN4F (4 port mini-sas) or A2SDi-8C(+)-HLN4F (8 port mini sas) would do the job. You don't need much RAM for file serving (4GB would absolutely suffice), but if you want to use some more intensive features, you many need to add more (and consider the 16-core CPU). Silverstone DS380 case take this and offer 8x 3.5" hotswap bays and 4 internal 2.5" bays. Get a stand alone SSD (either 2.5" or nvme) for OS, replace the Silverstone fans with Noctual, and you have yourself a quiet NAS box that will last a long time. FreeNAS is the OS of choice. It is a BSD based system that uses ZFS for storage. This is important - you want this. It offers the usual file shares and can be a backup target for a variety of systems. You can add modules to it that provide a bunch of other stuff I haven't looked at much - media server, and even OwnCloud.

One word of caution: drives fail, and when they fail in a NAS, the replacement drive has to be synced for the storage to be consistent. On modern drive sizes, this takes a very long time (days even) and increases the risk of another drive in the array failing. Therefore, I would strongly recommending picking a n+2 model. In traditional raid that's a 3-way mirror (RAID-1), or a RAID6, in ZFS parlance it's a 3-way mirror or raidz2.

For the second problem - how to interact with it:

This really depends on what you want out of it. If you just want to drag and drop stuff onto a share somehwere, any old NAS would do. If you want something that Windows Backup or TimeMachine can talk to, all the options I mentioned has that. There are a host of other features that may be of interest to you - best to make a list of what you're looking for and see what supports it.
 
Insertnamehere
Posts: 292
Joined: Tue Jul 24, 2018 3:44 am

Re: Personal cloud storage

Sat Jun 27, 2020 2:35 pm

seat64k wrote:
This is, by definition, not a cloud - cloud being someone else's computer ;)

One word of caution: drives fail, and when they fail in a NAS, the replacement drive has to be synced for the storage to be consistent. On modern drive sizes, this takes a very long time (days even) and increases the risk of another drive in the array failing. Therefore, I would strongly recommending picking a n+2 model. In traditional raid that's a 3-way mirror (RAID-1), or a RAID6, in ZFS parlance it's a 3-way mirror or raidz2.



Unless your array is like 50TB it shouldn't take days to rebuild your array (unless you got unlucky and have an SMR drive in your array)

I agree though a NAS is the way to go. If you are slightly tech savvy and on a budget you can re-purpose an old computer, chuck some NAS specific drives. I would always recommend Linux for file storage over Windows as Linux is a lot more stable. If you have the time to learn some Linux I would recommend FreesNAS which is a OS designed specifically to act as a home file server.

A much simpler, but more expensive approach is to go ahead and buy a Synology NAS, they have a simple interface over a web browser and have similar features.
 
User avatar
WarRI1
Posts: 13847
Joined: Thu Sep 20, 2007 10:51 am

Re: Personal cloud storage

Sun Jun 28, 2020 1:00 am

My Grandson rents out storage to his friends, he is pretty well versed in technology. Not me of course.
It is better to die on your feet, than live on your knees.
 
dtw2hyd
Posts: 8234
Joined: Wed Jan 09, 2013 12:11 pm

Re: Personal cloud storage

Sun Jun 28, 2020 1:38 pm

IMHO, commercial cloud shouldn't be an option to store personal data, particularly free cloud storage, you lose rights to your own data once you store for free.

This may sound complicated/weird setup but it is very simple to set up and get you out of trouble if something goes wrong after storing 5 TB of family photo on personal cloud.

1) A router with OpenVPN feature. Most do. You can open ports to outside world, which I wouldn't recommend.
2) NAS with RAID-1 config along with
3) An external USB HDD.
Optional: Buy a second NAS and install at a close family/friends location.

Config NAS in RAID-1 (simple mirroring)
Attach USB HDD to NAS directly and set up a daily backup job to copy/overwrite new/changed files.
Setup OpenVPN on your router and install clients on devices you want external access.
Optional: Setup secondary NAS at a remote location, set up a replication job on primary NAS.

If a drive in NAS fails, mirroring will help.
If NAS fails (they do), you will still have a USB HDD with all your data. You don't have to deal with unfamiliar file systems used by NAS to recover your data.
If you set up an optional secondary NAS at a remote location, even better, they wouldn't even know it is there.
All posts are just opinions.
 
User avatar
readytotaxi
Posts: 7261
Joined: Mon Dec 11, 2006 2:09 am

Re: Personal cloud storage

Sun Jun 28, 2020 3:24 pm

And don't forget to encrypt your data if it's of any value. ;)
you don't get a second chance to make a first impression!
Growing older, but not up.
 
Dogman
Posts: 88
Joined: Fri Mar 16, 2018 4:47 pm

Re: Personal cloud storage

Sun Jun 28, 2020 4:19 pm

einsteinboricua wrote:
I'm thinking about getting a personal cloud so that I can have a single place of file storage and not be forking cash over to different companies for space that may be unused, not to mention the privacy of the items (not have to worry about hacks getting account information and stealing items).

Does anyone use a personal cloud? Which brand would you recommend? What alternatives are there to cloud?

I'm not keen on portable storage drives. Ever since a mishap with one a few years ago (I lost ALL my Flight Simulator files and data) and another instance where it was stolen in college, I'd rather not go down that route anymore. They're also not configured for Mac/Windows, meaning that if you format it for Windows you need add-ons to use it for Mac.

Would prefer a personal cloud I can set up at home, but if you make a convincing case for an online company, by all means go ahead.


It all depends what kind of info you are going to store and how you are going to use it. Is it only documents? Or are you also going to store a lot of video files? How much disk space do you need? Are you going to access it only from your home or from the outside as well? Forget about SAS drives, you only need them if you are planning to host a streaming site. For most home users you can do with an old computer without any RAID, using something like Drive Pool https://stablebit.com/DrivePool and a USB disk for backups.
 
seat64k
Posts: 608
Joined: Fri Mar 14, 2014 11:48 pm

Re: Personal cloud storage

Mon Jun 29, 2020 11:09 am

Agreed on SMR drives - that's why I singled out WD RED. (I don't know which other drives are SMR - haven't been paying attention that closely)

Insertnamehere wrote:
Unless your array is like 50TB it shouldn't take days to rebuild your array (unless you got unlucky and have an SMR drive in your array)


50TB is not crazy today. Where I live 10-12TB is the price/GB sweetspot. 8 of those in an n-2 configuration is 60GB+ usable. For some that might seem like unlimited storage, for other it's barely enough for a few projects.

With regards to build time, I'll share some recent experience. I had a 8 x 6TB RAID-6 set. A few months ago I had two drives fail, and given that I've had numerous failures with the particular drive (WD RED), I decided to replace the lot. So that entails replacing and rebuilding one at a time. Here's the last rebuild log entries:

Apr 22 12:01:57 lab6 kernel: [ 1587.085848] md: recovery of RAID array md0
Apr 23 21:22:28 lab6 kernel: [121615.111819] md: md0: recovery done.


That's 33 hours - 1 day and 9 hours. Some things to note.
1) The replacement drives are HSGT Ultrastar 12TB, which are a good deal faster than the WD REDs, but in this case the rebuild speed is limited by the write speed on the target drive (the array, even in degraded mode, can read at upwards of 400MB/s). The first rebuild and last rebuild completed within roughly the same amount of time - less than an hour difference and the variance is down to the NAS being in use.
2) The rebuild was for 6TB per disk only. Once the last disk completed, I could extend the array size to use the remaining free space. This means that a 12TB rebuild would take roughly double that - 66 hours/ 2 days and 18 hours.

There are, of course, other factors that impact build speed. This box wasn't idle (although not under any significant load - just some scheduled backups running once per night from two laptops), so it could be a little faster. The CPU impacts rebuild speed in software RAID systems, but frankly, this box is a powerhouse compared to the processors found in most consumer NAS systems. So YMMV.

My point was that one drive redundancy is a risk with bigger drives. I've seen enough double drive failures in RAID sets running on enterprise kit with enterprise drives that rebuild in a fraction of the time (e.g. 36GB 15krpm drives that rebuild in well under an hour), to know that it's not at all rare. Multi TB consumer drives? Especially if they end up being SMR? You'd be lucky if it doesn't happen to you.

Who is online

Users browsing this forum: No registered users and 29 guests

Popular Searches On Airliners.net

Top Photos of Last:   24 Hours  •  48 Hours  •  7 Days  •  30 Days  •  180 Days  •  365 Days  •  All Time

Military Aircraft Every type from fighters to helicopters from air forces around the globe

Classic Airliners Props and jets from the good old days

Flight Decks Views from inside the cockpit

Aircraft Cabins Passenger cabin shots showing seat arrangements as well as cargo aircraft interior

Cargo Aircraft Pictures of great freighter aircraft

Government Aircraft Aircraft flying government officials

Helicopters Our large helicopter section. Both military and civil versions

Blimps / Airships Everything from the Goodyear blimp to the Zeppelin

Night Photos Beautiful shots taken while the sun is below the horizon

Accidents Accident, incident and crash related photos

Air to Air Photos taken by airborne photographers of airborne aircraft

Special Paint Schemes Aircraft painted in beautiful and original liveries

Airport Overviews Airport overviews from the air or ground

Tails and Winglets Tail and Winglet closeups with beautiful airline logos