Skip to main content
Welcome guest. | Register | Login | Post

How Fast Can USB 2.0 Thumb Drives Be?

9 replies [Last post]
supermike's picture
Offline
Joined: 2006-02-17

What if you aggregated a bunch of 4GB USB 2.0 thumb drives and made a RAID-5 configuration (software RAID)? Would this have faster drive read/write than RAID-5 on regular hard drives?

Why do I ask? Well, I saw an advertisement in my supervisor's Processor.com magazine for a 2U rackmount web server that costs $65,000 and runs on non-volatile solid-state memory. It only has 128GB of memory. However, the ad said it could give you a website that's 1500% faster. The cash register in my brain started ringing...

* You take an ordinary OEM 2U rackmount web server with a single hard drive inside, 2GB of RAM, USB 2.0 port, CDROM, 2 10/100/1000 NIC cards, and 2 power supplies. The motherboard is a little small so that stuff can be attached to the chassis with machine screws.

* You get on the Internet and purchase a sheet of thin aluminum and another of thick aluminum. In your wood shop, you take a scroll saw and put a special blade on it that cuts metal really well. You cut the aluminum into the shape of an L, but the small end is a little fatter and you can drill holes in it. You then do this again. You then drill holes in such a way to attach the two L pieces together and then fashion a lid and a bottom to stay snug. You now have an aluminum box. You drill a hole such that a single, extremely high quality USB 2.0 cable can fit in there.

* Inside the aluminum box, you put 32 4GB thumb drives through a series of USB hubs in such a way that only the USB cable comes out. Then, to make it even better, you gently remove the chip drives from their casing, along with the USB hubs from their casing, and you gently attach this to the aluminum with really fine machine screws.

* Now take the aluminum box with the USB cable coming out and attach it to the chassis of the 2U rackmount server. Connect the USB cable to the motherboard's USB slot connector.

* Using the internal hard drive on it, now put Ubuntu Server Linux on it and build a web interface with PHP to administer it. Throw Mambo, PHP4 (cgi mode), PHP5 (module mode), and Apache2, PostgreSQL, MySQL, SSH, OpenSSL, and Postfix on it. Implement iptables firewall (and license it from Netfilter, grrr). Because web servers are sometimes in DMZs and don't have proper Internet access, make it so that in order to update it, one has to insert a CDR and connect to a web page to tell it to do an apt-get update; apt-get upgrade; from it. One gets the CDR through subscription. If you want to go nuts, put a download source for common dev tools from it and install Zend Accelerator.

* Mount the USB thumb drives as one great big 128 GB RAID-5 drive on ReiserFS.

* Make the web admin piece of it allow one to use the NICs on separate IP addresses, or aggregate the 2 NICs together on the same IP address for increased bandwidth.

* Let one use SCP to securely copy files to the web server, or perhaps enable FTP.

* Sell it for $40,000 (USA dollars). Your cost? Perhaps $6K to $8K (USA dollars) once you start to build on volume. If you sell 31 of these, you're a millionaire. Your customer? Go after the Fortune 500. They'd eat this stuff up in a heartbeat without even blinking.

* Downsides?

- These drives only last 10 years, so you have to tell the customer upfront and give them a renewal option that they can purchase now and store the renewal kit (with instructions) for that day, or they can just wait until that day comes and hope that you still have renewal kits available.

- You can only store 128 GB. If the customer wants more than that, cheaply, then they'll have to connect to another database instead of using the local one, or migrate some of the lesser-used tables to another database server. And of course that may slow down the app a little.

- If one drive blows in the RAID-5 configuration, and the 90 day warranty is up, they'll have to call out a technician from your company, expenses-paid, to replace it at some time before the second drive blows. (That is, unless you can figure out how to rip out the drives from hot-swappable disk bays and make the USB thumb drives work with that.)

a thing's picture
Offline
Joined: 2005-12-20

You're creative.

supermike's picture
Offline
Joined: 2006-02-17
"a thing" wrote:

You're creative.

I can't help it. My mom and dad were hippies and artists in the 60's in the USA. Dad flew the coop, but mom always encouraged me to be expressive with my ideas. I couldn't draw very well, but I have never been able to stop the volcano of ideas that pop in my head. I like to build things. By profession I would be a programmer, but my job got outsourced to Brazil, so now I'm a sysop who moonlights at home on a killer app programming project. I also was part of the team that finished the interior of my mansion of a house, installing doors, windows, flooring, drywall, cabinets, sinks, appliances, countertops, etc. (I got lucky during the dot-com era and could afford to build before the bubble burst.)

Since I don't have $5K to $6K floating around right now, and if I did I would be spending it on jacking up the back of my house (which has settled), I thought I would pass off this idea to anyone who gets off on this kind of stuff.

tbuitenh's picture
Offline
Joined: 2005-12-21

That reminds me of something: floppy RAID and memory stick RAID!!!

Offline
Joined: 2005-12-20

Shocked

i didnt't know solid state memoery only lasted for 10 years, I'm gonna have to go and kill the people who I boight my music player off of. Laughing out loud.

BTW, is it PHP you code in or what?

supermike's picture
Offline
Joined: 2006-02-17
"onlinebacon" wrote:

Shocked

i didnt't know solid state memoery only lasted for 10 years, I'm gonna have to go and kill the people who I boight my music player off of. Laughing out loud.

BTW, is it PHP you code in or what?

Yep. It's 10 years, they say. You can Google on that.

PHP? Yes. Love it. I used to do ASP and had it call VB6 DLLs on Windows. Then, I was trained on Microsoft .NET and felt that Microsoft had backtracked and done something stupid. In some areas, .NET was great, such as C#, but in other areas, such as XML parsing, database access, and security of objects, they made it way harder than it needed to be. Then, to boot, they admitted that until an operating system has .NET integrated into it, COM still runs faster in many cases. So, between the backstabbing Microsoft consultants I had dealt with, and the trouble with .NET, and Bill Gates lying under oath (clearly evident), and the tying of products and no room for competition, and the whole software patents defense that Microsoft was doing -- I had had it! I once said to a Microsoft consultant, "So, how come Microsoft is making their OS harder and harder to use, and all these lines of code, and their .NET is too bulky -- yet Linux is getting easier and easier, and their desktop manager is actually starting to be cool, and their languages are just as cool, and there's this brain trust moving to Linux?" The Microsoft consultant balked at me and was quite upset. He also didn't have any good answers. So I asked my company to lay me off since they kept me after the layoff, and I was granted it. That's when I joined another global company that loves Linux and PHP, and I started to learn that, and said, "Wow, this PHP has really come a long way since I first learned about it."

So here I am. I'm using Ubuntu as my office and home workstation. The servers in my office, because of corporate edict, are Suse 9. I use Bluefish, PHP, pgst (which I wrote), and PostgreSQL. I'm in a state of nirvana with it.

Another language I dabble in, which I actually don't like very well, is Python. And when I say that, I mean the PyGTK variety for making non-web apps in GNOME. So, I decided to make a reusable framework in PyGTK so that I could tolerate it. It's based on the work that I saw RedHat doing with their control panels. The reason I use it was because I didn't really see a functional kind of "VB6"-like language in Linux that was already pre-installed. But in GNOME, PyGTK is a big part of it already. I also didn't like the concept of Mono, which gives too much credence to Microsoft than is due, so I didn't want to go that route. So, with my reusable framework, I was able to whip out 'pgst' and put it on SourceForge. I have also used the framework to make a firewall tool that makes an iptables script, and a VPN tool. I released pgst, but have not released the firewall and VPN tools because I want to see if anyone wants to take the projects and finish them. If you know someone interested in PyGTK, I need someone to finish my code, make an RPM and apt package, and get it so that it works on Fedora and Ubuntu.

a thing's picture
Offline
Joined: 2005-12-20

As for a VB-like language, there's GAMBAS Almost Means BASIC and KDE4 will make heavy use of EMCAScript.

supermike's picture
Offline
Joined: 2006-02-17
"tbuitenh" wrote:

That reminds me of something: floppy RAID and memory stick RAID!!!

This was classic! I bust up laughing, holding my sides in, when I saw the stacked floppy drives. What a hoot! I also liked the picture of the Gremlin and how it related it to that. I sent the link to my boss and coworkers and said, "See? One more reason to switch to Linux. You can't do this in Windows!" They'll get a laugh out of it when they see that.

However, I do believe that if I'm not mistaken, USB 2.0 is pretty darn fast. And if the USB 2.0 thumb drives in the 4GB variety are faster than RAID-5 configured hard drives -- something I don't know yet -- then, well, that's something to think about. Note that I think RAID-5 slows hard drives down. So, when you compare USB 2.0 thumb drives to RAID-5 hard drives, I wonder which ones are faster?

BTW, the company doing solid-state rackmount CPUs is:

Texas Memory Systems
http://www.texmemsys.com/

This product they have is insane! From the looks of it, it looks like this company is choking in cash. Nobody else in the business is doing this yet.

supermike's picture
Offline
Joined: 2006-02-17

I have a hardware friend, Dave, who sometimes replies back to me from time to time. He has some info on my interest in USB 2.0. Here's what he says:

Quote:

RAM is *VASTLY* faster than Flash memory. Also, Flash
memory "wears out" if you write to it too many times,
and RAM memory does not.

It appears that these guys [sic Texas Memory Systems] are using RAM, not Flash memory, and they are backing it up to hard disk drives when the power fails:

# Redundant internal batteries power the system for 25 minutes after power loss
# Automatically backs up data to disk at 190 MB/sec

Also, USB 2.0 is slow. If you use a USB interface to a disk drive and copy the drive, it takes a lot longer than if you plug up the drive directly using an ATA/IDE cable.

However, SATA drives are generally hot-pluggable. Just be sure to plug in the power connector before plugging in the data connector; when unplugging, you must reverse the order, so unplug the data cable and then uplug the power cable:

http://www.maxtor.com/en/documentation/white_papers/sata_hot_plug_white_paper.pdf
http://www.asrock.com/news/SATA_HotPlug.pdf

Of course, how the software will handle that is an interesting problem. With Linux, you probably just need to umount it before unplugging it, and mount it after plugging it in, and then run the appropriate Software RAID rebuild command. But that is just an (optimistic) guess. There might be some interesting hurdles to overcome.

Offline
Joined: 2005-12-20

10 years is a lie! Solid state memory would be hard drives, right? I have 2 hard drives from 1994 that I use every day. I got a couple older ones too that still work, but ~100mb just isn't good for anything other than a paper weight.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.