Rant: GPT versus LVM

It hit medium-sized data centres approximately 4 years ago and now it's barreling down on home PCs with the force of a locomotive. Another virus? Worm? Trojan? Not at all - I'm talking about the 2TB partition size limitation for MS-DOS MBR-style partitions! Not too many have been affected but Linux users purchasing high-end machines are starting to hear the rumblings as their favorite distro's installer craps out on them. With 4TB consumer-grade drives out on the market it's only a matter of time until the rest of us are confronted with this. My first reaction was

"Damn! I won't be able to use cfdisk and will have to settle for GNU parted! I hate GNU parted!"

...but this is much more far-reaching than just affecting your choice of partition editor.

It is, of course, not the first time the industry has faced such barriers and overcome them. The 1MB RAM barrier faced by Intel's 16-bit processors (worked around with 24-bit addressing on the 80286 and finally dealt with by the fully 32-bit 80386), the 8GB partition size limitation imposed by CHS addressing (worked around with extended partitions and finally dealt with by LBA addressing), Y2K date limitation (worked around with a cutoff of 1930 instead of 1900 making it the Y2K30 limitation and resolved by a 4-digit year making it the Y9K999 limitation), the 4GB RAM barrier (worked around with PAE and resolved with 64-bit processors), and the "limit" of only 4,000,000,000 (2^32 minus reserved) 32-bit IPv4 addresses (worked around with NAT and resolved with 128-bit IPv6 which nobody's using), to name a few. As usual the pattern will be work around it and then finally resolve it with a new standard. Whether that standard is an elegant forward-looking "fresh start" solution or an inelegant back(ass)wards-compatible kludge which only delays the problem is yet to be seen.

So what can we do right now? Well...we can simply create more partitions and delay the inevitable. Using MBR you can splinter your data and create 4 x 2TB monsters leaving you with a "new" theoretical limit of 8TB or, better yet, create an extended partition and have an infinite number of 2TB slices at your disposal. In my opinion, that's hardly ideal. The "industry" response to this is GPT - which creates a sloppy kludge on top of the old MBR structure and comes with it's own "new" limitations. Essentially it takes the MBR and "reserves" space using a partition of type 0xEE then allowing it to create it's own odd (each partition has a "type" so that vendors can fight over the essentially meaningless UUID for each "type" - a concept carried over from the old MBR-style partitions), arbitrary (The partition tables at sector 34? Why 34?), limited (128 partitions to a maximum size of 9.4ZB, or 9,400,000TB), fixed-form structure (there is, and will only be a V1.0 of GPT).

Well...we get 9ZB partitions...it's supported in the Linux kernel...problem solved, right? OK...yes in a barely and kludgy backwards-compatible way but solved, right? Well let's think here. What would be considered forward looking and potentially a superior solution to GPT? It's best if we look at some challenges and caveats of the MBR-style partition system:

• Larger block sizes improve performance and increase storage capacity. Does it support block sizes other than the old DOS-style 512-bytes?
  
• Sometimes partitions run out of space and we need more - it would be nice just to buy another disk and have more. Does it allow partitions to be combined seamlessly into logically contiguous blocks? On a live system in real time? If they aren't side-by-side? On different disks? On different machines?
  
• We like to get the most from our hardware and losing data sucks. Does it allow for partition striping to gain performance or mirroring to gain reliability?
  
• Working with live filesystems prevents certain important activities such as backups. Does it allow for instant copies of partitions on a live system to ease backups, imaging? Or for safe trials of such activities as filesystem performance tweaking or high-risk repair tools?
  

The answer to all of these problems being resolved by GPT is a resounding "NO". It was this comment which really got me thinking: doesn't LVM as implemented in Linux solve all of these problems?
As a matter of fact Linux LVM is the penultimate evolution of the whole "partitioning" solution. While LVM is currently limited to "only" 8 Exabytes (0.008 Zetabytes) this is a small price to pay for all that other functionality. Additionally, and unlike GPT, LVM standards are also developed in a revisable manner so it would be trivial to increase this in future revisions. Why was GPT even designed when a superior solution already existed? Why was the wheel reinvented...as a triangle?

In the name of simplicity I'm hoping LVM will one day overtake GPT as a straightforward unified method to divide disk storage and give us all the flexibility we deserve. The main hurdle to this happening is a certain antagonist commonly named in the industry: Microsoft. I haven't been tallying, but this will be approximately the 15,000th time the industry has had to settle for mediocrity and inferiority in order to be compatible with the mediocrity and inferiority of their products and general incompetence. Since it's open-source, it would technically be easier for Microsoft (and to a lesser degree other vendors) to support LVM than to "create" a solution to this problem. The code is there for them to freely use and implement without the threat of getting sued over patents and/or copyrights. (As an aside, this is Microsoft's dirty little secret of course, since the case is the exact opposite for Linux, standards bodies, and other tech companies supporting their technologies and standards. After 15 years in this industry one sees Microsoft not choose external standards or technologies and instead create their own, not so much because of classic NIH, but rather because they seem to like kicking small puppies with any mighty newfound "Intellectual Property" powers they come to possess).

Personally, I don't need the added complexity of 2 impotent partitioning schemes (and all the related problems that pop up from time to time) and would like the possibility of simply having LVM manage my storage. Before this can happen though, it has one major hurdle to overcome: LVM is not supported by "standard" BIOS since it's not part of the EFI "standard" to which we can expect almost all current and future BIOSs to adhere. This means it is not accessible nor bootable from the BIOS and either MBR or GPT is required at least until a real OS of some kind is loaded.

Will this be the final result? GPT is here to stay because the "industry" has decided? In my opinion, yes and the two will coexist. However, there are many smart data centre administrators and savvy techies out there who will add some weight by choosing a superior configuration for their machines and making it work (custom BIOSs and booting from other media come immediately to mind). Second, booting from LVM becomes a greater possibility when you consider that computer BIOSs are no longer a fixed, burnt-in, permanence. They can be flashed and superior open alternatives are available to give Pheonix, AMI, Byosoft the kick-in-the-ass they've sorely needed for decades! Consider this a challenge to the coreboot community: help support superior solutions and make BIOSs LVM-aware!

The way I see it, MBR and the 2TB partition size limit is not a new limit at all but an expansion. I now can have up to 2TB of bootstrap code in which to get a running OS to use my LVM volumes on the rest of the drive ;- )

I'm curious if it's possible to create an MBR-style partition for /boot, an 0xEE partition to tell old-school tools to "piss off", and an LVM volume embedded directly into the hdd block device right after the /boot partition ends.

In the meantime is anybody else using LVM instead of GPT? I'd like to hear about different approaches people have taken and how they've panned out!

Rant: Breaking up with Gentoo

Dear Gentoo,

We've been together for 10 years now and there's a couple of things I've really been meaning to say. When I first fell for you I was enticed by how you were so fast and lean and full of charm (you also had a bigger set of packages than any other distro out there). It wasn't all looks either, you had a willingness to try anything and, unlike LFS whom I had just dumped for being a slob, you knew how to keep our living space neat. It really made me feel special and I was convinced that nobody else had done the things we did together. Although we were relatively new to it we learned and got better together.

Then what happened? I mean, look at yourself! You're threatening to tear apart my desk and stick plasmoids all over my walls, you constantly whine about how outdated your profile is (even though changing it could be hazardous to your health), every little makeover is followed with an argument about all our life's decisions, and you're always making veiled threats to change our baselayout!

When did you get to be like this? Your friends have developed a bad attitude to even your most ardent fans and instead of listening to criticism and accepting patches they defend your inequities! My family can't stand to see you anymore and you're an embarrassment.

I'm also constantly babysitting and you can't even make a basic decision anymore without talking too much, making bizarre demands, becoming unstable. Why did I need to reinstall python to install docutils to install upgrade eselect to upgrade sandbox and portage to upgrade cairo to upgrade XULrunner? Why do we always have to fight this way? All I wanted was the latest version of firefox without you going crazy on me and having a complete mental breakdown. Is that too much to ask? Speaking of psycho have you made up your mind about whether to organize the cmputer according to python 2.6 or 2.4? While I'm on the topic of your crazy whims, you decided some time ago to install everything onto the new python 2.6 you made me get and now you want everything to go back to 2.4 again because you've flippantly changed your mind? This isn't like picking and choosing between links and lynx - decisions like this should not be made lightly since our package manager depends on this! Speaking of packages your selection is poor and although you think overlaying this makes up for it, I think it only makes you that much more unstable! I hope you see my point, you have fundamental dependency issues and you're always revdep-rebuilding all over the tree instead of just keeping things neat and organized from the get-go. I can't live like this anymore! I'm trying to build an online life and that can't be done on shifting sands.

I guess what I'm trying to say is: we're through! I know it's tacky to break up via "blog" posting, but it's the new millenium. I'm not being a coward, it's just how this is done these days.

It probably isn't the best time to tell you that I've been seeing something else. No, it isn't a former X like Mandrake, LFS, or Ubuntu, and it certainly isn't that narcissistic, slutty, trailer-trash byetch Windows.

It's PCLinuxOS. It's down to earth, not very needy, and has been respectful to my family and friends. Although it's not as open to new ideas, it knows you have to be assertive to keep a tidy dependency tree. I understand if you don't want to stay friends after this.

As soon as I can sell our computer PCLinuxOS and I will be getting a new quad-core computer together. I need you to find another drive to stay on so you can have your bytes cleared out of mine by the end of the month.

Rant: He Neither "Misses" nor "Hits" the "Point"

Bill Gates recently gave a speech (here is the video) for a kinder, gentler, more merciful form of Capitalism. One which would benefit the poorest two-thirds of our fellow man while still being profitable to businesses. There have been arguments for both Gates being right and Gates being wrong and even the skeptic that I am cannot help but agree that we need to do something about this horrible divide between those who have and those who have-not.

What could be better than the world's wealthiest man dedicating himself to the poorest people in the world? This is fantastic news!

...except...

Sadly, there is a huge difference between what Bill Gates says, and what Bill Gates does. There is an unfortunate truth that the top 2% (probably more) of the wealthiest people in the world have obtained their status at the expense (summary) of the rest of the population, rather than through some mutually beneficial methods. I really had hoped that this was an exception to that rule but no matter how much I denied it - the facts, which I outline below, confirm it:
Bill Gates neither misses, hits, nor cares about the point - his speech is merely the preamble for a corporate welfare application being prepared by Microsoft. He hasn't gone insane, this is just "Business as Usual^TM".

In his speech Mr. Gates proclaims that "Companies should create businesses that focus on building products and services for the poor." as if oblivious to the incredible efforts during the last 3 years to provide those countries with modern educational tools! Efforts such as the Linux-based OLPC laptop project, which after becoming an object of Gates' ridicule, has been making progress in successfully providing the third world affordable computing and learning platforms posing a long-term threat to the Microsoft's Windows monopoly. How ironic, then, is the fact that educational tools and computers based on Windows are virtually non-existent in poor countries due to Microsoft's monopoly pricing and their lack of interest in those who cannot pay those prices? They have started focusing on products and businesses in these sectors very recently but only as a response to the competitive threat posed by Linux, OLPC, and others. Their recent efforts have included giving steep (and steeper) discounts on Windows licenses, attempting to develop a version for low-end hardware, and in more disturbing cases threatening and bribing smaller poor Governments to adopt their products. Microsoft's recent efforts are not about providing the bottom-third of our population modern learning tools, they are merely an extension of a classic business strategy that Microsoft has recently had to pursue for competitive reasons.

So now the million dollar question is: Why is Gates pushing for changes to our Capitalist system if there are already businesses and organizations making headway in providing the solutions for the 3rd world? Businesses doing exactly what he touts in his speech, albeit without Microsoft technology?

It is when Gates states that "...disburse funds to create financial incentives for businesses to improve the lives of the poor..." that I inevitably come to the conclusion that Gates isn't asking for a fundamental change to Capitalism. What he's asking for is a subsidy which will allow Microsoft to enjoy the types of profits it's used to while extending it's monopolistic tactics to a new, low-profit, market.

So as for Bill Gates' speech and all the hoopla surrounding it, I will write it off as Waggener Edstrom doing what they do best to earn their paycheque. They have elegantly crafted an argument which paints this whole effort as an off-the-cuff plan, supposedly written on a napkin during a flight, based on Bill Gates' mercy after witnessing the poverty in the African slum of Soweto. They have managed to wield our own emotions as if they were their own personal tool by using a combination of our pity for the poor and our wishful thinking to make it all but impossible to argue the point without coming off as an heartless cynic.

Well, this heartless cynic predicts that this campaign will be easily identifiable through the following:

• the repeatition of this "creative capitalism" message (through Waggener Edstrom and Microsoft's channels of influence)
• increasing focus on the creation of financial incentive programs from Governments and U.N. Agencies as the "right thing" to do
  
• "creative capitalism" becoming the newest buzzword on the lips of every pointy-haired boss while being nothing more than a poorly-disguised euphemism for corporate welfare

With that in mind I'm really hoping we will be able to resist the inevitable call to action spawned from these suggestions for 3rd World "Technology Outreach" programs. It is obvious that these challenges are best left in the hands of organizations genuinely aiming to improve the situation as well as to businesses with sustainable business models for the 3rd world. The issue with profit-obsessed companies, such as but not limited to Microsoft, who are only seeking to extend their market rather than commit, with a dedicated mission and/or business model, to helping the world's neediest is that they will only do the absolute minimum to obtain approval (i.e. check-off the "ticky boxes") on whatever reports are required to obtain their subsidies. I have no doubt that this will be done without any regard as to whether, in reality, their efforts help those people or not. This is in stark contrast to non-profit organizations devoted to such causes which make sure that the help they provide is empathetic, real, and effective! It is also a slap in the face for businesses which have opted to use creativity in their business models, rather than subsidies, to solve those countries' problems and still turn a profit. Sure their profits are not as excessive as those demanded by monopolists, but they are sustainable.

While I am somewhat impressed with the sheer talent in the design and execution of such a PR campaign I anticipate becoming extremely frustrated having to watch the irony of this whole "creative capitalism" speech come to bear:
the world's poorest being used as an excuse to line the coffers of the world's richest!