Friday, July 26, 2013

How to create a 322107-part Virtual Disk


I generated this meme to remind me of the consequences...

Several months back I switched from Arch Linux, to SuSE, then to Ubuntu (which I had used for many years in the past).  Ultimately I ended up running back to Arch.

When I re-installed Arch on my laptop  I decided that I was going to try me some Btrfs.  I hear all the time how it's "ready for production" and now SuSE is using it by default, and Netgear is using it in their newer ReadyNAS (and maybe ReadyDATA) products blah blah blah.  The install and everything was great, no real different than any other file system.

I installed Snapper and started doing hourly snapshots of my data and that worked well.  After about a month or two of use the hard disk would sit there and spin; my computer would grind to a halt and I had to wait until it was finished.  The only clues I had were the btrfs-endio processes that were apparently the culprits.

When this reached the 5 minute mark I decided that something was really wrong.  My first thought was Snapper is taking a snapshot - so I disabled Snapper.

When the problem reached the 10 minute mark I finally decided that this has to be something that I can fix. 

After I RTFM, err Wiki - I discovered that Btrfs doesn't play well with large files that have lots of small writes to them.  It totally makes sense once I thought about it.

Btrfs Wiki: Gotchas

Files with a lot of random writes can become heavily fragmented (10000+ extents) causing trashing on HDDs and excessive multi-second spikes of CPU load on systems with an SSD or large amount a RAM.
On servers and workstations this affects databases and virtual machine images. 

Btrfs is a copy-on-write file system; this means that instead of overwriting data it will write to an empty portion of the hard disk.  ZFS also behaves this way and it's a great feature that lets you take quick snapshots and protect your data by doing so.  If power dies while the file system is still writing, your original data is still intact because it's writing to an empty part of the disk leaving your original files alone.  After the write the filesystem then updates its big dataset of data (meta...) pointing to the new file/portion of data.  So a snapshot would leave all your files in place, then the filesystem would start writing to other areas of the disk, so all your files are there and just a few commands away from recovery.

With my virtual disk, however, this is terrible.  It's Windows 7 Pro virtual machine that I use for some Windows-only work applications.  Windows likes to write things to the hard disk constantly and each time btrfs writes to another area of the disk.

So a long story long I ran filefrag on my vdi file and blam - 322107 fragments in this one file.  That means each time I start my VM the computer would have to try to read from all of these different fragments that are in physically disparate parts of the hard disk..

I tried to defragment the file with the btrfs defragment tool ($btrfs fi defrag /path/to/file), but no luck; my system would just crash after a while and eventually reboot.  I ultimately ended up copying the vdi to a new file, and turning COW off for this file ($chattr +C /path/to/file).



Moral of the story - RTFM.



Monday, July 12, 2010

Tuesday, April 27, 2010

Sticks and Stones

Nice article about why you shouldn't care what people think and how you should examine why you care and/or if they really care...if that makes sense.

http://www.thesimpledollar.com/2010/04/26/who-judges-you

Posted via web from Dave's per-posterous

Thursday, January 28, 2010

Heard this yesterday...

Some 1.5 million children die each year from diarrhea, -- more than AIDS, malaria, and measles combined

Holy crap...no pun intended. This is pretty ridiculous. Something we take so whimsically.

Posted via web from Dave's per-posterous

Tuesday, September 23, 2008

Use Cash, Not Credit

I've been an avid Dave Ramsey fan for quite a while. I only really started buckling down with my budgeting over the past few months, but I've been following his principles and listening to his show for about a year now (now he's also on the Fox Business Network). At the beginning of the year I was almost $27,000 is debt (vehicle included). I'll be completely out of debt in January (this is partially dependent on my tax refund...I've apparently been paying way too much).

Here are a few things I like about Dave Ramsey and his "teachings" and why I definitely recommend them:

1. No gimmicks.
  • Dave Ramsey doesn't preach any gimmicks. He's not trying to sell you on a program, but tell you what has worked for him and hundreds of thousands of his fans.

2. Simple, common sense approach.
  • He teaches things that most people should already know. Spend less than you make, pay your debts, save for your future. Do a budget and stick to it. Simple, no?
3. Gives me a program/structure to follow.
  • I tend to be better at following something that has already been put in place and is being used by other people. If there is no template to follow I usually try to make my own way of doing things...and I never finish because I always think it could be better. Having something in place and someone to show me what to do and what not to do works for me.
4. Teaches that debt is dumb, cash is king...basically don't borrow money.
  • By the time I was 23 I was over $42,000 in debt, half of which was just stupid credit card debt. I didn't look broke because I bought a lot of crap, but before I knew it I was deeply in debt and had a huge negative net worth. I had "free money" that the the credit card company gave me and I could pay it back later. I didn't really realize that I'd be paying for it years later. I've been in debt as long as I can remember and I'm quite ready to be out of it and never get back in it. Dave teaches you that while debt is "normal," it's quite alright to not be normal. Imagine not having any payments to make, not having to worry about ruined credit, etc.
The points above are only a brief overview of the things that Dave Ramsey teaches. I'm no expert on personal finance, but for the first time in my life I feel like I have control over my money. I've tried to stick to the things Dave teaches as much as possible, but I still learning discipline when it comes to my finances (and a lot of other things).

You don't necessarily have to do everything Dave says, but it seems like quite a good thing. You have to get in the right mindset and forget about what society says about debt being normal and a part of life. It doesn't have to be! If you don't borrow money, you don't need credit...period.

In my life I've implemented quite a few things from Dave's books that have really helped me to gain control over my money and my life. This makes for a much less stressful experience with money. I don't have to worry about whether or not I'll have money for this or that. I'll write briefly about the major point that are helping me now.

1. Save for emergencies, don't borrow money for them.
  • Growing up an emergency fund was a credit card. If something broke and needed to get fixed you used a credit card to fix it. Dave Ramsey teaches to save money in an "emergency fund." It's money you have set aside for...emergencies! When you're paying off debt you start small at $1,000, but once you're out of debt save 3-6 months of expenses. DON'T TOUCH IT! It's tempting to dip into this fund to buy things, but this money is for emergencies only.
2. Spend all your money on paper (i.e.: do a budget).
  • I used to spend money throughout the month then go back and figure out what I spent it on. Then I would say to myself, "I should cut back on the coffee." That doesn't work for me. Dave teaches to do a budget as soon as you get your check. Spend all your money on paper, and stick to the budget. Budgeting is such a basic thing, but it's also so hard to do. He teaches that there isn't a perfect budget that you can write that will work from month to month. Each month is different and has different needs. This is by far the easiest way to save money...as long as you stick to the budget.
  • Another thing he teaches is to budget for infrequent expenses to spread the cost out throughout the year. For example; I budget about $144 per month throughout the year to pay for my car insurance (in a lump sum) and to pay for my vehicle registration. You know the registration; you get the letter from the DMV and suddeny remember you owe a couple hundred bucks next month. Saving throughout the year will let you not worry about it.
3. Use cash...the envelope system.
  • Don't use credit cards! It hurts to spend cash, so use it. The envelope system is basically this: put the cash for particular categories in an envelope; when the money from an envelope has run out, then you're done for the month. It might take a while to learn how much to budget for each envelope, but you'll get the hang of it. I have an emotional attachment to cash. It's hard for me to part with. Credit cards/debit cards on the other hand and very easy to swipe and thing about later. Get Rich Slowly wrote about this today.
4. Get out of debt!
  • This all really ties in. Budget, find extra money, pay off debt. Think about the feeling you'll get with not having any payments, not owing money to anyone, being completely financially independent.
Anyway...you don't have to follow Dave Ramsey, but at least take a good look at your finances and understand where your money is actually going. Don't be me and realize that you're dumb when it's too late. Luckily I'm still young and have learned this valuable lesson early...

Happy Birthday Jackson!