I Blame Thunderbird

 By Shamus Sep 11, 2008 60 comments

I know we just went through this a month or so ago, but for no discernible reason Thunderbird lost all my emails. I restored a 2-month old backup, and it managed to lose those. I restored the backup again, but I have no way of knowing how long Thunderbird will keep them around this time.

More to the point: Any personal emails sent to me in the last 2 months are gone. If you emailed me in the past week or so and didn’t hear back from me, please re-send. Several people have sent me links and “You might want to see this” articles over the last couple of weeks, and now those are gone. I’d planned on writing about some of them

Boring details of the misbehavior follows:
I would just write this off as some unfortunately trashed hard drive data, but Thunderbird is freaking out on me. After being open for a few minutes, it will suddenly stop showing me the contents of emails. I can click on them in the list, but they never open. The tiny progress bar at the bottom isn’t there, so it’s not compacting a mailbox. Worse, when I close the application, it doesn’t actually close. The window vanishes but the process keeps going in the background. I have to kill it manually, which is probably what triggered the loss of my mail.

It keeps re-compacting my mailboxes – a process which takes five minutes or so. It finishes, I close Thunderbird, and next next time I run Thunderbird it starts compacting everything again.

If I empty my Junk mail folder, it takes several minutes to move 42,000 emails from Junk to Trash. (Why doesn’t it give me an option to kill them outright?) Next time I start Thunderbird, the 42,000 emails are back in the Junk folder. So… it takes forever to throw them away, by they are instantly recovered, unlike my real, actual, valuable emails, which are instantly deleted and unrecoverable?

It’s always been a little touchy, but something happened yesterday to really push it over the edge. I can’t think of what it might be. I’d think I hit some magical size threshold in there, but now that the last 2 months are gone it seems like Id be below that limit again.

Thunderbird has 6GB of my data (and falling) and I no longer trust it with that responsibility. But I don’t know who else is up for the job. That’s a lot of data, but it’s broken up into yearly archives so that no one file gets too big.

I’m ready to switch, but… where do you go from here? I’m certainly not going back to Outlook. I have a gmail account, but there’s no way to get my eleven years of email archives up there. Heck, I don’t see a way to get this year’s archives (what’s left of them) onto Gmail in a way that preserves the date/sender info.

I installed Evolution, and found it to be a magnificent bag of suck. It started, I configured it, and then it went into a coma and I had to kill the process. (With little DOS windows flicking in and out in the background, an side-effect of its rough port from Linux, I’m sure.)

Man, I can’t wait until this new “email” technology settles down and we can get some decent software for it.

20202060 comments? This post wasn't even all that interesting.


  1. dagbrown says:

    Evolution for Windows? The very thought of that chills me to my bones.

    It’s bad enough on Linux without thinking about how a Windows port would work out.

    As for me? Uh, I use a stone-age mail program called “mutt”. On Linux. I’m one of those weirdos who has never seen a reason to use Windows for anything ever. I know that this makes me a weirdo, but at least my computer breaks in ways that I’m familiar with.

  2. Dave says:

    I use Eudora. Have for years. It works for me, but I’m not sure how it might handle your archives.

  3. Frank says:

    Dare I mentioned Windows Live Mail? I’ve been using it for about 6 months now, and it’s been handling my gigabytes of archives remarkably well. *ducks*

  4. Ben Orchard says:

    It’s like forum software. It all sucks. It’s a matter of choosing which one sucks the least. Outlook sucks, Thunderbird sucks, Evolution, Eudora, Opera, they all suck.

    I used to love thunderbird. These days I use gmail exclusively.

    May I humbly suggest that perhaps 6GB is a LOT of email and you could switch to a slightly different schema? It may be time to consult a professional on this and get a solution that involves hosting your own email server. I don’t know that much about it, but this would also mean that you could maintain a permanent archive.

  5. Primogenitor says:

    I’m afraid I have to vote with the “you have too much mail” tag. 6Gb. That’s a lot. That even enough to fill a GMail account (which you might be able to move to since it can be used with Thunderbird via IMAP).

    Are you using an IMAP server (where it stays on the server) or a POP server (where you download a copy)? Personally, I will only use IMAP systems these days.

    Oh, and when email began, 6GB was probably enough for all world-wide emails for the first decade or so. That’s just a wild guestimate though!

  6. Poet says:

    I get about six gigs of spam and porn (two very different things for me) every 12 hours or so. Worst case scenario, I don’t write my aunt because I think it’s a solicitation for credit card information from someone in Nigeria.
    My point is, there’s no point in crying over spilled electrons.

  7. dlowe says:

    Something you might want to try is to setup a local IMAP server and use fetchmail to pull your accounts into it. Then you could use Thunderbird in its infinitely safer IMAP mode.

  8. Nathon says:

    I think I have to second Ben. Setting up an email server (and a backup server while you’re at it) is probably worth while for you. And if you really only use the 6 GB of archives on that rare occasion when you need to find that recipe someone sent you 4 years ago, you might be able to get by with archiving and emptying your inbox every year and just keeping those archives separate. At least you wouldn’t have to wait a week while things get compacted. Is your thunderbird version the latest?

    I also used to love Thunderbird (or, at the time, Minotaur) until I saw gmail’s interface. I have mutt but only use it for reading root’s mail.

  9. Factoid says:

    Outlook for life. This may make me a heretic…but I really do like it.

    Yes I’ve tried others.

  10. Illiterate says:

    Shamus,

    I know it’s not your usual gaming project, but have you considered this as an opportunity to muck around with data warehousing?

    Turn your backlog of emails into a database and play around with sorts and queries?

  11. Andy Adams-Moran says:

    If you’re willing to give T’Bird another go (or are forced into that position), I suggest the following:

    * Archive the folders you want to keep (don’t bother if you use IMAP)
    * Uninstall T’Bird
    * Install T’Bird, but don’t hook up any IMAP accounts yet
    * Make sure that “Compact folders” setting is disabled
    * Make sure that “Delete mail marked as Junk” setting is enabled
    * IMAP: Hook up your IMAP and sit back …
    * POP: copy your folders back by hand

    I don’t have any Junk folders, just Trash, and it works great. Be very careful with IMAP: if you have a Junk folder on your IMAP (esp. one with 42k messages), you may be screwed. I would try deleting the Junk folder instead of moving the contents to Trash.

    The only time I’ve lost messages was when I moved messages between IMAP folders while offline; I wanted to see if the moves happened sensibly when I went online. It was like a bad transporter accident :-)

    I still find that T’Bird gets confused a lot, especially over flaky internet connections, but I haven’t lost anything (emails going back to when I started using IMAP). And I have 7Gb or so in my IMAP.

    Now, if you’re using POP, the same steps should work, but you’ll have to copy your folders back in by hand.

  12. mikeful says:

    I try to use IMAP protocol as much as possible. It keeps emails on server and if something goes wrong I can blame someone else. ;)

    Are you sure that every piece of that 6GB pile is necessary?

  13. Kevin says:

    Mac user. Time machine backs up everything that’s important onto an external hard drive. As long as I don’t have more than one drive go simultaneously (I have 3) then I’m all set.

    Yeah, I know that’s not any help. Sorry though. You have my sympathies.

  14. mc says:

    If sylpheed is available on windows, I heartily recommend it.

    /me checks.

    It is!

  15. I’m not sure why anyone would say that Eudora sucks; I’ve been using it for over a decade and it does everything I would expect a mail client to do–filters, address books, multiple identities, formatted or unformatted text. It’s stable, fast, and it uses the solid, unimpeachable MBOX standard for its local mailboxes.

  16. Shamus says:

    In response to the “Do you really need all 6GB?”:

    No. I’m sure well over half of it has no value whatsoever. It’s just that sorting through to weed out the crap is a massive job.

    T’Bird really is hosed, and no amount of coaxing seems to be able to get it to behave. The additional downside is that to evaluate a program I have to install it and import my archives – which is about as time-consuming as it gets.

  17. Andrew says:

    6GB?! You’re crazy to have that in a single file. Outlook, Lotus Notes, Evolution, Thunderbird, Mail (on Mac) anything will choke on that. All of these store them in flat files, so when you open your email it does scan the entire file (and is why compacting happens – if it does it repeatedly, something must go wrong, Thunderbird has an option to do it automatically).

    Seriously, the main thing you need to do even if you hate Thunderbird now and change is to export, backup or remove your email down to a managable size (even if you only add it to the “Local Folders” – create a new folder there and copy stuff to it, it won’t get altered or even opened if you don’t click on it – and it’s nice to order it by date or subject for easy searching later!).

    The only way you cna get away with that physical amount of email is typically IMAP, where you only download the headers until you open the actual email – and in fact, I’d recommend that anyway. Most website hosts provide it (if your host has cpanel, it’s a certainty), and if you want a backup you can create mail rules in almost all clients to “On new mail, copy to this folder…”. I’d recommend to start using it now :)

    This is better then POP based stuff (as you’ve just found out!), and also means it’s on a server as well as your PC :) – you can access it online, on a laptop (if you get one) and so forth, since it copies changes to the server.

    As for the client, find a suitable one for you (go look at all the features. I’m sure someones reviewed them), but Thunderbird has never lost a single email for me so I don’t have any other recommendations.

    IMAP and moving things local folders (or better yet; both) is a great way to do it though. Make a backup of that old set of emails though before starting :)

  18. Jonathan Grimwauld says:

    Shamus, I bumped a friend who’s an expert on porting email into gmail. He’ll either post or send a email toyou.
    sidenote: the nutscrape/TBird stuff has always been faulty with compacting of the folders – God only knows why. (it’s also a problem outlook pro has, but not express in my experience)

    If you end up deciding to continue to use T-Bird (prolly for imap – that’s what I do. no more trust isues, y’know?), just turn off the compaction option. it will take more room up – but I’m at 8 gigs of archived information, and I only have one bug that’s unrelated.

    Not the best of solutions..

    Anyways, watch for someone to come along and explain how to import local email archives to gmail. He’s a lifesaver :)

  19. Martin says:

    That’s odd, and unfortunate. I’m up to 11 GB at work since I catch a copy of all the filtered spam in a special magnet address.

    Have you tried to rebuild the Index on any folders?

    Any why compact? Disk space is cheap, and it’ll be faster uncompressed, right?

  20. Bai Shen says:

    AFAIK, Thunderbird will have issues with mail files over 2GB. I’m not sure why, but from what I’ve heard, that seems to be the reason mail disappears.

    I agree with the others about making a local mail server and then using Thunderbird(or whatever client you choose) in IMAP mode. You would set up fetchmail to download your POP3 accounts to the local mail server.

    I’m looking at doing this myself, that way my email isn’t tied to one particular machine. I haven’t had a chance to set it up yet, but everyone I’ve talked to has recommended that I use Citadel instead of setting it up myself.

    From what I can tell, Citadel is pretty slick. It basically takes over whatever machine you install it on(real or virtual) and sets up all the mail rules, plus other cool features.

  21. Bai Shen says:

    Martin,

    Thunderbird stores emails in files, one for each folder. When you delete an email, it doesn’t actually delete the email from the file, it just marks it as gone. Much like the way file systems work. When you compact your mail, it actually goes through and removes these flagged emails.

    So think of compacting your mail as defragging your hard drive.

  22. Shamus says:

    Andrew: It’s not one 6GB file. It’s broken into several email accts, archive folders, etc, pretty much the way you suggest. Added together they come to 6GB. Individual ones range from a few hundred MB to one that is 1.5GB. (Still to big, I know.)

  23. Delve says:

    “No. I’m sure well over half of it has no value whatsoever. It’s just that sorting through to weed out the crap is a massive job.”

    This may have been mentioned already, but jobs like this are *very* amenable to chopping up over time. Start at day one, and sift through those emails. Depending on your email volumes, maybe do two or three days or a week at a time. Then stop for the day and get on with life. Keep a chart on a legal pad. Tomorrow, pick the next day in your archive. Don’t spend more than 10 minutes (or 1 day’s worth, whichever is larger) a day on it and you should be able to slog it all out eventually. Be vigorous, if you *think* you *might* *one day* want something from an email, delete it. Keep contracts and possibly online purchase receipts (over a certain value). Destroy the rest, all the way up to a year ago. If you’ve got attachments in there you want… what the hell are you doing using email as a file backup???

    But of course, you probably already knew all that. As for email apps, I’ll add my vote for Gmail. “You are currently using 73 MB (1%) of your 7124 MB.” Yeah, it’s got room. Granted you have to pray that Google never goes down, but that’s about as close as you ever get to a garruntee right now. Next month? Well, that’s the risk you take.

  24. Brandon says:

    I hate to say it, but I’m not sure there are any programs out there that will healthily deal with 6 GB of email at a time. I think you might need to create several “archive” mailboxes, limited to no more than about 500 MB, which you can open and then remove from the email client when you’re done with them. Just never have them all open at the same time. No more than 3 or 4 at a time is probably the best policy.

    When you have that much email cleaning it out seems insurmountable, yet it is something you must do if you want healthy email again. Your problem is less Thunderbird and more poor email management. You are stuck in a Catch 22. Maybe you need to create a Linux virtual machine and run Pine. Something like Pine might actually be able to manage that much email. I suspect the most recent versions of Outlook might actually be able to tackle it a little better than Thunderbird, but you’re still dealing with almost 6 GB total of largely textual data.

    To those who have recommended Eudora, I strongly disagree. It’s a great program, but it can’t handle as much email as Shamus has. Thunderbird is better equipped for that volume than Eudora. Again, Outlook might handle it a little better but not much. Shamus has just broken through the practical upper limit of consumer email systems. What he needs now is a corporate email infrastructure. Most businesses which have that much email have some industrial or custom-coded archiving/storage system to help manage their stuff.

  25. Matt says:

    I’ve used Mulberry since probably 1996, and I’ve never really seen a good reason to switch. It’s got some quirks (such as it’s never handled HTML email well at all, but some people like that “feature”), but it’s the only client I’ve ever seen that really does IMAP right.

    It used to be commercial, but is open-sourced now. Oh, and it’s available for all 3 major OS’s, and basically works the same on each.

  26. Mayhem says:

    Being one of those people who keeps almost everything I’ve used Eudora since the late 90s when netscape communicator went all bloatware, and found it superb for handling large amounts of archived email.

    I split everything up into folders and it creates a separate mailbox file for each primary folder (I believe subfolders are contained within that mailbox).
    This means firstly that it is trivial to import and export my copious archives into other programs if needed, and secondly they can be backed up individually without any issues as they are easily identified files.

    Downside is it stores attachments in a separate directory to the email, and they are then linked to the email within the program.
    On the other hand, this also makes your mailbox files much smaller as they contain only text, so can be compressed very easily, and it saves hunting through years of emails for a particular attachment.

  27. Robyrt says:

    I’ve had Thunderbird eat my emails before. I imported it into Gmail and frankly never looked at the vast quantity of old emails ever again.

  28. One thing that might coax Thunderbird into at least working properly is deleting your profile directories (after making sure your backup is close at hand, of course).

    Go to “SystemDrive:\Documents and Settings\Username\Application Data” and you should see a Thunderbird directory. Pull anything out of there that you need and, before reinstalling, delete it completely.

    There’s another directory in “SystemDrive:\Documents and Settings\Username\Local Settings\Application Data” that you might want to deal with as well. I’m not well-versed enough with Thunderbird to know exactly what it keeps in each folder, though I’d imagine that the Application Data one would keep your contacts and settings and the Local Settings one would actually store your mail.

    After clearing those out, the simple act of restarting your already-installed copy of Thunderbird should actually work. You can always uninstall and reinstall just to be safe, though.

    This method also works wonders if you have any bizarre issues with Firefox (I’ve only personally experienced such issues once, but it’s still a good thing to know about). Firefox’s profiles are stored under the Mozilla directory, in the same places.

  29. James says:

    Loosing the mail is God’s way of saying “6GB Jesus Sonny-Boy!”

  30. Sungazer says:

    Hello,

    I’m one of those people that sent you a link in the last couple of weeks. I’d love to resend it to you but I’m having trouble tracking down your address again.

    Would you mind if I placed the link here in your comments area?

    Thanks!

  31. Have you checked with your webhost? Chances are that they already include a decent mail server in your package- I know I give my hosting clients two or three choices of preconfigured, and they can install their own if they want, as well (as long as it’ll run on a Linux server, of course). You should have IMAP support already in place, it may just be a matter of storage limits (not sure what your host charges, but these days, 6GB is typically free.

    Your other good option, if your host won’t co-operate, is to run your own mail server at home. Get a cheap (or free) older PC from Goodwill or something (something in the P3 range will do just fine) and install your favourite Linux distro on it (that was a little joke, Shamus, ha ha). In 20 minutes or so, you can have your very own mail server. And don’t forget to go to your domain registrar’s control panel and update your MX record! You wouldn’t believe how many DIY-ers forget that step.

    Sorry, I kinda made that option look a bit goofy, but honestly it’s not that hard. I’m sure you’ve got a couple dozen readers who would bang one off on a surplus bit of junk they have lying around and mail it to you for a few bits of link-love.

  32. Kevin says:

    Shamus,

    Toss me an email off line…I do email administration and have managed accounts that were over 6 Gig in size, so I should have some good info, which you can compile and share in a later post.

    I’ll keep an eye out for a message from you at my junk email address: waste.some.space@gmail.com and get back to you with my correct address.

    I’d suggest a new gmail account for “emergency contact” with people…in a case like this one.

    –Kev.

  33. Nazgul says:

    I’ve used Eudora on both Mac and PC and liked it a lot. Outlook and Thunderbird are both fine too but I liked Eudora better.

    I use (the not-so-creatively-named) Mail on the Mac and it too shows multiple signs of suffering under from seven years(!) of upgrades and/or the load from keeping most of my non-spam email forever.

    I’m too much of a pack rat and a privacy guy to let a company like Google or Yahoo store my email someplace remote.

    Have you tried managing the stored email? Sometimes I find that sorting the old messages so there aren’t a gazillion emails in any one folder sometimes helps the software avoid choking on those for whatever reason. Sorting by date (years or every 6 months) is pretty painless.

  34. Samrobb says:

    Shamus,

    I’ll echo the “run your own mail server and use IMAP” theme here. My wife and I have been running our own postfix server for about 8 years now, and it’s been fine. Setup was basically trivial – a few hours of reading and tweaking, and the occasional bit of maintenance. I’ve upgraded the postfix and imap servers several times without problems. We only have about 250 MB of mail between us, though, so YMMV.

    Best feature: when I need a throwaway email address, I can make one up on the spot. Very convenient :-) Also, since I have a dedicated server now, it’s the obvious place to set up other externally facing services – personal web server, wiki pages, ssh server, perforce server, and the like.

  35. onosson says:

    Whatever your ultimate choice of main email software to use, I highly recommend archiving your stuff with gmail. That way if you ever have a problem like this in the future, you can get your old versions back again.

    You can configure your email host to send copies to your software and gmail at the same time, so all incoming future emails will be taken care of.

    And, the search functionality in gmail is, well, as good as google, since that’s what it is! I use Outlook myself on my desktop, but certainly not for its “search”…

  36. John Lopez says:

    If you upgrade to Google Apps, you get 25GB per user. We have done that with one of the companies I work with; they have users who will fill that in under a year easily (many large attachments per day).

    We still use Outlook 2007 (which can support larger file without puking on a 2GB limit), but we archive off aggressively and leave archives closed unless they are needed. I would like an alternative, but Outlook is the best of a bad lot.

    For personal use, one can’t beat the “we really use the file system” nature of old Unix mail systems, which avoids arbitrary size limits. It somewhat blows my mind that everyone feels the need to reinvent the wheel (the file system in this case). I would *love* an e-mail client that stored folders of my e-mail in… (wait for it) FOLDERS!

  37. Telas says:

    I’ll chime in on the “too much email in one place” chorus. The size of your email file is so big and dynamic that it’s unstable.

    Before Gmail, I used Outlook with no problems. I would run monthly backups of the PST files, and quarterly I’d move everything older than a year to an archive PST. So I’ve got files named Archive 2001.pst, Archive 2002.pst, etc.

    Yes, it’s a pain in the ass to find that one email from 2-3 years ago, but none of my PSTs ever hit a gig at the time. These days, I’m pretty sure they’d still be under 2GB per year.

    Regardless of your chosen solution, good luck with it.

    Telas

  38. Factoid says:

    complete threadjack, but I got a beta key to Good Old Games today (gog.com, mentioned by Shamus a few days ago) Not many titles available, but the ones they have are excellent!

    Fallout 1 & 2 & Tactics
    Freespace 1 & 2
    Descent 1 – 3
    Shogo (An anime-style FPS, very fantastic game)

    Almost all the games are 5.99. A couple are 9.99

    I plan on picking up Fallout tonight.

  39. Susie says:

    I’m seconding the use of IMAP – it’s the only way to go for email. A dedicated mail server is nice too :-)

  40. The Lone Duck says:

    Just so you know, I have 7124 Mb of space on my gmail account. I don’t know how you would transfer the files, but the faqs might have an answer? I’ve never lost an e-mail. (At least, any e-mails I know. I do not know about the absences of unknown e-mails.) Hmmm… Worst case scenario, take a weekend (or two) and just do it manually. If you can find a better solution, great. It’s one thing to lose e-mail from us, I hope your job related e-mail is secure. Anyway, good luck.

  41. Clint says:

    @Jonathan Grimwauld: …I have been well and truly bumped, I guess.

    Anyways, as he said, IMAP seems to be the way to go for large quantities of email, and GMail makes a nice free place to store it all. The program you’ll want to use to import stuff is Mark Lyon’s GML (http://marklyon.org/gmail/). That allows you to take a standard MBox file (which is how Thunderbird stores its mail) and upload it directly to a GMail account. Once it’s there, it’s fairly trivial to set up filters that will tag all of the uploaded mail with something so you can search just the messages from a particular mailbox.

    When you have all of your mail stored on a server, you can then tell Thunderbird to connect with IMAP instead of POP. Now keeping your mail safe is your email provider’s problem, and they tend to be a lot better about it. Plus, once it’s on an Internet-connected server, you can easily grab a copy of all of the messages into the mail client of your choice. IMAP really is the universal portable mail format.

    If you want more details on any of this, feel free to email me at the address I used to post this.

    Clint

  42. Mark Lyon’s GML is better than nothing, but still pretty flawed. I gave it a try when I was dabbling with other email systems. The real killer is that GMail helpfully ignores the “Date” on email messages, replacing it with the date that GMail received it. This is good most of the time, since lots of people have misconfigured machines sending email from the distant past or from the future. But GML loads your email by remailing it all to GMail. So all of the dates are the date you ran GML. Want to dig up emails about the trip you went on two years ago? Well, you can’t use the date search functionality.

    I experimented with setting up a POP server and letting GMail pull it over. I remember it was painfully slow; it would take GMail something like two month to pull my archives over. There was some other problem as well, may be the date, that was a dealbreaker, but I don’t remember what it was.

    That GMail won’t let you easily import your old email is really, really frustrating.

  43. Did you keep a copy of the supposedly borked mailboxes? Mails may not actually be lost in those yet; can just be TB that can’t index past a certain point. Other tools should be able to.

    And I second the motion of uploading to an IMAP service (of which GMail will do fine).

  44. Veylon says:

    I have to wonder if this is somehow related to the issue with 32-bit values only being able to reach up to 4GB before rolling over. Something like compacting or indexing could easily need pointers that go higher than that if you have 6GB running around. Maybe you need some 64-bit application?

  45. Illiterate says:

    I suspect, trusted though they might be, that Shamus might balk at the idea of giving his 6 gb of email to google.

    Just as activation servers will fail when the company does, who is to say that gmail won’t unexpectedly become unavailable, or suffer some sort of massive data loss?

    My vote is for an onsite imap server, with regular backups to somewhere offsite and trusted.

  46. Ferrous Buller says:

    Downside is it stores attachments in a separate directory to the email, and they are then linked to the email within the program.

    I’m pretty sure you meant to say “upside” there. ;-) [Unless you mean it dumps all attachments into the same directory, regardless of how your e-mail folders are organized, which is a downside.] I’ve used Eudora for over 12 years now; and storing attachments separately from e-mail messages is a big part of why I haven’t switched. I hate how every other e-mail client I’ve tried embeds attachments within the messages themselves, usually with no way to separate or delete them without deleting the entire message. Lots of times I want to keep the e-mail but delete the attachment: Eudora is the only client I’ve tried which does that.

  47. Teamdest says:

    I deal with a large volume of mail every day. I don’t know the size but it’s several hundred thousand emails, most with ~.5-5MB attachments. For that, I use Wordperfect Mail. Yes, Wordperfect. Try it out, it’s got a very “retro” interface, but the backend is database-like in design, and it’s handled my huge volume of mail magnificently.

  48. blah blah blah says:

    Odds are in your favor those emails may still be there somewhere on your disk, you just have to find them.

    If you can boot from a Live Linux CD that reads your HD, you can grep[1] for a phrase that’s likely to appear in one of your most recent emails. Many people, myself included, who don’t use Linux normally keep around a Knoppix or whatever disc in case Windows gets hosed.

    I’ve never really liked Thunderbird, it really does have those occasional show-stopping bugs that wreck all your data that make it unusable for me. I still think OS X’s Mail is the least-crappy mail client I’ve ever used, but in spite of all its faults Outlook gets the job done acceptably well. I’ve heard good things about Pegasus Mail, too, but never tried it.

    [1] You may be able to do something similar on Windows, but the few times I’ve tried I’ve been less-than-impressed with its abilities to find things I know to exist.

  49. I’ve actually used Hotmail for the last… decade? Holy cow, it has been that long because I know I had my current address in August 1998.

    Yeah, it’s web-based, but it works. I know this probably doesn’t help you any, but it’s been pretty reliable with no data loss or anything like that.

  50. Andrew says:

    Odd. Local folders are not even actively opened by Thunderbird since they are entirely offline and not attached to an account – although the 1.5GB sounds much too big, I must admit, the thread seems to point towards IMAP one way or another. Big advantages, even if you do what someone said and run a “Local IMAP server” which I am sure can be done relatively easily, running one from your website domain is likely easier and allows you to get at the damn things remotely. I’m surprised the journalist in you doesn’t need on the road access to your account :)

    But it’s also odd because I have some similarly large files (4 years of University email and large attachments mainly), although split up like you mention, since they reside in the “local folders” part they just never move and I’ve had no trouble with them. It’s probably near 4 or 5GB at least.

    I can’t help on the compacting issue, which is best solved by moving everything to your local folders (you can move/rename folders/files to get them to appear there), and starting IMAP up, and copying back into the IMAP server everything you want. Only way to be sure really :) (and like someone else said, entirely reinstall Thunderbird – there’s a few files which if corrupted just crash it).

    Good luck anyway, I sounded a bit condescending before, and I still can’t offer an alternative to it, heh.

  51. Justin says:

    You’re still able to recieve mail though, right? I ask because I just finished Free Radical and I thought I might email the author.

  52. Shamus says:

    Justin: Yes.

    Update: I’m actually trying a commercial product, which is an odd move for me. I’m pretty used to hitting SourceForge when I need stuff these days. I’m using a program called “The Bat”.

    First impression: It’s interface is different and I fear it. Having said that, this thing handles my jumbo archives with less straining and labored breathing.

    So far so good.

  53. Miral says:

    Eudora is still the best email-handling client I’ve used thus far — certainly the most powerful when it comes to creating filters (which I do a lot), and the way it splits attachments off separately from the mail leads to nice small mailbox sizes (although admittedly it can also make it a little trickier to manage the attachments). It’s also a bit weaker when it comes to viewing HTML-format mail — but I view that as a feature, not a bug.

    Regarding getting Thunderbird back working again: create a new profile. Almost all T-Bird insanity can be tracked back to something odd in the profile, so this will almost certainly fix it. The tricky thing will be to get your mail back in afterwards — you shouldn’t just blindly copy the files or you may inadvertently reintroduce the problem.

    Otherwise setting up your own mail server might be a good idea. I’ve seriously considered doing that on more than one occasion (and in fact I even started once before getting distracted by something else).

  54. Nick says:

    Your most recent backup is 2 months old??

    I use Hamster as a personal IMAP server:

    http://tglsoft.de/freeware_hamster.html

    The page is in german but the app has an english translation.

    To permanently delete email from TB, do a Shift-Delete.

  55. Thunderbird mail storage seems luckily very simple.
    There is a “archive” and a “archive.msf”

    Not sure what the .msf is, seems to be checksums and lots of other stuff.

    But the archives without an extension are simple raw ascii text. So luckily it’s nothing proprietary like some mail programs.

    Each email starts with: From – Tue Aug 01 22:43:43 2006

    So in theory any email program that can import “raw” email archives should be able to read it.

    Might be fun to make a PHP script or similar and dump it all into a SQL database. :)

    As to why your thunderbird fails. Assuming it’s not a 2GB or similar limit rearing it’s head. My only guess is that the XUL/Javascript stuff in Thunderbird is corrupted somehow?

    It’s a shame there is no neutral email archive format so you could just use whatever email program you wanted.
    But I guess Thunderbirds raw text archives is the closest one gets?

    Hmm, I did loose a lot of file attachments in some really old emails once, I wonder if they are still in there someplace and it’s just the .msf that got corrupted. *scratches head*

  56. I tried to keep about eight years of e-mails, especially my sent ones, as that represents my written correspondence nad some stories I didn’t want to forget but had written down.

    This project has failed time and time again.

    Never mind including the part where my HD overheated and melted.

  57. Eric Ellingson says:

    I’d like to second the suggestion of trying out Pegasus Mail. My wife has over 2GB of saved mail (with 50K+ messages in some folders) and we rarely have problems (and have never lost a message).

  58. Roxysteve says:

    Gotta love the nescape nutz. They’ll argue all day how websites shouldn’t force choices of browser on a person, then turn around and tell you that the problem with the mail isn’t that Thunderbird is broken, which it obviously is, but that you have too much mail.

    Surely, the limit on one’s e-mail archives should be a matter of personal decision and practical storage limits, not some bug in the mail client its distributers cannot be arsed to fix?

    Wait…I feel a stirring in the force…someone…someone is about to type “What do you expect? It’s free!”.

    :o)

    Steve.

Leave a Reply

Comments are moderated and may not be posted immediately. Required fields are marked *

*
*

Thanks for joining the discussion. Be nice, don't post angry, and enjoy yourself. This is supposed to be fun.

You can enclose spoilers in <strike> tags like so:
<strike>Darth Vader is Luke's father!</strike>

You can make things italics like this:
Can you imagine having Darth Vader as your <i>father</i>?

You can make things bold like this:
I'm <b>very</b> glad Darth Vader isn't my father.

You can make links like this:
I'm reading about <a href="http://en.wikipedia.org/wiki/Darth_Vader">Darth Vader</a> on Wikipedia!