a glob of nerdishness

Although the NSFastEnumeration Protocol reference is pretty obtuse, the linked article tells all you need to know about how fast enumeration is implemented. It’s not really all that complicated — or pretty, for that matter — but it nonetheless allows for (... in ...) looping over any conforming class. All that said conforming class has to do is give out a pointer to its own internal structures when possible, or, at worst, copy a handful of pointers at a time into some pre-allocated stack space.

[Editors note: I'd been editing this article for a few days, and then John Gruber sent everybody to a similar article on 5thirtyone instead. I'm not jealous or anything — this glob is too typographically atrocious to merit a link from DF at present — but I still wanted to put this up for my own reference.]

Google recently rolled out free IMAP support to Gmail users. This is a neat gesture, but they twisted the IMAP protocol so that it works in The Way of the One True Algorithm. In their own words, “we’d like to make your IMAP experience match the Gmail web interface as much as possible”. Fortunately, Apple’s Mail provides the tools necessary to work around most of this Google IMAP “experience”.

The skinny

1. Set Mail.app to work with Gmail
2. Google has recommended settings. Ignore them. Well, DO uncheck “Store sent messages on the server”, unless you are using a non-Google SMTP server. But don’t uncheck “Store deleted messages on the server” or “Store junk messages on the server”.
3. Map Gmail IMAP Folders to Mail.app Default Folders:
   

   Use mailbox	For Mail’s…
   [GMAIL]/Trash	Trash
   [GMAIL]/Spam	Junk
   [GMAIL]/Drafts	Drafts
   [GMAIL]/Sent Mail	Sent, only if you are using a non-Gmail SMTP server

4. Google has a table showing what actions in your email client do to your Gmail. Read them, but realize half of them are wrong or irrelevant with the way Mail.app is now set up. Here’s some corrections:
   

   If you want Gmail to…	Do this in mail
   Apply a star to a message.	Flag the message.
   Apply a label to a message.	Copy the message to the corresponding folder.
   Remove a label from a message.	Move the message to “[GMAIL]/All Mail”. Don’t delete the message.
   Undelete a message.	Move it to “[GMAIL]/All Mail” or another label.
   Make “All Mail” not match what Mail.app shows	Delete a message from “[GMAIL]/All Mail”. Don’t do this!

5. As you use it, sometimes things get out-of-sync for a bit since Gmail is changing folder contents behind the scenes in ways that Mail.app doesn’t expect. If you want to make sure that what you see in Mail reflects the way things are in Gmail right away, use the “Synchronize All Accounts” item in the “Mailbox” menu.

The key to understanding how Google changed IMAP is to realize that the folders it presents are never locations in the sense that folders usually are, which is the sense that Mail.app treats them. Every Gmail IMAP folder is just a “Label”, a tag. Because of this, you may find (or place) copies of the same message in multiple folders. To Mail.app, these each look like individual messages with an identity of their own. To Gmail, they are just differing representations of one underlying object, which can only be deleted via the Trash.

The gory details

So what’s the problem with just doing it the way Google says to? Having folders represent labels means that Mail.app’s “Delete” button won’t work like it does for with normal IMAP accounts. When you “delete” a message in Mail, it removes it from whatever folder it was in and puts it in a deleted items folder. Since it doesn’t know about Gmail’s Trash folder, it creates a new one named “Deleted Messages” and moves your deleted items there. This is a problem, because to Gmail you’re just removing the “Inbox” tag and adding a tag called “Deleted Messages”. You haven’t really deleted the item, and it will still show up in “[GMAIL]/All Mail” and any other Label folders it was in. Then when you empty Mail’s trash, Gmail just sees you removing the “Deleted Messages” label and the message lives on, even if orphaned.

Deleting

To actually delete the underlying message, you must place a representation of it into the “[GMAIL]/Trash” folder-aka-label. (That’s what the mapping in step 3 is about.) While it hangs out there, the message will be hidden from all the other label folders it was in. If you move it back to another “folder”, it will reappear in all the Labels it previously had. There is one catch, though.

Not deleting, when we don’t mean to

What if we want to keep our message, but just remove a particular Label? If we hadn’t told Mail to use the “[GMAIL]/Trash” folder for storing deleted messages, we could just delete a message from the corresponding folder to clear that label. But if we do that now, Gmail will get not only a message saying “remove this message from Label” but also “add this message to [GMAIL]/Trash”. This will cause the message to be hidden from ALL labels, and when we empty the trash it will disappear for good. So we can’t do that, despite Google’s suggestion. Instead we move it, which sends two messages to the server: “remove this message from Label” (thus accomplishing our goal) and “add this message to the [GMAIL]/All Mail folder” (where it probably already is anyway). The same trick can be used to undelete a message as well.

When to move, when to copy

When copying message within Mail.app, Gmail is smart about maintaining only a single underlying identity. This is important, because to add a label you can’t really “Move” a message from one folder to another, because that would also remove the label you moved it from. So, generally, to add a label to a message copy it instead of moving by holding down the command key while you drag. If you do want to remove the Inbox label (for example), then by all means do move instead of copy.

Regarding “Inbox” and “All Mail”

Both “Inbox” and “All Mail” are just tags. If you remove an item from either, it stays on Google’s server (unless you move a “copy” from any folder into “[GMAIL]/Trash” or “[GMAIL]/Spam”, which we’ve set up Mail to do). There seems to be a discrepancy between “All Mail” IMAP folder and the “All Mail” view online: if you delete messages from the folder in Mail.app it still shows up in the web interface.

In thinking about wrapping up…

If you’ve got any other questions, tips or corrections feel free to leave them in the comments. Or in the comments on my new arch-nemesis’s article. But keep in mind, when I find out which friend in Omsk told friend in Tomsk the results of my research, there will be great suffering in Guilder. (Kidding, kidding!)

Let’s say you come into work and find a note on your desk from your boss: “The mail server went belly up.” All the messages on the IMAP server are gone. What to do?

The first step, and this is very important: Do not open Mail.app until you’ve made a copy of the offline mailbox cache. (You can hold Shift while logging in to keep it from automatically opening.) If you let Mail sync to your now-empty IMAP account, it will erase your offline copies lickety-split. As long as this doesn’t happen, it’s pretty easy to restore the server from your local backup.

1. Find the corresponding IMAP-user@host folder inside of ~/Library/Mail/. Make a copy somewhere safe, like your Desktop.
2. Rename all the .imapmbox folders inside of your new copy to have the .mbox extension instead.
3. Now you can open Mail, and import the main backup folder. Select “File > Import Mailboxes…”, choose the “Mail for OS X” option and then select the modified IMAP-user@host folder.
4. Move folders back onto the IMAP server. You might need to make one new folder (”Add Mailbox”) on the IMAP server so that it shows in the sidebar, and then you can drag the rest from the Import folder. Any sent messages or todos can be moved to those special mailboxes as well.
5. If Mail complains that the folders you are trying to drag in already exist, one workaround I found is to delete the IMAP account and set it up again. A simple “synchronize” might have also done the trick.

Once Mail is done uploading the messages, you can delete your “Import” copies of each Mailbox. Then you can get back to seizing the day, whilst hoping you don’t have to do any of this again.

As reported by Gus Mueller, remembered by Fraser Speirs and reverse engineered by dragging messages from Mail.app into a rich TextEdit document: Mail.app now supports permalink URLs to messages.

The URLs are just “message:” followed by the Message-Id, which should be URL encoded. So if your Message-Id is “<abc%20071105@sender.org>” — the angle brackets are considered part of the Message-Id — the URL becomes “message:%3Cabc%2520071105@sender.org%3E”, although “message:<abc%2520071105@sender.org>” will also work from Safari’s address bar.

This is basically just the mid: URL scheme with a different scheme name. Why they used “message” instead of “mid” is strange, especially since on the Flying Meat forum there’s a discussion of links provided by an add-on called MailTags that use a similar URL scheme. These have an extra “//” before the Message-Id, and make Mail grumble that “No associated application could be found”. Update: On second try the extra “//”s seem to work as well, so the real question is why MailTags, not Apple, didn’t use the “mid:” form.

To get the Message-Id, select “Long headers” or “Raw Source” in Mail’s “View > Message” menu. You can also drag a message from Mail into a rich text field to get the hyperlink. Enjoy!

Update: John Gruber has put together a more definitive article on this topic, though he doesn’t make mention of the (seemingly historical) “mid:” URL scheme.

If you’re able to connect to a folder on a Windows SMB share via smb://beigeboxname, you can find its IP address via nmblookup beigeboxname. Handy as Remote Desktop Connection doesn’t seem to be able to lookup the machine by that same name.

I’ve been using Leopard for just over a week now, sans Time Machine even, and it’s good stuff. For the record, a lot of the glitches seem to disappear after one reboot. I’m getting used to Spaces, though I still wish there were some little onscreen indication of which space I’m currently in. And I already might be attached to the glitz of the Dock/menubar.

• Top feature so far: Xcode 3.0, hands down.
• Top little fix: No intermediate .tar file from .tar.gz extraction.
• Coolest spectacle: iChat switching between local and shared screens.

Over and out.