by

Looking Backwards: Eight Lessons from Twenty Years of MIME

After this week’s celebration of MIME’s 20th anniversary, I expected to feel sated enough leave it alone for another 20 years. But I think it might be worth writing just a bit more, summarizing the lessons MIME might teach about how to create a successful technology standard.

1.  Where you work matters. I devoted roughly 2 years of my life to defining MIME. Not that many employers would tolerate that, but I was a researcher at Bellcore, with a broad mandate to promote more bandwidth use in the future. Other companies support standards work, but few to the extent that Bellcore supported me. It would have been hard to create MIME while working for most technology companies.

2.  Address a real need. Most people didn’t know it yet, but the world really needed an interoperable, open standard for multimedia data; almost everything on today’s Internet reflects this reality. I realized it early because I had built a multimedia email system at Carnegie Mellon, and Steve Jobs had followed up with something similar at NeXT, but the two systems couldn’t exchange multimedia data with each other. I knew that some day I wanted to get pictures of my grandchildren by email, but I didn’t want my kids and I to have to use the same email software.

3.  Address another real need. Any standard will face barriers to adoption, at least from the inertia of the installed base; meeting two major needs can increase the number of people who care, and hence the pressure for adoption. In the case of MIME, multimedia junkies like me were able to make common cause with the deep desire of people around the world to send email in languages other than English. These problems could have been solved separately, but a standard that solved both surely hastened adoption, perhaps even making the difference between success and failure.

4.  Connect the dots and share the credit. Some successful teams self-assemble, but behind most successful teams is a visionary who figured out what parts needed to be brought together. In the case of MIME, the visionary was the late Einar Stefferud, who introduced me to Ned Freed and suggested that we collaborate on the work that became MIME.

Sharing the credit is remarkably useful in leading argumentative technology gurus to consensus. At the end of the MIME standard, there’s a long list of acknowledgements of people who helped draft the standard. I found that adding someone to this list made them less argumentative. There’s no downside to sharing credit generously.

5.  Keep your goals modest, realistic, and limited. I know, extending email to include all human languages and all media types doesn’t sound like a limited goal, but the truth is that we achieved those goals via a very limited mechanism. We avoided trying to settle as many battles as we could, preferring instead to create a framework for the debate to continue. Thus, MIME doesn’t declare  JPEG a better image format than GIF, or PDF superior to HTML and DOC; we just made it possible to unambiguously define labels for these types, such as image/gif and image/jpeg. (The wisdom of this approach is clearest when you consider applying it to the natural language problem: had we tried to specify that everyone should always speak English, or Chinese, we would never have found consensus.)

6.  Acknowledge that your vision is limited. Standards designers tend to overspecify; MIME was designed in the aftermath of X.400, a proposed email standard that failed in large part due to its complexity. Rather than try to imagine every future use of MIME, we created an initial set of media types, and a registry for defining new ones. The result is that the number of media types has grown from under 20 in the original standard to over 1,300 today.

7.  Worry about branding and marketing. This is the lesson I find hardest to convey to technically-oriented people, who tend to dismiss anything non-technical as fluff. The fact is, technologies are adopted (or not) by people, who are subject to a wide range of influences. Good publicity and catchy names really matter.

In fact, the best advice I’ve gotten in my entire career came from Dave Crocker, the author of the original Internet email standards, who convinced me to come up with a clever name or acronym. I laughed, but he was insistent, so after 15 minutes I came up with “Multipurpose Internet Mail Extensions” — MIME which, because it is much catchier than, say, RFC 1341, is often used conversationally.

Essentially, because people have heard the name MIME and perhaps have a vague idea what it is, I have instant credibility with total strangers.

8.  Give it away. If you want to see a standard adopted, it helps to produce a solid implementation and release it as open source software. I built a software package called metamail, a standalone MIME implementation for UNIX that could be plugged into any mail reader, and released it to the world when the MIME spec was stable. Combine real need and free software, and things happen fast. Within a few days, I received patches that made it work on DOS, while Macintosh, Amiga, and others were not far behind. Again, credit is due Bellcore, for supporting building such software only to give it away.

There are other lessons, I’m sure, but most relate to technical details and are unlikely to be of wider value. So now, perhaps, I can stop writing about MIME for another ten or twenty years and see what it looks like then.

Photo CC via Len RadinDave GrayÞorgerður Olafsdottir on Flickr

 

by

Why Is Email So Complicated? Part 362: Too Many Lazy Idiots

I’m probably going to spend the rest of my life completing this series of essays about why email is so complicated, but I’d really hate to go to my grave without mentioning one of the deepest, least soluble, and most frustrating reasons:  lazy idiots.  I rarely call anyone names, so let me explain.

As is probably clear to anyone reading these essays by now, email is inherently very complicated.  The best minds in the email world — and some others — have worked tirelessly and endlessly to figure out how to make it work well.  This has resulted in a host of standards, most of them well specified in excruciating detail.  They’re not perfect, but if everyone complied with them fully, the world of email would be much more reliable and significantly less complicated.

Unfortunately, a remarkable number of people who write software that sends email not only don’t follow the standards, and not only don’t read the standards, but don’t even know that the standards exist.  They look at a few messages, say “I can do that,” and write software that produces things that look, from a distance, like email messages.  The myriad ways such software violates the standards is a significant, ongoing, and probably permanent factor in the complexity of email.

Consider, for example, the hotshot young programmer building a web site for his uncle.  He designs an HTML form that includes a box to input an email address, and when he gets one, he sends his uncle’s information to that address in the form of something with a From, To, Subject, and Date field, but which only follows the standards if he gets extremely lucky. This happens all the time, every day.

A few such messages are an annoyance.  But now and then, the kid and his uncle get lucky.  Whatever it is they’re doing, it catches on, and soon they’re sending out tens of thousands of malformed messages every day.  What happens on the receiving end?

To a certain type of person, it might seem obvious that a mail server should simply reject or discard any malformed mail.  That, surely, would teach the offending party to pay more attention to the standard.  But traditionally, the Internet has been built on the philosophy of “be liberal in what you accept, and conservative in what you send.”  This is the philosophy that has made the net function, and many implementors take it as gospel.

Even if this weren’t the case, the marketplace forces mail servers to race each other to the bottom.   Consider the case of a third party that regularly sends out malformed mail that the recipients actually want.  If mail server A accepts it, and the customer changes to mail server B, the customer will be outraged if mail server B doesn’t also accept the message.  They will consider mail server B’s attempts to enforce the standard to be a bug, and inevitably B will be changed in order to keep the customer happy.  This happens all the time, every day.

The result is that every mail server in the world has to take its best guess about what certain messages mean.  Is an incorrectly wrapped line supposed to be part of the previous header, a new header, or the beginning of the body?  Is an incorrect newline indicator supposed to be a newline or part of the previous line?  (I’ll write more about these specific cases later, if I live long enough.)  Inevitably, different servers make different guesses; once you’re outside the standard, there’s no right answer.  And also inevitably, sometimes these guesses are wrong, and result in messages that appear malformed or unreadable to the end user.

This happens all the time, every day.

It’s unlikely that the world will ever run out of lazy idiots.  But it’s relatively rare for one of them to decide to start naively spewing poor imitations of TCP/IP packets.  Unfortunately, sending email is such a user-visible function, and the format looks sufficiently simple, that email generation seems to be the project of choice for the protocol-challenged.

We can do our best to educate them, but I think there’s an endless supply.  We can cope with it, of course.  But it does make things more complicated.

Image via Robbo-Man and CHRISTOPHER MACSURAK on Flickr

by

What are the different types of Cloud Standards?

Mimecast Chief Scientist Nathaniel Borenstein was in London a couple of weeks ago- and I took the opportunity to interview him about Cloud standards- since he knows a thing or two about standards.

One of the benefits of experience is perspective- and being an industry veteran gives Nathaniel a fair amount of it- certainly more than a young whipper-snapper like me. I was interested- were any applications waiting for Standards? He went on to turn my question on it’s head and ask- what type of standard was I talking about. I’m ashamed to say I didn’t know the difference- I do now.

The second of a two part series- the first is here.

httpv://www.youtube.com/watch?v=2b9WIGV77tU

Justin Pirie: Hi. This is Justin Pirie. I’m here with Nathaniel Borenstein, Chief Scientist of Mimecast. Nathaniel, you’ve been here in the UK this week talking about the standards, I was wondering, are there any potential cloud applications that might be moved to the cloud that might be stuck waiting for standards.

Nathaniel Borenstein: I suspect there are but I’m actually not aware of any. I can cite a couple of examples where specific applications where standards would really help. So for example there’s a lot of Cloud services now that provide virtual machines, on which you can run your own applications and so there’s a growing business in moving from one of those vendor’s to one another and it would help it there were better standards for encapsulating those virtual machines and making them portable.

Having said that the fact that it’s a thriving business now sort of indicates that while those would be nice their not essential and I think that is true for most applications, precisely because the Cloud… the big difference between a Cloud application and a pre-Cloud application is that the data centres are being centralized with a third party, but the actual technology behind that differs from the classic client server only in the multi- tenancy approach really.

Only in the fact that a given vendor is handling and keeping separate multiple customer’s data and keeping them insulated from each other and so on. So you don’t need very many formats, you don’t need very many protocols.

Justin Pirie: So turning this on its head are there any standards that would be helpful for applications that are currently on the Cloud that maybe would help differentiate vendors and give some confidence to people going to the cloud for stuff that’s already there.

Nathaniel Borenstein: Well there’s two parts in my answer there because yes they absolutely are I’ll mention a couple of examples but it’s very hard in general for vendors to strongly differentiate themselves with standards. The whole idea of a standard is that you want everybody to do it, and if everybody’s doing it the standards aren’t differentiating them very much. Although, one of the grand old men of the internet and one of my mentors Einar Stefferud used to say that the rallying cry  of the future would be “We inter operate with everyone else better that anyone else” and I think that’s a reasonable approach here.

In terms of what standards would help, the biggest ones that come to mind are those related to management. You have complex systems that you need to manage and we have a lot of protocols that have already been derived for that in the pre-Cloud world, things like SNMP. In the Cloud world it gets more complicated with the multi-tenancy and things like that so there are extensions to those standards and perhaps some new ones that will make it easier for customers to monitor their applications at these remote sites. Those can be very helpful but again things are mostly working pretty well without them.

Justin Pirie: And I think I would add security to that list.

Nathaniel Borenstein: Well it’s yeah…. you mentioned that. This brings up a whole question of what are standards and I think there is some confusion because there are two types of standards or really two things are called standards that are barely the same thing at all. I’ve been mostly talking about what I would call definitional standards and perhaps that shows a bias that comes from my having worked on definitional standards from most of my career. Things like defining the MIME protocol or the DKIM protocol for signing email messages. A definitional standard defines a protocol or a data format or basically tells people how to do things in an inter-operable way. They’re very essential and they’re very detailed. The other kind of standard is what you might call an evaluative standard and this is something like ISO 9000 where you investigate a service and you score it on a couple of hundred dimensions sometimes and say oh they doing pretty well, they’re doing badly and this is really much less precise.

You try to make it precise but each person doing the evaluation might slightly differently. It’s still useful. It’s very useful in developing the ability to compare vendors in being able to say that vendor is 90% secure and that vendor is 60% secure. So when you mean evaluative standards then yes, they’re extremely useful and security is one of the biggest things they’re useful, for but with ISO 9000 you can also evaluate reliability for example, up time stuff like that and that’s equally important.

Justin Pirie: Well thank you very much that’s been really enlightening. You can check us out on the blog: blog.mimecast.com or @mimecast on twitter. Thank you Nathaniel.

Nathaniel Borenstein: Thank you.

by

Do Standards Mask the Real Issues holding back Cloud Adoption?

Mimecast Chief Scientist Nathaniel Borenstein was in London a couple of weeks ago- and I took the opportunity to interview him about Cloud standards- since he knows a thing or two about standards.

Nathaniel had an interesting perspective- that a lack of standards aren’t necessarily the problem with Cloud Adoption

httpv://www.youtube.com/watch?v=HOSkmauQhgE

Transcript:

Justin Pirie: Hi. This is Justin Pirie. I’m here with Nathaniel Borenstein, Chief Scientist of Mimecast. Nathaniel, you’ve been here in the UK this week talking about the standards and I wondered how much do you think standards are an inhibitor to businesses adopting to the cloud?

Nathaniel Borenstein: I think that standards are often an excuse that masks other concerns about cloud option. There are certainly some issues around standards, but the large number of organizations and applications that have moved to the cloud already are an existence proof that a lot things are reasonable to do in the cloud now. What happens is that a lot of times if an organization is leery of the cloud or has various fears around the cloud, standards are an easy thing to latch on to say “Oh we’re going to wait for more standards” and if you can’t be more specific about what standards are you waiting for you’re probably not thinking very clearly.

Justin Pirie: So what is a good example of an application that isn’t good for the Cloud, because we’re always talking about things that are good of the Cloud, but what isn’t?

Nathaniel Borenstein: Well there some things that really no amount of standards or engineering can possibly make suitable for the Cloud and my favourite example it’s a bit fanciful because it’s so obviously a bad idea but try to imagine if you have an implantable defibrillator that’s sitting in your chest and has the job of restarting your heart if it stops. You really don’t want any of that to depend on the Cloud, because any external dependency could kill you. So that is the worst possible application move to the Cloud and its characteristic of a class of applications, I’d call it real time, non-networked applications.

If you have an application that needs to act in real time, and isn’t already dependent on the network moving into the Cloud adds to that dependency, it’s a bad idea. If it’s real time but already depends on the network, and you do it right the Cloud isn’t likely to or won’t necessarily make things worse, but if you’re already not dependent on the network best stay that way.

Justin Pirie: So taking another angle, are standards an important part of preventing vendor lock-in, in the Cloud?

Nathaniel Borenstein: Well that’s one of the most commonly cited reasons for holding off to the Cloud and for more standards are needed and it’s certainly a valid concern. And what I think that people fear is that their vendor will implement the services in such a way that is extremely hard to transfer their data and application to another vendor.

That can be the case but it rarely is. More often there are other factors that go to vendor lock-in. A good example that Mimecast does we do email management in the Cloud and if anyone ever needed to leave our service and go to another vendor, we would certainly work with them and make sure they can get the data but the issue wouldn’t be one of format for the data or data transmission protocol. Formats for archive data have been around for a long time. Microsoft Exchange uses PST files. Lotus notes uses NSF files and while it might be nice to have a single standard for that, the pre-Cloud world got along pretty well without it and so can the Cloud world.

In fact the harder thing in that kind of migration is just dealing with the sheer quantities of data and you often have to send out technicians with disks because the network isn’t up to the terabytes or petabytes of mail you might be trying to transfer. So what really turns out to matter more than the standards, not that the standards don’t sometimes matter here, is your relationship and trust with the vendor. If you don’t believe your vendor will help you leave them if it ever comes to that, then you probably shouldn’t be with that vendor in the first place. You probably shouldn’t go to that vendor.

Mimecast makes a promise to its customers that if they ever need to leave, we’ll be sorry to see them go but we’ll help them get on to the new location because that’s a part of our professional service and it should be part of the professional standards for the industry and it should be perhaps in some cases even a legal requirement that vendors make this kind of data portability and service portability easy. Standards can be a part of that but I think very relatively small part of that.

Justin Pirie: Absolutely. I mean when I told people that as I say you got to think about the birth, marriage and death of contracts or divorce, and it’s a really important part of contracts that people don’t necessarily spend enough time thinking about. But, if you think about the divorce before you get married i.e. with a pre-nup, then you can make sure you can get your data out well.

Nathaniel Borenstein: That’s exactly right. The commitment that vendors like Mimecast try to make to our customers that we will help you if you leave it’s a lot like a pre-nuptial agreement and its designed to reduce the inevitable pain of the divorce.

Justin Pirie: Absolutely and of course we don’t want a divorce.

Nathaniel Borenstein: No of course and I think we found that the more clearly and believably we make those promises the less often we have to fulfil them.

Justin Pirie: Well thank you very much that’s been really enlightening. You can check us out on the blog: blog.mimecast.com or @mimecast on twitter. Thank you Nathaniel.

Nathaniel Borenstein: Thank you.