July 8, 2002
BOOM TOWN
By LEE GOMES  
The Wall Street Journal online 
 
Hotmail Has Quite a Job to Save its E-Mail Empire From Spam

You think you've got a problem with spam e-mails because of the dozen or so you
get every day? Welcome to Hotmail, where they get more than a billion.

Hotmail, owned by Microsoft, is, by virtue of its 110 million users, among the
world's biggest e-mail providers. It is, therefore, one of the world's biggest
spam buckets. The number of messages it gets each day is closing in on two billion.
Up to 80% are spam.
Spam, for someone in the e-mail business today, is like cold for someone at 
the North Pole. It's everywhere, and if you forget about it even for a minute, 
it can kill you. Hotmail engineers constantly monitor their machines. A sudden
deluge of spam, if not tended to, will take down the whole system.

SURVIVING THE FLOOD

Is your junk e-mail out of control? If so, what pitches bother you most? What have
you done to stop the flood? See our junk e-mail page, or join a discussion 
with other readers.
 
The Internet wasn't supposed to be this way. But then again spam, though 
universally despised, is the purest expression of the Internet's egalitarian 
technical vision. Everyone is connected; communications are free; anyone can be
a communicator.

"Spam is the Internet's only indigenous folk art," is how Steve G. Steinberg,
a San Francisco researcher, consoles himself while deleting his daily spam.

Internet arithmetic favors spam. Type "bulk email" in Yahoo; you'll see 
a long list of offers to sell you millions of addresses for a few hundred bucks. 
That means the tiniest acceptance rate puts you in the black. And so the
contemporary spammer is not some shadowy pornmonger, but a debt-plagued 
middle classer who decides to try spam instead of, say, Amway.

Hotmail started out free of charge. One reason Microsoft is now trying to persuade 
users to pay for it is that the drastic increase in spam has made free e-mail very,
very expensive to offer.

Bengt-Erik Norum, a Hotmail operations manager, says spam, by nature, arrives
all at once -- in torrents -- rather than in a steady stream, like regular e-mail. 
To handle these peaks, he said, you need to greatly overbuild your computer system.

This overbuilding is evident in Hotmail's bunker-like operations center in 
San Jose, Calif. In some ways, it's the house that spam built. The rooms -- icy cold 
from air conditioning -- house row upon row of stacked gear, panel lights all a-blinking.
New gear is added almost weekly to keep up with demand. There are thousands of servers, 
and many pedabytes of storage. (Each pedabyte is a million gigabytes; you probably have 
10 or 20 gigabytes on your PC.) The electricity used here could power 7,000 homes.

When you talk to Hotmail staffers, it's obvious how proud they are of their technical
accomplishment: keeping Hotmail humming along despite, for example, the tripling of 
e-mail in the past year. Too bad much of that victory involves simply not letting 
the spammers win.

Hotmail and the spammers play the inevitable technical cat-and-mouse game. 
Hotmail sets up dummy e-mail accounts to monitor spam. So, of course, do the 
spammers, monitoring the monitoring.

The graphs for spam at Hotmail show a sharp downturn whenever a new antispam
feature goes live. But soon, the line starts moving up again, and after a few weeks, 
it's nearly back to normal.

Hotmail blocks messages from an IP number it identifies with a spammer, 
who promptly moves on to a new one. Hotmail's spam filters might start to flag
e-mails with, say, more than 50 names on the "To" line. The spammers see that,
and try again, this time with 49. It's the same game with certain words in the
subject line. Hence the constant new ones:

Lose weight by summer;
we'll pay you to eat and shop:
do you desire more?

Hey! A spam haiku.

Spam is a great artificial intelligence problem: A person can spot it right away, 
but not the smartest computer. Of course, computer programmers keep trying to make 
better spam-killers. Rick Holzli, Hotmail's development director, said there are a
number of new ones out there being considered.

Something else Hotmail is considering: redesigning its system so most users 
can get e-mail only from preapproved addresses: friends, family, etc. That's 
already an option, but used by less than 10% of Hotmailers.

That would be radical change to the sociology of e-mail. But one day, all e-mail 
might have to work this way. Thanks, spammers!

It's unclear how, in such a world, you'd ever see that out-of-the-blue e-mail 
from a long-lost pal. But think of all the other things you'd never see -- and 
never miss.

Write to Lee Gomes at lee.gomes@wsj.com