-- thom (email)

Response to Advice on website attack

Hi Edward,

Do you have the user-agent string from these requests or a small excerpt from the web logs?

The IP adddress you referenced belongs to Microsoft and when I did some searching for parts of the domain name I found weblogs that listed the domain name bl1sch*******.phx.gbl with a user-agent of "msnbot/1.0 (+http://search.msn.com/msnbot.htm)". If this matches what you are seeing in your weblog and the traffic is coming from an MSN search spider then you'll probably need to either contact Microsoft or attempt to limit the traffic from this spider using your robots.txt file. http://search.msn.com/docs/siteowner.aspx?t=SEARCH_WEBMASTER_FAQ_MSNBotIndexing.htm#D

-- Paul (email)

Still more

On Saturday, March 15, there were 238,447 more pageviews to our ecommerce/shopping cart link from bl1sch4081711.phx.gbl (65.55.107.116), before we blocked this Microsoft IP and sent it off to our toxic IP garden of spammers, trolls, sockpuppets, and Nigerian bots.

The grand total came to 714,032 pageviews. The staff will try to figure things out this week.

The flooded link takes orders for, among other things, my essay "The Cognitive Style of PowerPoint." It was apparently not the case, however, that Microsoft sought to order 714,032 copies of the PP booklet.

-- Edward Tufte

That IP is indeed owned by Microsoft. Each block of address is registered with a central authority (the IANA) when it is given out, and this can be examined by using the WHOIS service.

All the IPs in the range 65.52-55.* are registered with Microsoft:

OrgName:    Microsoft Corp
OrgID:      MSFT
Address:    One Microsoft Way
City:       Redmond
StateProv:  WA
PostalCode: 98052
Country:    US

NetRange:   65.52.0.0 - 65.55.255.255
CIDR:       65.52.0.0/14
NetName:    MICROSOFT-1BLK
NetHandle:  NET-65-52-0-0-1
Parent:     NET-65-0-0-0-0
NetType:    Direct Assignment
NameServer: NS1.MSFT.NET
NameServer: NS5.MSFT.NET
NameServer: NS2.MSFT.NET
NameServer: NS3.MSFT.NET
NameServer: NS4.MSFT.NET
Comment:
RegDate:    2001-02-14
Updated:    2004-12-09

RTechHandle: ZM23-ARIN
RTechName:   Microsoft Corporation
RTechPhone:  +1-425-882-8080
RTechEmail:  noc@microsoft.com

OrgAbuseHandle: ABUSE231-ARIN
OrgAbuseName:   Abuse
OrgAbusePhone:  +1-425-882-8080
OrgAbuseEmail:  abuse@microsoft.com

OrgAbuseHandle: HOTMA-ARIN
OrgAbuseName:   Hotmail Abuse
OrgAbusePhone:  +1-425-882-8080
OrgAbuseEmail:  abuse@hotmail.com

OrgAbuseHandle: MSNAB-ARIN
OrgAbuseName:   MSN ABUSE
OrgAbusePhone:  +1-425-882-8080
OrgAbuseEmail:  abuse@msn.com

OrgNOCHandle: ZM23-ARIN
OrgNOCName:   Microsoft Corporation
OrgNOCPhone:  +1-425-882-8080
OrgNOCEmail:  noc@microsoft.com

OrgTechHandle: MSFTP-ARIN
OrgTechName:   MSFT-POC
OrgTechPhone:  +1-425-882-8080
OrgTechEmail:  iprrms@microsoft.com

# ARIN WHOIS database, last updated 2008-03-15 19:10
# Enter ? for additional hints on searching ARIN's WHOIS database.

-- David Magda (email)

Conclusion

Our expert admin reports:

"Those requests have stopped. I added a record to our /robots.txt
file that tells robots to stay away from the shopping cart page,
and as soon as this robot re-read the file, these requests stopped.

I'm not sure if this is a Microsoft robot or a malicious robot
posing as the Microsoft bot. It would certainly be weird for a
malicious robot to stop after we asked it to.

It's more likely this behavior was the result of some bug in the
robot. The shopping cart is different from other pages on the site
in that we require the user to accept cookies before going on. So
it's possible that the robot got stuck in some kind of loop.

In any case, we went from 4 requests/second to none in the last
20 minutes, so I think this problem has been solved."

These sorts of things happen a couple of times each year;
I get all excited; then our expert diagnoses the problem and
restores tranquility to the website.

I regret that Microsoft created this problem, especially
in a robot that presumably visits a great many websites.
In the future perhaps MS will take greater care in unleashing
their robots onto the world. Perhaps the MS robot director
could respond to let us know if we figured this out correctly.

Thanks everyone for all the thoughtful contributions.

ET

-- Edward Tufte

Ed,

I heard about this thread from some Microsoft employees who recently attended a session in Seattle. I work for Live Search and I wanted to apologize for your site being hit so hard in the Fall. We had an issue with one of our crawlers that was brought to our attention by numerous customers. We do believe we fixed the issue with the crawler in question, but I wanted to check in and make sure you haven't had any additional issues with Microsoft hitting your site too hard. We try very hard to respect peoples sites and are open to any feedback you may have.

You or any of your readers who are having issues can contact me directly at jandrick@microsoft.com and we will look into the issue immediately. Again I am very sorry for any inconvenience this may have caused you.

Jeremiah Andrick Program Manager, Live Search Webmaster Center

-- Jeremiah Andrick (email)

700,000 shopping cart visits in 3 days

Since the economic damage to our shopping cart access was modest,
I appreciate this gracious and thoughtful response.

ET

-- Edward Tufte

What is really amusing to me is that I can't even get to edwardtufte.com from inside the Microsoft corporate network... yet our servers are pinging away at yours?

-- Brian (email)

phx.gbl stands for phoenix global intelligence. http://en.wikipedia.org/wiki/Phoenix_Global_Intelligence_Systems Yeah, it is the U.S. governments spy service. And they use it to moniter and watch people through messenger. Be careful it came up on my computer but as 10.7.something.100 not the usual 64.*.*.* and it does not resolve it is a blackhole. Don't answer the emails or anything if Messenger is connecting to it, they are spying on you. I have also seen that many email scams and other hacking incidents have been related to it.

-- Charles (email)