Just over a year ago, when I went through the logs of an IPS located at a dedicated and internal network (not facing the Internet or any external networks), I saw DHCP activity.
Since this is a servers based environment, presumably they all should have a fixed IP address, so any DHCP activity may indicate a non-authorized activity on the network, so I went into the details of the events.

So yes, they were DHCP Discovery events, but their details included three disturbing attributes:

First, the MAC addresses were obviously fake and not belonging to any networking vendor. For example, 4d:c8:43:bb:8b:a6 or 45:3b:13:0d:89:0, which did not return any match at a MAC-to-Vendor search site.

Second, the MAC addresses changed every 3 seconds …

Third, The Domain Name was DETECTIVE … (which of course did not exist on the network in question)

Oops, I thought to myself, there is someone or something bad going on around here…

Immediately I turned to the Internet, and I saw that I was not alone, nor the first, to encounter this phenomenon, wondering about a possible breach.
I searched but found nothing at Microsoft’s web sites, both regular and support, not by these MAC addresses, nor by the word DETECTIVE.

I turned to the employee responsible for the relevant VLAN in which the activity occurred and he said that at the date and time in question he was working on a particular “Windows 2003” based server. I checked the MAC addresses of the server and none of them matched the IPS detected MAC addresses, so I investigated the server’s running processes and all the processes that load at boot time – but nothing unusual was found.
As I was unable to go further in this investigation, I filed this case as a computerized voodoo and moved on with my work.

But after a month or so, the phenomenon returned, with exactly the same behavior. Again the same employee and again a “Windows Server 2003” but this time in another server.
This time, I said to myself, I am going all the way.

I thoroughly questioned the employee, whom I know as a knowledgeable and responsible person, and he told that he thinks that in both cases he used the “Configure Your Server Wizard” of the Windows server.
If so, I said to myself, let’s move up in the food chain.

I turned to the integration company that gave us support, with all the relevant links I found online, but no one there knew anything about this, so I asked to escalate the case to the Premier Support of Microsoft in Israel, and so it was.

But guess what? The Premier supporter didn’t really knew what to do or give an answer, although I handed to him all of the online references to similar cases and directed him to the problem applet. Still he denied that Windows Server 2003 has any issue and he strongly claimed that our network has a live malicious code or sophisticated intruder inside of it. He probably did not try to reproduce the case in a laboratory with a Sniffer or something even close to that.
I demanded to escalate the case higher in the support chain, abroad, but he refused, and even refused to transfer his answer in writing.

He did not know who he is messing with… 😉 Since I am known as a nagger who doesn’t rest until he reaches the bottom of thing, I turned to “Microsoft Israel” and demanded that the case will be escalated to a higher support level, abroad, because something is going on here.
It took them a while and I had to “yell” a bit over email, but eventually they agreed.

And then, at last, enlightenment! Paul from the premier support in England, who fortunately had access to the source code of all of Microsoft’s products, confirmed, in his first reply email, that the word “Detective” exists only once in all of MS products’ code, and only at the source code of “Windows Server 2003” and it is located in the applet of “Configure Your Server Wizard”…
In short, he confirmed my findings and those found on the internet, and said it was the way of the Wizard to locate active DHCP servers on the network, to impersonate as a client that does not exist, only for the discovery phase.
I explained to him that this method can cause some information security and network administrators a heart attack, seeing forged MAC addresses appear and change rapidly in succession plus a fake domain called DETECTIVE … and so Microsoft should publish a support knowledge-base article on the subject, so customers will receive “All Clear signal” from the manufacturer itself. He agreed with me and said he would see to it.
Pleasure. That’s support.

I tried to ask for public credit to me in the coming KB article, like the previous times, but apparently this time I didn’t reach the needed threshold to accept a public credit.

So, overall it took MS a few good months, and the first version was too technical and confused, but in the end they balanced with a reasonable version though not very friendly or relevant to the issue, with KB 945948, which appears as the first search result if you search using Google for the words DETECTIVE and DHCP at the support site of Microsoft – so the risk of a heart attack has decreased, and so I hope I could save at least one admin’s life…

I’ve done my part.
.

(This post is a translation of a post from my Hebrew information security blog, from 10-August-2008)
.