Jorge's Quest For Knowledge!

All You Need To Know About Identity And Security On-Premises And In The Cloud. It's Just Like An Addiction, The More You Have, The More You Want To Have!

(2005-11-14) It Works On Physical Hardware And It Does Not In VMware Virtualization Software

Posted by Jorge on 2005-11-14


Have you ever been in the situation where you had to make a choice which Virtual Software to use for your own Infrastructure? I can imagine yes, because at this moment you can choose between a Microsoft solution and an EMC VMware solution. One of the factors that may influence your choice is support of Microsoft products within a virtual environment. If you choose Microsoft Virtual PC or Microsoft Virtual Server as your virtual solution, the Microsoft products within that virtual environment have exactly the same support as if those Microsoft products were running on physical hardware. However, if you choose one of the EMC VMware solutions (ESX server, GSX server or workstation) the support for the Microsoft products in those virtual environments is somewhat different. Microsoft explains the "how’s" and "what’s" in KBQ897615 (http://support.microsoft.com/?id=897615). Simply put Microsoft says: "if you encounter issues with Microsoft products within non-Microsoft virtualization software, Microsoft requires you to reproduce the issue independently from the non-Microsoft virtualization software". The support for all these products also depends on the support agreements you have and how you purchased those products. More can be read at "http://www.vmware.com/pdf/ms_support_statement.pdf".

Well, the main reason for me to write this here, is that I found an issues with Windows Server and Active Directory within a VM Workstation 5.0 virtual environment. I, at first did not imagine there could be an issue and always thought "it will never happen that issues might occur with Microsoft products within a VMware virtual environment whereas they will not occur on physical hardware." I proved to be wrong, as an issue occurred. I was also able to solve the issue. As everyone else I searched the internet for a solution, but did not find any. So for those that experience the same here is the problem description, its solution and my experiences!

The story goes like this…

HOW THE ENVIRONMENT SHOULD LOOK LIKE (within VMware):
* Forest Root Domain
  –> Forest Root Domain: CORP.NET (DFL and FFL = W2k3)
  –> 1 DC = GC = DNS = FSMOs (it happened with Windows Server 2003 SP1 with or without R2)
  –> Name of DC = ROOTDC01
  –> DC points only to itself as prim. Server
  –> DNS server hosts zones: CORP.NET (in domain app part), _MSDCS.CORP.NET (in forest app part), .(root) (in custom app part called ‘ROOTDNSZONE.CORP.NET’ – the only enlisted DC = ROOTDC01), 0.10.in-addr.arpa (in forest app part)
  –> All zones are AD-I and secure DDNS is enabled
  –> NO WINS used!
  –> On DNS server a delegation has been created for the zone BRANCH.CORP.NET to CHUBDC01.BRANCH.CORP.NET

* Child Domain
  –> Domain: CHILD.CORP.NET
  –> 1 DC = GC = DNS = FSMOs (it happened with Windows Server 2003 SP1 with or without R2)
  –> Name of DC = CHLDDC01
  –> DC points only to itself as prim. Server and point to ROOTDC01.CORP.NET as an alternate
  –> DNS server hosts zones: BRANCH.CORP.NET (in domain app part), _MSDCS.CORP.NET (in forest app part), 0.10.in-addr.arpa (in forest app part)
  –> All zones are AD-I and secure DDNS is enabled
  –> NO WINS used!
  –> On DNS server forwarding has been configured to ROOTDC01.CORP.NET

Before installing AD, both servers were stand alone servers and ROOTDC01 had as local password CORP and CHLDDC01 had as local password CHILD.
I installed AD on ROOTDC01 and configured everything as mentioned above. All went OK, no issues. Just for a reminder, the password for the default administrator of the forest root domain was CORP and the password for the default administrator of the stand alone server CHLDDC01 was CHILD.
Before installing AD on CHLDDC01 I did some checks:
* On ROOTDC01.CORP.NET I ran "NETDIAG /V" -> all tests passed
* On ROOTDC01.CORP.NET I ran "DCDIAG /V" -> all tests passed
* On CHLDDC01.CHILD.CORP.NET I ran "dcdiag /test:dcpromo /dnsdomain:CHILD.CORP.NET /childdomain" it said: CHLDDC01 passed test DcPromo
* Pinging between both servers is OK!
* NSLOOKUP from both servers querying different type of records works OK
* No errors in event log
* Adding CHLDDC01 to the domain CORP.NET as a member server works OK
* No firewalls used

After removing CHLDDC01 from the forest root root domain as a member server a make it a stand alone server, I tried to install AD on CHLDDC01 by configuring it as a domain controller for a new domain in an existing tree. At some point I needed to enter credentials with enough permissions to install a DC in an additional domain. So I used CORP\Administrator with the password CORP install AD on the DC. That last action failed after entering credentials and clicking OK to continue. The error presented was:

————————————-
The wizard cannot gain access to the list of domains in the forest
This condition may be caused by a DNS lookup problem. For info… blababla….
The error ‘The RPC server is unavailable’

————————————-

In short, I was not able to make whatever DC configuration I wanted to for CHLDDC01 (additional DC for existing domain, new domain, etc.) as it kept presenting me the same error over and over again…Irritating!

After a while I started thinking about the error as it was described: "The wizard cannot gain access to the list of domains in the forest. This condition may be caused by a DNS lookup problem"… but NSLOOUPs were working OK. So it had to be something like permissions or corrupt data or whatever, but not name resolution because that proved to work. Imagine a Enterprise Admin not having enough permissions to do stuff in AD. Oh, well..
 
To test permissions and credentials I created a mapping (to the ADMIN$ share) from the stand alone server to the forest root DC and used username administrator and password CORP. result = OK
To test permissions and credentials I started LDP on the stand alone server and connected to the forest root DC and used username administrator and password CORP. result = OK. I was able to anything in the directory.
To test permissions and credentials and joined the stand alone server and made it a member server of the forest root domain using the username administrator and password CORP. result = OK.
 
I logged on to the stand alone server with administrator and password CHILD, which is obvious.
I started DCPROMO to install a DC for a child domain in an existing tree and as credentials I entered the credentials of the forest root domain administrator being administrator with password CORP. NO GO!

So the name resolution tests and permission tests proved OK, and I still was not able to promote the damn thing to a DC. WTF!

Finally I decided to do a network trace, but at first I did not believe it would give any hint as name resolution was working OK and authentication was also working OK as proved by the mapping and LDP. OK, lets see what it gives…
I started the trace, started DCPROMO and after entering the credentials it gave me the error. Did the same with a custom enterprise admin I created in AD and it also gave me the error. In the output of the sniffer I saw CHLDDC01 connect to
\ROOTDC01IPC$ and after that it presented the error:

————————————-
SMB (Server Message Block Protocol)
     SMB Header
          Server Component: SMB
          SMB Command: Session Setup AndX (0x73)
          NT Status: STATUS_LOGON_FAILURE (0xc000006d)        <<<<<—————–???????
————————————-
 
How could there be an auth. failure when authentication proved OK? Suddenly I changed the password from the stand alone server administrator to match the password of the forest root domain administrator. Started DCPROMO again AND IT WORKED!.

Stopped DCPROMO. Started it again and tried it with the custom enterprise admin and THAT ALSO WORKED! Changed the password for stand alone server administrator back to CHILD, started DCPROMO again and that ALSO WORKED!
 
The fun part: it never gave an access denied error, it just said that DNS was not OK! Grrr…
What I’m still trying to understand is: WHAT AND WHY? That is one I think for Microsoft and VMware to solve!
 
LESSON LEARNED: if everything else fails get that network sniffer working as soon as possible!

This issue occurred in VMware Workstation 5.0 and I have seen other posts on the internet that have a similar issue using GSX server and ESX server. However, the issue DID NOT occur on physical hardware (as those posts also mention).

 

Cheers,
Jorge
———————————————————————————————
* This posting is provided "AS IS" with no warranties and confers no rights!
* Always evaluate/test yourself before using/implementing this!
* DISCLAIMER:
https://jorgequestforknowledge.wordpress.com/disclaimer/
———————————————————————————————
############### Jorge’s Quest For Knowledge #############
#########
http://JorgeQuestForKnowledge.wordpress.com/ ########
———————————————————————————————

21 Responses to “(2005-11-14) It Works On Physical Hardware And It Does Not In VMware Virtualization Software”

  1. Hmm who was telling me that Vmware was better than Virtual PC ;P

    C

  2. Lets see if comments work now..
    Jorge

  3. Yes, comments work now!

    Ok, what I wanted to ask you: did you do your test on an internal VM NIC, or on a bridged interface? I have had vague networking issues in the past that were solved by going to the other; I forget which one gave the trouble.

  4. Willem,

    This occured in VMware Workstation 5.0 using one of the internal networks it has. The fun part was I was able to reproduce it, even after reinstalling tje whole thing from scratch. Still don’t understand why, but it looked like DCPROMO was not using the credentials entered and used the credentials of the logged on user. Don’t know what that has to do with VMware…!!??!!

  5. Hi Jorge,

    Thought I’d comment on this as I see you asked us (MS) to investigate. :: (BTW, next time you want us to investigate this, please let us know….we can’t read every blog you know :))

    Anyway, let’s look at the error again:
    The wizard cannot gain access to the list of domains in the forest
    This condition may be caused by a DNS lookup problem.

    And then you said:
    The fun part: it never gave an access denied error, it just said that DNS was not OK!

    I think the problem here is the reading of the error.
    The error told you what we have *commonly* observed the error to be…DNS. It did not say that the error was in fact a DNS problem. Rather, it said that the problem was:
    The wizard cannot gain access to the list of domains in the forest

    The DNS mention was only what we have typically observed the condition to be.
    Very often, we don’t know what is causing an issue. We know there is failure, but we can’t figure out the problem….be it for security reasons or just because the code was not written that way. In these cases, we try and at least give you a clue…..let you know what we have commonly observed the problem to me. In this case, we commonly saw it as a DNS problem, so we have you the clue we had.

    So, I guess what I’m saying is, when we say it *may* be a DNS problem, I’d point out that it *may* not be too.🙂

    On the error quality…if you show me the trace, I can take a look and see if we can improve the error in the next version of dcpromo. You know my email address….so please email me the trace if you don’t mind.

    Thx!
    ~Eric

  6. Hi Jorge,
    I have the same Problem with vmware GSX Server!
    The problem is first occur with Microsoft SP1 (for W2K3). Without the SP1 there is no problem!
    I have big trouble since SP1, because I can not test our automautomatic installation with VM!! I always must install on physical hardware ….grrr …. what a waste of time!

    If you have a workaround, can you mail me please?

    Joo.Meyer@gmx.net

    PS. I’m from germany!

  7. Hi Jorge,
    I have the same Problem with vmware GSX Server!
    The problem is first occur with Microsoft SP1 (for W2K3). Without the SP1 there is no problem! I have big trouble since SP1, because I can not test our automautomatic installation with VM!! I always must install on physical hardware … grrr … what a waste of time!

    If you have a workaround, can you mail me please?
    Joo.Meyer@gmx.net

  8. I found another situation where something goes wrong and where changing the password solved it. This time I wanted to create a trust. Read more at: http://blogs.dirteam.com/blogs/jorge/archive/2005/12/18/297.aspx

  9. Deji said

    I know that you specifically mentioned that this happened to you on VMWare. But I thought I’d just let you know anyway that – ready? – it works fine on Virtual Server. Hehe🙂

    Now, having said that ….. I am having troubles following your config here. I may be a bit slow, or just reading too fast for my brain cells to catch up. So, let me see if I understand you correctly.

    1 Forest root called Corp.net. Has only 1 DC that does everything. Correct?

    So, you have a server called CHLDDC01. You joined this server to corp.net as a member server, and there was no problem. You disjoined this server from corp.net with the intention of re-using it later. Correct so far?

    The local admin password on CHLDDC01 is different from corp.net’s administrator’s password. Correct?

    Now, you ran dcpromo on this server and tried to create a NEW child domain under corp.net. That’s when the trouble started?

    Did I get all that correct?

    If not, then I must have read too fast. If yes, then you are confusing me – please read on…..

    You posted the following:

    * Child Domain
    –> Domain: CHILD.CORP.NET
    –> 1 DC = GC = DNS = FSMOs (it happened with Windows Server 2003 SP1 with or without R2)
    –> Name of DC = CHLDDC01
    –> DC points only to itself as prim. Server and point to ROOTDC01.CORP.NET as an alternate
    –> DNS server hosts zones: BRANCH.CORP.NET (in domain app part), _MSDCS.CORP.NET (in forest app part), 0.10.in-addr.arpa (in forest app part)
    –> All zones are AD-I and secure DDNS is enabled
    –> NO WINS used!
    –> On DNS server forwarding has been configured to ROOTDC01.CORP.NET

    I am going to assume that this is the configuration you have AFTER you have succeeded in promoting the child DC. I said that because…..obviously you’d not have been able to do all that without a valid DC in place. So, if this is the config you have AFTER, what is your config BEFORE and DURING the DCPROMO. Again, maybe I’m reading too fast, but I did not see where you indicated what DNS server the server was pointing to after you disjoined it from corp.net and while you were running dcpromo.

    You also wrote:
    So I used CORPAdministrator with the password CORP install AD on the DC. That last action failed after entering credentials and clicking OK to continue.

    IIR this screen correctly, there would be 3 inout fields (username, password and domain). Do you remember ever trying administrator (instead of corpadministrator) and typing corp in the domain field?

    Mind you, I am not trying to imply that there is no bug in the process or that you don’t have a valid solution. I am just saying that the steps you have outlined above does not provide enough information to satisfactorily conclude that you are not the one fat-fingering stuff🙂

    Now, off to go test your “broken-trust” discovery – on Virtual Server since I don’t have VMWare. Something tells me that I will be able to gloat again and tell you “use Virtual Server instead”🙂

  10. Hi Deji,

    The environment/configuration I wanted to achieve is mentioned in “HOW THE ENVIRONMENT SHOULD LOOK LIKE (within VMware):”

    The environment/configuration I started with is mentioned in “Before installing AD, both servers were stand alone servers and ROOTDC01 had as local password CORP and CHLDDC01 had as local password CHILD”

    I first encountered the error when creating a new DC for a child domain. Then I went back and tried to create an additional DC for an existing domain. That did not work either. I also tried to just join the domain as a member server and that did work!

    When entering credentials in DCPROMO I did not enter CORPADMINISTRATOR (this is just a way of writing it down) I entered each one in the appropriate field:
    : administrator
    : CORP
    : CORP.LAN

    The first time I experienced this I thought I did something wrong, so tried it again and again and again. I checked credentials, permissions, nameresolution, everything. I even did the DCPROMO test in DCDIAG. Everything showed: OK! And DCPROMO showed me the error.

    So this case, no fat-fingered stuff involved. It is all real! Tony experienced the same issue with the creation of the trust between 2 SP1 forests.

    So there is only one conclusion:
    You have read it too fast AND you are confused! hehehehehe ;-))

  11. Deji said

    Jorge,

    looks like you are on your own🙂

    Quick question, though. When you have a free moment, could you try and repro my “differencing hard disk” SID duplication issue in VMWare. I am not sure if VMWare has something equivalent to Virtual Server’s “differencing virtual hard disk” concept, but I suspect they must have one.

    Try and see if you get the same behavior.

  12. Jorge,

    we had the same problem.

    just don’t install the “Shared Folders” from VMWare Tools setup!


    Magnus Schaefer
    magnus_schaefer@gmx.net

  13. Jorge,

    Magnus is correct. I had the same problem and removing the Shared Folders made it work. To be more precise, the driver hgfs.sys should not be loaded.

    In certain cases, I had to remove this driver manually from the VM.

  14. Cool,
    Magnus, you are a hero! I was looking for hours about that problem. Without hgfs.sys is everything ok.
    Thanks thousand times!

  15. Magnus, you are great. I spend really A LOT of time with that.
    The funny thing: In my scenario the problem is gone when you log on as an domain admin on the member server that is going to get a dc. But i did not want to do it that way.
    The problem is also gone when you map a drive to the FQDN of the first DC an then start dcpromo.

    Thanks a lot.
    frank

  16. it seems that with a newer version of vmware the problem is also gone…

  17. Blast!
    Thanks for your excellent tip!
    Had the same issue!

    Host ESX 3.0.2 b52542

    Had to change the local password of the member server (first tried standalone server) to the same as the domain admin.
    Then I could promote this server!!

    Cost me 3 hours! but then I found you .. (actually, been here before)

  18. […] Remember me writing about not being able to promote a new DC into the forest as an additional DC for an existing domain or as a DC for a new child domain? (read more about it at: https://jorgequestforknowledge.wordpress.com/2005/11/14/it-works-on-physical-hardware-and-it-does-not…) […]

  19. […] It Works On Physical Hardware And It Does Not In VMware Virtualization Software […]

  20. […] It Works On Physical Hardware And It Does Not In VMware Virtualization Software […]

  21. […] (2005-11-14) It Works On Physical Hardware And It Does Not In VMware Virtualization Software […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

 
%d bloggers like this: