For the server part you only need .NET framework 3.5. Make sure it is installed and patched.

Domain it not mandatory. i.e. you can run this vCenter installation in a workgroup if you like. For a home lab setup, a single disk should be fine, but as we need to install MS SQL, a separate disk for the SQL server is perhaps a good idea.

For this run i will install the database separately, and when the db is up and running, i will install vCenter 5.0. I shouldn't be necessary - but for some reason, the default vCenter installation seems to do a fairly poor job with the SQL setup. Or at least I don’t understand the setup and consequently Update Manager fails.

SQL 2008 R2 Express is supported for vCenter 5.0.

VMware Product Interoperability Matrixes (http://partnerweb.vmware.com/comp_guide2/sim/interop_matrix.php?)

0. Add feature
1. Install MS SQL 2008 R2 Express
2. vCenter (simple) installation
3. VMware vSphere Update Manager

0. Add feature: .NET Framework 3.5.1

For a 2008 R2 adding the .NET 3.5 feature shouldn't be a problem. If you are testing this on 2012 it is a different matter: Make sure only the .NET 3.5 feature is installed - nothing else.

1. Install SQL 2008 R2 Express

1.1 Local user for the SQL service.
Create a new local user for the SQL service. Note the username and password for this user. Computer Management -> Local Users and Groups -> Users. Add user.

1.2 Install MS SQL 2008 R2 Express (64 bit)
Download and start the installation. If you like, you can start off by running the Planning -> System Configuration Checker. It does simply what the name implies: It check your system for inconsistent with the SQL server. If you are running this test from a vanilla 2008 you should not have any issues. If the Configuration Checker didn't find any problems, start the installation: Installation -> New installation or add feature to an existing installation.
a) Accept the license terms.
b) Feature Selection: All features are default selected, so leave it this way (Detour: If you want to install the database on a different drive - now is the time to change drive)
c) Instance Configuration: This is the name of the server. Not the database itself. If no other database server is already running on the system, leave the default values, unless you need to make things complicated. If a database server is already is running on this box, you need a simple name for this instance - like SQLExpress. Again - remember to change the drive, if you are running the database server on a different drive.
d) Server Configuration: You will now add the Windows user created in step 1.1. Click “Use the same account for all SQL Server services” and add the user and provide a valid password. Also make sure “Startup Type” for “SQL Server database Engine” is set to “Automatic”
e) Database Engine Configuration: Select ‘Mixed Mode’. You must specify who is SA for this db server. This is not the sql user we added in step 1.1 - you must add the administrator account. In the bottom part of the windows you should see something like: [server name]\Administrator (administrator). Add the administrator password. Click ‘Next’ and things should be running along. Hopefully it all ends up with a confirmation that the installation completed successfully.
f) After installation is completed, you should check Windows update. Also check firewall status for this box. If you are running vCenter and SQL on separate computers you must add a inbound rule for port 1433 (tcp). Please check relevant resources how to do this.

1.3 Preparing SQL server
The previous steps have installed a basic SQL server on the system. From here on we need to prepare it for vCenter Server.

a) Open SQL Server management Studio. SQL Manager Studio is a bit flakey sometimes. You should be able to use ‘localhost’ as server name. Sometimes this doesn't work. Try [server name]\SQLEXPRESS or [IP]\SQLEXPRESS. You can use both Windows or SQL Server Authentication. Either your administrator account or the SA added in step 1.2 e).
b) Create a database for the vCenter installation. In Object Explorer, Right click on ‘Databases’ -> New Database. Simply provide a name for the db (vcenter_db)
c) Add a user. In Object Explorer open the Security note. Find “Logins”. Right click on Logins and choose ‘New login ...’ In the ‘General page select “SQL Server Authentication”. Define a name (vcenter_user) and password. De-select “Password policy” for this user and select default database. Default database must be database created in step b). Open the “User mapping” page in the left side.

SQL Manager Studio - User Mapping

You must do the same thing for both ‘msdb’ and the database your have just created (vcenter_db) 1) Map the database. 2) Select ‘dbo’ as default schema and 3) Select ‘db_owner’. That is for both databases.

1.4 32 or 64 bit DSN (Database source name)
This is a bit strange: vCenter 5.0 is okay with a default 64 bit ODBC system DSN. vCenter 5.1 is not. If you are installing vCenter 5.0 you can use default administrator system ODBC DSN from Data Sources (Start menu -> Administrative tools -> Data Sources (ODBC)The default ODBC system DSN on a 64 bit system is … 64 bit. For some reason vCenter 5.1 can’t use a 64 bit ODBC DNS. vCenter 5.1 will need a 32 bit DSN instead. Therefore you will need to add a 32 bit ODBC DSN. You cannot use the normal ODBC data sources administration tool on a Windows 2008 R2 server too add 32 bit ODBC system DSN. Use instead the odbcad32.exe utility. Should be around here: C:\Windows\SysWOW64\odbcad32.exe. Either way you use, do the following:

1) Select ‘System DSN’. and add a ‘SQL Server Native Client X.X’ to the mix.
2) Name: Simple name for this data source (vcenter_db). Use the drop-down to select your database server. It should be localhost or [server name]\SQLEXPRESS. Description is optional.
3) Authentication: Select SQL authentication and select the user created in 1.3 c).
4) Accept the default values.
5) And accept default values for the language settings.
6) Now test the data source connection. If the test failed, please delete the connection and start over. If not - you have now a working 32/64 bit system DSN on your box.

2. vCenter (simple) installation

Time for the actual vCenter installation: Start the ‘VMware vCenter Simple install’. Accept domain restriction and patent agreements.
1) Customer registration: Add whatever you like. Without registration key, vCenter will simply be running in evaluation mode for 60 days. That’s what we are after.
2) Database options: Choose ‘Use existing supported database’. In the drop-down list you should now see the name of the system DSN you just created.
3) Server credentials: Use the username created in 1.3 c) You will possible get a connection error at this point. Please re-boot your server, if the JDBC URL connection string is correct and try again.
4) Accept defaults and use the system account. FQDN (fully qualified domain name) should be [server name]. (server name plus a dot) if you are running your vCenter server in a workgroup. The installer will properly not be able to resolve this name.
5) Destination folders. Accept defaults unless you have a very good reason not too.
6) Linked Mode Options: Create a standalone instance.
7) Configure ports: Accepts defaults (Perhaps paste a screenshot to your evernotes would be a good idea) The ports can easily be changed - but there should not be any port conflicts. If you do get them, you are running services which shouldn't be here. Unless you are very sure these services are not a problem to the vCenter installation - please remove these services and re-start the installation.
8) Ports for Inventory service: Accepts defaults and take a note which ports you are using.
9) JVM Memory: Using a small size inventory. Use common sense.
10) You can try out an ephemeral port or just start the installation.
11) Installation Completed.

3. VMware vSphere Update Manager

Part 1 and 2 are successfully completed, so you will now start the Update Manager installation.
1) Enter location and credentials for vCenter. IP address: The IP address of the 2008 server you are installing vCenter on. Port: Default is port 80. Username and password is admin user for the server. It is not the SQL user created previous.
2) Database options: Use an existing database. From the drop-down list those the system DSN created in step 1.4
3) Database Information: Use the SQL user created previous in 1.3 c).
4) Connection information: Accepts defaults.
5) Destination folders: Accepts defaults. For a test run you will definitely do no need 120 GB. A least 10 Gb for less than 20 VM should do it.
6) Succes!