iThinkVirtual™

How To: Create A VMFS5 Datastore On A USB Drive

Create A VMFS5 Datastore On A USB Drive

Ever wondered if it was possible to use a USB Drive as a VMFS5 datastore in VMware vSphere 6.0?  I sure know that I have!  Not that I would like to run any VM’s on said datastore, as I’m sure performance would not be optimal, but instead to test its functionality and use it for storing ESXi host logs for example.  Well, I ran into an issue today where I needed to unmount all of my NFS mounts on ESXi 6.0 U2 in order to recreate some of the volumes before remounting them.  The problem was that I was unable to unmount one of my volumes because it was bound to the ESXi host for scratch logs.  As I didn’t have a spare drive of any sort to attach to my host so that I could reconfigure the location for scratch logs, I began tinkering with the idea of using a small USB drive as a temporary datastore for these logs.

After doing a little research, I came across a post from Florian Grehl aka @virten  showing exactly how to accomplish this so I figured I’d share the process of doing so.  Keep in mind that this should only be used for testing purposes and should not be used in production environments.  This is unsupported by VMware.  Here we go!

Ensure that the USB device is unplugged from the ESXi host then begin by connecting to your ESXi host and stopping the USB arbitrator service.  This service is responsible for allowing USB device passthrough from an ESXi host to a virtual machine, so keep in mind that you will no longer be able to pass through USB devices to VM’s until this is restarted.  (Note: restarting service after creating and mounting USB datastore will break connectivity and recognition of the USB datastore).   To stop the service, run the following command:

/etc/init.d/usbarbitrator stop

2016-08-11_17-12-192016-08-11_17-12-35

Optionally, if you’d like to permanently disable the service so it persists thru reboots, run the following command:

chkconfig usbarbitrator off

Plug the USB drive into your ESXi host.  For the purposes of this tutorial, I am using a small Lexar 8GB USB device.  If you navigate to the storage devices section on your host, you should now see the connected USB device is recognized by the hypervisor.  Make note of the device identifier number (mpx.vmhbaXX) for this device.

vSphere Client:

 2016-08-14_16-53-34

vSphere Web Client:

2016-08-14_17-02-22

vSphere HTML5 Web Client:

2016-08-14_17-02-44

You can also list the device information to determine the identifier by running the following command:

ls /dev/disks/

2016-08-11_17-13-09

As we can see, my identifier is mpx.vmhba40:C0:T0:L0 for this device which also matches the identifier from the GUI pics above.  Note: The other USB (mpx.vmhba32:C0:T0:L0) is a separate USB where ESXi is installed on.

2016-08-14_16-48-33

Next, we need to create a GPT (GUID Partition Table) label on the device.  To do so, run the following command using the correct identifier for the drive.  In my case, I will run with mpx.vmhba40 for all of the following commands.  Be sure to change this to your correct ID.

partedUtil mklabel /dev/disks/mpx.vmhba40\:C0\:T0\:L0 gpt

2016-08-14_17-04-20

Now run the following command to get the partition table information.

partedUtil getptbl /dev/disks/mpx.vmhba40\:C0\:T0\:L0

2016-08-14_17-05-36

This returned the following output for me…

gpt

973 255 63 15634432

2016-08-14_17-06-01

Next, we need to create a partition in which you will need to know the start sector and end sector which all depend on the size of the device drive and GUID.  As an FYI…

  • The start sector is always 2048
  • The GUID for VMFS is always AA31E02A400F11DB9590000C2911D1B8
  • The end sector is calculated using the values obtained by running the previous command.
    • Formula: 973 x 255 x 63 – 1 = 15631244

2016-08-11_17-33-01

We can also run the following command to calculate the end sector value.  This should return an identical value that matches the previous calculation. 

eval expr $(partedUtil getptbl /dev/disks/mpx.vmhba40\:C0\:T0\:L0 | tail -1 | awk '{print $1 " \\* " $2 " \\* " $3}') - 1

2016-08-14_17-10-052016-08-14_17-10-45

If everything has gone smoothly so far, you should be ready to create the VMFS partition.  Run the following command, ensuring to replace the identifier and end sector values with your own.

partedUtil setptbl /dev/disks/mpx.vmhba40\:C0\:T0\:L0 gpt "1 2048 15631244 AA31E02A400F11DB9590000C2911D1B8 0"

2016-08-14_17-12-022016-08-14_17-12-42

Lastly, we need to format the partition with VMFS using vmkfstools.  Do so by running the following (Note: “” in the command below can be any name you like so feel free to use a different name for your datastore):

vmkfstools -C vmfs5 -S USB_Datastore /dev/disks/mpx.vmhba40\:C0\:T0\:L0:1

2016-08-14_17-13-58

Sit tight…wait about one minute…and…voila!  

2016-08-14_17-16-35

After a quick rescan/refresh you should now have and see your mounted VMFS5 USB Datastore!

vSphere Client:

2016-08-14_17-17-40

vSphere Web Client:

2016-08-14_17-18-21

vSphere HTML5 Web Client:

2016-08-14_17-18-52

After I changed the “Syslog” configuration for my scratch logs to use this new datastore, I was finally able to unmount my NFS datastores.  I hope this helps so please feel free to comment below.

Shoutout to Florian Grehl for his wonderful post!

Cheers!

-virtualex-

Pingbacks:

USB Devices as VMFS Datastore in vSphere ESXi 6.0

PernixData FVP Freedom Woes With Missing Supermicro System UUID

PernixData FVP Freedom Woes With Missing System UUID

Recently, I’ve been wanting to give PernixData FVP Freedom a run in my HomeLab Datacenter to better familiarize myself with the product and see how much of a performance improvement I’d get if any at all.  I’ve heard from so many people how much they love the product so I figured “why not”?

For those who are not familiar with PernixData FVP, it accelerates Storage and Virtual Machines by moving read and write operations to the server tier, instead of the storage tier, using Flash or RAM to ensure the fastest VM performance.  This, in turn, reduces VM latency by a claimed 10x and overall SAN utilization by over 80%.

 To start off, I visited the PernixData website and went ahead to register for the free FVP Freedom product.  A short time later  I received an email and obtained my download and license key information, along with all the documentation needed to get it up and running.  I installed the ESXi host VIBs and opted to deploy the .ova appliance version so that the deployment would be a piece of cake.  Once I got the product up and running, I logged into the Management appliance and attempted to configure my cluster and add resources, but for some reason, none of my hosts’ were showing up.  I kept getting the “No PernixData compatible hosts have been detected in the cluster“, and only (1) of my (5) hosts was detected but it was not part of the cluster that I was configuring yet.

This is where I ran into a snag that took quite a bit of time to research and find a fix.  Luckily, another blogger by the moniker “vWilmo” who’d experienced this same issue and described how to fix it, so I figured I’d write a similar entry for my own reference, and to help others who may frequent my blog.  I will also be sharing his link at the bottom of this page.

Ultimately, the issue stemmed from the fact that Supermicro did not generate any system UUIDs for my boards and FVP needs them to detect the hosts to use as resources.  KB 1006250 references the situation of an ESXi host not having a unique UUID but did not offer a solution other than to contact the manufacturer (which I did via email and am still awaiting a reply).  To confirm this, I ran a script I found online called Get-VMHostUUID to pull the UUID’s from all of my hosts connected to my vCenter server.  Upon review, it only returned a value for my “white box” host, and returned all “zero” values for my remaining (4) Supermicro systems.  I also ran prnxcli via SSH connection to my host which returned an error as well.

 2016-08-05_15-10-50 2016-08-05_19-20-02

As my Supermicro systems run an AMI (American Megatrends) BIOS, there is a BIOS utility that can be used to generate a new UUID for the system which can be found here.  Download this file and extract the contents.  The file we need to use is named AMIDEDOS.exe, so I took this file and placed it on a DOS formatted USB drive that I had created with Rufus back when I needed to flash my BIOS and upgrade my Intel NIC firmware.

 2016-08-05_19-39-09

Insert and boot into the USB, then navigate to the directory that houses the file mentioned above. 

 2016-08-05_20-18-14

Enter the following command:

AMIDEDOS.exe /su auto

 2016-08-05_20-18-25

If successful, this will generate a new system UUID for you and you should then receive an output like this:

 2016-08-05_20-19-25

Reboot your host, SSH into it and run the prnxcli command.  If it runs successfully you should see an output like this:

 2016-08-05_20-30-04

After I completed this process on all of my (4) impacted hosts, I ran the Get-VMHostUUID script again and was happy to see that I now had a valid UUID for each host which match each prnxcli output.

 2016-08-05_20-52-42

Upon logging into the vCenter Web Client, I noticed that there is now a PernixData plugin icon in the vCenter Web Client interface which can be selected to launch the PernixData Management Console or access the FVP dashboard from within the Web Client.

 2016-08-06_11-02-462016-08-06_11-03-07

Lastly, I logged into the PernixData FVP Management console again, and I was now able to create my cluster and assist hosts as resources.  The only caveat is that there is a single-cluster limitation with the FVP Freedom version license, so if have all of your hosts in a single cluster then you are good.  Unfortunately for me, I have (3) clusters so I need to pick which one I want to use FVP with.  I decided to use my Management Cluster since that houses the majority of my VMs at the moment.

 2016-08-05_20-54-12

After letting it work its magic for a few hours, I noticed that the VM latency had reduced drastically to an average ~ +/- 2.0 ms and overall performance was great!  I must say that I am really impressed, satisfied, and glad that I gave this program a shot!

2016-08-06_16-18-52 2016-08-06_16-19-00 2016-08-06_16-19-25

Well,  I hope that you have found this useful, thanks for stopping by!

Special thanks to Geoff Wilmington aka @vWilmo for helping me to solve this as I am lucky I found your post.  Another shout out to Andy Daniel aka @vNephologist from PernixData for his willingness to communicate with and try to assist me with this problem.

Pingbacks:

Installing phpIPAM on Ubuntu 16.04

| 08/05/2016 | Tags: , , ,

Installing phpIPAM on Ubuntu 16.04

I have been thinking, for a while now, about deploying an IP Address Management (IPAM) system in my Home Lab environment to keep track of my assigned addresses across my various VLANs.  In looking for the right solution, I came across many different choices, from Infloblox to Microsoft’s very own IPAM feature within Microsoft Windows Server 2012 R2.  I read many articles, and kept seeing rave reviews and tons of praise about phpIPAM and that it was simple to install and get it running (at least that’s how it’s advertised).  I went to the phpIPAM website to lookup more information and noticed they have an installation guide available.  Upon observing it, I quickly became disappointed at the lack of detailed instructions to actually deploy it on a system.  I guess they assume everyone has adequate knowledge of Linux operating systems, but myself personally, I’m still pretty novice at Linux and am looking to become more proficient with it.  I figured this is a good opportunity to get some hands-on Linux experience since I already knew how to, at the ver least, install an OS!  

So like any normal “noob” at this, I started consulting “Mr. Google” searching for easy to follow guides on installing phpIPAM and stumbled across one that made it all look very simple (I will share all links at the end of this post).  I spun up an Ubuntu VM and followed the guide precisely, step-by-step, and was disappointed when I could not access the phpIPAM installation web page.  After more searching, I stumbled on even more articles and each one used different methods to get it to work properly.  I thought to myself, “there has to be an easier way to get this working right?”  

So after countless hours of trial and error, taking little tid-bits from several references, I managed to finally get phpIPAM successfully deployed and working on Ubuntu 16.04.  At this point, I figured it would be a great idea to document my installation steps so that I can share my experience with all of you and hope that this guide will be of some value.  Let’s get to it!

Prerequisites:

  • Ubuntu Server 16.04 64-bit
    • Linux, Apache, MySQL, PHP (LAMP) configuration
      • apache2
      • MariaDB (MySQL replacement) or MySQL
      • php7.0 + modules
        • libapache2-mod-php7.0
        • php7.0-cli
        • php7.0-curl (optional)
        • php7.0-gmp
        • php7.0-json
        • php7.0-ldap
        • php7.0-mcrypt (optional, for phpmyadmin)
        • php7.0-mysql
        • php7.0-xml
      • php-pear
      • php-apcu (to speed-up php)
      • phpmyadmin (optional)
        • php-mbstring
        • php-gettext
  • phpIPAM 
  • Web Browser

For the purposes of this guide, I will not cover the actual OS installation steps and am confident that you can easily get an OS installed and running.

I will first configure the server with a LAMP configuration.  I began with a “vanilla” or shall I say “minimal” installation of Ubuntu Server 16.04 64-bit and I will be running all of the commands as the root user.  Having had Linux installed and my server ready means that the “L” part of the “LAMP” configuration is already done.  FYI – features that define a LAMP configuration and be installed in any order.

 2016-05-08_14-04-29

Log in with your local account then enter:

sudo su

 2016-05-08_14-07-02

First, I updated apt-get by running:

apt-get update

 2016-05-08_14-24-48

Once completed, we will move on to the “M” phase of the configuration and install the MySQL database.  I chose to use MariaDB instead of MySQL as I’ve read there are many performance improvements over MySQL.

To install MariaDB, run the following:

apt-get -y install mariadb-server mariadb-client wget

 2016-05-08_14-32-37

When the components have finished installing, we can set a root password for MariaDB by entering the following:

mysql_secure_installation

2016-05-08_14-36-52

You will then be asked the following series of questions:

  • Enter current password for root (enter for none): <– press enter
  • Set root password? [Y/n] <– y
  • New password: <– Enter the new MariaDB root password here
  • Re-enter new password: <– Repeat the password
  • Remove anonymous users? [Y/n] <– y
  • Disallow root login remotely? [Y/n] <– y
  • Remove test database and access to it? [Y/n] <–y
  • Reload privilege tables now? [Y/n] <– y

Next, test the login to MariaDB by entering the following:

mysql -u root -p

Enter the root users password that you previously configured.  If successful, you should see a screen similar to this:

2016-05-08_14-41-05

To exit MariaDB, type quit and press Enter

 

Now, I have just completed the “M” phase of our LAMP configuration and can move on to the “A” phase and install Apache2.

To install apache2, simply run the following command:

apt-get -y install apache2

2016-05-08_14-45-15

When that has finished, test apache to make sure it works by opening a web browser and browse to the VM’s IP or FQDN (http://ipaddress or http://FQDN).  I will use Google Chrome and access it via hostname (FQDN) since I’ve created the DNS record already.

2016-05-08_14-51-32

Success!  This now completes the “A” phase of the LAMP configuration and I can now move on to the final “P” phase by installing PHP7.0

I will begin by simply installing php7.0 and the Apache2 php module.  To do this, enter the following:

apt-get -y install php7.0 libapache2-mod-php7.0

 

2016-05-08_14-57-02

When that finishes, restart apache by running:

systemctl restart apache2

To test that PHP7.0 installed successfully, I will make an info.php file in the web server directory by running the following:

vim /var/www/html/info.php

2016-05-08_15-03-21

You can also use nano instead of vi or vim.  Then I add the following lines by first pressing “I” for “Insert

<?php
phpinfo();
?>

2016-05-08_15-07-54

Save the file by pressing “esc” followed by “Shift :” then type the letters “wq” and press Enter.  Next, run the following command to change ownership of the file:

chown www-data:www-data /var/www/html/info.php

2016-05-08_15-08-42

Now, I can test to ensure PHP is running under Apache2 by opening a web browser and navigating to the IP or FQDN /info.php link (http://ipaddress/info.php or http://FQDN/info.php).  If successful, you should see a page like this.

2016-05-08_15-11-57

Perfect!  Now I will add some additional php modules that will be needed for phpIPAM to work along with some others to add support for MariaDB.  I’ll start with the following command to list the available php7.0 modules.

apt-cache search php7.0

I’ll then install the necessary php modules that are needed by phpIPAM to add support for the database (MariaDB) by entering the following (some of them may have already been installed via php 7.0 installation earlier) :

apt-get -y install php7.0-cli php7.0-curl php7.0-gmp php7.0-json php7.0-ldap php7.0-mcrypt php7.0-mysql php7.0-xml php-pear

2016-05-08_15-24-36

Restart apache2 by running:

systemctl restart apache2

Point your web browser to the /info.php page again and reload it.  If all is well, you should see the new modules installed along with “mysqli“.  Now I know that MariaDB is supported in my php 7.0 installation

2016-05-08_15-28-47

At this point, I have finished the “P” phase in the LAMP configuration and can move on to installing phpIPAM.  But before doing that, I want to add a few extra modules to my PHP configuration to make it run faster via APCU, and to add support for PHPMyAdmin and SSL.

Start by entering the following to speed up PHP

apt-get -y install php-apcu

Then restart apache2 with

systemctl restart apache2

2016-05-08_15-34-20

If you want to ensure it’s installed and running, again load the /info.php site in your web browser and look for the following

2016-05-08_15-34-44

If you’d like, and for security concerns, you can delete the info.php at this time by running 

rm -f /var/www/html/info.php

Now, I am going to enable SSL so that I have (https://) access to my web server as well.  Do this by running the following:

a2enmod ssl
a2ensite default-ssl

Then restart apache2 again with

systemctl restart apache2

2016-05-08_15-39-22

Test it by launching your apache web server link in your web browser using (https://ipaddress or https://FQDN)

2016-05-08_15-41-262016-05-08_15-41-53

The last thing to officially complete my “P” phase of my LAMP configuration is to install phpMyAdmin to allow me to manage my database easily from a web browser.  To install, enter:

apt-get -y install phpmyadmin php-mbstring php-gettext

2016-05-08_15-44-27

You will be presented with the following screen.  Select the “apache2” option by pressing the “space bar” on the highlighted object, and press Enter.

 2016-05-08_15-46-14

Next, you will be presented with this screen.  Select “Yes” and press Enter.

 2016-05-08_15-48-01

On this next screen, just press Enter as a random password will be generated for the phpmyadmin account

 2016-05-08_15-49-29

Next, I need to explicitly enable “crypt” and “mbstring” or the web page will not load properly.  Do this by running the following:

phpenmod mcrypt
phpenmod mbstring

Then, restart apache 2 again with:

systemctl restart apache2

2016-05-08_15-55-34

And the final command to finish the installation is to run the following:

echo "update user set plugin='' where User='root'; flush privileges;" | mysql --defaults-file=/etc/mysql/debian.cnf mysql

2016-05-08_15-57-37

Test phpMyAdmin by navigating to the web server /phpmyadmin page (http(s)://ipaddress/phpmyadmin or http(s)://FQDN/phpmyadmin).  The login is root and the database password you created earlier.

2016-05-08_16-00-27

Excellent!  Now, my LAMP configuration is complete and I can start with the phpIPAM configurations and installation.

Since I’ve already installed all of the required modules, the next thing to do is download the phpipam file and then extracting it to the web servers directory (/var/www/html).  Start by changing over to the /tmp directory

cd /tmp

Next, download phpipam.  I am using the latest version which, at the time of this writing, is phpipam-1.2.1.tar

wget https://sourceforge.net/projects/phpipam/files/phpipam-1.2.1.tar

2016-05-08_16-10-20 2016-05-08_16-10-48

Extract the file to the web server directory:

tar -xvf phpipam-1.2.1.tar -C /var/www/html

2016-05-08_16-19-56

The files have been extracted to a new folder at /var/www/html/phpipam.  Now we need to edit the config.php file in that directory.  But there is no such file so we have to create it by copying the default config.dist.php file to config.php.  Do this by running:

cp /var/www/html/phpipam/config.dist.php /var/www/html/phpipam/config.php

2016-05-08_16-20-12

Now we can edit this file with vim, vi, or nano:

vim /var/www/html/phpipam/config.php

2016-05-08_16-21-03

By default, the file will look like this and I will need to change the following selections:

2016-05-08_16-22-09 2016-05-08_16-22-56

I will make the following changes before saving and exiting the file:

2016-05-08_16-25-16 2016-05-08_16-25-53

Since I have defined the ‘BASE’, it also explicitly said to change this in the .htaccess file.  To open and edit this file, enter:

vim /var/www/html/phpipam/.htaccess

2016-05-08_16-29-01

By default, the file looks like this and I will be changing the following line

2016-05-08_16-29-18

To look like this before saving and exiting the file:

2016-05-08_16-30-03

Next, I will edit the default apache web file (000-default.conf) by entering:

vim /etc/apache2/sites-available/000-default.conf

2016-05-08_16-35-38

By default it looks like this:

2016-05-08_16-35-55

I will be adding the following lines before saving and exiting the file.  This will also allow you enable “Prettify Links” while using an HTTP connection.

<Directory "/var/www/html">
     Options FollowSymLinks
     AllowOverride all
     Require all granted
     Order allow,deny
     Allow from all
</Directory>

If you would like to enable “Prettify Links” while using an HTTPS connection, you need to edit the default apache https web file (default-ssl.conf) by entering:

vim /etc/apache2/sites-available/default-ssl.conf

Look for the same “Directory” area as in the previous step (scroll about halfway down).  By default it will look like this:

Change the entry to the following:

<Directory "/var/www/html">
     Options FollowSymLinks
     AllowOverride all
     Require all granted
     Order allow,deny
     Allow from all
     SSLOptions +StdEnvVars
</Directory>

Now I have to restart the apache2 service again but before doing so, I need to enable “mod_rewrite” by first entering the following and then restarting apache2 as described earlier:

a2enmod rewrite

systemctl restart apache2

2016-05-08_16-40-41

If all is successful, you can now open your web browser and navigate to your web server’s /phpipam URL (http(s)://ipaddress/phpipam or http(s)://FQDN/phpipam) and be presented with the following:

2016-05-08_16-42-29

This is awesome!!  Now, I can select “Automatic database installation“, enter the “root” username and password and click “Install phpipam database

2016-05-08_16-44-07

After a few brief moments, you should see the “Database installed successfully” and you can press “Continue” to log in.

2016-05-08_16-45-56

Enter a password to set the “Admin Password” then click “Save Settings

2016-05-08_16-47-47

After another brief few seconds, you will see a “Settings updated, installation complete!” message and you can click “Proceed to login

2016-05-08_16-49-08

At this point, you will be presented with the phpipam login screen, where you can enter “Admin” and the password you’ve created for the account, then begin configuring your subnets, etc within the dashboard!  I will not go over the configurations in this post as I still need to poke around it a bit, but I’m sure you will find it pretty easy and self-explanatory.

2016-05-08_16-51-32 2016-05-08_16-51-54

Well, that is it!  I hope that you’ve all found this guide to be useful and I welcome any feedback.  Please feel free to rate this post above and share!

**Update**

  • If you would like to check the status by running a ping check, resolve IP addresses, and add the ability to automatically scan for new hosts to automatically add to phpIPAM every 15 minutes, you must add the following cronjob…
crontab -e

Then enter the following at the end of the file…

*/15 * * * * /usr/bin/php -c /etc/php/7.0/cli/php.ini /var/www/html/phpipam/functions/scripts/pingCheck.php

*/15 * * * * /usr/bin/php -c /etc/php/7.0/cli/php.ini /var/www/html/phpipam/functions/scripts/resolveIPaddresses.php

*/15 * * * * /usr/bin/php -c /etc/php/7.0/cli/php.ini /var/www/html/phpipam/functions/scripts/discoveryCheck.php

(Optional)

Instead of running a Discovery Check at the specified 15-minute interval, I also added a rule to do a check every day at 11 AM (see code below).  Please note that I currently have the rule disabled by adding a “#” at the beginning of the line, but if I ever do decide to use that instead of the 15-minute check, I can remove the hashtag and place it in front of the 15-minute check rule.

0 11 * * * /usr/bin/php -c /etc/php/7.0/cli/php.ini /var/www/html/phpipam/functions/scripts/discoveryCheck.php

  • If you’d like to force phpIPAM to always use HTTPS, edit the .htaccess file again:
vim /var/www/html/phpipam/.htaccess

Enter the following:

RewriteCond %{HTTPS} !=on

RewriteRule ^/?(.*) https://%{SERVER_NAME}/$1 [R,L]

Cheers!

-virtualex-

Ping backs:

Creating a Single-Node VSAN

| 02/05/2016 | Tags: , ,

Creating a Single-Node VSAN

 

Many of us homelab enthusiasts tend to build “whitebox” systems from spare PC parts and a few internal hard drives for local storage that we’ve either ordered or had laying around in order install ESXi and run a single-node lab environment.  VMware Virtual SAN (VSAN) enables the ability to build a local SAN environment utilizing the local hard drives in the host.  The only downside/caveat is that you need a minimum of (3) ESXi hosts in a cluster to enable and configure VSAN.  Bummer!  

Well, thanks to some very smart people in the community, there is a way to create a VSAN on your single-node host!  

Keep in mind that this is not supported by VMware and is recommended for testing purposes only and should not be done in a production environment so use at your own risk.

I will mention that this topic has been covered by other bloggers (one of my favorites, William Lam, and a few others…) in the community but for my own knowledge and sharing, I decided to write this post detailing how I configured this in my environment.

Below, I will show you how to configure a hybrid and an all-flash VSAN on your ESXi host.  Again, I remind you that running a single-node VSAN is not supported by VMware and you run the risk losing all your data in the event of a disaster scenario (system crash, etc.)  Please be sure to understand the risks when deciding to use this in your own home lab.

Prerequisites:

  • (1) vCenter Server (Windows-based or VCSA)
  • (1)ESXi host in a cluster
  • (1) vmk for VSAN Traffic
  • (1) SSD for Caching Tier
  • (1) SSD for Capacity Tier (All-Flash configuration)…or…
  • (1) HDD for Capacity Tier (Hybrid Configuration)
  • SSH access to Host

For simplicity’s sake, I will be using a VCSA and ESXi VM deployed in VMware Workstation 12.1.1 Pro on Windows 10 for this demonstration.

1 2

Be sure that you have connected your ESXi host to your vCenter Server and have added it to a cluster, you do not need to enable VSAN on the cluster yet.  Add an additional vmk to your vSwitch for VSAN traffic.  Also ensure you have started the SSH service on the ESXi host.

Open an SSH session to your ESXi Host.  If you’ve added a vmk for VSAN but have not enabled it for VSAN traffic yet, enter the following command.

esxcli vsan network ipv4 add -i vmkN

(Where “N” is the number of your vmk port – ie: vmk1)

In my environment, I already created a VSAN vmk and enabled it for VSAN traffic so I was able to skip the command above.

3

Using the vSphere Web Client or C# client, verify the hard drive that you want to use for your VSAN datastore.  I will be using these drives, the 30GB will be my cache disk and the 120GB will be the capacity disk.

4 5

Back in your SSH session, enter the following command to determine and confirm the eligibility of the disks intended for use to create your VSAN.

vdq -q

6

Next, enter the following command to get the current default VSAN policy.

esxcli vsan policy getdefault

7

We will need to change the current policy by running the following commands.

esxcli vsan policy setdefault -c cluster -p "((\"hostFailuresToTolerate\" i0) (\"forceProvisioning\" i1) (\"stripeWidth\" i1))"
esxcli vsan policy setdefault -c vdisk -p "((\"hostFailuresToTolerate\" i0) (\"forceProvisioning\" i1) (\"stripeWidth\" i1))"
esxcli vsan policy setdefault -c vmnamespace -p "((\"hostFailuresToTolerate\" i0) (\"forceProvisioning\" i1) (\"stripeWidth\" i1))"
esxcli vsan policy setdefault -c vmswap -p "((\"hostFailuresToTolerate\" i0) (\"forceProvisioning\" i1) (\"stripeWidth\" i1))"

8

Run this command again to confirm that the policy has been changed.

esxcli vsan policy getdefault

9

Run the following command to create a new VSAN cluster

esxcli vsan cluster new

10

Now, since my disks are all SSD, I am creating an All-Flash VSAN configuration.  I need to run the following command to tag the capacity SSD as the data disk.  The “-d” represents the “capacity disk” and you need to specify the identifier of the disk to tag.  You can simply copy the identifier number directly from the ESXi hosts storage devices section in Web Client/C# Client, or from the SSH session where we ran the “vdq -q” command.  

Note – If you are deploying a Hybrid VSAN, this command is not needed so you can skip to the next command to add the “cache & capacity” disks to your VSAN.

esxcli vsan storage tag add -d <disk identifier> -t capacityFlash

11

If you’d like to confirm that the disk has been tagged for “capacityFlash” simply run the “vdq -q” command again and check the disk.

12

Next, run the following command to add both your disks (cache & capacity) to your VSAN storage volume.  The “-s” represents the SSD “cache disk”, and the “-d” represents the “capacity disk”.  Be sure to enter the correct identifier number for the respective disks.

esxcli vsan storage add -s <disk identifier> -d <disk identifier>

13

 Run the following command to show the VSAN cluster info.

esxcli vsan cluster get

14

Run the following command to list the VSAN storage

esxcli vsan storage list

15

Congratulations, if everything has been followed correctly, you should now have created a single-node VSAN datastore!

16 17

But we are not quite finished just yet.  Even though I can see the VSAN datastore, I still want to officially enable VSAN on the cluster in vCenter.  Do the following…

18 19 20

In my environment, I have an extra disk in my host, but I do not want to claim this as part of my VSAN.  So, from the drop-down menu, I selected “Do not claim” and hit Next then Finish. 

21 22

Now I can see that VSAN is “Turned On” and can see the disks that are associated with the Disk Group.

23 24

But, there’s still a bit more to be done for me to be able to provision VMs on this datastore.  I need to edit the VSAN VM Storage Policy.  Personally, I prefer to leave the default policies intact and instead create a new policy for my single-node datastore.  I will show both editings of the default policy, for those who do not want to bother with creating a new policy, as well as creating a new policy.  First, let’s check the VASA storage provider and ensure it has been synchronized so that we can edit/create our VSAN Storage policy.

25 26

 

Editing Virtual SAN Default Storage Policy

27

Here we need to simply change:

  • Number of failures to tolerate = 0 (Default is 1)
  • Force provisioning = Yes (Default is No)

28

 

Creating a new Virtual SAN Storage Policy

30

Give it a Name and a Description then hit Next.  

29

Select VSAN from the “Rules based on data services” drop-down, then add all the rules from the drop-down and configure the same settings mentioned above, then hit Next and Finish.

  • Number of failures to tolerate = 0 (Default is 1)
  • Force provisioning = Yes (Default is No)

31 32

And, there you have it!  A fully functional Single-Node VSAN to provision VMs on.  You still have to add a VSAN license, but that will not be covered here as you should already be familiar with the licensing process.

33

 

The Finishing touches

The following optimizations commands are optional but highly recommended for better performance and stability in your VSAN environment.

Since this is a homelab, the disks I used may not be on the official VMware HCL and can potentially impact the performance of the lab environment.  Corman Hogan wrote a great blog and included a tip on how to disable VSAN device monitoring.  Open an SSH session to your host again and run the following command.

esxcli system settings advanced set -o /LSOM/VSANDeviceMonitoring -i 0

To confirm that the command was successful, run the following command.  It should return a value of “0” as the default value is “1”.

esxcfg-advcfg -g /LSOM/VSANDeviceMonitoring

34

Cormac Hogan also wrote another great post about the new “Sparse VM Swap Object”.  ESXi 6.0 Update 2 (aka 6.2) brings a new setting in VSAN 6.2 which allows VSAN to provision a VM swap object as thin instead of thick, where thick has historically been the default.  So if you’d like to disable thick provisioning and use thin, run the following command.

esxcli system settings advanced set -o /VSAN/SwapThickProvisionDisabled -i 1

To confirm, run the following command.  It should return a value of “1” as the default value is “0”.

esxcfg-advcfg -g /VSAN/SwapThickProvisionDisabled

35

And last, but definitely not least, if you intend on running any Nested ESXi VMs on your newly created VSAN, be sure to run the following command to prevent any errors when trying to create SCSI disks for your ESXi VM.  This will enable an advanced ESXi setting that will “fake” SCSI reservations.  William Lam has a nice post about this here.

esxcli system settings advanced set -o /VSAN/FakeSCSIReservations -i 1

And to confirm it took, run the following command.  It should return a value of “1” as the default value is “0”.

esxcfg-advcfg -g /VSAN/FakeSCSIReservations

36

That’s all folks!  I hope that you’ve found this post to be of use to you and I hope you come back for more content.  Feel free to comment, share, and subscribe!

Giving credit where it is due, shout out to William Lam and Cormac Hogan!

 

-virtualex-

Create a Synology VM with XPEnology

| 30/04/2016 | Tags: , ,

Create a Synology VM with XPEnology

 

I’m a huge fan of Synology NAS systems, but I must say, they do often put a gaping hole in your wallet.  Well, fortunately the folks over at XPEnology have created an alternative way for us to create your own Synology devices, whether it be deployed on a bare-metal system or as a virtual machine.  I currently own a few Synology NAS devices, but I love having the ability to spin up a working VM version quickly and with ease, for use in my nested lab environments.

 

In this post, I am going to show you how to create your very own Synology VM on VMware ESXi, Workstation, and Fusion hypervisors.  As I mentioned previously, you can also deploy this onto a bare-metal system, but since I do not have a spare system to test this with, I will not cover that deployment.  So without any further hesitation, let’s get to it!

Prerequisites:

  • VMware ESXi
  • VMware Workstation
  • VMware Fusion
  • XPEnology files
    • DSM 5.x PAT file – (please note that DSM 6.0 remains unsupported with XPEnology at this time)
    • XPEnoboot 5.x VMDK file
    • Synology Assistant (optional but recommended)
    • NFS Plug-in for VMware VAAI (optional but recommended)
    • Open-VM-tools (optional but recommended)

The type of Synology system that XPEnology emulates is a DS3615xs 12-bay unit.  Let’s begin by heading over to the XPEnology website and grabbing what we need.  At the time of this writing, the current version of DSM is the newly released DSM 5.2-5967, but the XPEnology team has not yet created an updated bootloader (XPEnoboot) that supports this version.  The current stable version is DSM 5.2.-5644 Update 5 (and can be manually updated to Update 8 at a later time).  The current stable version of XPEnoboot is 5.2-5644.5 so I’ve downloaded the following files…

2016-04-29_19-13-15

Now the fun begins, and I will start with the VMware ESXi deployment.  I will be deploying on the newest version of ESXi 6.0 Update 2 via the vSphere Web Client.  For those of you using the vSphere C# client, the directions are the same just the interface is different.

 

ESXi 6.0 Update 2

Part 1:

Create a new virtual machine

2016-04-29_19-43-31

Give it a name and select a folder to place the VM in

2016-04-29_19-48-30

Select a compute resource

2016-04-29_19-49-15

Select a storage location

2016-04-29_19-50-35

Select the proper Compatibility version

2016-04-29_19-52-01

Select the following Guest OS settings

2016-04-29_19-55-03

Customize the VM with the following settings, remove the floppy, and select the appropriate network 

2016-04-29_19-57-01

Finish and let it build the “Shell” VM 

2016-04-29_19-58-42

 

Part 2:

Now that the VM has been built, we need to add the XPEnboot VMDK that we downloaded earlier to the VM

Browse to the datastore where you created the VM and upload the XPEnoboot VMDK 

2016-04-29_22-27-40

Once uploaded, we need to add the disk to the VM 

2016-04-29_22-29-15 2016-04-29_22-29-562016-04-29_22-30-58

At this point, I also like to add an additional SCSI disk which will be used as an NFS volume, or iSCSI LUN.  You can make this any size you like but for the purposes of this post, I’m just going to add a simple 50GB Thin Provisioned disk. The 16GB disk we created with the VM will be used to install application packages (i.e. – VMware Tools, etc.)

Keep in mind that this is a 12-bay device, so you can technically add 10 more disks to fill all the drive bays

2016-04-29_22-37-21

Now power-on the VM and open the console window.  Keep an eye on the window for the IP address assigned to the VM.  You can connect to this IP using a web browser instead of using the Synology Assistant to detect and connect to it.

2016-04-29_22-51-55

Open you favorite web browser and type in the IP address of the VM, and hit enter.  This will connect you to the VM’s Synology Web UI

Click the Set up button, then click the Manual Install link and browse for the DSM .pat file we downloaded earlier. Then click the Install Now button

2016-04-29_23-03-382016-04-29_23-04-27 2016-04-29_23-06-12

This will prompt you that it will erase all data on all disks, including the XPEnoboot disk we uploaded and attached earlier.  Note, I have experienced failures at this point and received a message that the .pat file may be corrupt.  In the even this also happens to you, please use this alternative XPEnology download link.

2016-04-29_23-11-55

Once complete, the VM will be rebooted.  But since we erased all of the disks during installation, the VM will fail to boot properly and this is expected

2016-04-29_23-17-15 2016-04-29_23-17-36

Power off the VM and access the datastore that the VM is stored on.  Delete the XPEnoboot VMDK then re-upload the VMDK that we originally downloaded

2016-04-29_23-19-41 2016-04-29_22-27-40

Power on the VM and select the Install/Upgrade option

2016-04-29_23-24-42

When it finishes it boot, this time you will notice that it will not display the IP address we saw the last time.  The IP should have remained the same but I’m going to use the Synology Assistant to detect it and help me connect since my web browser was not connecting to the same address

2016-04-29_23-25-50 2016-04-29_23-44-17

Now we are presented with the Web UI screen and we can login with admin and a blank password

2016-04-29_23-45-29

Click next and give the Synology VM a name and change the admin user password or enter a new username and password then click Next

2016-04-29_23-48-13 2016-04-29_23-50-20

In order to prevent updates from installing automatically and possibly breaking the boot up of the Synology VM, I chose “Download DSM updates but let me choose whether to install them” and clicked next

2016-04-29_23-52-26

I also chose to skip the “Set up QuickConnect”

2016-04-29_23-53-442016-04-29_23-54-26

And….You’re Done!  You now have a fully functional Synology virtual machine.  You can feel free to add additional disks or what have you.

2016-04-29_23-57-16

At this point, I like to go ahead and install the Open-VM-Tools package so that I can use the VMware tools to gracefully power off the VM as needed.  To do so, you will need to have downloaded the package from the XPEnology website.

Open Package Center, then click the Settings button.

2016-04-30_0-02-06

On the General tab, set the Trust Level to Any Publisher and click OK

2016-04-30_0-03-04

Back in the Package Center, click Manual Install.  Browse to the package and click Next

2016-04-30_0-04-51

Since we have yet to create a volume, it will prompt you to click OK to launch Storage Manager and create a volume to install the package on.

2016-04-30_0-07-21

In Storage Manager, click Volume the click Create.  Keep the default of “Quick”, Next, select Disk 3, Next, OK to the warning message, Yes for disk check, Next, Apply….Done!

2016-04-30_0-08-57 2016-04-30_0-11-08 2016-04-30_0-12-02 2016-04-30_0-13-10 2016-04-30_0-14-49 2016-04-30_0-15-13

Now, repeat step 3 above, click Apply, and you will have VMware tools installed.

2016-04-30_0-17-23

Wow, that seemed like an awful lot of steps, but it really wasn’t all that much.  In the next sections, I will just go over the deployment of the “Shell VMs”  and the steps to add the XPEnoboot VMDK to the VMs in VMware Workstation and VMware Fusion as the rest of the post-boot setup steps are the same.

 

VMware Workstation Pro 12.1.1

Part 1:

Create the Shell VM…

2016-04-30_1-07-28 2016-04-30_1-07-43 2016-04-30_1-08-01 2016-04-30_1-08-17 2016-04-30_1-10-34 2016-04-30_1-10-46 2016-04-30_1-11-05 2016-04-30_1-11-47 2016-04-30_1-12-14 2016-04-30_1-12-27 2016-04-30_1-12-39 2016-04-30_1-13-17 2016-04-30_1-13-40 2016-04-30_1-15-42

Part 2: 

Add the XPEnoboot VMDK to the VM…

2016-04-30_1-17-08

2016-04-30_1-18-32 2016-04-30_1-18-46 2016-04-30_1-19-02 2016-04-30_1-19-14 2016-04-30_1-19-54 2016-04-30_1-21-00

2016-04-30_1-21-20

Power on the VM and continue with same process outlined above for ESXi deployment.  Use the Synology Assistant to find and connect as the VM may not display the IP address right away.

 

VMware Fusion Pro 8.1.1

Part 1:

Create the Shell VM…

2016-04-30_09-21-41 2016-04-30_09-22-04 2016-04-30_09-22-36 2016-04-30_09-22-58 2016-04-30_09-24-14

Part 2:

Add the XPEnoboot VMDK to the VM…

2016-04-30_09-25-28 2016-04-30_09-26-35 2016-04-30_09-27-02 2016-04-30_09-29-08 2016-04-30_09-30-49 2016-04-30_09-31-51 2016-04-30_09-32-41 2016-04-30_09-33-14 2016-04-30_09-34-58 2016-04-30_09-36-10 2016-04-30_09-36-45 2016-04-30_09-37-39

Power on the VM and continue with same process outlined above for ESXi deployment.  Use the Synology Assistant to find and connect as the VM may not display the IP address right away.

 

Now you should all feel confident in creating your own Synology VM systems for whatever your use case may be and hopefully you will all love using Synology products as much as I do.  If you would like the true experience, I’d recommend purchasing a real system.  But in the interim, this will give you a way to create your own and play around with it.

Keep in mind that when updates are released, it is always wise to first check on the XPEnology site and/or forum to see if the updates break anything as sometimes they will cause the boot process to break since the XPEnoboot cannot support it yet.  Usually you are safe to update as long as the update is within the same DSM version, but consult the site/forum first as I am not responsible for any upgrade impacts.

Well, I hope that you all have enjoyed this read/guide and come back for more!

And giving credit where it is due, I’d like to give a shout out to Erik Bussink as his guide is what inspired me to write this updated guide for all of you!  

Cheers!

-virtualex-

Home Lab 2016 – Part 3

| 23/04/2016 | Tags:

Home Lab 2016 – Part 3

 

Hello all!  My sincere apologies for the brief hiatus, but I am back to continue my Home Lab 2016 series.  In my previous posts, I covered the components that make up my new Home Lab.  In this post I will quickly cover my Storage and Network solutions that connect my lab.  Let’s get to it!

I will begin by covering my networking components used in my home LAN and LAB.  

 

My WAN connects to my ISP modem, which then connects to my amazing Ubiquiti EdgeRouter Lite (ERLite-3) via eth0.  My hardwired LAN connects from eth1 on the ERLite-3 to port 1 on the SG300-10 (core1).   Lastly, I changed my ASUS router to Access Point mode and connected my Wifi LAN from port 1 of the ASUS to eth2 on my ERLite-3.

2016-04-23_17-34-38

EdgeRouter Connections:

  • eth0 – WAN
  • eth1 – LAN (configured as a 192.x.x.x network)
  • eth2 – WLAN (configured as a 172.x.x.x network)

2016-04-23_17-57-472016-04-23_17-59-46

On to the Cisco SG300-10 (core1), 10-port managed switch, this is configured in Layer 3 (L3) mode and is where I created all of my VLANs and DHCP scopes, etc….

2016-04-23_17-19-20

  • port 1 – connects to ERLite-3
  • ports 2-5 configured in an LACP/EtherChannel trunk to Cisco SG300-52 (core 2)
  • ports 6-10 connect to different rooms in my home LAN

 

I created the following VLANs on core 1 and allowed them to traverse the trunk to core 2.  

  • VLAN10 – IPMI
  • VLAN20 – ESXi Management
  • VLAN30 – vMotion
  • VLAN40 – VM Traffic
  • VLAN50 – NFS Traffic
  • VLAN55 – VSAN Traffic
  • VLAN60 – DEV-VM Traffic
  • VLAN65 – DEMO-VM Traffic
  • VLAN99 – UPS/ATS/DPU

The Cisco SG300-52 is configured in its default Layer 2 (L2) mode and I set up the proper settings, trunk ports, and access ports for each VLAN.  I understand that I could’ve also configured this in L3-mode and reduce the extra hop to core 1, but I didn’t feel the need to do so for my use case. I may change my mind at some point, but it works for me…for now.  

Due to the way the ethernet cables connected from the switch to each ESXi host, I started configuring the switch ports at the end of the switch and worked my way towards to the beginning of the switch ports.

2016-04-23_16-42-35

 

  • ports 49-52 (LAG 8): LACP/EtherChannel trunk from SG300-10 (core 1)
  • ports 23-24, 47-48: IMPI
  • ports 1, 19-22, 43-47: ESXi Management Traffic
  • ports 15-18, 25, 39-42: vMotion Traffic
  • ports 11-14, 26: NFS Traffic
  • ports 26, 35-38: VSAN/iSCSI Traffic
  • ports 2, 7-10, 31-34: VM Traffic
  • ports 6, 30 (LAG 7): Synology
  • ports 5, 29: UPS/ATS/DPU
  • ports 3-4, 27-28: Unassigned

Switch_Connections_clean

 

Next, let’s take a look at Shared Storage.  I run my shared storage infrastructure on Synology DiskStation hardware, because…they’re flat out awesome, and give you a ton of bang for your buck!

I have to say I absolutely love the Synology products, and my DS415+ rocks!  I have this running 4 SSD’s and a DX213 expansion unit attached with 2 HDD’s in in.  It runs DSM 6.0-* update *. The interface is slick and setting up the device is a breeze.  I am currently using NFS only in my lab, and plan to incorporate VSAN soon, as well as testing out iSCSI vs NFS performance.  For my disk setup, I decided to use the Micron 500DC SSD drives as my first Disk Group (Disk Group 1) for my performance volume (Volume 1), the Micron 510DC drives for SSD Cache, and the HGST drives as my 2nd Disk Group (Disk Group 2) for all other storage volumes (Volume 2-x; ISOs, Backups, etc.)

2016-04-23_17-26-532016-04-23_17-31-30

Well, there you have it.  In my next posts, I will go over the basic setup and configurations of my Home Lab.  

I hope you enjoyed the read!  

Don’t forget to comment and subscribe!

Configure OpenDNS on Ubiquiti EdgeRouter Lite

| 27/03/2016 | Tags:

I recently picked up a new router/firewall for my home, and chose the Ubiquiti EdgeRouter Lite (ERLite-3).  This device comes with a lot of bells and whistles and if you would like more information on it, please see here.

I am a huge fan of speed and security, and for this purpose I always choose to configure my home network to use OpenDNS name servers rather than my ISP’s (Internet Service Provider) name servers.  OpenDNS has some great setup guides available for users to configure their devices and you can view the setup guides here and choose the solution that best suits your needs.  Now, I prefer to configure OpenDNS right on the router so that it applies to any and all devices on my network that use the internet.  Unfortunately though, I did not find a setup guide for Ubiquiti devices…but since I was already familiar with the process, I tinkered a bit around the EdgeOS and managed to figure it out.

 

In this post, I am going to cover how to set up your EdgeRouter to use OpenDNS name servers.  By default, the router is configured to forward DNS queries to the name server IP addresses obtained from your ISP via DHCP.  So let’s go ahead and change that!

 

First, let’s head over to the following OpenDNS page to test our settings and ensure we are not already configured to use OpenDNS’s DNS name servers somehow (this is just for good measure).  On the right-hand site there is a link that reads:

“click here to test your settings”

2016-03-27_10-56-18

Go ahead and click it, and it should return the following message:

2016-03-27_10-56-39

Next, login to your EdgeOS dashboard WebGUI (by default it’s configured as 192.168.1.1) and login using the default user credentials (ubnt/ubnt).

2016-03-27_11-18-00

On the bottom left corner, click the System tab to open it up.

2016-03-27_11-19-21

In the Name Server area, add the following OpenDNS name server IP addresses then scroll to the bottom and click Save before closing this window:

208.67.222.222

208.67.220.220

2016-03-27_11-20-232016-03-27_11-20-40

Great, Step 1 is done!  Now onto Step 2…this can be done via the CLI or the Config Tree.  (I chose the latter…)

Click on the Config Tree button and navigate to the following from the left-side Configuration pane:

service / dns / forwarding / system 

Click the triangle-like icon direct to the left of the word system, then click the “+” icon to the right of the word system. If done correctly you should see the following header on the top of the page and two buttons, Discard & Preview, on the bottom of the page.

2016-03-27_11-21-41

Next, click the Preview button at the bottom of the page.  This will show you the command that would be used in the CLI to apply the name servers.  The great thing about this is that it also give you the option to “Apply” it now via the GUI!  I chose to hit Apply.  

2016-03-27_10-58-20

Once, completed, there should be some green text at the bottom reading:

“The configuration has been applied successfully”

2016-03-27_11-13-02

Now, the final step is to go back to the OpenDNS page and test the settings again.  If everything is applied correctly, you will be presented with the following:

2016-03-27_11-03-45

Now your router is fully configured to use OpenDNS!  

I hope that you have found this to be informative so please be sure to comment below and share this post.

Intel NIC not detected by ESXi

| 21/03/2016 | Tags: ,

Intel NIC Not Detected by ESXi

 

In this post I am going to cover a random issue I encountered after installing ESXi 6.0 Update 2 on one of my new Home Lab 2016 hosts.  The actual installation of ESXi was extremely easy and painless (I may cover that in another post).  After I had completed the installation, I was attempting to configure my Management network interfaces and suddenly noticed that only 4 network interfaces were being detected!  

2016-03-18_19-07-54

As I then thought to myself, “I wonder what is going on here since I didn’t get any POST issues?”, I noticed that I was getting a message during POST regarding the initialization of the Intel Boot Agent for PXE booting.  The message stated:

“…PXE-E05: The LAN adapter’s NVM configuration is corrupted or has not been initialized.  The Boot Agent cannot continue.” 

 

2016-03-18_18-02-50

Immediately, I began to consult “Mr. Google” and see if there was anything I could find related to this particular problem.  After reading a few threads, many users had mentioned and/or suggested that the NIC’s firmware was corrupted and needed to be “re-flashed”.  I quickly got to work and researched a bit further to understand the process of flashing the NIC firmware.  I downloaded the latest version of PREBOOT (at the time of this writing, it was version 20.7)which contains the “BootUtility” needed to perform the flash.

Next, I prepared myself a DOS-bootable USB using Rufus.  I then extracted the PREBOOT.exe file using 7zip and placed the contents on the newly created USB.  This would allow me to either boot into the USB and access DOS or boot into the UEFI: Built-in Shell on Supermicro motherboards, and access the necessary files.  Once I had my drive ready, I went ahead and plugged it into my server and initiated a reboot.  During POST, I invoked the boot menu so and chose the option to boot into the Built-in Shell.

 2016-03-21_21-19-54

Once in the shell, I determined that my USB was mounted at fs4:

 2016-03-21_21-21-36

I navigated through the directories so I can see the contents of each folder until I saw the BootIMG.FLB file which is the new flash image I want to apply.  I then navigated to the location of the BootUtility.  Since I am using the built-in shell, I needed to ensure that I used the BootUtil for x64-bit EFI so I navigated to the following location:

...\PREBOOT\APPS\BootUtil\EFIx64

 2016-03-21_21-24-17

Running the BOOTUTIL64E.EFI file will simply list your network interfaces and I could then see the current firmware version for all of my interfaces, although for some reason the ones in question are displaying “Not Present”.  Adding a “-?” suffix will bring up the help and list all the parameters to execute the commands properly.  I found a great reference article here which made it easier for me to see what parameters I needed in my command.  

2016-03-21_21-25-48

To begin, I entered the following command since I wanted to enable flash on all of my NICs.  

BOOTUTIL64E.EFI -ALL -FLASHENABLE

2016-03-21_21-29-17

Or, if you wanted to do each one individually, you could specify the NIC  number (referenced as X below) manually.

BOOTUTIL64E.EFI -NIC=X -FLASHENABLE

A reboot is required after successful completion of this command before proceeding, so I went ahead and rebooted my system and then booted back into my USB via the built-in shell.  

2016-03-21_21-29-51

Afterwards, simply running the utility again showed that NIC ports 1-4 were all PXE ready.  

2016-03-21_21-34-35

Now it was time to run the following flash command.  Note: Specifying the file parameter is optional.  Without it, it will assume that the BOOTIMG.FLB file is in the same location you are executing the command from.  Since I left the file in its originating location, I had to specify it manually.

BOOTUTIL64E.EFI -UP=PXE -ALL -FILE=\SuperMicro\PREBOOT\APPS\BootUtil\BOOTIMG.FLB

You will then be prompted to create a restore image, in the event something goes awry but I chose not to create a restore image.

 2016-03-21_21-38-31

Upon successful completion, I can now see that my firmware version has been upgraded from version 1.3.98 and is now running version 1.5.78

 2016-03-21_21-40-55

Now that my firmware has been upgraded, I rebooted the host and accessed the Management Network settings screen, and to my delight, ESXi was now detecting all of my network interfaces!  Woohoo!!  

2016-03-21_22-17-38

I am still trying to figure out why flash is not present on the NICs that previously were not detected, and my assumption is that it’s due to these being relatively cheap $80 network cards instead of full priced (~$300) network cards.  In any case, they still work and I am quite happy with them.  I hope you have found this information useful.  Thanks for reading!

Home Lab 2016 – Part 2

| 15/03/2016 | Tags:

Home Lab 2016 – Part 2

Welcome back for Part 2 of my Home Lab 2016 Series.  I hope that you enjoyed my previous post, Part 1 from last week, where I covered the basis of my home lab and presented the Bill of Materials (BOM) for my mini-datacenter environment.

Today I am bringing you Part 2 and will cover the actual physical build process, putting together the components to build each ESXi host server.  I hope you’re as excited as I am!

Beginning with the case, I chose to go with the Supermicro CSE-504-203B which has the motherboard backplane and all connections at the rear of the case, instead of the CSE-505-203B which has everything in the front of the case.  I wanted to have more of a cleaner look to my rack enclosure, and the best thing about these cases is that they come with a 200W High-efficiency “80 Gold Level Certified” power supply!

 2_1_FullSizeRender2 FullSizeRender

The next component to go into this case is the motherboard.  I chose the Supermicro A1SAi-2750 with an Intel ATOM “System on a Chip” (SoC) CPU.  This is a 20W, 8-Core processor, is compatible with “Westmere” VMware Enhanced vMotion Compatibility mode, and supports a maximum of 64GB DDR3  RAM in (4) DIMM sockets!  I went ahead and maxed the RAM on each board with (4) 16GB Micron MEM-DR316L-CL02-ES16 DDR3 1600MHz ECC 204-pin 1.35V SO-DIMM chips.

 IMG_1667 IMG_1670

Since I wanted to have redundancy for all my network connections, as per “best practices”, I decided to install an Intel I350-T4 quad-port NIC.  Unfortunately, even with the low-profile mounting brackets that come with the cards, they simply would not fit in a small 1U case, as they are designed to be installed horizontally.  I picked up a couple of Supermicro RSC-RR1u-E8 PCI-E x8 riser cards which would allow me to insert the NICs properly.

  

Next, came the disk drives to run ESXi as well as VM’s, in a VSAN cluster, for management machines if I wanted to move them off of my shared storage device.  I also wanted to have the ability to create a VSAN environment for testing and educational purposes (i.e.: VCP/VCAP certifications).  I decided to utilize the onboard USB 3.0 socket and installed a SanDisk Ultra Fit 16GB USB 3.0 flash drive to run ESXi, after all…this is a lab right?  For my VSAN drives, I decided to pair a Kingston SSDNow V300 series 120GB SATA III SSD with an HGST Travelstar Z7K500 500GB 7200RPM HDD

   

In order to stack them together, I picked up a Supermicro MCP-220-00044-0N HDD Converter bracket.

Here is the end result of the insides after all the components above were installed.

Once I had the first server built, I powered it on to ensure it was in working order before continuing on and building the remaining (3) hosts.  Afterwards, I decided to tidy things up a bit further, zip-tying cables, etc. for a cleaner look, before closing up the cases to place them in my rack enclosure.

Please stay tuned for Part 3, where I will quickly cover my networking and storage solutions!  Thanks for stopping by!

Home Lab 2016 – Part 1

| 04/03/2016 | Tags:

Home Lab 2016 – Part 1

 

 

Having a home lab is every IT enthusiasts dream come true, and now I can finally say that I have fulfilled that dream!  I previously was (and currently still am…) using a 1-node “white box” system I had built from a spare gaming machine I had laying around, running on an open-air tech bench from TopDeck.  It’s comprised of the following:

And even though it runs great, I simply felt it wasn’t enough as I basically wanted to replicate a mini-datacenter for my lab which would help tremendously with my VMware studies and overall VMware knowledge.

So I quickly got to work and embarked on the adventure of creating my new lab.  I started off by opening a Feedly account and subscribing to numerous other user and community blogs, reading what others did to create and build/setup their homelabs, and also checked out some youtube channels.

Lot’s of good reads out there…

TinkerTry

Wahl Network

VMware Front Experience

VirtualJad

virtuallyGhetto

Just to name a few…

I also spent the last year+ researching, planning, designing, and purchasing the equipment for my new lab.  And since I wanted somewhat of a low power solution (as to not incur outrageous electric bill charges) I settled on SuperMicro’s A1SAi-2750 ATOM SOC (System-on-a-chip) Mini-ITX motherboards.  Boy, do these things boast a boatload of features (not getting into specifics as you guys know how to use Google I’m sure…)!  Since I also wanted to have them in a rack to replicate a mini-datacenter, I went with a Navepoint 9U rack enclosure.  I bought some Sandisk USB’s, some SSD’s & HDD’s (for eventual VSAN setup), and extra NIC’s (for redundancy and best practices), 1U cases, and some Synology NAS devices.  Here’s my entire part’s list…

And for NAS storage…..

The Networking components…

And last, but definitely not least…. a slew of Monoprice Cat6 24AWG Flexboot cables (various lengths)

Phew!…what a list!  Wait!…am I missing anything??

The end result…my new mini-datacenter homelab 2016!! (with previous Dev “white box” system to the side)

 

Stay tuned for Part 2 ( I hope) where I plan on “Putting it all together”!

Feel free to comment and let me know your thoughts/feedback…and words of encouragement so I can continue on this new blogging adventure!