How to use Foundation to set up a Nutanix cluster

Field Installation Overview Nutanix installs the Acropolis hypervisor (AHV) and the Nutanix Controller VM at the factory before shipping a node to a customer. To use a different hypervisor (ESXi or Hyper-V) on factory nodes or to use any hypervisor on bare metal nodes, the nodes must be imaged in the field. This guide provides step-by-step instructions on how to use the Foundation tool to do a field installation, which consists of installing a hypervisor and the Nutanix Controller VM on each node and then creating a cluster. You can also use Foundation to create just a cluster from nodes that are already imaged or to image nodes without creating a cluster.

Before you start – Requirements Physical elements, including power and network switches Network segmentation, including VLANs and subnets Network components, including DNS servers Available IP addresses for the Nutanix cluster

Power and Networking Requirements A Nutanix block contains two redundant power supplies that provide power to all nodes in the block. A 208V power source is recommended for these power supplies if you want full redundancy. A 110V power source is supported, but not fully redundant. Check the specific power supply requirements for the block on the Nutanix portal for more information. By default, each node of a Nutanix cluster contains five network adapters: Two 10 GbE adapters - empty SFP+ adapters Two 1 GbE adapters One 10/100 or 1 GbE Ethernet adapter for lights-out management

Network Requirements Existing Customer Network Default gateway Network mask DNS server NTP server New IP Addresses (3 IPs per node) IPMI interface Hypervisor host Nutanix Controller VM 1 IP address for Virtual IP of the Nutanix cluster 1 IP address for “External Data services” If VLAN exists, please specify tag ID

SFP+ TwinAx SFP+/SFP+ 1 / 3 / 5 meters Transceiver 2x by link Identify network adapters and cables UTP Cat 6a/7 Up to 100 mt. CAT level related

Nutanix Series - Front view NX-1050NX-3060-G4/G5 SX-1065-G5, NX-1020, NX-1065S, NX- 1065S-G5, NX-1065-G4/G5 NX-3155G-G4

Nutanix Series - Front view NX-6035-G4/G5NX-3175-G4/G5 NX-6020, NX-6035C NX-6155-G5

Nutanix Series - Front view NX-8150-G4/G5 NX-8035-G4/G5

Nutanix Series - Rear view SX-1065-G5, NX-1065-G4/G5, NX-3060-G4/G5 NX-1050, NX-3050, NX-3060 NX-3155G-G4 NX-3175-G4/G5 Pay attention to shared IPMI Port Some models only has 10GbE ports, use 10GbE > 1GbE Transceiver

Nutanix Series - Rear view NX-6035-G4/G5 NX-6155-G5 NX-8150-G4/G5 NX-8035-G4/G5

Foundation applet

Creating a Cluster using Foundation applet This procedure describes how to install a selected hypervisor and the Nutanix Controller VM on discovered nodes and how to configure the nodes into a cluster. "Discovered nodes" are factory prepared nodes on the same subnet that are not part of a cluster currently. This procedure runs the Foundation tool through the Nutanix Controller VM. Note: This method creates a single cluster from discovered nodes. If you want to image discovered nodes without creating a cluster or image bare metal nodes, see Creating a Cluster using Foundation VM, which is discussed next.

Foundation applet - Setup Download the Foundation applet from the Nutanix Support Portal : Networking Scenarios Use dumb 1Gb switch Connect just eth0 on shared IPMI port (no other port, including IPMI – refer to previous slides to identify the correct port –> framed in red) Workstation and IPMI/CVM/Hypervisor on the same subnet

Foundation applet 15 Nodes will present itself Select one node that will be the starting point Follow the simple wizard Start the Foundation applet 1 Select one of the free nodes 2 Launch the Foundation tool 3

Foundation applet – Discovering nodes 16 Foundation will automatically discover every nodes ready to be imaged Nodes must be powered on with a CVM up and running Nodes can’t be or have been part of a cluster

Foundation applet – Node selection 17 Select all nodes that must be part of the cluster 1 Click Next 2

18 Fill each blank field with cluster information 1 Enable IPMI to be able to set IP addresses/Netmask/G ateway 2 Set the CVM’s Memory size (depending on features that will be used, like dedup) 3 Click Next 4 Foundation applet – Cluster information

19 Foundation applet – Nodes configuration Enter Hypervisor hostnames, and IP ranges for CVM, Hypervisor and IPMI 1 Click Validate Networks 2

20 Foundation applet – Checking connectivity Running on the CVM enable Foundation to test the actual network links the cluster will use. This should prevent about 40% of support calls.

21 Foundation applet – Hypervisor and AOS upload Upload a more recent AOS version if needed (available on Nutanix Support Portal) 1 Click Create Cluster 4 Select your hypervisor and upload the corresponding ISO (available on VMware & Microsoft websites or Nutanix Support Portal for AHV) 2 For Microsoft Hyper-V, select the SKU: Free/Standard/ Datacenter, core only or with GUI. 3 If needed, Nutanix hosts an up-to-date OS installer whitelist on the Support Portal

22 Foundation applet – Whitelist

23 Foundation applet – Work in progress

Foundation cannot reimage the node it’s on, so we image in stages Host CVM + Foundation Host CVM + Foundation Host CVM + Foundation Host CVM + Foundation 1. User says go 2. image a single node 3. image remaining nodes Foundation applet – Under the hood: distributed installation

25 Foundation applet – Process is complete When the process is over, you can access the Prism web console directly from Foundation 1

Foundation applet – Prism web console Log in with default credentials User: admin Password: admin 1

Foundation VM

Imaging nodes and/or creating a Cluster using Foundation VM This procedure describes how to install a selected hypervisor and the Nutanix Controller VM on discovered and/or manually added nodes and how to configure the nodes into a cluster. While "Discovered nodes" are factory prepared nodes that are not part of a cluster currently, you can also use Foundation VM to add Bare Metal nodes and image them. This procedure runs the Foundation tool on your own laptop inside a VM, through VMware Workstation/Fusion or Oracle Virtual Box.

Foundation VM - Setup Download the Foundation VM from the Nutanix Support Portal :

Foundation VM – Network settings Find “shared” port and connect only one cable Networking Scenarios Use dumb 1Gb switch Connect just eth0 on shared IPMI port (no other port, including IPMI – refer to previous slides to identify the correct port –> framed in red) Workstation and IPMI/CVM/Hypervisor on the same subnet

Foundation VM – Deployment 1.Run Oracle VM VirtualBox (or VMware Workstation/Fusion) on your laptop 2.Deploy the Foundation VM OVF file inside it (File > Import) 3.Run the new VM and connect with credentials: nutanix / nutanix/4u 4.Install VMwareTools or VBoxAdditions If you need to upgrade your Foundation VM, download the foundation-version#.tar.gz bundle from the Support Portal, then: a.Copy the tarball (foundation-version#.tar.gz) inside /home/nutanix b.Go to /home/nutanix c.Upgrade Foundation: $./foundation/bin/foundation_upgrade -t foundation-version#.tar.gz

1.Download the NOS tarball (nutanix_installer_package-version#.tar.gz) from Support Portal 2.Copy the NOS tarball inside /home/nutanix/foundation/nos 3.Copy Hypervisor’s ISOs in their own directories: ESXi: /home/nutanix/foundation/isos/hypervisor/esx Hyper-V: /home/nutanix/foundation/isos/hypervisor/hyperv AHV (already included inside Foundation): /home/nutanix/foundation/isos/hypervisor/kvm Foundation VM – Hypervisor & NOS files

33 Foundation VM – Network configuration Run the set_foundation_ip_address script in Terminal to configure the Foundation VM network 1

34 Foundation VM – Network configuration Set the correct Network settings for your Foundation VM 1

35 Foundation VM – Run Run Nutanix Foundation 1

36 Foundation VM – Discovering nodes Foundation will automatically discover every nodes ready to be imaged If some nodes are not discovered (Bare Metal Nodes), you can manually add them: select Add Blocks

37 Foundation VM – Add Bare Metal nodes Add the number of Blocks (2U chassis) 1 Enter the correct number of Nodes to add in each Block 2

Foundation VM – Add Bare Metal nodes Get the IPMI MAC addresses from the stickers located at the rear of each Bare Metal Nodes

39 Foundation VM – Add Bare Metal nodes Fill the IPMI MAC address field of each node with the information previously gathered And follow the process described on the next slides 1

40 Foundation VM – Nodes configuration Enter Hypervisor hostnames, and IP ranges for CVM, Hypervisor and IPMI 1 Click Next 2

41 Set the Netmask/Gateway/DNS of each componentFill each blank field with cluster information 1 Set the CVM’s Memory size (depending on features that will be used, like dedup) 2 Click Next 3 Foundation VM – Global configuration

42 Multi-Homing is needed if different subnets are requested for IPMI and HV+CVM IPs If this box is not checked, Foundation requires that either all addresses are on the same subnet or that the configured IPMI, hypervisor, and Controller VM IP addresses are routable. Your laptop\VM must be in the same HV+CVM subnet. Foundation VM – Multi-Homing

43 Foundation VM – Hypervisor and AOS selection Select the AOS package to install 1 Select the hypervisor version to install 2 For Microsoft Hyper-V, select the SKU: Free/Standard/ Datacenter, core only or with GUI. 3 If needed, Nutanix hosts an up-to-date OS installer whitelist on the Support Portal Click Next 4

44 Foundation VM – Whitelist

45 Foundation VM – Cluster creation Select Create New Cluster 1

46 Foundation VM – Cluster creation Enter the cluster name, VIP and DNS/NTP 2 Select all nodes that must be part of the new cluster 3 Click Verify and Install to start the imaging process 4

47 Foundation VM – Proceed

48 Foundation VM – Work in progress

49 Foundation VM – Process is complete

Foundation VM – Prism web console Log in with default credentials User: admin Password: admin Then set a new password 2 Use a browser to connect to Cluster’s VIP 1

Foundation VM – Reset configuration If everything is OK, don’t forget to Reset Configuration in your Foundation VM 1

Tips & Tricks

53 Foundation Tips & Tricks Foundation VM: Localization (keyboard input), Timezone Always use latest version available on Support Portal Always use latest iso_whitelist.json available on Support Portal Maintain ISOs & NOSs /home/Nutanix/foundation/isos /home/Nutanix/foundation/nos All relevant info in /home/Nutanix/foundation/config/ Collect Foundation logs: Foundation applet: Youtube video:

54 Foundation Tips & Tricks Discover all CVM on local network: ~]$ cd foundation/bin/ bin]$./discovery.py discover INFO: Discovering nodes. Expect this to take some time. INFO: Wrote nodes to discovered_nodes.txt bin]$ cat discovered_nodes.txt 15SM NX-1065-G4 A fe80::20c:29ff:fe9d:207c eth0 15SM NX-1065-G4 C fe80::20c:29ff:fe31:965a eth0 15SM NX-1065-G4 D fe80::20c:29ff:fe50:30a7 eth0 15SM NX-1065-G4 B fe80::20c:29ff:fe67:3c16 eth0 Create Phoenix ISO (on Foundation VM): /home/nutanix/foundation/bin/foundation --generate_phoenix --nos_package=/home/nutanix/foundation/nos/nutanix_installer_package-release-danube-4.7-stable.tar