Assumptions:
– You have already installed OL7 on a virtual machine
– assigned at least 4G of memory, otherwise the CVU will complain.
– 2 network cards
=============== BEGIN ERRORS DESCRIPTION =======================
Note: Along the installation of the CLusterware 12cR1 I encountered couple of issues.
- This is actually the 2nd one, but should be fixed right at this step.
Solution:
VBoxManage list vms
"rac1" {83ff3550-ced8-48be-8f8e-85d4363fb047}
"rac2" {3ae7500c-53b4-4ee6-aa8e-152869b6d473}
VBoxManage setextradata 3ae7500c-53b4-4ee6-aa8e-152869b6d473 "VBoxInternal/CPUM/HostCPUID/Cache/Leaf" "0x4"
VBoxManage setextradata 3ae7500c-53b4-4ee6-aa8e-152869b6d473 "VBoxInternal/CPUM/HostCPUID/Cache/SubLeaf" "0x4"
VBoxManage setextradata 3ae7500c-53b4-4ee6-aa8e-152869b6d473 "VBoxInternal/CPUM/HostCPUID/Cache/eax" "0"
VBoxManage setextradata 3ae7500c-53b4-4ee6-aa8e-152869b6d473 "VBoxInternal/CPUM/HostCPUID/Cache/ebx" "0"
VBoxManage setextradata 3ae7500c-53b4-4ee6-aa8e-152869b6d473 "VBoxInternal/CPUM/HostCPUID/Cache/ecx" "0"
VBoxManage setextradata 3ae7500c-53b4-4ee6-aa8e-152869b6d473 "VBoxInternal/CPUM/HostCPUID/Cache/edx" "0"
VBoxManage setextradata 3ae7500c-53b4-4ee6-aa8e-152869b6d473 "VBoxInternal/CPUM/HostCPUID/Cache/SubLeafMask" "0xffffffff"
2. This is the first error that we’re going to get but this can be fixed while we’re doing the installation:
Solution:
As soon as the installer has finished with the step “Copying files” do this:
cp $ORACLE_HOME/javavm/jdk/jdk6/lib/lib*12.a $ORACLE_HOME/lib/
This library (libjavavm12.a) wasn’t copied at the right location
==============================END ERRORS =============================
If you have cloned the VM (as I do), this is the command to change the name of the VM:
This is the syntax:
# hostnamectl set-hostname Your-New-Host-Name-Here # hostnamectl set-hostname "Your New Host Name Here" --pretty # hostnamectl set-hostname Your-New-Host-Name-Here --static # hostnamectl set-hostname Your-New-Host-Name-Here --transient
And here is what I did:
[root@rac1 ~]# hostnamectl set-hostname rac1.home-oracle
[root@rac1 ~]# hostnamectl status
Static hostname: rac1.home-oracle
Icon name: computer-vm
Chassis: vm
Machine ID: 2334d0c81bb84132a4f15d6b594a6647
Boot ID: 8f7baf3c1ef14f52aa819ab822127fa5
Virtualization: kvm
Operating System: Oracle Linux Server 7.1
CPE OS Name: cpe:/o:oracle:linux:7:1
Kernel: Linux 3.8.13-55.1.6.el7uek.x86_64
Architecture: x86_64
Let’s start with prereqs (all on the new VM):
- – Download the Oracle Public Yum Repository Files
As root, change directory to /etc/yum.repos.d. # cd /etc/yum.repos.d Use the wget utility to download the repository configuration file that is appropriate for your system. # wget http://public-yum.oracle.com/public-yum-release.repo For Oracle Linux 7, enter: # wget http://public-yum.oracle.com/public-yum-ol7.repo The /etc/yum.repos.d directory is updated with the repository configuration file, in this example, public-yum-ol7.repo. You can enable or disable repositories in the file by setting the value of the enabled directive to 1 or 0 as required.
- – Install the oracle-rdbms-server preinstall on Oracle Linux 7
Install the oracle-rdbms-server-12cR1-preinstall package as the root user: # yum install oracle-rdbms-server-12cR1-preinstall Install the oracle-rdbms-server-11gR2-preinstall package as the root user: # yum install oracle-rdbms-server-11gR2-preinstall
A best practice after the oracle-rdbms-server-*-preinstall RPM installation is to set the password for the oracle user that is created during the install. As root, type the following:
$ passwd oracle Changing password for user oracle. New password: Retype new password: passwd: all authentication tokens updated successfully. $
- Install the ASM required RPMs (you can also download them from here)
yum localinstall -y oracleasm-support-2.1.8-3.el7.x86_64.rpm oracleasmlib-2.0.12-1.el7.x86_64.rpm
- – Disable iptables/firewall
[root@rac1 ~]# systemctl status firewalld firewalld.service - firewalld - dynamic firewall daemon Loaded: loaded (/usr/lib/systemd/system/firewalld.service; enabled) Active: inactive (dead) since Sun 2016-04-24 21:15:33 CEST; 14h ago Process: 669 ExecStart=/usr/sbin/firewalld --nofork --nopid $FIREWALLD_ARGS (code=exited, status=0/SUCCESS) Main PID: 669 (code=exited, status=0/SUCCESS) CGroup: /system.slice/firewalld.service Apr 24 10:36:21 rac1.home-oracle systemd[1]: Starting firewalld - dynamic firewall daemon... Apr 24 10:36:22 rac1.home-oracle systemd[1]: Started firewalld - dynamic firewall daemon. Apr 24 21:15:27 rac1.home-oracle systemd[1]: Stopping firewalld - dynamic firewall daemon... Apr 24 21:15:33 rac1.home-oracle systemd[1]: Stopped firewalld - dynamic firewall daemon. [root@rac1 ~]# systemctl stop firewalld [root@rac1 ~]# systemctl disable firewalld rm '/etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service' rm '/etc/systemd/system/basic.target.wants/firewalld.service' [root@rac1 ~]#
- – Disable SELinux
Edit /etc/selinux/config. It should look like this: [root@rac1 ~]# cat /etc/selinux/config # This file controls the state of SELinux on the system. # SELINUX= can take one of these three values: # enforcing - SELinux security policy is enforced. # permissive - SELinux prints warnings instead of enforcing. # disabled - No SELinux policy is loaded. SELINUX=disablede # SELINUXTYPE= can take one of three two values: # targeted - Targeted processes are protected, # minimum - Modification of targeted policy. Only selected processes are protected. # mls - Multi Level Security protection. SELINUXTYPE=targeted
- – Configure the /etc/hosts File
Mine is looking like this (and I do use a DNS server as a separate VM):
[root@rac1 ~]# cat /etc/hosts 127.0.0.1 localhost.home-oracle localhost # Public 192.168.100.101 rac1.home-oracle rac1 192.168.100.102 rac2.home-oracle rac2 # Private 192.168.200.101 rac1-priv.home-oracle rac1-priv 192.168.200.102 rac2-priv.home-oracle rac2-priv # Virtual 192.168.100.111 rac1-vip.home-oracle rac1-vip 192.168.100.112 rac2-vip.home-oracle rac2-vip # SCAN 192.168.100.121 rac-scan.home-oracle rac-scan 192.168.100.122 rac-scan.home-oracle rac-scan 192.168.100.123 rac-scan.home-oracle tac-scan
- – Configure SSH and X11 Forwarding
Install xauth and the xorg-x11-apps RPMs, as root type:
# yum -y install xorg-x11-apps xauth.
Enable X11 forwarding in the /etc/ssh/sshd_config by adding “ForwardX11 yes”… something like this:
......... #AllowAgentForwarding yes #AllowTcpForwarding yes #GatewayPorts no X11Forwarding yes #X11DisplayOffset 10 .........
Now restart the sshd service
systemctl restart sshd.service
- – Disable IPv6
At the end of /etc/sysctl.conf file add these lines:
net.ipv6.conf.all.disable_ipv6 = 1 net.ipv6.conf.default.disable_ipv6 = 1
- – Create the necessary directories
# mkdir -p /u01/app/oracle/product/12.1.0/dbhome_1 # mkdir -p /u01/app/oracle/12.1.0/grid/ # cd /u01 # chown -R oracle:oinstall app # chmod -R 775 app/*
- – Adapt ~/.bash_profile
Mine is looking like this
[oracle@rac1 ~]$ cat .bash_profile # .bash_profile # Get the aliases and functions if [ -f ~/.bashrc ]; then . ~/.bashrc fi # User specific environment and startup programs PATH=$PATH:$HOME/.local/bin:$HOME/bin export PATH # Oracle variables TMP=/tmp; export TMP TMPDIR=$TMP; export TMPDIR ORACLE_HOSTNAME=rac1.home-oracle; export ORACLE_HOSTNAME ORACLE_BASE=/u01/app/oracle; export ORACLE_BASE ORACLE_HOME=$ORACLE_BASE/product/12.1.0/dbhome_1; export ORACLE_HOME AGENT_HOME=$ORACLE_BASE/product/agentr4/agent_inst; export AGENT_HOME OMS_HOME=$ORACLE_BASE/product/MiddlewareR4/oms; export OMS_HOME ORACLE_INSTANCE=/u01/app/oracle/product/gc_inst/WebTierIH1; export ORACLE_INSTANCE ORACLE_SID=ORCL; export ORACLE_SID ORACLE_UNQNAME=ORCL1; export ORACLE_UNQNAME PATH=/usr/sbin:$PATH; export PATH PATH=$ORACLE_HOME/bin:$PATH; export PATH LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib; export LD_LIBRARY_PATH CLASSPATH=$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib; export CLASSPATH
- Clone your RAC1 to RAC2
Note: Don’t forget to adapt the name and the network IPs
- Run CVU to insure that your environment is complying with all prerequisites.
I used in the beginning the executable delivered within the 12c grid zip file which I have downloaded from OTN. I ran into these issues:
- a) The CVU was failing at this point:
[oracle@rac1 grid]$ ./runcluvfy.sh stage -pre crsinst -n rac1,rac2 -verbose ............. Check: TCP connectivity of subnet "192.168.200.0" Source Destination Connected? ------------------------------ ------------------------------ ---------------- rac1 : 192.168.200.101 rac1 : 192.168.200.101 passed rac2 : 192.168.200.102 rac1 : 192.168.200.101 failed ERROR: PRVG-11850 : The system call "connect" failed with error "113" while executing exectask on node "rac2" No route to host rac1 : 192.168.200.101 rac2 : 192.168.200.102 failed ERROR: PRVG-11850 : The system call "connect" failed with error "113" while executing exectask on node "rac1" No route to host rac2 : 192.168.200.102 rac2 : 192.168.200.102 passed Result: TCP connectivity check failed for subnet "192.168.200.0" WARNING: Could not find a suitable set of interfaces for VIPs WARNING: Could not find a suitable set of interfaces for the private interconnect Checking subnet mask consistency... Subnet mask consistency check passed for subnet "192.168.100.0". Subnet mask consistency check passed for subnet "192.168.200.0". Subnet mask consistency check passed. Result: Node connectivity check failed Checking multicast communication... Checking subnet "192.168.100.0" for multicast communication with multicast group "224.0.0.251"... PRVG-11138 : Interface "192.168.100.101" on node "rac1" is not able to communicate with interface "192.168.100.101" on node "rac1" over multicast group "224.0.0.251" PRVG-11138 : Interface "192.168.100.101" on node "rac1" is not able to communicate with interface "192.168.100.102" on node "rac2" over multicast group "224.0.0.251" PRVG-11138 : Interface "192.168.100.102" on node "rac2" is not able to communicate with interface "192.168.100.101" on node "rac1" over multicast group "224.0.0.251" PRVG-11138 : Interface "192.168.100.102" on node "rac2" is not able to communicate with interface "192.168.100.102" on node "rac2" over multicast group "224.0.0.251" Checking subnet "192.168.200.0" for multicast communication with multicast group "224.0.0.251"... PRVG-11138 : Interface "192.168.200.101" on node "rac1" is not able to communicate with interface "192.168.200.101" on node "rac1" over multicast group "224.0.0.251" PRVG-11138 : Interface "192.168.200.101" on node "rac1" is not able to communicate with interface "192.168.200.102" on node "rac2" over multicast group "224.0.0.251" PRVG-11138 : Interface "192.168.200.102" on node "rac2" is not able to communicate with interface "192.168.200.101" on node "rac1" over multicast group "224.0.0.251" PRVG-11138 : Interface "192.168.200.102" on node "rac2" is not able to communicate with interface "192.168.200.102" on node "rac2" over multicast group "224.0.0.251"
Solution
In my case, it was the firewall which I missed to disable it (on both nodes):
[root@rac2 ~]# systemctl stop firewalld [root@rac2 ~]# systemctl disable firewalld rm '/etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service' rm '/etc/systemd/system/basic.target.wants/firewalld.service' [root@rac2 ~]#
- b) The CVU was constantly failing at this step:
Starting check for /dev/shm mounted as temporary file system ... ERROR: PRVE-0426 : The size of in-memory file system mounted as /dev/shm is "2097152k" megabytes which is less than the required size of "2048" megabytes on node "" PRVE-0426 : The size of in-memory file system mounted as /dev/shm is "20631552k" megabytes which is less than the required size of "2048" megabytes on node "" Check for /dev/shm mounted as temporary file system failed
And this is my /dev/shm looking like:
[oracle@rac1 grid]$ cat /etc/fstab # # /etc/fstab # Created by anaconda on Thu Apr 21 21:53:57 2016 # # Accessible filesystems, by reference, are maintained under '/dev/disk' # See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info # /dev/mapper/ol-root / xfs defaults 0 0 UUID=f323280a-75df-40a1-a180-654334b3fdfc /boot xfs defaults 0 0 /dev/mapper/ol-home /home xfs defaults 0 0 /dev/mapper/ol-tmp /tmp xfs defaults 0 0 /dev/mapper/ol-u01 /u01 xfs defaults 0 0 /dev/mapper/ol-usr /usr xfs defaults 0 0 /dev/mapper/ol-var /var xfs defaults 0 0 /dev/mapper/ol-swap swap swap defaults 0 0 tmpfs /dev/shm tmpfs size=2048M 0 0
Now, the value of the shm is the one required by Oracle. As Oracle stated, this is a Bug related to a script (checktmpfs.sh) which is not processing correctly the size of the shm. It’s all described in this doc: Doc ID 1601354.1
In order to still check my environment I have downloaded the latest version on CVU from OTN, and all went just fine and I’m ready with all my prerequisite checks.
Configure ASM disk
I have created 5 shareable disks and attached them on both VMs.
I have formatted them and created 1 partition on each disk and this is the outcome:
[root@rac1]# fdisk -l | grep /dev/sd Disk /dev/sdb: 5368 MB, 5368709120 bytes, 10485760 sectors /dev/sdb1 2048 10485759 5241856 83 Linux Disk /dev/sdd: 5368 MB, 5368709120 bytes, 10485760 sectors /dev/sdd1 2048 10485759 5241856 83 Linux Disk /dev/sde: 5368 MB, 5368709120 bytes, 10485760 sectors /dev/sde1 2048 10485759 5241856 83 Linux Disk /dev/sdc: 5368 MB, 5368709120 bytes, 10485760 sectors /dev/sdc1 2048 10485759 5241856 83 Linux
On the second rac execute this:
[root@rac2 ]# partprobe [root@rac2 ]# fdisk -l | grep /dev/sd Disk /dev/sdb: 5368 MB, 5368709120 bytes, 10485760 sectors /dev/sdb1 2048 10485759 5241856 83 Linux Disk /dev/sdc: 5368 MB, 5368709120 bytes, 10485760 sectors /dev/sdc1 2048 10485759 5241856 83 Linux Disk /dev/sdd: 5368 MB, 5368709120 bytes, 10485760 sectors /dev/sdd1 2048 10485759 5241856 83 Linux Disk /dev/sde: 5368 MB, 5368709120 bytes, 10485760 sectors /dev/sde1 2048 10485759 5241856 83 Linux
On RAC1 & RAC2 as root configure the ASMlib:
[root@rac1 ~]# /etc/init.d/oracleasm configure
Configuring the Oracle ASM library driver.
This will configure the on-boot properties of the Oracle ASM library
driver. The following questions will determine whether the driver is
loaded on boot and what permissions it will have. The current values
will be shown in brackets ('[]'). Hitting without typing an
answer will keep that current value. Ctrl-C will abort.
Default user to own the driver interface []: oracle
Default group to own the driver interface []: dba
Scan for Oracle ASM disks on boot (y/n) [y]:
Writing Oracle ASM library driver configuration: done
Initializing the Oracle ASMLib driver: [ OK ]
Scanning the system for Oracle ASMLib disks: [ OK ]
[root@rac1 ~]#
On RAC1 as root create the ASM disks:
[root@rac1 ~]# oracleasm createdisk CRS /dev/sdb1 Writing disk header: done Instantiating disk: done [root@rac1 ~]# oracleasm createdisk ASM1 /dev/sdc1 Writing disk header: done Instantiating disk: done [root@rac1 ~]# oracleasm createdisk ASM2 /dev/sdd1 Writing disk header: done Instantiating disk: done [root@rac1 ~]# oracleasm createdisk ASM3 /dev/sde1 Writing disk header: done Instantiating disk: done [root@rac1 ~]# oracleasm scandisks Reloading disk partitions: done Cleaning any stale ASM disks... Scanning system for ASM disks... [root@rac1 ~]# oracleasm listdisks ASM1 ASM2 ASM3 CRS [root@rac1 ~]#
On RAC2 as root we scan and list the disks:
[root@rac2 ~]# oracleasm scandisks Reloading disk partitions: done Cleaning any stale ASM disks... Scanning system for ASM disks... Instantiating disk "CRS" Instantiating disk "ASM1" Instantiating disk "ASM2" Instantiating disk "ASM3" [root@rac2 ~]# oracleasm listdisks ASM1 ASM2 ASM3 CRS [root@rac2 ~]#
Just before running the GUI, I ran again a CVU check:
As oracle user on RAC1:
[oracle@rac1 bin]$ ./cluvfy stage -pre crsinst -n rac1,rac2 -fixup -verbose Performing pre-checks for cluster services setup ............... Pre-check for cluster services setup was successful. NOTE: No fixable verification failures to fix [oracle@rac1 bin]$
It looks good to go for the GUI 🙂
Install the clusterware release 12cR1
Go to the folder where you have unzipped the archives:
[oracle@rac1 ~]cd /media/sf_oracle_stuff/12cR1/grid [oracle@rac1 grid]$ ./runInstaller
Choose the first option and click Next
I configure a Standard cluster for fun
For fun I chose the first option. Click next.
Here I added also the second node to the configuration. Click next.
Checked also the subnet. Click next.
Put the correct Software location; As I like using ASM I chose the “Oracle Automatic Storage Management” option. I have set a very difficult password to guess. Don’t forget about the OSSASM group which should be the same as the one you have set while configuring the asm disks.
Choose the disk(s) where you want to put the Clusterware configuration. Pay attention that you need to allocate more than 5G (you’re going to store 2 DB instances on it)
Easy. Click next
In 12cR1 you have the “luxury” to choose whether you want the installer to run the last “root.sh” file(s) on both nodes for you…or not. In the end I chose not to…..I was too curious to see what’s doing behind. But it’s good to know you have the option though.
Now the installer is running the CVU script identifying all the problems that you may occur and for some, will offer you a fix.
As expected, the CVU didn’t “forget” about SHM issue from the beginning of my RAC installation. Being a known problem/bug, I chose “Ignore All” and moved on.
And this is the confirmation. I chose Yes.
OK. We’re now ready to install the Clusterware 12cR1. Click Install.
This is how the “progress” is looking like.
If you it does get done with “Copy files”, run the fix I told you in the beginning…. I mean this:
cp $ORACLE_HOME/javavm/jdk/jdk6/lib/lib*12.a $ORACLE_HOME/lib/
And you won’t get the image bellow :).
Well…if you still get the image from above, you can still apply the fix and click “Yes”
Now, we got to the root script executions and this is how it looks in my case:
[root@rac1 ~]# /u01/app/grid/12.1.0/root.sh Performing root user operation. The following environment variables are set as: ORACLE_OWNER= oracle ORACLE_HOME= /u01/app/grid/12.1.0 Enter the full pathname of the local bin directory: [/usr/local/bin]: The contents of "dbhome" have not changed. No need to overwrite. The contents of "oraenv" have not changed. No need to overwrite. The contents of "coraenv" have not changed. No need to overwrite. Entries will be added to the /etc/oratab file as needed by Database Configuration Assistant when a database is created Finished running generic part of root script. Now product-specific root actions will be performed. Using configuration parameter file: /u01/app/grid/12.1.0/crs/install/crsconfig_params 2016/04/28 11:03:13 CLSRSC-4001: Installing Oracle Trace File Analyzer (TFA) Collector. 2016/04/28 11:03:35 CLSRSC-4002: Successfully installed Oracle Trace File Analyzer (TFA) Collector. 2016/04/28 11:03:35 CLSRSC-363: User ignored prerequisites during installation OLR initialization - successful root wallet root wallet cert root cert export peer wallet profile reader wallet pa wallet peer wallet keys pa wallet keys peer cert request pa cert request peer cert pa cert peer root cert TP profile reader root cert TP pa root cert TP peer pa cert TP pa peer cert TP profile reader pa cert TP profile reader peer cert TP peer user cert pa user cert 2016/04/28 11:04:07 CLSRSC-330: Adding Clusterware entries to file 'oracle-ohasd.service' CRS-4133: Oracle High Availability Services has been stopped. CRS-4123: Oracle High Availability Services has been started. CRS-4133: Oracle High Availability Services has been stopped. CRS-4123: Oracle High Availability Services has been started. CRS-2672: Attempting to start 'ora.evmd' on 'rac1' CRS-2672: Attempting to start 'ora.mdnsd' on 'rac1' CRS-2676: Start of 'ora.mdnsd' on 'rac1' succeeded CRS-2676: Start of 'ora.evmd' on 'rac1' succeeded CRS-2672: Attempting to start 'ora.gpnpd' on 'rac1' CRS-2676: Start of 'ora.gpnpd' on 'rac1' succeeded CRS-2672: Attempting to start 'ora.cssdmonitor' on 'rac1' CRS-2672: Attempting to start 'ora.gipcd' on 'rac1' CRS-2676: Start of 'ora.cssdmonitor' on 'rac1' succeeded CRS-2676: Start of 'ora.gipcd' on 'rac1' succeeded CRS-2672: Attempting to start 'ora.cssd' on 'rac1' CRS-2672: Attempting to start 'ora.diskmon' on 'rac1' CRS-2676: Start of 'ora.diskmon' on 'rac1' succeeded CRS-2676: Start of 'ora.cssd' on 'rac1' succeeded ASM created and started successfully. Disk Group DATA created successfully. CRS-2672: Attempting to start 'ora.crf' on 'rac1' CRS-2672: Attempting to start 'ora.storage' on 'rac1' CRS-2676: Start of 'ora.storage' on 'rac1' succeeded CRS-2676: Start of 'ora.crf' on 'rac1' succeeded CRS-2672: Attempting to start 'ora.crsd' on 'rac1' CRS-2676: Start of 'ora.crsd' on 'rac1' succeeded CRS-4256: Updating the profile Successful addition of voting disk a052a4e6e22d4fe2bf39cb34f155796f. Successfully replaced voting disk group with +DATA. CRS-4256: Updating the profile CRS-4266: Voting file(s) successfully replaced ## STATE File Universal Id File Name Disk group -- ----- ----------------- --------- --------- 1. ONLINE a052a4e6e22d4fe2bf39cb34f155796f (ORCL:ASM1) [DATA] Located 1 voting disk(s). CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'rac1' CRS-2673: Attempting to stop 'ora.crsd' on 'rac1' CRS-2677: Stop of 'ora.crsd' on 'rac1' succeeded CRS-2673: Attempting to stop 'ora.crf' on 'rac1' CRS-2673: Attempting to stop 'ora.ctssd' on 'rac1' CRS-2673: Attempting to stop 'ora.evmd' on 'rac1' CRS-2673: Attempting to stop 'ora.storage' on 'rac1' CRS-2673: Attempting to stop 'ora.mdnsd' on 'rac1' CRS-2673: Attempting to stop 'ora.gpnpd' on 'rac1' CRS-2677: Stop of 'ora.storage' on 'rac1' succeeded CRS-2673: Attempting to stop 'ora.asm' on 'rac1' CRS-2677: Stop of 'ora.crf' on 'rac1' succeeded CRS-2677: Stop of 'ora.ctssd' on 'rac1' succeeded CRS-2677: Stop of 'ora.gpnpd' on 'rac1' succeeded CRS-2677: Stop of 'ora.mdnsd' on 'rac1' succeeded CRS-2677: Stop of 'ora.evmd' on 'rac1' succeeded CRS-2677: Stop of 'ora.asm' on 'rac1' succeeded CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'rac1' CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'rac1' succeeded CRS-2673: Attempting to stop 'ora.cssd' on 'rac1' CRS-2677: Stop of 'ora.cssd' on 'rac1' succeeded CRS-2673: Attempting to stop 'ora.gipcd' on 'rac1' CRS-2677: Stop of 'ora.gipcd' on 'rac1' succeeded CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'rac1' has completed CRS-4133: Oracle High Availability Services has been stopped. CRS-4123: Starting Oracle High Availability Services-managed resources CRS-2672: Attempting to start 'ora.mdnsd' on 'rac1' CRS-2672: Attempting to start 'ora.evmd' on 'rac1' CRS-2676: Start of 'ora.evmd' on 'rac1' succeeded CRS-2676: Start of 'ora.mdnsd' on 'rac1' succeeded CRS-2672: Attempting to start 'ora.gpnpd' on 'rac1' CRS-2676: Start of 'ora.gpnpd' on 'rac1' succeeded CRS-2672: Attempting to start 'ora.gipcd' on 'rac1' CRS-2676: Start of 'ora.gipcd' on 'rac1' succeeded CRS-2672: Attempting to start 'ora.cssdmonitor' on 'rac1' CRS-2676: Start of 'ora.cssdmonitor' on 'rac1' succeeded CRS-2672: Attempting to start 'ora.cssd' on 'rac1' CRS-2672: Attempting to start 'ora.diskmon' on 'rac1' CRS-2676: Start of 'ora.diskmon' on 'rac1' succeeded CRS-2676: Start of 'ora.cssd' on 'rac1' succeeded CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'rac1' CRS-2672: Attempting to start 'ora.ctssd' on 'rac1' CRS-2676: Start of 'ora.ctssd' on 'rac1' succeeded CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'rac1' succeeded CRS-2672: Attempting to start 'ora.asm' on 'rac1' CRS-2676: Start of 'ora.asm' on 'rac1' succeeded CRS-2672: Attempting to start 'ora.storage' on 'rac1' CRS-2676: Start of 'ora.storage' on 'rac1' succeeded CRS-2672: Attempting to start 'ora.crf' on 'rac1' CRS-2676: Start of 'ora.crf' on 'rac1' succeeded CRS-2672: Attempting to start 'ora.crsd' on 'rac1' CRS-2676: Start of 'ora.crsd' on 'rac1' succeeded CRS-6023: Starting Oracle Cluster Ready Services-managed resources CRS-6017: Processing resource auto-start for servers: rac1 CRS-6016: Resource auto-start has completed for server rac1 CRS-6024: Completed start of Oracle Cluster Ready Services-managed resources CRS-4123: Oracle High Availability Services has been started. 2016/04/28 11:08:18 CLSRSC-343: Successfully started Oracle Clusterware stack CRS-2672: Attempting to start 'ora.asm' on 'rac1' CRS-2676: Start of 'ora.asm' on 'rac1' succeeded CRS-2672: Attempting to start 'ora.DATA.dg' on 'rac1' CRS-2676: Start of 'ora.DATA.dg' on 'rac1' succeeded 2016/04/28 11:09:27 CLSRSC-325: Configure Oracle Grid Infrastructure for a Cluster ... succeeded [root@rac1 ~]#
And this is for the second node:
[root@rac2 ~]# /u01/app/grid/12.1.0/root.sh Performing root user operation. The following environment variables are set as: ORACLE_OWNER= oracle ORACLE_HOME= /u01/app/grid/12.1.0 Enter the full pathname of the local bin directory: [/usr/local/bin]: Copying dbhome to /usr/local/bin ... Copying oraenv to /usr/local/bin ... Copying coraenv to /usr/local/bin ... Creating /etc/oratab file... Entries will be added to the /etc/oratab file as needed by Database Configuration Assistant when a database is created Finished running generic part of root script. Now product-specific root actions will be performed. Using configuration parameter file: /u01/app/grid/12.1.0/crs/install/crsconfig_params 2016/04/28 11:09:40 CLSRSC-4001: Installing Oracle Trace File Analyzer (TFA) Collector. 2016/04/28 11:10:02 CLSRSC-4002: Successfully installed Oracle Trace File Analyzer (TFA) Collector. 2016/04/28 11:10:03 CLSRSC-363: User ignored prerequisites during installation OLR initialization - successful 2016/04/28 11:11:13 CLSRSC-330: Adding Clusterware entries to file 'oracle-ohasd.service' CRS-4133: Oracle High Availability Services has been stopped. CRS-4123: Oracle High Availability Services has been started. CRS-4133: Oracle High Availability Services has been stopped. CRS-4123: Oracle High Availability Services has been started. CRS-4133: Oracle High Availability Services has been stopped. CRS-4123: Starting Oracle High Availability Services-managed resources CRS-2672: Attempting to start 'ora.mdnsd' on 'rac2' CRS-2672: Attempting to start 'ora.evmd' on 'rac2' CRS-2676: Start of 'ora.mdnsd' on 'rac2' succeeded CRS-2676: Start of 'ora.evmd' on 'rac2' succeeded CRS-2672: Attempting to start 'ora.gpnpd' on 'rac2' CRS-2676: Start of 'ora.gpnpd' on 'rac2' succeeded CRS-2672: Attempting to start 'ora.gipcd' on 'rac2' CRS-2676: Start of 'ora.gipcd' on 'rac2' succeeded CRS-2672: Attempting to start 'ora.cssdmonitor' on 'rac2' CRS-2676: Start of 'ora.cssdmonitor' on 'rac2' succeeded CRS-2672: Attempting to start 'ora.cssd' on 'rac2' CRS-2672: Attempting to start 'ora.diskmon' on 'rac2' CRS-2676: Start of 'ora.diskmon' on 'rac2' succeeded CRS-2676: Start of 'ora.cssd' on 'rac2' succeeded CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'rac2' CRS-2672: Attempting to start 'ora.ctssd' on 'rac2' CRS-2676: Start of 'ora.ctssd' on 'rac2' succeeded CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'rac2' succeeded CRS-2672: Attempting to start 'ora.asm' on 'rac2' CRS-2676: Start of 'ora.asm' on 'rac2' succeeded CRS-2672: Attempting to start 'ora.storage' on 'rac2' CRS-2676: Start of 'ora.storage' on 'rac2' succeeded CRS-2672: Attempting to start 'ora.crf' on 'rac2' CRS-2676: Start of 'ora.crf' on 'rac2' succeeded CRS-2672: Attempting to start 'ora.crsd' on 'rac2' CRS-2676: Start of 'ora.crsd' on 'rac2' succeeded CRS-6017: Processing resource auto-start for servers: rac2 CRS-2672: Attempting to start 'ora.net1.network' on 'rac2' CRS-2676: Start of 'ora.net1.network' on 'rac2' succeeded CRS-2672: Attempting to start 'ora.ons' on 'rac2' CRS-2676: Start of 'ora.ons' on 'rac2' succeeded CRS-2673: Attempting to stop 'ora.LISTENER_SCAN1.lsnr' on 'rac1' CRS-2677: Stop of 'ora.LISTENER_SCAN1.lsnr' on 'rac1' succeeded CRS-2673: Attempting to stop 'ora.scan1.vip' on 'rac1' CRS-2677: Stop of 'ora.scan1.vip' on 'rac1' succeeded CRS-2672: Attempting to start 'ora.scan1.vip' on 'rac2' CRS-2676: Start of 'ora.scan1.vip' on 'rac2' succeeded CRS-2672: Attempting to start 'ora.LISTENER_SCAN1.lsnr' on 'rac2' CRS-2676: Start of 'ora.LISTENER_SCAN1.lsnr' on 'rac2' succeeded CRS-6016: Resource auto-start has completed for server rac2 CRS-6024: Completed start of Oracle Cluster Ready Services-managed resources CRS-4123: Oracle High Availability Services has been started. 2016/04/28 11:13:59 CLSRSC-343: Successfully started Oracle Clusterware stack 2016/04/28 11:14:13 CLSRSC-325: Configure Oracle Grid Infrastructure for a Cluster ... succeeded [root@rac2 ~]#
Now…we’re good to go further.
Now is creating all the other stuff it needs such as the listener/tnsnames/ management instance…etc
And ……we’re done.
