X11 forwarding request failed on channel 0

I got this error while trying to redirect my display to the local host.

marius$ ssh -X oracle@rac1
Warning: the ECDSA host key for 'rac1' differs from the key for the IP address '192.168.100.101'
Offending key for IP in /Users/marius/.ssh/known_hosts:10
Matching host key in /Users/marius/.ssh/known_hosts:13
Are you sure you want to continue connecting (yes/no)? yes
oracle@rac1's password:
X11 forwarding request failed on channel 0
Last login: Mon Apr 25 18:02:35 2016 from 192.168.100.1
[oracle@rac1 ~]$

The solution that I’ve found is this one: just set the X11UseLocalhost to “no” (in my case was set to “yes”):

vi /etc/ssh/sshd_config
...............
X11Forwarding yes
#X11DisplayOffset 10
X11UseLocalhost no
................

How to enable a user to access a shared folder within Virtualbox?

This is what I did to be able to access shared folders on OL7 from my VM (Virtualbox):

# as root:
# gpasswd -a oracle vboxsf
# reboot

Oracle RAC12c on OL 7 using Virtualbox

Assumptions:
– You have already installed OL7 on a virtual machine
– assigned at least 4G of memory, otherwise the CVU will complain.
– 2 network cards

=============== BEGIN ERRORS DESCRIPTION =======================

Note: Along the installation of the CLusterware 12cR1 I encountered couple of issues.

1. This is actually the 2nd one, but should be fixed right at this step. 

Solution:

VBoxManage list vms
"rac1" {83ff3550-ced8-48be-8f8e-85d4363fb047}
"rac2" {3ae7500c-53b4-4ee6-aa8e-152869b6d473}

 VBoxManage setextradata 3ae7500c-53b4-4ee6-aa8e-152869b6d473 "VBoxInternal/CPUM/HostCPUID/Cache/Leaf" "0x4"
 VBoxManage setextradata 3ae7500c-53b4-4ee6-aa8e-152869b6d473 "VBoxInternal/CPUM/HostCPUID/Cache/SubLeaf" "0x4"
 VBoxManage setextradata 3ae7500c-53b4-4ee6-aa8e-152869b6d473 "VBoxInternal/CPUM/HostCPUID/Cache/eax" "0"
 VBoxManage setextradata 3ae7500c-53b4-4ee6-aa8e-152869b6d473 "VBoxInternal/CPUM/HostCPUID/Cache/ebx" "0"
 VBoxManage setextradata 3ae7500c-53b4-4ee6-aa8e-152869b6d473 "VBoxInternal/CPUM/HostCPUID/Cache/ecx" "0"
 VBoxManage setextradata 3ae7500c-53b4-4ee6-aa8e-152869b6d473 "VBoxInternal/CPUM/HostCPUID/Cache/edx" "0"
 VBoxManage setextradata 3ae7500c-53b4-4ee6-aa8e-152869b6d473 "VBoxInternal/CPUM/HostCPUID/Cache/SubLeafMask" "0xffffffff"

2. This is the first error that we’re going to get but this can be fixed while we’re doing the installation:

Solution:
As soon as the installer has finished with the step “Copying files” do this:

cp $ORACLE_HOME/javavm/jdk/jdk6/lib/lib*12.a $ORACLE_HOME/lib/

This library (libjavavm12.a) wasn’t copied at the right location

==============================END ERRORS =============================

If you have cloned the VM (as I do), this is the command to change the name of the VM:
This is the syntax:

# hostnamectl set-hostname Your-New-Host-Name-Here
# hostnamectl set-hostname "Your New Host Name Here" --pretty
# hostnamectl set-hostname Your-New-Host-Name-Here --static
# hostnamectl set-hostname Your-New-Host-Name-Here --transient

And here is what I did:

[root@rac1 ~]# hostnamectl set-hostname rac1.home-oracle
[root@rac1 ~]# hostnamectl status
   Static hostname: rac1.home-oracle
         Icon name: computer-vm
           Chassis: vm
        Machine ID: 2334d0c81bb84132a4f15d6b594a6647
           Boot ID: 8f7baf3c1ef14f52aa819ab822127fa5
    Virtualization: kvm
  Operating System: Oracle Linux Server 7.1
       CPE OS Name: cpe:/o:oracle:linux:7:1
            Kernel: Linux 3.8.13-55.1.6.el7uek.x86_64
      Architecture: x86_64

Let’s start with prereqs (all on the new VM):

• – Download the Oracle Public Yum Repository Files

As root, change directory to /etc/yum.repos.d.

# cd /etc/yum.repos.d
Use the wget utility to download the repository configuration file that is appropriate for your system.

# wget http://public-yum.oracle.com/public-yum-release.repo
For Oracle Linux 7, enter:

# wget http://public-yum.oracle.com/public-yum-ol7.repo
The /etc/yum.repos.d directory is updated with the repository configuration file, in this example, public-yum-ol7.repo.

You can enable or disable repositories in the file by setting the value of the enabled directive to 1 or 0 as required.

• – Install the oracle-rdbms-server preinstall on Oracle Linux 7

Install the oracle-rdbms-server-12cR1-preinstall package as the root user:
# yum install oracle-rdbms-server-12cR1-preinstall
 
Install the oracle-rdbms-server-11gR2-preinstall package as the root user:
# yum install oracle-rdbms-server-11gR2-preinstall

A best practice after the oracle-rdbms-server-*-preinstall RPM installation is to set the password for the oracle user that is created during the install. As root, type the following:

$ passwd oracle
Changing password for user oracle.
New password:
Retype new password:
passwd: all authentication tokens updated successfully.
$

• Install the ASM required RPMs (you can also download them from here)

yum localinstall -y oracleasm-support-2.1.8-3.el7.x86_64.rpm oracleasmlib-2.0.12-1.el7.x86_64.rpm

• – Disable iptables/firewall

[root@rac1 ~]# systemctl status firewalld
firewalld.service - firewalld - dynamic firewall daemon
   Loaded: loaded (/usr/lib/systemd/system/firewalld.service; enabled)
   Active: inactive (dead) since Sun 2016-04-24 21:15:33 CEST; 14h ago
  Process: 669 ExecStart=/usr/sbin/firewalld --nofork --nopid $FIREWALLD_ARGS (code=exited, status=0/SUCCESS)
 Main PID: 669 (code=exited, status=0/SUCCESS)
   CGroup: /system.slice/firewalld.service

Apr 24 10:36:21 rac1.home-oracle systemd[1]: Starting firewalld - dynamic firewall daemon...
Apr 24 10:36:22 rac1.home-oracle systemd[1]: Started firewalld - dynamic firewall daemon.
Apr 24 21:15:27 rac1.home-oracle systemd[1]: Stopping firewalld - dynamic firewall daemon...
Apr 24 21:15:33 rac1.home-oracle systemd[1]: Stopped firewalld - dynamic firewall daemon.
[root@rac1 ~]# systemctl stop firewalld
[root@rac1 ~]# systemctl disable firewalld
rm '/etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service'
rm '/etc/systemd/system/basic.target.wants/firewalld.service'
[root@rac1 ~]# 

• – Disable SELinux

Edit /etc/selinux/config. It should look like this:
[root@rac1 ~]# cat /etc/selinux/config

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=disablede
# SELINUXTYPE= can take one of three two values:
#     targeted - Targeted processes are protected,
#     minimum - Modification of targeted policy. Only selected processes are protected. 
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted 

• – Configure the /etc/hosts File

Mine is looking like this (and I do use a DNS server as a separate VM):

[root@rac1 ~]# cat /etc/hosts
127.0.0.1 localhost.home-oracle localhost
# Public
192.168.100.101 rac1.home-oracle rac1
192.168.100.102 rac2.home-oracle rac2
# Private
192.168.200.101 rac1-priv.home-oracle rac1-priv
192.168.200.102 rac2-priv.home-oracle rac2-priv
# Virtual
192.168.100.111 rac1-vip.home-oracle rac1-vip
192.168.100.112 rac2-vip.home-oracle rac2-vip
# SCAN
192.168.100.121 rac-scan.home-oracle rac-scan
192.168.100.122 rac-scan.home-oracle rac-scan
192.168.100.123 rac-scan.home-oracle tac-scan

• – Configure SSH and X11 Forwarding

Install xauth and the xorg-x11-apps RPMs, as root type:

# yum -y install xorg-x11-apps xauth. 

Enable X11 forwarding in the /etc/ssh/sshd_config by adding “ForwardX11 yes”… something like this:

.........
#AllowAgentForwarding yes
#AllowTcpForwarding yes
#GatewayPorts no
X11Forwarding yes
#X11DisplayOffset 10
.........

Now restart the sshd service

systemctl restart sshd.service

• – Disable IPv6

At the end of /etc/sysctl.conf file add these lines:

net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1

• – Create the necessary directories

# mkdir -p /u01/app/oracle/product/12.1.0/dbhome_1
# mkdir -p /u01/app/oracle/12.1.0/grid/
# cd /u01
# chown -R oracle:oinstall app
# chmod -R 775 app/*

• – Adapt ~/.bash_profile

Mine is looking like this

[oracle@rac1 ~]$ cat .bash_profile 
# .bash_profile

# Get the aliases and functions
if [ -f ~/.bashrc ]; then
	. ~/.bashrc
fi

# User specific environment and startup programs

PATH=$PATH:$HOME/.local/bin:$HOME/bin

export PATH

# Oracle variables
TMP=/tmp; export TMP
TMPDIR=$TMP; export TMPDIR
ORACLE_HOSTNAME=rac1.home-oracle; export ORACLE_HOSTNAME
ORACLE_BASE=/u01/app/oracle; export ORACLE_BASE
ORACLE_HOME=$ORACLE_BASE/product/12.1.0/dbhome_1; export ORACLE_HOME
AGENT_HOME=$ORACLE_BASE/product/agentr4/agent_inst; export AGENT_HOME
OMS_HOME=$ORACLE_BASE/product/MiddlewareR4/oms; export OMS_HOME
ORACLE_INSTANCE=/u01/app/oracle/product/gc_inst/WebTierIH1; export ORACLE_INSTANCE
ORACLE_SID=ORCL; export ORACLE_SID
ORACLE_UNQNAME=ORCL1; export ORACLE_UNQNAME
PATH=/usr/sbin:$PATH; export PATH
PATH=$ORACLE_HOME/bin:$PATH; export PATH
LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib; export LD_LIBRARY_PATH
CLASSPATH=$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib; export CLASSPATH

• Clone your RAC1 to RAC2

Note: Don’t forget to adapt the name and the network IPs

• Run CVU to insure that your environment is complying with all prerequisites.

I used in the beginning the executable delivered within the 12c grid zip file which I have downloaded from OTN. I ran into these issues:

• a) The CVU was failing at this point:

[oracle@rac1 grid]$ ./runcluvfy.sh stage -pre crsinst -n rac1,rac2 -verbose
.............

Check: TCP connectivity of subnet "192.168.200.0"
  Source                          Destination                     Connected?      
  ------------------------------  ------------------------------  ----------------
  rac1 : 192.168.200.101          rac1 : 192.168.200.101          passed          
  rac2 : 192.168.200.102          rac1 : 192.168.200.101          failed          

ERROR: 
PRVG-11850 : The system call "connect" failed with error "113" while executing exectask on node "rac2"
No route to host
  rac1 : 192.168.200.101          rac2 : 192.168.200.102          failed          

ERROR: 
PRVG-11850 : The system call "connect" failed with error "113" while executing exectask on node "rac1"
No route to host
  rac2 : 192.168.200.102          rac2 : 192.168.200.102          passed          
Result: TCP connectivity check failed for subnet "192.168.200.0"


WARNING: 
Could not find a suitable set of interfaces for VIPs

WARNING: 
Could not find a suitable set of interfaces for the private interconnect
Checking subnet mask consistency...
Subnet mask consistency check passed for subnet "192.168.100.0".
Subnet mask consistency check passed for subnet "192.168.200.0".
Subnet mask consistency check passed.

Result: Node connectivity check failed

Checking multicast communication...

Checking subnet "192.168.100.0" for multicast communication with multicast group "224.0.0.251"...
PRVG-11138 : Interface "192.168.100.101" on node "rac1" is not able to communicate with interface "192.168.100.101" on node "rac1" over multicast group "224.0.0.251"
PRVG-11138 : Interface "192.168.100.101" on node "rac1" is not able to communicate with interface "192.168.100.102" on node "rac2" over multicast group "224.0.0.251"
PRVG-11138 : Interface "192.168.100.102" on node "rac2" is not able to communicate with interface "192.168.100.101" on node "rac1" over multicast group "224.0.0.251"
PRVG-11138 : Interface "192.168.100.102" on node "rac2" is not able to communicate with interface "192.168.100.102" on node "rac2" over multicast group "224.0.0.251"
Checking subnet "192.168.200.0" for multicast communication with multicast group "224.0.0.251"...
PRVG-11138 : Interface "192.168.200.101" on node "rac1" is not able to communicate with interface "192.168.200.101" on node "rac1" over multicast group "224.0.0.251"
PRVG-11138 : Interface "192.168.200.101" on node "rac1" is not able to communicate with interface "192.168.200.102" on node "rac2" over multicast group "224.0.0.251"
PRVG-11138 : Interface "192.168.200.102" on node "rac2" is not able to communicate with interface "192.168.200.101" on node "rac1" over multicast group "224.0.0.251"
PRVG-11138 : Interface "192.168.200.102" on node "rac2" is not able to communicate with interface "192.168.200.102" on node "rac2" over multicast group "224.0.0.251"

Solution
In my case, it was the firewall which I missed to disable it (on both nodes):

[root@rac2 ~]# systemctl stop firewalld
[root@rac2 ~]# systemctl disable firewalld
rm '/etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service'
rm '/etc/systemd/system/basic.target.wants/firewalld.service'
[root@rac2 ~]# 

• b) The CVU was constantly failing at this step:

Starting check for /dev/shm mounted as temporary file system ...

ERROR: 

PRVE-0426 : The size of in-memory file system mounted as /dev/shm is "2097152k" megabytes which is less than the required size of "2048" megabytes on node ""
PRVE-0426 : The size of in-memory file system mounted as /dev/shm is "20631552k" megabytes which is less than the required size of "2048" megabytes on node ""

Check for /dev/shm mounted as temporary file system failed

And this is my /dev/shm looking like:

[oracle@rac1 grid]$ cat /etc/fstab 

#
# /etc/fstab
# Created by anaconda on Thu Apr 21 21:53:57 2016
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
/dev/mapper/ol-root     /                       xfs     defaults        0 0
UUID=f323280a-75df-40a1-a180-654334b3fdfc /boot                   xfs     defaults        0 0
/dev/mapper/ol-home     /home                   xfs     defaults        0 0
/dev/mapper/ol-tmp      /tmp                    xfs     defaults        0 0
/dev/mapper/ol-u01      /u01                    xfs     defaults        0 0
/dev/mapper/ol-usr      /usr                    xfs     defaults        0 0
/dev/mapper/ol-var      /var                    xfs     defaults        0 0
/dev/mapper/ol-swap     swap                    swap    defaults        0 0
tmpfs       		/dev/shm        	tmpfs   size=2048M      0 0

Now, the value of the shm is the one required by Oracle. As Oracle stated, this is a Bug related to a script (checktmpfs.sh) which is not processing correctly the size of the shm. It’s all described in this doc: Doc ID 1601354.1

In order to still check my environment I have downloaded the latest version on CVU from OTN, and all went just fine and I’m ready with all my prerequisite checks.

Configure ASM disk

I have created 5 shareable disks and attached them on both VMs.
I have formatted them and created 1 partition on each disk and this is the outcome:

[root@rac1]# fdisk -l | grep /dev/sd
Disk /dev/sdb: 5368 MB, 5368709120 bytes, 10485760 sectors
/dev/sdb1            2048    10485759     5241856   83  Linux
Disk /dev/sdd: 5368 MB, 5368709120 bytes, 10485760 sectors
/dev/sdd1            2048    10485759     5241856   83  Linux
Disk /dev/sde: 5368 MB, 5368709120 bytes, 10485760 sectors
/dev/sde1            2048    10485759     5241856   83  Linux
Disk /dev/sdc: 5368 MB, 5368709120 bytes, 10485760 sectors
/dev/sdc1            2048    10485759     5241856   83  Linux

On the second rac execute this:

[root@rac2 ]# partprobe 
[root@rac2 ]# fdisk -l | grep /dev/sd
Disk /dev/sdb: 5368 MB, 5368709120 bytes, 10485760 sectors
/dev/sdb1            2048    10485759     5241856   83  Linux
Disk /dev/sdc: 5368 MB, 5368709120 bytes, 10485760 sectors
/dev/sdc1            2048    10485759     5241856   83  Linux
Disk /dev/sdd: 5368 MB, 5368709120 bytes, 10485760 sectors
/dev/sdd1            2048    10485759     5241856   83  Linux
Disk /dev/sde: 5368 MB, 5368709120 bytes, 10485760 sectors
/dev/sde1            2048    10485759     5241856   83  Linux

On RAC1 & RAC2 as root configure the ASMlib:

[root@rac1 ~]# /etc/init.d/oracleasm configure
Configuring the Oracle ASM library driver.

This will configure the on-boot properties of the Oracle ASM library
driver.  The following questions will determine whether the driver is
loaded on boot and what permissions it will have.  The current values
will be shown in brackets ('[]').  Hitting  without typing an
answer will keep that current value.  Ctrl-C will abort.

Default user to own the driver interface []: oracle
Default group to own the driver interface []: dba
Scan for Oracle ASM disks on boot (y/n) [y]: 
Writing Oracle ASM library driver configuration: done
Initializing the Oracle ASMLib driver:                     [  OK  ]
Scanning the system for Oracle ASMLib disks:               [  OK  ]
[root@rac1 ~]# 

On RAC1 as root create the ASM disks:

[root@rac1 ~]# oracleasm createdisk CRS /dev/sdb1
Writing disk header: done
Instantiating disk: done
[root@rac1 ~]# oracleasm createdisk ASM1 /dev/sdc1
Writing disk header: done
Instantiating disk: done
[root@rac1 ~]# oracleasm createdisk ASM2 /dev/sdd1
Writing disk header: done
Instantiating disk: done
[root@rac1 ~]# oracleasm createdisk ASM3 /dev/sde1
Writing disk header: done
Instantiating disk: done
[root@rac1 ~]# oracleasm scandisks
Reloading disk partitions: done
Cleaning any stale ASM disks...
Scanning system for ASM disks...
[root@rac1 ~]# oracleasm listdisks
ASM1
ASM2
ASM3
CRS
[root@rac1 ~]# 

On RAC2 as root we scan and list the disks:

[root@rac2 ~]# oracleasm scandisks
Reloading disk partitions: done
Cleaning any stale ASM disks...
Scanning system for ASM disks...
Instantiating disk "CRS"
Instantiating disk "ASM1"
Instantiating disk "ASM2"
Instantiating disk "ASM3"
[root@rac2 ~]# oracleasm listdisks
ASM1
ASM2
ASM3
CRS
[root@rac2 ~]# 

Just before running the GUI, I ran again a CVU check:

As oracle user on RAC1:

[oracle@rac1 bin]$ ./cluvfy stage -pre crsinst -n rac1,rac2 -fixup -verbose

Performing pre-checks for cluster services setup 
...............
Pre-check for cluster services setup was successful. 


NOTE: 
No fixable verification failures to fix
[oracle@rac1 bin]$ 

It looks good to go for the GUI 🙂

Install the clusterware release 12cR1

Go to the folder where you have unzipped the archives:

[oracle@rac1 ~]cd /media/sf_oracle_stuff/12cR1/grid
[oracle@rac1 grid]$ ./runInstaller 

Choose the first option and click Next

I configure a Standard cluster for fun

For fun I chose the first option. Click next.

Here I added also the second node to the configuration. Click next.

Checked also the subnet. Click next.

Put the correct Software location; As I like using ASM I chose the “Oracle Automatic Storage Management” option. I have set a very difficult password to guess. Don’t forget about the OSSASM group which should be the same as the one you have set while configuring the asm disks.

Choose the disk(s) where you want to put the Clusterware configuration. Pay attention that you need to allocate more than 5G (you’re going to store 2 DB instances on it)

Easy. Click next

In 12cR1 you have the “luxury” to choose whether you want the installer to run the last “root.sh” file(s) on both nodes for you…or not. In the end I chose not to…..I was too curious to see what’s doing behind. But it’s good to know you have the option though.

Now the installer is running the CVU script identifying all the problems that you may occur and for some, will offer you a fix.

As expected, the CVU didn’t “forget” about SHM issue from the beginning of my RAC installation. Being a known problem/bug, I chose “Ignore All” and moved on.

And this is the confirmation. I chose Yes.

OK. We’re now ready to install the Clusterware 12cR1. Click Install.

This is how the “progress” is looking like.

If you it does get done with “Copy files”, run the fix I told you in the beginning…. I mean this:

cp $ORACLE_HOME/javavm/jdk/jdk6/lib/lib*12.a $ORACLE_HOME/lib/

And you won’t get the image bellow :).

Well…if you still get the image from above, you can still apply the fix and click “Yes”

Now, we got to the root script executions and this is how it looks in my case:

[root@rac1 ~]# /u01/app/grid/12.1.0/root.sh
Performing root user operation.
The following environment variables are set as:
    ORACLE_OWNER= oracle
    ORACLE_HOME=  /u01/app/grid/12.1.0
Enter the full pathname of the local bin directory: [/usr/local/bin]: 
The contents of "dbhome" have not changed. No need to overwrite.
The contents of "oraenv" have not changed. No need to overwrite.
The contents of "coraenv" have not changed. No need to overwrite.
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Using configuration parameter file: /u01/app/grid/12.1.0/crs/install/crsconfig_params
2016/04/28 11:03:13 CLSRSC-4001: Installing Oracle Trace File Analyzer (TFA) Collector.
2016/04/28 11:03:35 CLSRSC-4002: Successfully installed Oracle Trace File Analyzer (TFA) Collector.
2016/04/28 11:03:35 CLSRSC-363: User ignored prerequisites during installation
OLR initialization - successful
  root wallet
  root wallet cert
  root cert export
  peer wallet
  profile reader wallet
  pa wallet
  peer wallet keys
  pa wallet keys
  peer cert request
  pa cert request
  peer cert
  pa cert
  peer root cert TP
  profile reader root cert TP
  pa root cert TP
  peer pa cert TP
  pa peer cert TP
  profile reader pa cert TP
  profile reader peer cert TP
  peer user cert
  pa user cert
2016/04/28 11:04:07 CLSRSC-330: Adding Clusterware entries to file 'oracle-ohasd.service'
CRS-4133: Oracle High Availability Services has been stopped.
CRS-4123: Oracle High Availability Services has been started.
CRS-4133: Oracle High Availability Services has been stopped.
CRS-4123: Oracle High Availability Services has been started.
CRS-2672: Attempting to start 'ora.evmd' on 'rac1'
CRS-2672: Attempting to start 'ora.mdnsd' on 'rac1'
CRS-2676: Start of 'ora.mdnsd' on 'rac1' succeeded
CRS-2676: Start of 'ora.evmd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'rac1'
CRS-2676: Start of 'ora.gpnpd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'rac1'
CRS-2672: Attempting to start 'ora.gipcd' on 'rac1'
CRS-2676: Start of 'ora.cssdmonitor' on 'rac1' succeeded
CRS-2676: Start of 'ora.gipcd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'rac1'
CRS-2672: Attempting to start 'ora.diskmon' on 'rac1'
CRS-2676: Start of 'ora.diskmon' on 'rac1' succeeded
CRS-2676: Start of 'ora.cssd' on 'rac1' succeeded
ASM created and started successfully.
Disk Group DATA created successfully.
CRS-2672: Attempting to start 'ora.crf' on 'rac1'
CRS-2672: Attempting to start 'ora.storage' on 'rac1'
CRS-2676: Start of 'ora.storage' on 'rac1' succeeded
CRS-2676: Start of 'ora.crf' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'rac1'
CRS-2676: Start of 'ora.crsd' on 'rac1' succeeded
CRS-4256: Updating the profile
Successful addition of voting disk a052a4e6e22d4fe2bf39cb34f155796f.
Successfully replaced voting disk group with +DATA.
CRS-4256: Updating the profile
CRS-4266: Voting file(s) successfully replaced
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   a052a4e6e22d4fe2bf39cb34f155796f (ORCL:ASM1) [DATA]
Located 1 voting disk(s).
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'rac1'
CRS-2673: Attempting to stop 'ora.crsd' on 'rac1'
CRS-2677: Stop of 'ora.crsd' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.crf' on 'rac1'
CRS-2673: Attempting to stop 'ora.ctssd' on 'rac1'
CRS-2673: Attempting to stop 'ora.evmd' on 'rac1'
CRS-2673: Attempting to stop 'ora.storage' on 'rac1'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'rac1'
CRS-2673: Attempting to stop 'ora.gpnpd' on 'rac1'
CRS-2677: Stop of 'ora.storage' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'rac1'
CRS-2677: Stop of 'ora.crf' on 'rac1' succeeded
CRS-2677: Stop of 'ora.ctssd' on 'rac1' succeeded
CRS-2677: Stop of 'ora.gpnpd' on 'rac1' succeeded
CRS-2677: Stop of 'ora.mdnsd' on 'rac1' succeeded
CRS-2677: Stop of 'ora.evmd' on 'rac1' succeeded
CRS-2677: Stop of 'ora.asm' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'rac1'
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'rac1'
CRS-2677: Stop of 'ora.cssd' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'rac1'
CRS-2677: Stop of 'ora.gipcd' on 'rac1' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'rac1' has completed
CRS-4133: Oracle High Availability Services has been stopped.
CRS-4123: Starting Oracle High Availability Services-managed resources
CRS-2672: Attempting to start 'ora.mdnsd' on 'rac1'
CRS-2672: Attempting to start 'ora.evmd' on 'rac1'
CRS-2676: Start of 'ora.evmd' on 'rac1' succeeded
CRS-2676: Start of 'ora.mdnsd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'rac1'
CRS-2676: Start of 'ora.gpnpd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.gipcd' on 'rac1'
CRS-2676: Start of 'ora.gipcd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'rac1'
CRS-2676: Start of 'ora.cssdmonitor' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'rac1'
CRS-2672: Attempting to start 'ora.diskmon' on 'rac1'
CRS-2676: Start of 'ora.diskmon' on 'rac1' succeeded
CRS-2676: Start of 'ora.cssd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'rac1'
CRS-2672: Attempting to start 'ora.ctssd' on 'rac1'
CRS-2676: Start of 'ora.ctssd' on 'rac1' succeeded
CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'rac1'
CRS-2676: Start of 'ora.asm' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.storage' on 'rac1'
CRS-2676: Start of 'ora.storage' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.crf' on 'rac1'
CRS-2676: Start of 'ora.crf' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'rac1'
CRS-2676: Start of 'ora.crsd' on 'rac1' succeeded
CRS-6023: Starting Oracle Cluster Ready Services-managed resources
CRS-6017: Processing resource auto-start for servers: rac1
CRS-6016: Resource auto-start has completed for server rac1
CRS-6024: Completed start of Oracle Cluster Ready Services-managed resources
CRS-4123: Oracle High Availability Services has been started.
2016/04/28 11:08:18 CLSRSC-343: Successfully started Oracle Clusterware stack
CRS-2672: Attempting to start 'ora.asm' on 'rac1'
CRS-2676: Start of 'ora.asm' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.DATA.dg' on 'rac1'
CRS-2676: Start of 'ora.DATA.dg' on 'rac1' succeeded
2016/04/28 11:09:27 CLSRSC-325: Configure Oracle Grid Infrastructure for a Cluster ... succeeded
[root@rac1 ~]#

And this is for the second node:

[root@rac2 ~]# /u01/app/grid/12.1.0/root.sh
Performing root user operation.
The following environment variables are set as:
    ORACLE_OWNER= oracle
    ORACLE_HOME=  /u01/app/grid/12.1.0
Enter the full pathname of the local bin directory: [/usr/local/bin]: 
   Copying dbhome to /usr/local/bin ...
   Copying oraenv to /usr/local/bin ...
   Copying coraenv to /usr/local/bin ...
Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Using configuration parameter file: /u01/app/grid/12.1.0/crs/install/crsconfig_params
2016/04/28 11:09:40 CLSRSC-4001: Installing Oracle Trace File Analyzer (TFA) Collector.
2016/04/28 11:10:02 CLSRSC-4002: Successfully installed Oracle Trace File Analyzer (TFA) Collector.
2016/04/28 11:10:03 CLSRSC-363: User ignored prerequisites during installation
OLR initialization - successful
2016/04/28 11:11:13 CLSRSC-330: Adding Clusterware entries to file 'oracle-ohasd.service'
CRS-4133: Oracle High Availability Services has been stopped.
CRS-4123: Oracle High Availability Services has been started.
CRS-4133: Oracle High Availability Services has been stopped.
CRS-4123: Oracle High Availability Services has been started.
CRS-4133: Oracle High Availability Services has been stopped.
CRS-4123: Starting Oracle High Availability Services-managed resources
CRS-2672: Attempting to start 'ora.mdnsd' on 'rac2'
CRS-2672: Attempting to start 'ora.evmd' on 'rac2'
CRS-2676: Start of 'ora.mdnsd' on 'rac2' succeeded
CRS-2676: Start of 'ora.evmd' on 'rac2' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'rac2'
CRS-2676: Start of 'ora.gpnpd' on 'rac2' succeeded
CRS-2672: Attempting to start 'ora.gipcd' on 'rac2'
CRS-2676: Start of 'ora.gipcd' on 'rac2' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'rac2'
CRS-2676: Start of 'ora.cssdmonitor' on 'rac2' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'rac2'
CRS-2672: Attempting to start 'ora.diskmon' on 'rac2'
CRS-2676: Start of 'ora.diskmon' on 'rac2' succeeded
CRS-2676: Start of 'ora.cssd' on 'rac2' succeeded
CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'rac2'
CRS-2672: Attempting to start 'ora.ctssd' on 'rac2'
CRS-2676: Start of 'ora.ctssd' on 'rac2' succeeded
CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'rac2' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'rac2'
CRS-2676: Start of 'ora.asm' on 'rac2' succeeded
CRS-2672: Attempting to start 'ora.storage' on 'rac2'
CRS-2676: Start of 'ora.storage' on 'rac2' succeeded
CRS-2672: Attempting to start 'ora.crf' on 'rac2'
CRS-2676: Start of 'ora.crf' on 'rac2' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'rac2'
CRS-2676: Start of 'ora.crsd' on 'rac2' succeeded
CRS-6017: Processing resource auto-start for servers: rac2
CRS-2672: Attempting to start 'ora.net1.network' on 'rac2'
CRS-2676: Start of 'ora.net1.network' on 'rac2' succeeded
CRS-2672: Attempting to start 'ora.ons' on 'rac2'
CRS-2676: Start of 'ora.ons' on 'rac2' succeeded
CRS-2673: Attempting to stop 'ora.LISTENER_SCAN1.lsnr' on 'rac1'
CRS-2677: Stop of 'ora.LISTENER_SCAN1.lsnr' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.scan1.vip' on 'rac1'
CRS-2677: Stop of 'ora.scan1.vip' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.scan1.vip' on 'rac2'
CRS-2676: Start of 'ora.scan1.vip' on 'rac2' succeeded
CRS-2672: Attempting to start 'ora.LISTENER_SCAN1.lsnr' on 'rac2'
CRS-2676: Start of 'ora.LISTENER_SCAN1.lsnr' on 'rac2' succeeded
CRS-6016: Resource auto-start has completed for server rac2
CRS-6024: Completed start of Oracle Cluster Ready Services-managed resources
CRS-4123: Oracle High Availability Services has been started.
2016/04/28 11:13:59 CLSRSC-343: Successfully started Oracle Clusterware stack
2016/04/28 11:14:13 CLSRSC-325: Configure Oracle Grid Infrastructure for a Cluster ... succeeded
[root@rac2 ~]#

Now…we’re good to go further.

Now is creating all the other stuff it needs such as the listener/tnsnames/ management instance…etc

And ……we’re done.

 

Virtualbox automation scripts

As I’m trying to speed up things while I’m creating my ORACLE environments I started to build up some scripts to help me on doing it faster.
1. Adapt the network and the hosts file
Note: I’m running this script from my local server and not from VM, like this:

# root>ssh root@192.168.1.130 < $script_location/adapt_network.sh

================================== START =========================================
#!/bin/bash

################################################
# Adapting the /etc/sysconfig/network-scripts 

################################################

# Regenerate the IP rules
#mv /etc/udev/rules.d/70-persistent-net.rules ~/70-persistent-net.rules.bck
#/sbin/udevadm trigger --type=devices --action=add

### adapt the IPADDR 
cd /etc/sysconfig/network-scripts
grep -q '^IPADDR' ifcfg-eth1 && sed -i 's/^IPADDR.*/'IPADDR=192.168.100.10'/' ifcfg-eth1 || echo 'IPADDR=192.168.100.10' >> ifcfg-eth1
grep -q '^IPADDR' ifcfg-eth2 && sed -i 's/^IPADDR.*/'IPADDR=192.168.101.11'/' ifcfg-eth2 || echo 'IPADDR=192.168.101.10' >> ifcfg-eth2

### adapt the Name
grep -q '^NAME' ifcfg-eth1 && sed -i 's/^NAME.*/NAME="eth1"/' ifcfg-eth1 || echo 'NAME="eth1"' >> ifcfg-eth1
grep -q '^NAME' ifcfg-eth2 && sed -i 's/^NAME.*/NAME="eth2"/' ifcfg-eth2 || echo 'NAME="eth2"' >> ifcfg-eth2

### adapt the BOOTPROTO and remove the UUID
for f in ifcfg-eth{1..2}; do
grep -q '^BOOTPROTO' $f && sed -i 's/^BOOTPROTO.*/BOOTPROTO=none/' $f || echo 'BOOTPROTO=none' >> $f
sed -i '/UUID/d' $f
done

==================================== END =========================================

Note: I’m still searching for a way to use variables instead the “fixed” IP addresses; didn’t find a way yet.
Looking forwards for your comments.

How to change the network device name from eth5 to eth0

As I’m using a lot Virtualbox, I have often encountered the situation where I had to adapt the network adapters to follow some naming standards (to match my scripts 🙂 of course).

I’ve created some time ago a BASE VM where I deployed all the packages that I would possibly need for any ORACLE installation. On this BASE VM I have created 3 network adapters (1 bridged one and 2 host-only ones). Each time I need to create a VM, I’m just cloning the BASE VM – it saves A LOT of time. However, I still need to apply some changes, such as the hostname and network adapters.

So, here is what I did:

After the new VM is up&running, grab the MAC addresses first:

[root@base rules.d]# ifconfig | grep HWadd
eth3      Link encap:Ethernet  HWaddr 08:00:27:50:9C:9D
eth4      Link encap:Ethernet  HWaddr 08:00:27:B6:B4:99
eth5      Link encap:Ethernet  HWaddr 08:00:27:B1:8F:0A
[root@base rules.d]#

Go to /etc/udev/rules.d/ and edit 70-persistent-net.rules

This is what I have:

[root@base rules.d]# cat 70-persistent-net.rules
# This file was automatically generated by the /lib/udev/write_net_rules
# program, run by the persistent-net-generator.rules rules file.
#
# You can modify it, as long as you keep each rule on a single
# line, and change only the value of the NAME= key.
# PCI device 0x8086:0x100e (e1000)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="08:00:27:22:74:f9", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"
# PCI device 0x8086:0x100e (e1000)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="08:00:27:b1:b2:5c", ATTR{type}=="1", KERNEL=="eth*", NAME="eth2"
# PCI device 0x8086:0x100e (e1000)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="08:00:27:89:47:fd", ATTR{type}=="1", KERNEL=="eth*", NAME="eth1"
# PCI device 0x8086:0x100e (e1000)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="08:00:27:50:9c:9d", ATTR{type}=="1", KERNEL=="eth*", NAME="eth3"
# PCI device 0x8086:0x100e (e1000)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="08:00:27:b6:b4:99", ATTR{type}=="1", KERNEL=="eth*", NAME="eth4"
# PCI device 0x8086:0x100e (e1000)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="08:00:27:b1:8f:0a", ATTR{type}=="1", KERNEL=="eth*", NAME="eth5"
[root@base rules.d]#

As you can see, the new names are eth3 – eth4 – eth5. I’m going to delete the previous ones and rename the new ones as eth0 – eth1 – eth2. My file is going to look like this:

[root@base rules.d]# cat 70-persistent-net.rules
# This file was automatically generated by the /lib/udev/write_net_rules
# program, run by the persistent-net-generator.rules rules file.
#
# You can modify it, as long as you keep each rule on a single
# line, and change only the value of the NAME= key.
# PCI device 0x8086:0x100e (e1000)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="08:00:27:50:9c:9d", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"
# PCI device 0x8086:0x100e (e1000)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="08:00:27:b6:b4:99", ATTR{type}=="1", KERNEL=="eth*", NAME="eth1"
# PCI device 0x8086:0x100e (e1000)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="08:00:27:b1:8f:0a", ATTR{type}=="1", KERNEL=="eth*", NAME="eth2"
[root@base rules.d]#

The second step is to go here “/etc/sysconfig/network-scripts” and adapt the ifcfg-eth* files accordingly.

[root@base network-scripts]# ll ifcfg-eth*
-rw-r--r--. 1 root root 231 Jun 18  2014 ifcfg-eth0
-rw-r--r--. 1 root root 231 Jun 18  2014 ifcfg-eth1
-rw-r--r--. 1 root root 231 Jun 18  2014 ifcfg-eth2
[root@base network-scripts]#

Delete in each file the UUID line, replace the MAC address with the one u get with “ifconfig | grep HW” and adapt the Name with a proper one.

The last step is to REBOOT the server and check the network adapters.