Exadata Faked Storage Server Setup

The Exadata Server is the high-end Hardware and Software of Oracle’s Engineered Systems. Part of the Exadata is the Storage Server also called Cell Server. The complete Software stack of the Cell Server runs normally only on cell server hardware.

But there is a chance to do a setup of a fake Exadata Storage Server. This article describes how to do the setup.

required software

  • Virtualbox
  • Exadata Storage Server Software
    • Download from Oracle e-delivery
    • Version 12.1.1.0
    • V42777-01.zipOracle Database Machine Exadata Storage Cell (X4-2L, X4270M3, X4270M2, X4275) Image 12c Release 1 (12.1.1.1.0) for Linux x86_64
    • cell1
  • Oracle Linux
    • Download from Oracle e-delivery
    • Oracle Linux Release 5 Update 10 for x86_64 (64 Bit)  Name:V40139-01

The setup step by step

Create a new VM called for example cell1:

  • 1 core, 6GB RAM, 20 GB Diskspace
  • 2 network cards
    • eth0 dhcp
    • eth1 manually managed in my example
      • IP Adress: 192.168.178.200
      • Netmask: 255.255.255.0
      • Gateway: 192.168.178.1
      • (eth1 is the “Infiniband emulation”)
    • DVD medium is the V40139-01 Linux image
      • Setup the Linux including “Software Development”

The Linux installation is straight forward. After the setup install the Virtualbox Guest additions to be able to use “Shared Folders”. The “Shared Folders” are used as Software storage and is shown to the Linux machine as folder “/media/sf_Software”.

Finishing the Linux 5.10 installation I prepare for the cell server installation. Hostname is “cell1.localdoain”

faked cell server setup:

  • Login as root user
  • unzip V42777-01.zip image
    • 1.1.1.0_LINUX.X64_131219-1.x86_64.tar
    • untar the cellImageMaker File
    • two files are relevant
      • dl180/boot/cellbits/cell.bin
        • This is the cell software
      • dl180/boot/cellbits/cellrpms.tbz
        • This cellrpms.tbz file contains the jdk-1.7.0_25-fcs.x86_64.rpm
  • Installation as root
    • mkdir –p /var/log/oracle
    • rpm -ivh jdk-1.7.0_25-fcs.x86_64.rpm
    • rpm -ivh cell-12.1.1.1.0_LINUX.X64_131219-1.x86_64.rpm
    • rpm installation finished with

After the installation of the rpm the cell server starts.


Starting RS and MS... as user celladmin

Done. Please Login as user celladmin and create cell to startup CELLSRV to complete cell configuration.
If this is a manual installation, please stop and restart ExaWatcher to pick up newly installed binaries.You can run "/opt/oracle.ExaWatcher/ExaWatcher.sh --stop" and then /opt/oracle.
ExaWatcher/ExaWatcher.sh --fromconf" to stop and restart ExaWatcher. Logout and then re-login to use the new cell environment.

 

  • Create a set of disks for the cell server
    • This could also done before you install the cell rpms
    • 12 cell disks with a size of 1GB
    • 4 flash disks with a size of 1GB

cell2

 

 

 

 

 

 

 

  • Next reboot the Linux system
  • Login as root again
  • fdisk -l 2>/dev/null |grep „Disk“
  • Faked cell disk
    • Disk /dev/sdb: 1073 MB, 1073741824 bytes
    • Disk /dev/sdc: 1073 MB, 1073741824 bytes
    • Disk /dev/sdd: 1073 MB, 1073741824 bytes
    • Disk /dev/sde: 1073 MB, 1073741824 bytes
    • Disk /dev/sdf: 1073 MB, 1073741824 bytes
    • Disk /dev/sdg: 1073 MB, 1073741824 bytes
    • Disk /dev/sdh: 1073 MB, 1073741824 bytes
    • Disk /dev/sdi: 1073 MB, 1073741824 bytes
    • Disk /dev/sdj: 1073 MB, 1073741824 bytes
    • Disk /dev/sdk: 1073 MB, 1073741824 bytes
    • Disk /dev/sdl: 1073 MB, 1073741824 bytes
    • Disk /dev/sdm: 1073 MB, 1073741824 bytes
  • Faked Flash Disk
    • Disk /dev/sdn: 1073 MB, 1073741824 bytes
    • Disk /dev/sdo: 1073 MB, 1073741824 bytes
    • Disk /dev/sdp: 1073 MB, 1073741824 bytes
    • Disk /dev/sdq: 1073 MB, 1073741824 bytes

next create symbolic links

  • mkdir /opt/oracle/cell12.1.1.1.0_LINUX.X64_131219/disks/raw
  • create symbolic links
    • name the disks beginning with the name of your cell server
    • in my example “cell1” like the hostname
    • ln –s /dev/sdb cell1_DISK01
    • ln –s /dev/sdc cell1_DISK02
    • ln –s /dev/sdd cell1_DISK03
    • ln –s /dev/sde cell1_DISK04
    • ln –s /dev/sdf cell1_DISK05
    • ln –s /dev/sdg cell1_DISK06
    • ln –s /dev/sdh cell1_DISK07
    • ln –s /dev/sdi cell1_DISK08
    • ln –s /dev/sdj cell1_DISK09
    • ln –s /dev/sdk cell1_DISK10
    • ln –s /dev/sdl cell1_DISK11
    • ln –s /dev/sdm cell1_DISK12
  • the faked flash disks
    • ln –s /dev/sdn cell1_FLASH01
    • ln –s /dev/sdo cell1_FLASH02
    • ln –s /dev/sdp cell1_FLASH03
    • ln –s /dev/sdq cell1_FLASH04
  • Check if symbolic links are correct

change Linux kernel parameters

vi /etc/sysctl.conf

    • file-max = 65536
    • aio-max-nr=50000000
    • core.rmem_default=262144
    • core.rmem_max=4194304
    • core.wmem_default=262144
    • core.wmem_max=2097152
  • /etc/sysctl –p
  • vi /etc/security/limits.conf
    • * soft nofile 65536
    • * hard nofile 65536

 

  • Now enable RDS over InfiniBand modules
    • modprobe rds
    • modprobe rds_tcp
    • modprobe rds_rdma
    • to be sure that these modules are loaded after a reboot
    • [root@cell1 raw]# vi etc/modprobe.d/rds.conf
    • install rds /sbin/modprobe –ignore-install rds && /sbin/modprobe rds_tcp && /sbin/modprobe rds_rdma

Installation done

 

Start the faked Cell Server

Login as celladmin

  • su – celladmin
  • cellcli
    • alter cell restart services all
      • Restart the Services
    • Check if Cell Server is up and running

cell3

As I said Oracle marked the installation as “makeModel: Fake hardware”.

But the Cell Server is up and running

 

  • Listing of the Cell disks

cell4

 

  • Grid Disks

cell5

Start / Stop des Cell Servers

Startup

cell6

Shutdown

cell7

 

Summary

Based on the blog of dbaesp it is easy to make a setup of an Oracle Cell Server. I did a setup with Oracle Linux 5.10 and the Cell Server Relase 12.1.1.1.0. This is a very good environment to made some small workshops especially in the Cell Server administration world.

Next I will setup a 12.1 RAC Cluster and use the Cell Server as Storage “Sub-System”.

Have fun with the Faked Storage Server! :-)

Advertisements

Über spa

Oracle and Unix Professional, main focus on Oracle HA - Systems also an Exadata enthusiasts
Dieser Beitrag wurde unter Exadata abgelegt und mit , , , , , , verschlagwortet. Setze ein Lesezeichen auf den Permalink.

Kommentar verfassen

Trage deine Daten unten ein oder klicke ein Icon um dich einzuloggen:

WordPress.com-Logo

Du kommentierst mit Deinem WordPress.com-Konto. Abmelden / Ändern )

Twitter-Bild

Du kommentierst mit Deinem Twitter-Konto. Abmelden / Ändern )

Facebook-Foto

Du kommentierst mit Deinem Facebook-Konto. Abmelden / Ändern )

Google+ Foto

Du kommentierst mit Deinem Google+-Konto. Abmelden / Ändern )

Verbinde mit %s