‘LIBCELL’ (EXADATA)

LIBCELL in its name tells us that LIB stands for library and CELL stands for storage cell

LIBCELL acts as a liquid layer which has the source code that allows the oracle kernel to communicate to the storage cells using network protocol called RDS(reliable datagram sockets)

Do you know that there is small library file used in exadata machine which is the core library which speaks with the iDB(intelligence database protocol)/RDS to get the data from storage cell

Let us get some practical insights from the machine

Locate to cell home and point to lib directory

[root@exceladm00 lib]# env|egrep 'OSS_HOME|LD_'

LD_LIBRARY_PATH=/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/cellsrv/lib
OSS_HOME=/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/cellsrv

Below is the libcell file which we look for!

[root@exceladm00 raw]# locate libcell
/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/cellsrv/lib/libcell11.so

Lets read the content of the file for insights

[root@exceladm00 ~]# strings /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/cellsrv/lib/libcell11.so > libcell

There are set of kernel parameters and error codes inside the file

This Libcell file actively interacts between kernel in the OS layer and storage layer and has the intelligence to ask for the data from storage cells via iDB which is required by the database or end user

[root@exceladm00 ~]# head libcell && echo "=================================================" &&tail libcell
_DYNAMIC
_GLOBAL_OFFSET_TABLE_
__gmon_start__
__cxa_finalize
_Jv_RegisterClasses
dbgc_get_gp
kgefac_
kgeasnmierr
__assert_fail
dbgdChkEventInt
=================================================
^^^^^^^^Error:  Buffer overrun occurred, forced exit
^^^^^^^^Initialization of symbol handler failed. Error code %d
^^^^^^^^RtlCaptureContext function not found in ntdll.dll
^^^^^^^^StackWalk is terminated abnormally. Error code %d
^^^^^^^^Exception is raised during stack walking
^^^^^^^^Signal %s is raised at 0x%p
^^^^^^^^Intel(R) Pentium(R) M and compatible Intel processors
^^^^^^^^Fatal Error: Can not initiate the Heap
basic
disabled

Lets view the process

[root@exceladm00 ~]# lsof|grep libcell
cellrssrm  4312      root  mem       REG                8,1  1717370     587643 /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/cellsrv/lib/libcell11.so
cellrssrm  4321      root  mem       REG                8,1  1717370     587643 /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/cellsrv/lib/libcell11.so
cellrssrm  4322      root  mem       REG                8,1  1717370     587643 /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/cellsrv/lib/libcell11.so
java       4323      root  mem       REG                8,1  1717370     587643 /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/cellsrv/lib/libcell11.so
cellrsbkm  4324      root  mem       REG                8,1  1717370     587643 /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/cellsrv/lib/libcell11.so
cellrsbkm  4339      root  mem       REG                8,1  1717370     587643 /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/cellsrv/lib/libcell11.so

Lets take a close look at trace for libcell process during a sql transaction and observe the insights from it

I triggered a select statement and grabbed the pid from the session for libcell

we can see lseek () system call which has the file pointer and increment the seek set from 0 to 512

we observe read() and write() system call which performs the data read from storage cell and write it to the buffer

Libcell constructs the sql statement over iDB/RDS and requests the data from the storage layer

you can observe the db block size as 8208 in the read system call which is shown in trace

 
    0.000000 read(11, "\2s\0\0\6\0\0\0\0\0\20\27\0\0\0U5gd\307M\272n|XU\315%\36\304\327x"..., 8208) = 627
  6171.714101 write(10, "\0\25\0\0\6\0\0\0\0\0\3\5y\3\0\0\0\17\0\0\0", 21) = 21
     0.000222 read(11, "\2\200\0\0\6\0\0\0\0\0\6\1\2\263\4\0\0\0\0\0\17\0\0\0\0\0\0\0\0\0\0\0"..., 8208) = 640
     0.000216 open("/data/oracle/11.2.0/db_home/rdbms/mesg/oraus.msb", O_RDONLY) = 9
     0.000141 fcntl(9, F_SETFD, FD_CLOEXEC) = 0
     0.000093 lseek(9, 0, SEEK_SET)     = 0
     0.000093 read(9, "\25\23\"\1\23\3\t\t\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 256) = 256
     0.000121 lseek(9, 512, SEEK_SET)   = 512
     0.000087 read(9, "f\31\2603gJ>h\265z\342\207C\226]\310m\374\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 512) = 512
     0.000107 lseek(9, 2560, SEEK_SET)  = 2560
     0.000085 read(9, "\241J\254J\264J\270J\273J\275J\277J\301J\303J\306J\312J\315J\321J\332J\340J\351J"..., 512) = 512
     0.000105 lseek(9, 486912, SEEK_SET) = 486912
     0.000126 read(9, "\v\0\27_\0\0J\0\30_\0\0e\0\31_\0\0\203\0\32_\0\0\254\0\33_\0\0\324\0"..., 512) = 512
     0.000121 close(9)                  = 0
     0.000134 write(10, "\0\25\0\0\6\0\0\0\0\0\3\5z\3\0\0\0\17\0\0\0", 21) = 21
     0.000193 read(11, "\2k\0\0\6\0\0\0\0\0\6\0\2\263\3\0\0\0\0\0\17\0\0\0\0\0\0\0\260\255l\371"..., 8208) = 619
     0.000072 write(10, "\0\25\0\0\6\0\0\0\0\0\3\5{\3\0\0\0\17\0\0\0", 21) = 21
     0.000363 read(11, "\2b\0\0\6\0\0\0\0\0\6\0\2\263\3\0\0\0\0\0\17\0\0\0\0\0\0\0\260\255l\371"..., 8208) = 610
     0.000212 write(10, "\0\25\0\0\6\0\0\0\0\0\3\5|\3\0\0\0\17\0\0\0", 21) = 21
     0.000189 read(11, "\2\200\0\0\6\0\0\0\0\0\6\0\2\263\3\0\0\0\0\0\17\0\0\0\0\0\0\0\260\255l\371"..., 8208) = 640

The contents shown in this article are just for general audience for understanding purpose and not standard operation procedure.Never ever try this in a production machine