At home I have a small atom-based server that was running Oracle VM Manager 3, installed using simple installation. Simple installation is the option where you just enter a password and the Oracle VM Manager installer installs : Oracle XE database, WebLogic Server and the Oracle VM Manager container. The same password is used for the database user, Oracle VM Manager database schema user, weblogic user and admin user for the manager instance.

The manager instance stores its data as objects inside the database. To do that, there is something called a datasource defined in weblogic during installation. It's basically a jdbc connection from weblogic to the database. This DS requires the following information : database hostname, database instance name, database listener port number, schema username and schema password. In my default install this was localhost, XE, 1521, ovs, mypassword.

Now that I re-organized my machines a bit, I have a larger server that runs a normal database 11.2.0.3, which I also happen to use for EM12c. So I figured I would take some load off the little atom server, keep it running Oracle VM Manager but shut down XE and move the schema over to my dedicated database host. This is a straightforward process so I just wanted to list the steps.

1) shut down Oracle VM Manager so that it does not continue updating the repository.
as root : /etc/init.d/ovmm stop

2) export the schema user using the exp command for Oracle XE
as oracle : 
cd /u01/app/oracle/product/11.2.0/xe
export ORACLE_HOME=`pwd`
export ORACLE_SID=XE
export PATH=$ORACLE_HOME/bin:$PATH
exp
(enter user ovs and its password)
export user (option 2)
export everything including data
this will create (by default) a file called expdat.dmp
copy this file over to the other server with the other database
The schema name is also in /u01/app/oracle/ovm-manager-3/.config (OVSSCHEMA)

3) shutdown oracle-xe as it's no longer needed  
as root : /etc/init.d/oracle-xe stop

4) import the ovs user into the new database. I like to do it as the user. 
I just simply pre-create the schema before starting import
as oracle : 
sqlplus '/ as sysdba'
create user ovs identified by MyPassword;
grant connect,resource to ovs;
at this point, run the imp utility on the box to import the expdat.dmp
import asks for username/password, enter ovs and its password
import yes on all data and tables and content.

At this point you have a good complete repository. 
Now let's make the Oracle VM Manager weblogic instance point to the new database.

5) on the original system, restart weblogic
as root :/etc/init.d/ovmm start
wait a few minutes for the instance to come online

6) use the ovm_admin tool
as oracle : 
cd /u01/app/oracle/ovm-manager-3/bin
./ovm_admin --modifyds orcl wopr8 1521 ovs mypassword
My new host name for the 11.2.0.3 database is called wopr, 
the database instance is orcl and listener is still 1521 with schema ovs
The admin tool asks for a password, this is the weblogic user password. 
In a simple install, this would be the same as your admin or ovs account password.

7) restart to have everything take effect.
as root : 
/etc/init.d/ovmm stop  ; sleep 5 ;/etc/init.d/ovmm start ;

8) edit the config file and update the new data 
vi /u01/app/oracle/ovm-manager-3/.config 
modify :
DBHOST=
SID=
LSNR=
OVSSCHEMA=
and leave the rest as is. 

that should do it !

It's been a while since we last posted something about ocfs2 on our Oracle blogs but that doesn't mean the filesystem hasn't evolved. I had to write a bit of a summary for a customer so I figured it would be a good idea to just add a blog entry and document it here as well.

OCFS2 is a native Linux cluster filesystem that has been around for quite a few years now and was developed at Oracle and also has had a ton of contributions from the folks at SuSE over several years. The filesystem got officially merged into 2.6.16 and all the changes since have been going into mainline first and then trickled down into versions we build for Linux distributions. So we have ocfs2 versions 1.2, 1.4, 1.6 (1.8 for Oracle VM 3) which are specific snapshots of the filesystem code and then released for specific kernels like 2.6.18 or 2.6.32.

SLES has a version of ocfs2 that they build, other vendors decided to not compile in the filesystem so for Oracle Linux, we of course make sure we have current versions available as well. We also provide support for the filesystem as part of Oracle Linux support. You do not need to buy extra clustering or filesystem add-on options, the code is part of Oracle Linux and the support is part of our regular Oracle Linux support subscriptions.

Many of the ocfs2 users, use the filesystem as an alternative to nfs, when I read the articles on the ocfs2 public maillists, this is a comment that comes back frequently. So there must be some truth to it as these are all unsolicited 3rd party comments :)... One nice thing with ocfs2 is that it's so very easy to set up. Just a simple text file (config file) on each node with the list of hostnames, ip addresses and you're basically good to go. One does need shared storage as it's a real cluster filesystem. This shared storage can be iscsi, san/fc or shared scsi and we highly recommend a private network so that you can isolate the cluster traffic. The main problem reports we get tend to be due to overloading servers. In a cluster filesystem you have to ensure that you know really what is going on with all servers, otherwise there is the potential for data corruption. This means that if a node gets in trouble, overloaded network or running out of memory, it will likely end up halting or rebooting the node so that the other servers can happily continue. A large percentage of customer reports tend to be related to misconfigured networks (share the interconnect / cluster traffic with everything else) or bad/slow disk subsystems that get overloaded and the heartbeat IOs cannot make it to the device.

One of the reasons ocfs2 is so trivial to configure, is that the entire ecosystem is integrated. It comes with its own embedded clustering stack, o2cb. This mini, specialized clusterstack provides node membership, heartbeat services and a distributed lock manager. This stack is not designed to be a general purpose userspace clusterstack but really tailored towards the basic requirements for our filesystem. Another really cool feature, I'd say it's in my top 3 cool features list for ocfs2, is dlmfs. dlmfs is a virtual filesystem that exposes a few simple locktypes : shared read, exclusive and trylock. There's a libo2dlm to use this in applications or you can simply use your shell to create a domain and locks just by doing mkdir and touch of files. Someone with some time on their hands could theoretically with some shell magic hack together a little userspace cluster daemon that could monitor applications or nodes and handle start, stop, restart. It's on my todo list but I haven't had time :) anyway, it's a very nifty feature.

Anyway, I digress... so one of the customer questions I had recently was about what's going on with ocfs2 and has there been any development effort. I decided to go look at the linux kernels since 2.6.27 and collect the list of checkins that have happened since. These features are also in our latest ocfs2 as part of Oracle VM 3.0 and for the most part also in our kernel (Unbreakable Enterprise Kernel). Here is the list, I think it's pretty impressive :

    Remove JBD compatibility layer 
    Add POSIX ACLs
    Add security xattr support (extended attributes for SELinux)
    Implement quota recovery 
    Periodic quota syncing 
    Implementation of local and global quota file handling
    Enable quota accounting on mount, disable on umount 
    Add a name indexed b-tree to directory inodes 
    Optimize inode allocation by remembering last group
    Optimize inode group allocation by recording last used group.
    Expose the file system state via debugfs 
    Add statistics for the checksum and ecc operations.
    Add CoW support. (reflink is unlimited inode-based (file based) writeable snapshots - very very useful for virtualization)
    Add ioctl for reflink. 
    Enable refcount tree support. 
    Always include ACL support 
    Implement allocation reservations, which reduces fragmentation significantly 
    Optimize punching-hole code, speeds up significantly some rare operations 
    Discontiguous block groups, necessary to improve some kind of allocations. It is a feature that marks an incompatible bit, ie, it makes a forward-compatible change
    Make nointr ("don't allow file operations to be interrupted") a default mount option 
    Allow huge (> 16 TiB) volumes to mount   (support for huge volumes)
    Add a mount option "coherency=*" to handle cluster coherency for O_DIRECT writes.
    Add new OCFS2_IOC_INFO ioctl: offers the none-privileged end-user a possibility to get filesys info gathering 
    Add support for heartbeat=global mount option (instead of having a heartbeat per filesystem you can now have a single heartbeat)
    SSD trimming support 
    Support for moving extents (preparation for defragmentation)

There are a number of external articles written about ocfs2, one that I found is here.

Have fun...

A feature that's not often talked about in Linux is kexec. kexec is part of an infrastructure that allows the Linux kernel to load a kernel directly. Basically jump right into executing the new kernel immediately instead of going to a standard reset -> system power-on -> bios/firmware initialize -> memory/device discovery -> bootloader -> linux kernel.

kexec's mechanism is most commonly used with kdump. Basically with kdump, when a crash or panic occurs, a new kernel is booted after the crash while the memory is preserved from the previous kernel's runtime. The new kernel can then capture this data and generate the dump which then can go to local disk, remote disk or anywhere else for that matter. In order to use kdump, you basically have to allocate/reserve memory for this dump kernel. This is done by adding crashkernel=xxx@yyy to the grub command line when booting. The crash kernel image is then loaded and will be executed when a crash or panic occurs. Even though kdump is a bit cumbersome to set up, it allows for really great flexibility and is very powerful in helping with debugging issues.

For those interested in kdump, there's a good blog out there test kdump on Oracle Linux.
Or for those that just want to read the documentation that's part of the Linux kernel tree : kdump.

Anyway, this entry is not about kdump. kdump is great but I wanted to talk about the use of kexec proper and how it can help with doing fast reboots of your systems. Both Oracle Linux 5 and Oracle Linux 6 have support for reboot to use kexec as the reboot mechanism (see /etc/init.d/halt for details). When a standard reboot command is executed, init goes to 6 and /etc/init.d/halt gets run. This script, when it sees that kexec has been configured with a kernel image, will just execute kexec -e. In a standard reboot (not reboot -f) the normal shutdown scripts get executed and at the end where the system normally does a reset.
This reset then makes the system hard reset, jump into the bios, does a memory test, finds devices, initialize the devices and firmware, boot the bootdevice bootloader, start the kernel.

To set up kexec you should run the following command shortly after you boot the system. If you want to automate this, it makes sense to add this to your rc scripts. We will look at integrating this more into the OS management scripts for Oracle Linux to make it easier for the system administrators.

kexec -l --append="`cat /proc/cmdline`" 
    --initrd=/boot/initrd-`uname -r`.img /boot/vmlinuz-`uname -r`

In my case I am running 2.6.32-200.13.1.el5uek.

Once this is done, the new kernel image is prepared, memory is allocated and you now can do one of 2 things :

- run reboot : halt at the tail end of a normal reboot (shutdown all services) will execute directly into this new kernel image, exactly the same way as you booted the OS to get to this point.

- you wish to do a very fast reboot without shutdown (reboot -f). then you do

sync; umount -a ; kexec -e

In this case, you bypass all the service shutdown scripts and instantly jump start the new kernel, this is by far the fastest way to restart your box.

The total amount of time saved is highly dependent on your server. Basically time a system startup all the way to grub executing the kernel image, that's the amount of time you will save on a subsequent reboot. This can range from a number of seconds (15,20) to, sometimes, several minutes.

One caveat with the use of kexec and instant restarts without going through device resets is that, in some cases, the devices might act badly or the driver might not be doing the right thing. Before you really use this on your system, test it first to ensure that the drivers for the hardware you have and the devices themselves are doing the right thing (tm).

You can find more info about kexec in this article written a number of years ago : kexec article.

I am planning on writing a entry next about how to use this with Oracle VM Server 3.

A few days ago I wrote this blog entry. It was a little example on how to use a container on Oracle Linux with our 2.6.39 kernel.

We just pushed the RPMs to both public-yum and ULN. So you can subscribe to the UEK2 beta channel or configure your yum repository for public-yum and get the packages.

To try out what I did in my blog, make sure you install the 2.6.39 uek2 kernel, get the latest btrfs-progs and get the lxc tools.

At Oracle OpenWorld we talked about Linux Containers. Here is an example of getting a Linux container going with Oracle Linux 6.1, UEK2 beta and btrfs. This is just an example, not released, production, bug-free... for those that don't read README files ;-)

This container example is using the existing Linux cgroups features in the mainline kernel (and also in UEK, UEK2) and lxc tools to create the environments.

Example assumptions :
- Host OS is Oracle Linux 6.1 with UEK2 beta.
- using btrfs filesystem for containers (to make use of snapshot capabilities)
- mounting the fs in /container
- use Oracle VM templates as a base environment
- Oracle Linux 5 containers

I have a second disk on my test machine (/dev/sdb) which I will use for this exercise.

# mkfs.btrfs  -L container  /dev/sdb

# mount
/dev/mapper/vg_wcoekaersrv4-lv_root on / type ext4 (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
tmpfs on /dev/shm type tmpfs (rw)
/dev/sda1 on /boot type ext4 (rw)
/dev/mapper/vg_wcoekaersrv4-lv_home on /home type ext4 (rw)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
/dev/mapper/loop0p2 on /mnt type ext3 (rw)
/dev/mapper/loop1p2 on /mnt2 type ext3 (rw)
/dev/sdb on /container type btrfs (rw)

lxc tools installed...

# rpm -qa|grep lxc
lxc-libs-0.7.5-2.x86_64
lxc-0.7.5-2.x86_64
lxc-devel-0.7.5-2.x86_64

lxc tools come with template config files :

# ls /usr/lib64/lxc/templates/
lxc-altlinux  lxc-busybox  lxc-debian  lxc-fedora  lxc-lenny  
lxc-ol4  lxc-ol5  lxc-opensuse  lxc-sshd  lxc-ubuntu

I created one for Oracle Linux 5 : lxc-ol5.

Download Oracle VM template for OL5 from http://edelivery.oracle.com/linux. I used OVM_EL5U5_X86_PVM_10GB.
We want to be able to create 1 environment that can be used in both container and VM mode to avoid duplicate effort.

Untar the VM template.

# tar zxvf OVM_EL5U5_X86_PVM_10GB.tar.gz

These are the steps needed (to be automated in the future)...
Copy the content of the VM virtual disk's root filesystem into a btrfs subvolume in order to easily clone the base template.

My template configure script defines :
template_path=/container/ol5-template

- create subvolume ol5-template on /containers

# btrfs subvolume create /container/ol5-template
Create subvolume '/container/ol5-template'

- loopback mount the Oracle VM template System image / partition

# kpartx -a System.img 
# kpartx -l System.img 
loop0p1 : 0 192717 /dev/loop0 63
loop0p2 : 0 21607425 /dev/loop0 192780
loop0p3 : 0 4209030 /dev/loop0 21800205

I need to mount the 2nd partition of the virtual disk image, kpartx will set up loopback devices for each of the virtual disk partitions. So let's mount loop0p2 which will contain the Oracle Linux 5 / filesystem of the template.

# mount /dev/mapper/loop0p2 /mnt

# ls /mnt
bin  boot  dev  etc  home  lib  lost+found  media  misc  mnt  opt  proc  
root  sbin  selinux  srv  sys  tftpboot  tmp  u01  usr  var

Great, now we have the entire template / filesystem available. Let's copy this into our subvolume. This subvolume will then become the basis for all OL5 containers.

# cd /mnt
# tar cvf - * | ( cd /container/ol5-template ; tar xvf ; )

In the near future we will put some automation around the above steps.

# pwd
/container/ol5-template

# ls
bin  boot  dev  etc  home  lib  lost+found  media  misc  mnt  opt  proc  
root  sbin  selinux  srv  sys  tftpboot  tmp  u01  usr  var

From this point on, the lxc-create script, using the template config as an argument, should be able to automatically create a snapshot and set up the filesystem correctly.

# lxc-create -n ol5test1 -t ol5

Cloning base template /container/ol5-template to /container/ol5test1 ...
Create a snapshot of '/container/ol5-template' in '/container/ol5test1'
Container created : /container/ol5test1 ...
Container template source : /container/ol5-template
Container config : /etc/lxc/ol5test1
Network : eth0 (veth) on virbr0
'ol5' template installed
'ol5test1' created

# ls /etc/lxc/ol5test1/
config  fstab

# ls /container/ol5test1/
bin  boot  dev  etc  home  lib  lost+found  media  misc  mnt  opt  proc  
root  sbin  selinux  srv  sys  tftpboot  tmp  u01  usr  var

Now that it's created and configured, we should be able to just simply start it :

# lxc-start -n ol5test1
INIT: version 2.86 booting
                Welcome to Enterprise Linux Server
                Press 'I' to enter interactive startup.
Setting clock  (utc): Sun Oct 16 06:08:27 EDT 2011         [  OK  ]
Loading default keymap (us):                               [  OK  ]
Setting hostname ol5test1:                                 [  OK  ]
raidautorun: unable to autocreate /dev/md0
Checking filesystems
                                                           [  OK  ]
mount: can't find / in /etc/fstab or /etc/mtab
Mounting local filesystems:                                [  OK  ]
Enabling local filesystem quotas:                          [  OK  ]
Enabling /etc/fstab swaps:                                 [  OK  ]
INIT: Entering runlevel: 3
Entering non-interactive startup
Starting sysstat:  Calling the system activity data collector (sadc): 
                                                           [  OK  ]
Starting background readahead:                             [  OK  ]
Flushing firewall rules:                                   [  OK  ]
Setting chains to policy ACCEPT: nat mangle filter         [  OK  ]
Applying iptables firewall rules:                          [  OK  ]
Loading additional iptables modules: no                    [FAILED]
Bringing up loopback interface:                            [  OK  ]
Bringing up interface eth0:  
Determining IP information for eth0... done.
                                                           [  OK  ]
Starting system logger:                                    [  OK  ]
Starting kernel logger:                                    [  OK  ]
Enabling ondemand cpu frequency scaling:                   [  OK  ]
Starting irqbalance:                                       [  OK  ]
Starting portmap:                                          [  OK  ]
FATAL: Could not load /lib/modules/2.6.39-100.0.12.el6uek.x86_64/modules.dep: No such file or directory
Starting NFS statd:                                        [  OK  ]
Starting RPC idmapd: Error: RPC MTAB does not exist.
Starting system message bus:                               [  OK  ]
Starting o2cb:                                             [  OK  ]
Can't open RFCOMM control socket: Address family not supported by protocol

Mounting other filesystems:                                [  OK  ]
Starting PC/SC smart card daemon (pcscd):                  [  OK  ]
Starting HAL daemon:                                       [FAILED]
Starting hpiod:                                            [  OK  ]
Starting hpssd:                                            [  OK  ]
Starting sshd:                                             [  OK  ]
Starting cups:                                             [  OK  ]
Starting xinetd:                                           [  OK  ]
Starting crond:                                            [  OK  ]
Starting xfs:                                              [  OK  ]
Starting anacron:                                          [  OK  ]
Starting atd:                                              [  OK  ]
Starting yum-updatesd:                                     [  OK  ]
Starting Avahi daemon...                                   [FAILED]
Starting oraclevm-template...
Regenerating SSH host keys.
Stopping sshd:                                             [  OK  ]
Generating SSH1 RSA host key:                              [  OK  ]
Generating SSH2 RSA host key:                              [  OK  ]
Generating SSH2 DSA host key:                              [  OK  ]
Starting sshd:                                             [  OK  ]
Regenerating up2date uuid.
Setting Oracle validated configuration parameters.

Configuring network interface.
  Network device: eth0
  Hardware address: 52:19:C0:EF:78:C4

Do you want to enable dynamic IP configuration (DHCP) (Y|n)? 

... 

This will run the well-known Oracle VM template configure scripts and set up the container the same way as it would an Oracle VM guest.

The session that runs lxc-start is the local console. It is best to run this session inside screen so you can disconnect and reconnect.

At this point,I can use lxc-console to log into the local console of the container, or, since the container has its internal network up and running and sshd is running, I can also just ssh into the guest.

# lxc-console -n ol5test1 -t 1

Enterprise Linux Enterprise Linux Server release 5.5 (Carthage)
Kernel 2.6.39-100.0.12.el6uek.x86_64 on an x86_64

host login: 

I can simple get out of the console entering ctrl-a q.

From inside the container :

# mount
proc on /proc type proc (rw,noexec,nosuid,nodev)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)

# /sbin/ifconfig
eth0      Link encap:Ethernet  HWaddr 52:19:C0:EF:78:C4  
          inet addr:192.168.122.225  Bcast:192.168.122.255  Mask:255.255.255.0
          inet6 addr: fe80::5019:c0ff:feef:78c4/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:141 errors:0 dropped:0 overruns:0 frame:0
          TX packets:19 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:8861 (8.6 KiB)  TX bytes:2476 (2.4 KiB)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:8 errors:0 dropped:0 overruns:0 frame:0
          TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:560 (560.0 b)  TX bytes:560 (560.0 b)

# ps aux
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         1  0.0  0.0   2124   656 ?        Ss   06:08   0:00 init [3]  
root       397  0.0  0.0   1780   596 ?        Ss   06:08   0:00 syslogd -m 0
root       400  0.0  0.0   1732   376 ?        Ss   06:08   0:00 klogd -x
root       434  0.0  0.0   2524   368 ?        Ss   06:08   0:00 irqbalance
rpc        445  0.0  0.0   1868   516 ?        Ss   06:08   0:00 portmap
root       469  0.0  0.0   1920   740 ?        Ss   06:08   0:00 rpc.statd
dbus       509  0.0  0.0   2800   576 ?        Ss   06:08   0:00 dbus-daemon --system
root       578  0.0  0.0  10868  1248 ?        Ssl  06:08   0:00 pcscd
root       610  0.0  0.0   5196   712 ?        Ss   06:08   0:00 ./hpiod
root       615  0.0  0.0  13520  4748 ?        S    06:08   0:00 python ./hpssd.py
root       637  0.0  0.0  10168  2272 ?        Ss   06:08   0:00 cupsd
root       651  0.0  0.0   2780   812 ?        Ss   06:08   0:00 xinetd -stayalive -pidfile /var/run/xinetd.pid
root       660  0.0  0.0   5296  1096 ?        Ss   06:08   0:00 crond
root       745  0.0  0.0   1728   580 ?        SNs  06:08   0:00 anacron -s
root       753  0.0  0.0   2320   340 ?        Ss   06:08   0:00 /usr/sbin/atd
root       817  0.0  0.0  25580 10136 ?        SN   06:08   0:00 /usr/bin/python -tt /usr/sbin/yum-updatesd
root       819  0.0  0.0   2616  1072 ?        SN   06:08   0:00 /usr/libexec/gam_server
root       830  0.0  0.0   7116  1036 ?        Ss   06:08   0:00 /usr/sbin/sshd
root      2998  0.0  0.0   2368   424 ?        Ss   06:08   0:00 /sbin/dhclient -1 -q -lf /var/lib/dhclient/dhclient-eth0.leases -pf /var/run/dhc
root      3102  0.0  0.0   5008  1376 ?        Ss   06:09   0:00 login -- root     
root      3103  0.0  0.0   1716   444 tty2     Ss+  06:09   0:00 /sbin/mingetty tty2
root      3104  0.0  0.0   1716   448 tty3     Ss+  06:09   0:00 /sbin/mingetty tty3
root      3105  0.0  0.0   1716   448 tty4     Ss+  06:09   0:00 /sbin/mingetty tty4
root      3138  0.0  0.0   4584  1436 tty1     Ss   06:11   0:00 -bash
root      3167  0.0  0.0   4308   936 tty1     R+   06:12   0:00 ps aux

From the host :

# lxc-info -n ol5test1
state:   RUNNING
pid:     16539

# lxc-kill -n ol5test1

# lxc-monitor -n ol5test1
'ol5test1' changed state to [STOPPING]
'ol5test1' changed state to [STOPPED]

So creating more containers is trivial. Just keep running lxc-create.

# lxc-create -n ol5test2 -t ol5

# btrfs subvolume list /container
ID 297 top level 5 path ol5-template
ID 299 top level 5 path ol5test1
ID 300 top level 5 path ol5test2

lxc-tools will be uploaded to the uek2 beta channel to start playing with this.

Oracle Linux 4 example

Here is the same principle for Oracle Linux 4. Using the template create script lxc-ol4. I started out using the OVM_EL4U7_X86_PVM_4GB template and followed the same steps.

# kpartx -a System.img 

# kpartx -l System.img 
loop0p1 : 0 64197 /dev/loop0 63
loop0p2 : 0 8530515 /dev/loop0 64260
loop0p3 : 0 4176900 /dev/loop0 8594775

# mount /dev/mapper/loop0p2 /mnt

# cd /mnt

# btrfs subvolume create /container/ol4-template
Create subvolume '/container/ol4-template'

# tar cvf - * | ( cd /container/ol4-template ; tar xvf - ; )

# lxc-create -n ol4test1 -t ol4

Cloning base template /container/ol4-template to /container/ol4test1 ...
Create a snapshot of '/container/ol4-template' in '/container/ol4test1'
Container created : /container/ol4test1 ...
Container template source : /container/ol4-template
Container config : /etc/lxc/ol4test1
Network : eth0 (veth) on virbr0
'ol4' template installed
'ol4test1' created

# lxc-start -n ol4test1
INIT: version 2.85 booting
/etc/rc.d/rc.sysinit: line 80: /dev/tty5: Operation not permitted
/etc/rc.d/rc.sysinit: line 80: /dev/tty6: Operation not permitted
Setting default font (latarcyrheb-sun16):                  [  OK  ]

                Welcome to Enterprise Linux
                Press 'I' to enter interactive startup.
Setting clock  (utc): Sun Oct 16 09:34:56 EDT 2011         [  OK  ]
Initializing hardware...  storage network audio done       [  OK  ]
raidautorun: unable to autocreate /dev/md0
Configuring kernel parameters:  error: permission denied on key 'net.core.rmem_default'
error: permission denied on key 'net.core.rmem_max'
error: permission denied on key 'net.core.wmem_default'
error: permission denied on key 'net.core.wmem_max'
net.ipv4.ip_forward = 0
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.default.accept_source_route = 0
kernel.core_uses_pid = 1
fs.file-max = 327679
kernel.msgmni = 2878
kernel.msgmax = 8192
kernel.msgmnb = 65536
kernel.sem = 250 32000 100 142
kernel.shmmni = 4096
kernel.shmall = 1073741824
kernel.sysrq = 1
fs.aio-max-nr = 3145728
net.ipv4.ip_local_port_range = 1024 65000
kernel.shmmax = 4398046511104
                                                           [FAILED]
Loading default keymap (us):                               [  OK  ]
Setting hostname ol4test1:                                 [  OK  ]
Remounting root filesystem in read-write mode:             [  OK  ]
mount: can't find / in /etc/fstab or /etc/mtab
Mounting local filesystems:                                [  OK  ]
Enabling local filesystem quotas:                          [  OK  ]
Enabling swap space:                                       [  OK  ]
INIT: Entering runlevel: 3
Entering non-interactive startup
Starting sysstat:                                          [  OK  ]
Setting network parameters:  error: permission denied on key 'net.core.rmem_default'
error: permission denied on key 'net.core.rmem_max'
error: permission denied on key 'net.core.wmem_default'
error: permission denied on key 'net.core.wmem_max'
net.ipv4.ip_forward = 0
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.default.accept_source_route = 0
kernel.core_uses_pid = 1
fs.file-max = 327679
kernel.msgmni = 2878
kernel.msgmax = 8192
kernel.msgmnb = 65536
kernel.sem = 250 32000 100 142
kernel.shmmni = 4096
kernel.shmall = 1073741824
kernel.sysrq = 1
fs.aio-max-nr = 3145728
net.ipv4.ip_local_port_range = 1024 65000
kernel.shmmax = 4398046511104
                                                           [FAILED]
Bringing up loopback interface:                            [  OK  ]
Bringing up interface eth0:                                [  OK  ]
Starting system logger:                                    [  OK  ]
Starting kernel logger:                                    [  OK  ]
Starting portmap:                                          [  OK  ]
Starting NFS statd:                                        [FAILED]
Starting RPC idmapd: Error: RPC MTAB does not exist.
Mounting other filesystems:                                [  OK  ]
Starting lm_sensors:                                       [  OK  ]
Starting cups:                                             [  OK  ]
Generating SSH1 RSA host key:                              [  OK  ]
Generating SSH2 RSA host key:                              [  OK  ]
Generating SSH2 DSA host key:                              [  OK  ]
Starting sshd:                                             [  OK  ]
Starting xinetd:                                           [  OK  ]
Starting crond:                                            [  OK  ]
Starting xfs:                                              [  OK  ]
Starting anacron:                                          [  OK  ]
Starting atd:                                              [  OK  ]
Starting system message bus:                               [  OK  ]
Starting cups-config-daemon:                               [  OK  ]
Starting HAL daemon:                                       [  OK  ]
Starting oraclevm-template...
Regenerating SSH host keys.
Stopping sshd:                                             [  OK  ]
Generating SSH1 RSA host key:                              [  OK  ]
Generating SSH2 RSA host key:                              [  OK  ]
Generating SSH2 DSA host key:                              [  OK  ]
Starting sshd:                                             [  OK  ]
Regenerating up2date uuid.
Setting Oracle validated configuration parameters.

Configuring network interface.
  Network device: eth0
  Hardware address: D2:EC:49:0D:7D:80

Do you want to enable dynamic IP configuration (DHCP) (Y|n)? 
...
...

# lxc-console -n ol4test1

Enterprise Linux Enterprise Linux AS release 4 (October Update 7)
Kernel 2.6.39-100.0.12.el6uek.x86_64 on an x86_64

localhost login: 

More sad news - Dennis Ritchie passed away. Not covered in the media like the passing of Steve Jobs (equally sad) but dmr definitely deserves the recognition of all the stuff he invented, coded, built and provided to the tech world.

Rob Pike wrote a short entry here.

respect.

sorry, Bryan smacked me on the fingers because I misspelled DTrace :) so wherever I wrote Dtrace, do s/Dtrace/DTrace :) keeps Bryan happy :) Bryan we good now? ;) my bad for misspelling the name...

Another Oracle Openworld revelation was the availability of version 2 of the Unbreakable Enterprise Kernel, in short uek2. Let me put together a few notes on this as well.

UEK2 is based on Linux 2.6.39 with applied changesets (not backports) up to Linux 3.0.3. The change from 2.6 to 3.0, by Linus, was pretty much arbitrary this time around. There were no major changes in design that went into the kernel post 2.6.39. However, versions changes like that do have an effect on userspace programs, so in order to maintain our true compatibility and refrain from code changes, sticking to 2.6.39 was the best solution.

UEK2 beta is freely available to everyone. All you need is Oracle Linux 6 installed (also freely available from edelivery) and connect to the Unbreakable Linux Network (ULN) if you are an existing customer or if you just want to play with cool new stuff, use our public yum repository.

The source code for our git repo is also publicly available on oss.oracle.com. http://oss.oracle.com/git/?p=linux-2.6-unbreakable-beta.git;a=summary. Indeed, not just a tarball of junk but the entire changelog, all the checkins, all the history, nicely linked up with the original git repo from kernel.org.

There is a huge list of changes in the mainline kernel since 2.6.32, too long to sum up here. Some of the things we discussed as new features are the following :

- btrfs production.
btrfs is an awesome filesystem which is going through rigorous testing in order to be able to announce it production quality with uek2. The feature list of the filesystem is great, you can read some of my earlier blog entries to get a feel of it. Use it for root filesystem and do snapshots while upgrading packages is definitely an interesting enhancement to deliver.The existing Oracle Linux 6 btrfs progs work just fine but the UEK2 channel will soon include an updated version of the btrfs userspace programs to give you full access to all features in uek2.

- Linux containers
This is not a port of Solaris Zones. Linux containers is based on lxc and all the cgroups work that has gone into the kernel over the last several years. All the kernel features required to handle container functionality is in uek2 and as with btrfs progs, we are going to update the beta channel with the lxc tools in userspace to enable easy creation of Oracle Linux containers. Oracle VM templates will be easily converted to running as a container as well as a VM in Oracle VM.

- OpenVswitch
Openvswitch is a very interesting virtual switch product that can help a lot in particular in a virtual world but is a good Linux bridge replacement. These packages will also be made available with uek2 in the beta channel. More information about openvswitch can be found here.

- many new enhancements including even better performance than uek today.
More details will be posted in upcoming blogs with detailed data and use cases to show how this will help running Oracle software (and non-Oracle software) better than on any other Linux distribution out there.

Getting started from ULN :

Register your Oracle Linux 6 server with ULN using uln_register. Click on your server and Manage subscriptions and add the ol6_x86_64_UEK_BETA channel to the list of channels for this server. Then on the server just run yum update kernel-uek, reboot and you are done.

Getting started from the public-yum repository :

# cd /etc/yum.repos.d
# wget http://public-yum.oracle.com/beta/public-yum-ol6-beta.repo

Edit the repo file and change enabled=0 to enabled=1

# yum update kernel-uek
Loaded plugins: refresh-packagekit, rhnplugin
This system is not registered with ULN.
ULN support will be disabled.
uek2_beta                                                |  951 B     00:00
uek2_beta/primary                                        | 339 kB     00:01
uek2_beta                                                                   8/8
Setting up Update Process
Resolving Dependencies
--> Running transaction check
---> Package kernel-uek.x86_64 0:2.6.39-100.0.12.el6uek set to be updated
--> Processing Dependency: kernel-uek-firmware >= 2.6.39-100.0.12.el6uek for pac
kage: kernel-uek-2.6.39-100.0.12.el6uek.x86_64
--> Running transaction check
---> Package kernel-uek-firmware.noarch 0:2.6.39-100.0.12.el6uek set to be updat
ed
--> Finished Dependency Resolution

Dependencies Resolved

================================================================================
 Package                Arch      Version                    Repository    Size
================================================================================
Installing:
 kernel-uek-firmware    noarch    2.6.39-100.0.12.el6uek     uek2_beta    1.1 M
     replacing  kernel-firmware.noarch 2.6.32-71.el6
     replacing  kernel-uek-firmware.noarch 2.6.32-100.28.5.el6
Updating:
 kernel-uek             x86_64    2.6.39-100.0.12.el6uek     uek2_beta     25 M

Transaction Summary
================================================================================
Install       1 Package(s)
Upgrade       1 Package(s)

Total download size: 26 M
Is this ok [y/N]:
Downloading Packages:
(1/2): kernel-uek-2.6.39-100.0.12.el6uek.x86_64.rpm      |  25 MB     00:51
(2/2): kernel-uek-firmware-2.6.39-100.0.12.el6uek.noarch | 1.1 MB     00:05
--------------------------------------------------------------------------------
Total                                           452 kB/s |  26 MB     00:58
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
Warning: RPMDB altered outside of yum.
  Installing     : kernel-uek-firmware-2.6.39-100.0.12.el6uek.noarch        1/5
  Updating       : kernel-uek-2.6.39-100.0.12.el6uek.x86_64                 2/5
  Cleanup        : kernel-uek-2.6.32-100.28.5.el6.x86_64                    3/5
  Cleanup        : kernel-uek-firmware-2.6.32-100.28.5.el6.noarch           4/5
  Erasing        : kernel-firmware-2.6.32-71.el6.noarch                     5/5

Installed:
  kernel-uek-firmware.noarch 0:2.6.39-100.0.12.el6uek

Updated:
  kernel-uek.x86_64 0:2.6.39-100.0.12.el6uek

Replaced:
  kernel-firmware.noarch 0:2.6.32-71.el6
  kernel-uek-firmware.noarch 0:2.6.32-100.28.5.el6

Complete!

At this point, your new kernel is installed and ready to be used. /etc/grub.conf contains an entry for this beta kernel. Just a simple reboot and you are ready to go.

# reboot

One of the things we talked about at Oracle Openworld, last week, was Dtrace for Oracle Linux. There's not been much information on it yet so I wanted to write up what you need to do to give it a go.

We released a preview of Dtrace and we will provide updates as we include new features on an ongoing basis. The biggest project to implement is userspace tracing and many side projects to include and enhance various Dtrace providers in kernel. So it's not all there yet but it is going to continuously be enhanced with new updates.

Dtrace is made available to Oracle Linux support subscribers and currently requires you to do the following :

- have your system registered with the Unbreakable Linux Network (ULN) (or a local yum repo that mirrors the channels on ULN)
- run Oracle Linux 6, of course with the Unbreakable Enterprise Kernel
- register your server with the Dtrace channel (ol6_x86_64_Dtrace_BETA)
- install the updated version of 2.6.32 and the dtrace kernel module and userspace programs
- run it

Here are some of the detailed steps :
(1) register your server with ULN, run the uln_register command on your server as the root user. It requires you to enter your single sign-on ID, password and Oracle Linux support ID (CSI).
(2) once your system(s) is registered, login to the ULN website, click on Systems and find the server(s) you just registered
(3) Click on the server and go to Manage Subscriptions
(4) add the Dtrace for Oracle Linux 6 channel to your server and click Save
(5) back on the server, double check if everything worked by typing yum repolist it should now also show ol6_x86_64_Dtrace_BETA
(6) install the required packages :
... (a) yum install dtrace-modules
... (b) yum install dtrace-utils
... (c) yum install kernel-uek-2.6.32-201.0.4.el6uek
You have to install that specific kernel from the dtrace ULN channel because it is the kernel with the instrumentation.
(7) reboot into the new kernel
(8) load the kernel modules modprobe dtrace and modprobe systrace

Now you are ready to start playing with dtrace. We will publish a number of useful dtrace scripts to get you going. I have one quick example. When running an Oracle Database on my machine, I want to find out which files the oracle executable opens on the system. This simple command does it for me :

dtrace -n 'syscall::open*:entry/execname == "oracle"/{ printf("%s %s", execname, copyinstr(arg0)); }'

Any syscall that's open* executed by "oracle" will print the first argument being the path in this case.

A lot more to come but for the existing customers that want to start taking a look at it, check it out. The 2.6.32-201.0.4 kernel source is available on ULN, the dtrace kernel module source is under the CDDL license and is also available on ULN. The userspace tools are as is. Dtrace for our UEK2 Beta kernel is not there yet but will come in an update of the UEK2 beta kernel. Another thing we are evaluating is the use of the ksplice technology to "splice in" the probes/providers at runtime. So you would be able to run a kernel without the extra code, if you want to enable dtrace, you first apply the ksplice dtrace update, run dtrace and after the fact unload it. The word evaluate is key here, however :).

a little sample :

# dtrace -n 'syscall::open*:entry/execname == "oracle"/{ printf("%s %s", execname, copyinstr(arg0)); }'
dtrace: description 'syscall::open*:entry' matched 2 probes
CPU     ID                    FUNCTION:NAME
  2      8                       open:entry oracle /proc/2672/stat
  2      8                       open:entry oracle /proc/2674/stat
  2      8                       open:entry oracle /proc/2678/stat
  2      8                       open:entry oracle /proc/2682/stat
  2      8                       open:entry oracle /proc/2686/stat
  2      8                       open:entry oracle /proc/2688/stat
  2      8                       open:entry oracle /proc/2690/stat
  2      8                       open:entry oracle /proc/2692/stat
  2      8                       open:entry oracle /proc/2694/stat
  3      8                       open:entry oracle /proc/loadavg
  1      8                       open:entry oracle /u01/app/oracle/oradata/XE/system.dbf

...

p13036236_30_Linux-x86-64.zip(about 50MB), just unzip the file and mount the included ISO image somewhere, or burn a CD and as user root start the runUpgrader.sh script.

The upgrade utility will be able to upgrade from an existing Oracle VM 3.0.1 installation.
As always with an upgrade, it's recommended to do a full database repository backup. Or just run the exp utility against your Oracle VM Manager repository database and export the schema that's used to store the repository (defaults to ovs). Just in case.

Oracle VM Manager 3.0.1 has to be up and running for this process to start, the upgrade process will undeploy the application containers and replace them with the newer version. Most of the information to upgrade will be automatically detected. In version 3.0.1 we save the installation configuration to a config file in /u01/app/oracle/ovm-manager-3/.config. The only thing you will need to provide is the password for the database repository schema.

We do export the metadata into xml files and the upgrade script backs up these files as well, it would be possible to import this backup into the database again, however I would still recommend and exp of the database, it will be more convenient.(/tmp/ovm-manager-3-backup-(date).zip)
After the xml files are created, the upgrade tool will transform the data into the new format, delete the old data in the database and import the new version.

To upgrade the Oracle VM servers, you can register with the ovm3_3.0.2_x86_64_base channel on the Unbreakable Linux Network(ULN) and yum upgrade the servers.

The detailed documentation can be found here.

If you do not already have Oracle VM 3.0.1 installed, a new installer ISO image for Oracle VM 3.0.2 will also be made available in the next several days. If you can't wait, download 3.0.1 + the 3.0.2 patch and you're good to go as well.Last Friday (9/30) we uploaded the first patch for Oracle VM 3.0 to My Oracle Support. You can download the upgrade ISO from the website. Just look for patch 13036236. When you download the patch file

Another day another btrfs feature :) Compression

btrfs has built-in compression. (support for both lzo and zlib) This allows you to automatically compress data for the entire filesystem, subvolumes or even down to individual files. A mount option is all that's needed to specify which compression type to use: compress=lzo,zlib.

To try this out, I took a simple setup where I used a linux kernel tree as a testcase. Here are the results:

mount the volume on /mnt and untar a linux-2.6.32.tar.bz2 tree onto /mnt

# btrfs filesystem df /mnt
Data, RAID1: total=1.00GB, used=369.81MB
Data: total=8.00MB, used=0.00
System, RAID1: total=8.00MB, used=4.00KB
System: total=4.00MB, used=0.00
Metadata, RAID1: total=1.00GB, used=57.75MB
Metadata: total=8.00MB, used=0.00

Now, lets do this again,

# mount -o compress=lzo /dev/sdf /mnt

# btrfs filesystem df /mnt
Data, RAID1: total=1.00GB, used=188.95MB
Data: total=8.00MB, used=0.00
System, RAID1: total=8.00MB, used=4.00KB
System: total=4.00MB, used=0.00
Metadata, RAID1: total=1.00GB, used=41.37MB
Metadata: total=8.00MB, used=0.00

So, 188.95M instead of 369.81M - pretty darned cool... go lzo. but.. what about zlib!

# mount -o compress=zlib /dev/sdf /mnt

# btrfs file df /mnt
Data, RAID1: total=1.00GB, used=128.75MB
Data: total=8.00MB, used=0.00
System, RAID1: total=8.00MB, used=4.00KB
System: total=4.00MB, used=0.00
Metadata, RAID1: total=1.00GB, used=35.45MB
Metadata: total=8.00MB, used=0.00

Down to 128M! Of course from a filesystem point of view you don't have to do anything, just pass a mount option or set the attribute. As I mentioned earlier, btrfs lets you do this at a file level, a directory level (and inherit down for that directory) or all the way to the top level.

More cool bits : The compression uses kernel threads and will make use of as many threads as there are cpus. So compression gets loadbalanced/spread out across all threads in a server even if it's a single big file, we will split up the big files into 128kb chunks and compress in parallel.
Another cool bit : if you have an existing uncompressed filesystem, and want to compress it, or even just compress a file on it, you can do that with btrfs filesystem defragment. The defragment command has an option -c that lets you specify zlib or lzo.

Another day, another btrfs entry. I'm trying to learn all the in's and out's of the filesystem here.

As many of you know, btrfs supports CRC for data and metadata. I created a simple btrfs filesystem :

# mkfs.btrfs -L btrfstest -d raid1 -m raid1 /dev/sdb /dev/sdc

then created a file on the volume :

# dd if=/dev/urandom of=foo bs=1M count=100

# md5sum /btrfs/foo 
76f4c03dc7a3477939467ee230696b70  /btrfs/foo

so now lets play the bad guy and write over the disk itself, underneath the filesystem so it has no idea. This could be a shared device with another server that accidentally had data written on it, or a bad userspace program that spews out to the wrong device or even a bug in kernel...

Step 1: find the physical layout of the file :

# filefrag -v /btrfs/foo
Filesystem type is: 9123683e
File size of /btrfs/foo is 104857600 (25600 blocks, blocksize 4096)
 ext logical physical expected length flags
   0       0   269312           25600 eof
/btrfs/foo: 1 extent found

# echo $[4096*269312]
1103101952

The filesystem is 4k blocksize and we know it's at block 269312. Now we call btrfs-map-logical to find out what the physical offsets are on both the mirrors (/dev/sdb /dev/sdc) so I can happily overwrite it with junk.

# btrfs-map-logical -l 1103101952 -o scratch /dev/sdb
mirror 1 logical 1103101952 physical 1083179008 device /dev/sdc
mirror 2 logical 1103101952 physical 1103101952 device /dev/sdb

there we go. now. let's scribble :

# dd if=/dev/urandom of=/dev/sdc bs=1 count=50000 seek=1083179008

so we just wrote 50k bytes of random stuff to /dev/sdc at the offset of its copy of file foo
accessing the file gives the right md5sum still but now we have this command called scrub that can be run at any time and it will go through the filesystem you specific and check for any nasty errors and recover them. This happens through creating a kernel thread that does this in the background and then you can just use scrub status to see where it's at later.

# btrfs scrub start /btrfs

# btrfs scrub status /btrfs
scrub status for 15e213ad-4e2a-44f6-85d8-86d13e94099f
scrub started at Wed Sep 28 12:36:26 2011 and finished after 2 seconds
     total bytes scrubbed: 200.48MB with 13 errors
     error details: csum=13
     corrected errors: 13, uncorrectable errors: 0, unverified

As you can see above, the scrubber found 13 errors. A quick peek in dmesg shows the following :

btrfs: fixed up at 1103101952
btrfs: fixed up at 1103106048
btrfs: fixed up at 1103110144
btrfs: fixed up at 1103114240
btrfs: fixed up at 1103118336
btrfs: fixed up at 1103122432
btrfs: fixed up at 1103126528
btrfs: fixed up at 1103130624
btrfs: fixed up at 1103134720
btrfs: fixed up at 1103138816

# md5sum /btrfs/foo 
76f4c03dc7a3477939467ee230696b70  /btrfs/foo

Everything got repaired. This happens on both data and metadata. If there was a true IO error reading from one of the 2 sides we'd have handled that in the filesystem as well. If you don't have mirroring then with CRC it would have told you it was bad data and given you an IO error (instead of reading junk).

ok so now it's Monday and I found a few minutes to continue my weekend project at work :)

Today, I want to take my OL6.1 with UEK setup and convert the root ext4 partition to btrfs. Then use yum update to create a snapshot before rpm installs/updates so that if something goes wrong, one can revert back to the original state.
here's my story :

The default OL6 install uses ext4 for the root fileystem(/). So the first step in my test is to convert the ext4 filesystem into a btrfs filesystem. The cool thing is that btrfs actually lets you do that, there's a tool called btrfs-convert which takes a volume as an argument and then converts ext[2,3,4] to btrfs and leaves the original ext[2,3,4] as a snapshot so you can even go back to it if you want to.

In order to do this I did the following :

- prepared my initrd to have btrfs built in. rebuilt it running mkinitrd using --with-module=btrfs. this way, the kernel module for the btrfs filesystem is included in the initrd.
- find a boot ISO that has btrfs-convert on it (not yet on the OL6 ISOs)
- reboot the machine in rescue mode off of the ISO image
- run btrfs-convert on the root volume in my case it was /dev/mapper/vg_wcoekaersrv3-lv_root
- edit etc/fstab

/dev/mapper/vg_wcoekaersrv3-lv_root /                       ext4    defaults        1 1
to
/dev/mapper/vg_wcoekaersrv3-lv_root /                       btrfs    defaults        1 1

- reboot OL6 again
- at reboot OL presents a message saying that selinux has to re-label the files. This will take a few minutes and a reboot will automatically follow again

From this point on, you have OL6 running with btrfs as root filesystem.

# mount
/dev/mapper/vg_wcoekaersrv3-lv_root on / type btrfs (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
tmpfs on /dev/shm type tmpfs (rw,rootcontext="system_u:object_r:tmpfs_t:s0")
/dev/sda1 on /boot type ext4 (rw)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)

The original ext snapshot is still available as a subvolume :

# btrfs subvolume list /
ID 256 top level 5 path ext2_saved

I don't need it any more so I am just going to throw it out :

# btrfs subvolume delete /ext2_saved
Delete subvolume '//ext2_saved'

# btrfs subvolume list /

Just to run optimally, it's a good idea to de-fragment the volume as we inherit the old ext4 layout.

# btrfs filesystem defragment /

There. done.
Next up - make sure the yum-plugin-fs-snapshot is installed

# rpm -qa|grep yum-plugin
yum-plugin-fs-snapshot-1.1.30-6.el6.noarch

If not, then just run yum install yum-plugin-fs-snapshot it's on the OL6 media/ULN

So, now the big experiment. I want to do a yum update. Thanks to the installed plugin, yum will detect that the filesystem is btrfs and it will automatically, prior to installing new rpms, create a snapshot, then install.
In this case a long list, I just added bold fonts to the interesting tidbits...

# yum update
Loaded plugins: fs-snapshot
Setting up Update Process
Resolving Dependencies
--> Running transaction check
---> Package binutils.x86_64 0:2.20.51.0.2-5.20.el6 will be updated
---> Package binutils.x86_64 0:2.20.51.0.2-5.20.el6_1.1 will be an update
---> Package ca-certificates.noarch 0:2010.63-3.el6 will be updated
---> Package ca-certificates.noarch 0:2010.63-3.el6_1.5 will be an update
---> Package certmonger.x86_64 0:0.42-1.el6 will be updated
---> Package certmonger.x86_64 0:0.42-1.el6_1.2 will be an update
---> Package cifs-utils.x86_64 0:4.8.1-2.el6 will be updated
---> Package cifs-utils.x86_64 0:4.8.1-2.el6_1.2 will be an update
---> Package cups.x86_64 1:1.4.2-39.el6 will be updated
---> Package cups.x86_64 1:1.4.2-39.el6_1.1 will be an update
---> Package cups-libs.x86_64 1:1.4.2-39.el6 will be updated
---> Package cups-libs.x86_64 1:1.4.2-39.el6_1.1 will be an update
---> Package ipa-client.x86_64 0:2.0.0-23.el6_1.1 will be updated
---> Package ipa-client.x86_64 0:2.0.0-23.el6_1.2 will be an update
---> Package ipa-python.x86_64 0:2.0.0-23.el6_1.1 will be updated
---> Package ipa-python.x86_64 0:2.0.0-23.el6_1.2 will be an update
---> Package kernel-uek-devel.x86_64 0:2.6.39-100.0.5.el6uek will be installed
---> Package kernel-uek-headers.x86_64 0:2.6.32-100.34.1.el6uek will be updated
---> Package kernel-uek-headers.x86_64 0:2.6.32-200.16.1.el6uek will be updated
---> Package kernel-uek-headers.x86_64 0:2.6.39-100.0.5.el6uek will be an update
---> Package kpartx.x86_64 0:0.4.9-41.0.1.el6 will be updated
---> Package kpartx.x86_64 0:0.4.9-41.0.1.el6_1.1 will be an update
---> Package nss.x86_64 0:3.12.9-9.0.1.el6 will be updated
---> Package nss.x86_64 0:3.12.9-12.0.1.el6_1 will be an update
---> Package nss-sysinit.x86_64 0:3.12.9-9.0.1.el6 will be updated
---> Package nss-sysinit.x86_64 0:3.12.9-12.0.1.el6_1 will be an update
---> Package nss-tools.x86_64 0:3.12.9-9.0.1.el6 will be updated
---> Package nss-tools.x86_64 0:3.12.9-12.0.1.el6_1 will be an update
---> Package perf.x86_64 0:2.6.32-131.6.1.el6 will be updated
---> Package perf.x86_64 0:2.6.32-131.12.1.el6 will be an update
---> Package phonon-backend-gstreamer.x86_64 1:4.6.2-17.el6 will be updated
---> Package phonon-backend-gstreamer.x86_64 1:4.6.2-17.el6_1.1 will be an update
---> Package portreserve.x86_64 0:0.0.4-4.el6 will be updated
---> Package portreserve.x86_64 0:0.0.4-4.el6_1.1 will be an update
---> Package qt.x86_64 1:4.6.2-17.el6 will be updated
---> Package qt.x86_64 1:4.6.2-17.el6_1.1 will be an update
---> Package qt-sqlite.x86_64 1:4.6.2-17.el6 will be updated
---> Package qt-sqlite.x86_64 1:4.6.2-17.el6_1.1 will be an update
---> Package qt-x11.x86_64 1:4.6.2-17.el6 will be updated
---> Package qt-x11.x86_64 1:4.6.2-17.el6_1.1 will be an update
---> Package rsyslog.x86_64 0:4.6.2-3.el6_1.1 will be updated
---> Package rsyslog.x86_64 0:4.6.2-3.el6_1.2 will be an update
---> Package samba-client.x86_64 0:3.5.6-86.el6 will be updated
---> Package samba-client.x86_64 0:3.5.6-86.el6_1.4 will be an update
---> Package samba-common.x86_64 0:3.5.6-86.el6 will be updated
---> Package samba-common.x86_64 0:3.5.6-86.el6_1.4 will be an update
---> Package samba-winbind-clients.x86_64 0:3.5.6-86.el6 will be updated
---> Package samba-winbind-clients.x86_64 0:3.5.6-86.el6_1.4 will be an update
---> Package selinux-policy.noarch 0:3.7.19-93.0.1.el6_1.2 will be updated
---> Package selinux-policy.noarch 0:3.7.19-93.0.1.el6_1.7 will be an update
---> Package selinux-policy-targeted.noarch 0:3.7.19-93.0.1.el6_1.2 will be updated
---> Package selinux-policy-targeted.noarch 0:3.7.19-93.0.1.el6_1.7 will be an update
---> Package tzdata.noarch 0:2011h-2.el6 will be updated
---> Package tzdata.noarch 0:2011h-3.el6 will be an update
---> Package tzdata-java.noarch 0:2011h-2.el6 will be updated
---> Package tzdata-java.noarch 0:2011h-3.el6 will be an update
---> Package xmlrpc-c.x86_64 0:1.16.24-1200.1840.el6 will be updated
---> Package xmlrpc-c.x86_64 0:1.16.24-1200.1840.el6_1.4 will be an update
---> Package xmlrpc-c-client.x86_64 0:1.16.24-1200.1840.el6 will be updated
---> Package xmlrpc-c-client.x86_64 0:1.16.24-1200.1840.el6_1.4 will be an update
--> Finished Dependency Resolution
--> Running transaction check
---> Package kernel-uek-devel.x86_64 0:2.6.32-100.28.9.el6 will be erased
--> Finished Dependency Resolution

Dependencies Resolved

=================================================================================================
 Package                  Arch   Version                   Repository                       Size
=================================================================================================
Installing:
 kernel-uek-devel         x86_64 2.6.39-100.0.5.el6uek     kernel-uek-2.6.39-100.0.5-alpha 7.3 M
Updating:
 binutils                 x86_64 2.20.51.0.2-5.20.el6_1.1  ol6_latest                      2.8 M
 ca-certificates          noarch 2010.63-3.el6_1.5         ol6_latest                      531 k
 certmonger               x86_64 0.42-1.el6_1.2            ol6_latest                      193 k
 cifs-utils               x86_64 4.8.1-2.el6_1.2           ol6_latest                       41 k
 cups                     x86_64 1:1.4.2-39.el6_1.1        ol6_latest                      2.3 M
 cups-libs                x86_64 1:1.4.2-39.el6_1.1        ol6_latest                      314 k
 ipa-client               x86_64 2.0.0-23.el6_1.2          ol6_latest                       88 k
 ipa-python               x86_64 2.0.0-23.el6_1.2          ol6_latest                      491 k
 kernel-uek-headers       x86_64 2.6.39-100.0.5.el6uek     kernel-uek-2.6.39-100.0.5-alpha 716 k
 kpartx                   x86_64 0.4.9-41.0.1.el6_1.1      ol6_latest                       41 k
 nss                      x86_64 3.12.9-12.0.1.el6_1       ol6_latest                      772 k
 nss-sysinit              x86_64 3.12.9-12.0.1.el6_1       ol6_latest                       28 k
 nss-tools                x86_64 3.12.9-12.0.1.el6_1       ol6_latest                      749 k
 perf                     x86_64 2.6.32-131.12.1.el6       ol6_latest                      998 k
 phonon-backend-gstreamer x86_64 1:4.6.2-17.el6_1.1        ol6_latest                      125 k
 portreserve              x86_64 0.0.4-4.el6_1.1           ol6_latest                       22 k
 qt                       x86_64 1:4.6.2-17.el6_1.1        ol6_latest                      4.0 M
 qt-sqlite                x86_64 1:4.6.2-17.el6_1.1        ol6_latest                       50 k
 qt-x11                   x86_64 1:4.6.2-17.el6_1.1        ol6_latest                       12 M
 rsyslog                  x86_64 4.6.2-3.el6_1.2           ol6_latest                      450 k
 samba-client             x86_64 3.5.6-86.el6_1.4          ol6_latest                       11 M
 samba-common             x86_64 3.5.6-86.el6_1.4          ol6_latest                       13 M
 samba-winbind-clients    x86_64 3.5.6-86.el6_1.4          ol6_latest                      1.1 M
 selinux-policy           noarch 3.7.19-93.0.1.el6_1.7     ol6_latest                      741 k
 selinux-policy-targeted  noarch 3.7.19-93.0.1.el6_1.7     ol6_latest                      2.4 M
 tzdata                   noarch 2011h-3.el6               ol6_latest                      438 k
 tzdata-java              noarch 2011h-3.el6               ol6_latest                      150 k
 xmlrpc-c                 x86_64 1.16.24-1200.1840.el6_1.4 ol6_latest                      103 k
 xmlrpc-c-client          x86_64 1.16.24-1200.1840.el6_1.4 ol6_latest                       25 k
Removing:
 kernel-uek-devel         x86_64 2.6.32-100.28.9.el6       installed                        22 M

Transaction Summary
=================================================================================================
Install       1 Package(s)
Upgrade      29 Package(s)
Remove        1 Package(s)

Total size: 63 M
Is this ok [y/N]: y
Downloading Packages:
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
fs-snapshot: snapshotting /: /yum_20110926132957
  Updating   : nss-sysinit-3.12.9-12.0.1.el6_1.x86_64                                       1/61 
  Updating   : nss-3.12.9-12.0.1.el6_1.x86_64                                               2/61 
  Updating   : xmlrpc-c-1.16.24-1200.1840.el6_1.4.x86_64                                    3/61 
  Updating   : xmlrpc-c-client-1.16.24-1200.1840.el6_1.4.x86_64                             4/61 
  Updating   : samba-winbind-clients-3.5.6-86.el6_1.4.x86_64                                5/61 
  Updating   : samba-common-3.5.6-86.el6_1.4.x86_64                                         6/61 
  Updating   : certmonger-0.42-1.el6_1.2.x86_64                                             7/61 
  Updating   : nss-tools-3.12.9-12.0.1.el6_1.x86_64                                         8/61 
  Updating   : ca-certificates-2010.63-3.el6_1.5.noarch                                     9/61 
  Updating   : 1:qt-4.6.2-17.el6_1.1.x86_64                                                10/61 
  Updating   : 1:qt-sqlite-4.6.2-17.el6_1.1.x86_64                                         11/61 
  Updating   : 1:qt-x11-4.6.2-17.el6_1.1.x86_64                                            12/61 
  Updating   : 1:phonon-backend-gstreamer-4.6.2-17.el6_1.1.x86_64                          13/61 
  Updating   : portreserve-0.0.4-4.el6_1.1.x86_64                                          14/61 
  Updating   : ipa-python-2.0.0-23.el6_1.2.x86_64                                          15/61 
  Updating   : 1:cups-libs-1.4.2-39.el6_1.1.x86_64                                         16/61 
  Updating   : selinux-policy-3.7.19-93.0.1.el6_1.7.noarch                                 17/61 
  Updating   : selinux-policy-targeted-3.7.19-93.0.1.el6_1.7.noarch                        18/61 
  Updating   : 1:cups-1.4.2-39.el6_1.1.x86_64                                              19/61 
  Updating   : ipa-client-2.0.0-23.el6_1.2.x86_64                                          20/61 
  Updating   : samba-client-3.5.6-86.el6_1.4.x86_64                                        21/61 
  Updating   : tzdata-2011h-3.el6.noarch                                                   22/61 
  Updating   : cifs-utils-4.8.1-2.el6_1.2.x86_64                                           23/61 
  Updating   : rsyslog-4.6.2-3.el6_1.2.x86_64                                              24/61 
  Installing : kernel-uek-devel-2.6.39-100.0.5.el6uek.x86_64                               25/61 
  Updating   : kernel-uek-headers-2.6.39-100.0.5.el6uek.x86_64                             26/61 
  Updating   : binutils-2.20.51.0.2-5.20.el6_1.1.x86_64                                    27/61 
  Updating   : tzdata-java-2011h-3.el6.noarch                                              28/61 
  Updating   : perf-2.6.32-131.12.1.el6.x86_64                                             29/61 
  Updating   : kpartx-0.4.9-41.0.1.el6_1.1.x86_64                                          30/61 
  Cleanup    : selinux-policy-targeted-3.7.19-93.0.1.el6_1.2.noarch                        31/61 
  Cleanup    : selinux-policy-3.7.19-93.0.1.el6_1.2.noarch                                 32/61 
  Cleanup    : tzdata-2011h-2.el6.noarch                                                   33/61 
  Cleanup    : kernel-uek-headers.x86_64                                                   34/61 
  Cleanup    : kernel-uek-headers.x86_64                                                   35/61 
  Cleanup    : tzdata-java-2011h-2.el6.noarch                                              36/61 
  Cleanup    : perf-2.6.32-131.6.1.el6.x86_64                                              37/61 
  Cleanup    : kernel-uek-devel-2.6.32-100.28.9.el6.x86_64                                 38/61 
  Cleanup    : ipa-client-2.0.0-23.el6_1.1.x86_64                                          39/61 
  Cleanup    : certmonger-0.42-1.el6.x86_64                                                40/61 
  Cleanup    : 1:qt-x11-4.6.2-17.el6.x86_64                                                41/61 
  Cleanup    : 1:phonon-backend-gstreamer-4.6.2-17.el6.x86_64                              42/61 
  Cleanup    : samba-client-3.5.6-86.el6.x86_64                                            43/61 
  Cleanup    : 1:cups-1.4.2-39.el6.x86_64                                                  44/61 
  Cleanup    : samba-common-3.5.6-86.el6.x86_64                                            45/61 
  Cleanup    : 1:qt-sqlite-4.6.2-17.el6.x86_64                                             46/61 
  Cleanup    : 1:qt-4.6.2-17.el6.x86_64                                                    47/61 
  Cleanup    : xmlrpc-c-client-1.16.24-1200.1840.el6.x86_64                                48/61 
  Cleanup    : nss-tools-3.12.9-9.0.1.el6.x86_64                                           49/61 
  Cleanup    : ca-certificates-2010.63-3.el6.noarch                                        50/61 
  Cleanup    : nss-sysinit-3.12.9-9.0.1.el6.x86_64                                         51/61 
  Cleanup    : nss-3.12.9-9.0.1.el6.x86_64                                                 52/61 
  Cleanup    : xmlrpc-c-1.16.24-1200.1840.el6.x86_64                                       53/61 
  Cleanup    : samba-winbind-clients-3.5.6-86.el6.x86_64                                   54/61 
  Cleanup    : 1:cups-libs-1.4.2-39.el6.x86_64                                             55/61 
  Cleanup    : portreserve-0.0.4-4.el6.x86_64                                              56/61 
  Cleanup    : ipa-python-2.0.0-23.el6_1.1.x86_64                                          57/61 
  Cleanup    : cifs-utils-4.8.1-2.el6.x86_64                                               58/61 
  Cleanup    : rsyslog-4.6.2-3.el6_1.1.x86_64                                              59/61 
  Cleanup    : binutils-2.20.51.0.2-5.20.el6.x86_64                                        60/61 
  Cleanup    : kpartx-0.4.9-41.0.1.el6.x86_64                                              61/61 

Removed:
  kernel-uek-devel.x86_64 0:2.6.32-100.28.9.el6                                                  

Installed:
  kernel-uek-devel.x86_64 0:2.6.39-100.0.5.el6uek                                                

Updated:
  binutils.x86_64 0:2.20.51.0.2-5.20.el6_1.1                                                     
  ca-certificates.noarch 0:2010.63-3.el6_1.5                                                     
  certmonger.x86_64 0:0.42-1.el6_1.2                                                             
  cifs-utils.x86_64 0:4.8.1-2.el6_1.2                                                            
  cups.x86_64 1:1.4.2-39.el6_1.1                                                                 
  cups-libs.x86_64 1:1.4.2-39.el6_1.1                                                            
  ipa-client.x86_64 0:2.0.0-23.el6_1.2                                                           
  ipa-python.x86_64 0:2.0.0-23.el6_1.2                                                           
  kernel-uek-headers.x86_64 0:2.6.39-100.0.5.el6uek                                              
  kpartx.x86_64 0:0.4.9-41.0.1.el6_1.1                                                           
  nss.x86_64 0:3.12.9-12.0.1.el6_1                                                               
  nss-sysinit.x86_64 0:3.12.9-12.0.1.el6_1                                                       
  nss-tools.x86_64 0:3.12.9-12.0.1.el6_1                                                         
  perf.x86_64 0:2.6.32-131.12.1.el6                                                              
  phonon-backend-gstreamer.x86_64 1:4.6.2-17.el6_1.1                                             
  portreserve.x86_64 0:0.0.4-4.el6_1.1                                                           
  qt.x86_64 1:4.6.2-17.el6_1.1                                                                   
  qt-sqlite.x86_64 1:4.6.2-17.el6_1.1                                                            
  qt-x11.x86_64 1:4.6.2-17.el6_1.1                                                               
  rsyslog.x86_64 0:4.6.2-3.el6_1.2                                                               
  samba-client.x86_64 0:3.5.6-86.el6_1.4                                                         
  samba-common.x86_64 0:3.5.6-86.el6_1.4                                                         
  samba-winbind-clients.x86_64 0:3.5.6-86.el6_1.4                                                
  selinux-policy.noarch 0:3.7.19-93.0.1.el6_1.7                                                  
  selinux-policy-targeted.noarch 0:3.7.19-93.0.1.el6_1.7                                         
  tzdata.noarch 0:2011h-3.el6                                                                    
  tzdata-java.noarch 0:2011h-3.el6                                                               
  xmlrpc-c.x86_64 0:1.16.24-1200.1840.el6_1.4                                                    
  xmlrpc-c-client.x86_64 0:1.16.24-1200.1840.el6_1.4                                             

Complete!

Well, wasn't that easy! You can see the snapshot here :

# btrfs subvolume list /
ID 256 top level 5 path yum_20110926132957

So if something went wrong in the rpm update or you want to revert to the prior copy of the OS/filesystem, you can boot back into the snapshot, using subvolid=256 as filesystem mount options for / in fstab.

If you want to just default to the snapshot then you can run btrfs subvol set-default 256 and you are just running from the old snapshot state going forward.

Since I was playing with btrfs over the weekend, I figured I 'd keep a log of things I tried out and put them together in a little blog. Just to show off some of the really nifty stuff you can do with this filesystem :)

btrfs is included in Oracle Linux and we are working hard to help make this into a production supportable filesystem and make sure it's going through a huge amount of filesystem testing, recovery scenarios, performance etc.

Let's summarize a few of the features first :

- checksumming of data and metadata (CRC)
- built-in device/space management (spanned across devices) (so multiple device support no need for lvm)
- support for raid0, raid1, raid10 and single at this point (with raid5/6 in the works)
- ability to independently span metadata and data across these devices
- copy on write(COW) for both data and metadata
- writable snapshots
- create filesystem in existing btrfs pool without need to worry about device management
- online resize of filesystem (both grow and shrink)
- transparent compression, you can even specify for each file, or across all (lzo or zlib)
- ability to defrag files and/or directories
- balance command to balance filesystem chunks in a path across multiple devices if needed
- online add and remove devices to/from filesystems
- support for trim and SSD optimizations
- in place conversion from ext3/4 to btrfs
- file-based or object based cloning support with reflink (per file clone)
- file allocation is extent based with B-tree directory structures
- cool feature for cloning is that you can use filesystem seeding on read-only storage to then have a COW btrfs fs)
- for the little details :
- Max file size 16 EiB
- Max number of files 2^64
- Max volume size 16 EiB

Getting started is very easy. btrfs.ko is the kernel module that needs to be loaded and btrfs-progs is the package that has all the needed utilities to get started.

yum install btrfs-progs
modprobe btrfs

I added 3 8gb devices to my system, per /proc/partitions output : /dev/sdb, /dev/sdc, /dev/sdd.

   8       16    8388608 sdb
   8       32    8388608 sdc
   8       48    8388608 sdd
 253        0   16326656 dm-0
 253        1    4128768 dm-1

# cat /proc/partitions 
major minor  #blocks  name

   8        0   20971520 sda
   8        1     512000 sda1
   8        2   20458496 sda2

So let's create a btrfs filesystem on those 3 devices and label it btrfstest. I will also use -d raid10 and -m raid10 to show how easy it is to decide your spanning choices for both.

# mkfs.btrfs -L btrfstest -d raid10 -m raid10 /dev/sdb /dev/sdc /dev/sdd

WARNING! - Btrfs Btrfs v0.19 IS EXPERIMENTAL
WARNING! - see http://btrfs.wiki.kernel.org before using

adding device /dev/sdc id 2
adding device /dev/sdd id 3
fs created label btrfstest on /dev/sdb
        nodesize 4096 leafsize 4096 sectorsize 4096 size 24.00GB
Btrfs Btrfs v0.19

create a mountpoint /btrfs and mount the filesystem root there :

# mkdir /btrfs
# mount -t btrfs /dev/sdb /btrfs

/dev/sdb              25165824        28  25151488   1% /btrfs

as you can see, we now have a filesystem mounted that shows the diskspace of the 3 disks we added. btrfs filesystem show gives more detailed information including the device list and use :

# btrfs filesystem show
Label: 'btrfstest'  uuid: 123e52f9-87f2-4764-a347-5bedb3cb12df
        Total devices 3 FS bytes used 28.00KB
        devid    2 size 8.00GB used 0.00 path /dev/sdc
        devid    3 size 8.00GB used 0.00 path /dev/sdd
        devid    1 size 8.00GB used 20.00MB path /dev/sdb

Now to quickly show how easy it is to remove/add a device to an existing, mounted volume:
lets remove /dev/sdc

/dev/sdb              16777216        28  16501760   1% /btrfs

# btrfs device delete /dev/sdc /btrfs

# btrfs filesystem show
Label: 'btrfstest'  uuid: 123e52f9-87f2-4764-a347-5bedb3cb12df
        Total devices 3 FS bytes used 28.00KB
        devid    3 size 8.00GB used 0.00 path /dev/sdd
        devid    1 size 8.00GB used 20.00MB path /dev/sdb
        *** Some devices missing

# df
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/mapper/vg_wcoekaersrv3-lv_root
                      16070076   7494668   7759076  50% /
tmpfs                  1028992         0   1028992   0% /dev/shm
/dev/sda1               495844    107990    362254  23% /boot

as you can see, it now shows 8GB less of space available. so, let's add it back in :

/dev/sdb              25165824        28  24889344   1% /btrfs

# btrfs filesystem show
Label: 'btrfstest'  uuid: 123e52f9-87f2-4764-a347-5bedb3cb12df
        Total devices 3 FS bytes used 28.00KB
        devid    4 size 8.00GB used 0.00 path /dev/sdc
        devid    3 size 8.00GB used 256.00MB path /dev/sdd
        devid    1 size 8.00GB used 20.00MB path /dev/sdb

# btrfs device add /dev/sdc /btrfs

# df
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/mapper/vg_wcoekaersrv3-lv_root
                      16070076   7494668   7759076  50% /
tmpfs                  1028992         0   1028992   0% /dev/shm
/dev/sda1               495844    107990    362254  23% /boot

and it's back!
on to snapshots. I created a few files in /btrfs and now want to create a snapshot. so while using sync I will first sync the fs and then create a snapshot under /btrfs/.snapshot:

# btrfs filesystem sync /btrfs

# btrfs subvolume snapshot /btrfs /btrfs/.snapshot

# ls /btrfs/.snapshot
bar  baz  foo  test

# ls /btrfs
bar  baz  foo  test

and creating a new subvolume (a new possible mountpoint without any files, so not a snapshot just a mknewfs really)

# btrfs subvolume create /btrfs/test

# btrfs subvolume list /btrfs
ID 256 top level 5 path test

Some random commands to play with : 1) filesystem df shows a more detailed explanation of what's going on.

# btrfs filesystem df /btrfs
Data: total=8.00MB, used=0.00
System: total=4.00MB, used=8.00KB
Metadata: total=264.00MB, used=24.00KB

2) list subvolumes :

# btrfs subvolume list /btrfs
ID 256 top level 5 path test
ID 257 top level 5 path .snapshot

3) if your filesystem is unbalanced due to tons of file creates and possible add/remove of devices you can rebalance it online :

# btrfs filesystem balance /btrfs

4) use cp to clone files on btrfs with COW (so individual file clones not just volumes) :

# cp --reflink foo1 foo4

5) deferagment filesystem :

btrfs filesystem defragment /btrfs

There you go. A quick 5 minute overview of some of the nifty stuff this FS can do and you have full access to. A lot more is coming and I will make sure to showcase new features as we make them available. Use this to have a backup root filesystem for recovery purposes, to do updates of rpms and the ability to fall back to a good previous known state. Use it for virtual machine files and the power of reflink. So many possibilities and virtually no filesystem or volume limits.
happy btrfs'ing :)

Our good friend Saar has created and uploaded a new set of the super cool Oracle VM RAC templates.

The new version is available through My Oracle Support(MOS) and the patchset number is 12694014. Customers with MOS access can go download them and deploy the latest and greatest version of Oracle 11gR2 11.2.0.3 RAC.

As always these templates are highly automated, super easy ways to install large Oracle RAC clusters with little to no effort. They configure the database instances, ASM/storage, clusterware/grid infrastructure and everything. Soup to nuts so to speak. They're the perfect example of how full stack virtualization and product integration can save tremendous amounts of time (and money). In the best case, just dd a simple config file to a device and you don't even have to log into any of the VMs at all! All software available, download, install, test, production. No trial versions, no downloads from different vendors, all nicely combined.