A fun ride

Nexenta project has come  a long way over the time I’ve been a part of it. From the fledging distribution, to a mainstream storage distro, it has come a long way in features and community. From six employees to a hundred. From nothing to the fastest growing storage appliance. It’s been a fun ride.

To all the folks I’ve know via Nexenta and it’s community: thanks for all the fish I hope to keep an eye out on future developments.

There were some concerns raised about the activity (or the lack of it) around the NCP and NexentaStor Community Edition. While there haven’t been many new announcements, there is a team much larger than NCP2 and 3 working in the background on NCP4. The early previews for the next community edition were handed out at SCALE. I’m hoping these announcements will be made on the sites soon.

My nexenta ID is retired, so you can reach out to me via my gmail ID (in the about me).

The way ahead

I’ve chosen to spend some time on a non-tech project I’ve had in mind for a while, about which I hope to blog in the coming weeks and months. I have allocated some time to work with startups so I’m available for consulting on interesting projects.

More to come soon.

Below are specifics about various components.

Illumos

NCP4 will start using Illumos-gate as the base. The current build is b147.

Userland

We’ve had multiple discussions on which userland to choose going ahead.. the options being Ubuntu LTS or Debian. In the initial phase, we have decided to use Debian Sid as the base. (This isn’t set in stone.. but given the effort invested in this direction by StormOS, it makes sense to utilise these packages).

Also Nexenta will no longer remain a full Debian/Ubuntu port.

The NCP project was started to provide a familiar system to a Linux user migrating to OpenSolaris, to get the advantages ZFS, Dtrace, and other technologies. This involved providing

• The apt package system. SVR4 was too different to be comfortable
• GNU commands. Sun’s command line was different, and did not support the same switches that a linux user would be familiar to.

The latter of the above is no longer true. Over the past 5 years, the OpenSolaris commandline has improved, and added support for these missing switches, and is generally the same as GNU command set.

So NCP4 will default to using Illumos commands by default. It will also default to the Illumos libraries and packages provided by Illumos. We will be closer to an upstream in active development, and optimised for solaris.

As to the userland, StormOS has made significant progress in porting many Debian sid base packages. We intend to work with these packages to get basic NCP4 going. The initial focus of NCP will be to port enough package to build a solid ZFS platform. Binaries provided by Illumos will be packages. Those packages not provided by Illumos (apt/dpkg/etc) will be ported from Debian/Ubuntu. The GNU build tools will be ported.

So, NCP4 will

• Be an Illumos based distribution
• Use Illumos command by default
• Packages and libraries provided by Illumos
• Port any other required packages from Debian.
• /usr/bin – Largely Illumos commands.

SUN_PERSONALITY

NCP has long had the SUN_PERSONALITY environment switch that allowed for forcing a Sun Solaris environment. This was via a patch to execv. This feature of NCP will be discarded. While it provided for a way to shift between commandline ecosystems earlier, it was a hackish solution, and in light of the above changes, no longer makes sense.

NCP will, as ever, be fully open to community development. Build your favourite package, and make it available via the repository. This time around we also have the powerful auto-builder.

Join in

If you have any questions, feel free to join into the discussion on the developer mailing list, and our IRC room at #nexenta on freenode. And stay tuned for more updates.

You’ll first need to install xml headers (required to build dbus), and a perl package (used in the example scripts). Do this by

apt-get install libxml2-dev libcarp-assert-perl

Next, grab the SDK tarball from here.

Extract the tarball, cd into the directory, and build the required libraries

$ cd NexentaStor-SDK-Linux/
$ ./build_libs.sh

First Run

To run the example, you’ll first need to allow the box with the SDK (say 192.168.1.10) access to the NexentaStor instance (at say 192.168.1.5). On the NexentaStor box, as root, do.

nmc@nza304:/$ setup appliance authentication
Option ?  iptable
Option ?  add
Option ?  allow
Mask or IPv4 address : 192.168.1.10
MODE       MASK/IP
allow      192.168.1.10

That’s it! Now back on the SDK box, run the test with:

$ cd examples/perl/
$ ./test.pl 192.168.1.5

More articles on NexentaStor SDK and documentation to come soon.

As seen in the release notes, we have a new pricing policy for NCP3.

Fee Structure
—–
Starting with this release of NCP3, there will be a small charge for all users of NCP. This is required for security upgrades, and usage of apt-clone functionality.

Developing and maintaining a distribution the size of NCP takes a lot of resources, and we need your help to sponsor continued development of your favourite OpenSolaris distribution.

To learn more about the upcoming pricing policy, visit,
http://www.nexenta.org/projects/site/wiki/Pricing

This change has garnered widespread reaction from the community members.. but I hope the users see why we did this.

Grab the latest iso here.

We’ve also produced a short comic, with our very own hero, Nexentaman, to walk you through NCP. Click below to read the comic.

The Adventures of Nexentaman

Links

Nexenta homepage: http://www.nexenta.org

Artwork and Wallpapers: http://www.nexenta.org/os/SpreadTheWord

IRC : #nexenta on freenode

Go ahead, test drive Nexenta, and spread the word

There were weeks of speculation of a buyout by IBM, which generated loads of discussions on the opensolaris forums. The biggest fear was IBM buying Sun, and killing off competing products. Sun and IBM competed on many fronts.. Opensolaris vs AIX, Eclipse vs Netbeans, Mysql vs DB2, Power vs Sparc, etc.. not to mention the server hardware market.

Sun’s price-tag was nothing to IBM.. it could’ve bought Sun out with a week’s revenues. The talks fail, Oracle swoops in, and grabs Sun. A /. comment sums it up..

Opensolaris has had the capability to share an iscsi target for sometime. This was done in the userland via the iscsi target daemon.

The COMSTAR project on opensolaris was started to build a fast scsi target on the platform. From the project page:

COMSTAR is a software framework that enables you to turn any OpenSolaris host into a SCSI target that can be accessed over the network by initiator hosts. COMSTAR breaks down the huge task of handling a SCSI target subsystem into independent functional modules. These modules are then glued together by the SCSI Target Mode Framework (STMF).

COMSTAR provides

• Extensive LUN Masking and mapping functions
• Multipathing across different transport protocols
• Multiple parallel transfers per SCSI command
• Scalable design
• Compatible with generic HBAs

With intergration of COMSTAR to ZFS, Nexenta Core Platform provides for very easy setup as an iSCSI target. These latest changes to ZFS are available both in the latest beta2 release of NCP, as well as the commercial NexentaStor v1.1.5.

How to setup a NCP2 box as an iSCSI target

First, shutdown older iSCSI target service, and start the new one

svcadm disable -s iscsitgt
svccfg delete -f iscsitgt

Also if you have any zfs volumes with shareiscsi=on, set them to shareiscsi=off. This is required to setup the new zfs integrated scsi correctly.

Enable iscsi/target and it’s dependency

svcadm enable -s stmf
svcadm enable -s iscsi/target

Install the switch by creating the file:

touch /etc/shareiscsi.conf

Now create a dataset (volume) which is to be shared via iSCSI

zfs create -V 500m tank/zvol1

Using -V creates a volume type dataset, and it reserves the given amount of space on the pool for it’s data. In our case we reserved 500 Mb of space for our volume zol1. To create an SCSI target using this, simply run.

zfs set shareiscsi=on tank/zvol1

And we’re now set . To confirm that the target has been configured, check that output of

itadm list-target -v

Performance gain

This implementation of an iSCSI target provides 2-3 times the performance of the older userland implementation in opensolaris. The numbers stack up as follows:

Parameters:  4 workers x 8 outstanding IOs

COMSTAR iSCSI target

50000 IOPS, 512B, 97% CPU, 24MB/sec  <- Reads
40000 IOPS, 512B, 92% CPU, 17MB/sec  <-Writes

OLD UserLand iSCSI target

17000 IOPS, 512B, 98% CPU, 8MB/sec  <- Reads
15000 IOPS, 512B, 98% CPU, 7.5MB/sec  <-Writes

The tests were conducted with IOMeter, using a 100M zvol with zil disabled. Also these were done over a 1G network between the target and the initiator.

More information on COMSTAR can be found on it’s project page, and the changes can be found in the nexenta-on package in NCP2.

“NCP2 will be released in 1 month so file your bugs, make your additions and voice your opinion now or forever hold your peace!”

The blueprint you need to keep track of NCP2 release is up at launchpad titled “hardy release”.

So there. You have been informed.. if there was a particular bug, or change for NCP you had in mind, let us know! We’re so interested, we’ve created the following ways for you to let us know:

IRC : #nexenta on freenode

Mailing list : gnusol-devel

Launchpad bugs : File a bug..

There are a bunch of things to be done, so your help with development and polishing of NCP2 is welcome and appriciated.

Debtorrent on Nexenta

Find your packages downloading slow on Nexenta? Debtorrent is a technology that brings bittorrent to apt packaging system. We’ve added support for debtorrent on nexenta.org, so you can now seed packages you download for others.. this makes download faster for users living near you.

Setting up

To setup debtorrent, simply run

apt-get install debtorrent

Now tha you have debtorrent running (by default on port 9988, add the following lines to /etc/apt/sources.lst

deb debtorrent://localhost:9988/apt.nexenta.org hardy-unstable main contrib non-free
deb-src debtorrent://localhost:9988/apt.nexenta.org hardy-unstable main contrib non-free

Debtorrent is intelligent enough to fallback to normal http mode if debtorrent doesnt work (or there is no one seeding the required package)

Advanced configuration

Debtorrent works great in a LAN environment. So one PC would use the above configuration, and the rest would connect to this single one.

Instructions

Full instructions for setting up can be found on our debtorrent page.

To learn more about debtorrent, visit its homepage.

XFCE on NCP2, Click to see full size

To get this desktop running on your Nexenta install, run

apt-get install xorg
#configure X (or copy a working xorg.conf file over)
Xorg -configure
apt-get install xfce4

Once done, you can start the desktop runnign startx. To install a login manager, run

apt-get install gdm

Come over to #nexenta, and give your thanks to Jason (wavejumper) who has been working on XFCE packages.