[I've made a few small updates correcting typos and inserted additional information to turn off any existing shareiscsi=on zfs volumes]
Opensolaris has had the capability to share an iscsi target for sometime. This was done in the userland via the iscsi target daemon.
The COMSTAR project on opensolaris was started to build a fast scsi target on the platform. From the project page:
COMSTAR is a software framework that enables you to turn any OpenSolaris host into a SCSI target that can be accessed over the network by initiator hosts. COMSTAR breaks down the huge task of handling a SCSI target subsystem into independent functional modules. These modules are then glued together by the SCSI Target Mode Framework (STMF).
With intergration of COMSTAR to ZFS, Nexenta Core Platform provides for very easy setup as an iSCSI target. These latest changes to ZFS are available both in the latest beta2 release of NCP, as well as the commercial NexentaStor v1.1.5.
How to setup a NCP2 box as an iSCSI target
First, shutdown older iSCSI target service, and start the new one
svcadm disable -s iscsitgt
svccfg delete -f iscsitgt
Also if you have any zfs volumes with shareiscsi=on, set them to shareiscsi=off. This is required to setup the new zfs integrated scsi correctly.
Enable iscsi/target and it's dependency
svcadm enable -s stmf
svcadm enable -s iscsi/target
Install the switch by creating the file:
Now create a dataset (volume) which is to be shared via iSCSI
zfs create -V 500m tank/zvol1
Using -V creates a volume type dataset, and it reserves the given amount of space on the pool for it's data. In our case we reserved 500 Mb of space for our volume zol1. To create an SCSI target using this, simply run.
zfs set shareiscsi=on tank/zvol1
And we're now set :). To confirm that the target has been configured, check that output of
itadm list-target -v
This implementation of an iSCSI target provides 2-3 times the performance of the older userland implementation in opensolaris. The numbers stack up as follows:
Parameters: 4 workers x 8 outstanding IOs
COMSTAR iSCSI target
50000 IOPS, 512B, 97% CPU, 24MB/sec <- Reads
40000 IOPS, 512B, 92% CPU, 17MB/sec <-Writes
OLD UserLand iSCSI target
17000 IOPS, 512B, 98% CPU, 8MB/sec <- Reads
15000 IOPS, 512B, 98% CPU, 7.5MB/sec <-Writes
The tests were conducted with IOMeter, using a 100M zvol with zil disabled. Also these were done over a 1G network between the target and the initiator.