Anil Gulecha's Musings

I putback new packages into the NCP2 repository today, and you can now have SMF support for your favorite server packages in the repository. Before I list them, a little bit on SMF.

Service Management Framework (SMF)

SMF is the opensolaris replacement for the legacy services start unix framework (what /etc/init.d did until now, and continues to on Linux).

Services in Nexenta are handled using solaris SMF. This makes it very easy to start and stop services; you dont have to worry about it’s dependency on other services, which is taken care of by SMF automatically. This cheetsheet lists how easy it is to use SMF.

SMFed Packages

• apache2
• mysql 5
• postgresql-8.3
• exim4
• rsync

All of these are ports of their Ubuntu 8.04 counterparts.

If you want toadd SMF support to a Nexenta package, take a look at my guide on adding SMF support.

If you want to follow the latest in the nexenta codebase, bookmark this RSS feed. This is also visible on the planet homepage, if you frequent there. If you run into any issues, ping us on the -devel mailing list or on #nexenta@freenode

[Comments by clicking here]

It was nice to wake up to a Nexenta story on slashdot. Find the discussion here.

There were some things I wanted to address some comments that were brought up.

Dont call it a Ubuntu derivative; say Debian.

The Nexenta project is an derivative of Ubuntu much the same way as Edubuntu is a derivative.. It uses the same repository and packages (and version numbers), and as such the slogan “Ubuntu userland + Opensolaris kernel”.

That does not mean Debian isnt in the picture.. Nexenta is thankful to the project for the amazing work it has done, and the great packaging ecosystem it has created. But Nexenta as such is a direct port of packages in a particular Ubuntu Repository (NCP2 is based on Ubuntu 8.04).

“Proprietary” comments..

Nexenta is fully open source. Enough said.

GUI and the desktop

Refer to my earlier post.

Looks interesting..

Great! Give it a shot on your fileserver.

For more chatting on all things Nexenta join in at #nexenta on freenode or use the web based chat.

—-

And now for the “other updates”

FOSS.in

I’ll be at the FOSS.IN conference, which is the largest FOSS conference in India. It draws people from all around the country, and few from past the borders. This year’s conference lies from the 25th to the 29th of this month.

I’m looking forward to interacting with folks, and seeking folks interested in contributing to the project. My plans include conducting hands on sessions (BoFs) for those new to the project, as well as those who understand Debian’s packaging system. I’ll be there with other members from BOSUG, and will either at the FOSS stalls, or near the Hackcenter area.

If you’re nearby, drop in at the conference, and if you are attending, stop by and say Hi. Looking forward to meeting enthusiastic members of FOSS India.

Services in Nexenta are handled using solaris SMF. This makes it very easy to start and stop services; you dont have to worry about it’s dependency on other services, which is taken care of by SMF automatically. This cheetsheet lists how easy it is to use SMF.

dh_installsmf

dh_installsmf is one of Nexenta’s addition to Debian’s debhelper scripts. This makes it trivial to add SMF support in a debian package that installs a service at /etc/init.d/<service>

We’ll take a look at how we can add support to such a package.

XML Manifest file

The SMF framework requires a manifest file, which is an XML file, that provides information regarding the service. This information includes the services that this particular service depends on (or configurations files, etc).

It also lists the commands that need to be run to enable, disable the service, or to set various other parameters. The format of the manifest file is explained here.

The Blastwave package has a good collection of manifest files for various packages. Lets take a look at the manifest for apache2:

<?xml version=”1.0″?>
<!DOCTYPE service_bundle SYSTEM “/usr/share/lib/xml/dtd/service_bundle.dtd.1″>

<service_bundle type=”manifest” name=”apache2″>
<service name=”network/apache2″ type=”service” version=”4″>
<create_default_instance enabled=”false”/>
<single_instance/>

<!– First of all, if the config file is not present,
then we needn’t bother with anything else.  –>
<dependency name=”config-file” grouping=”require_all” restart_on=”none” type=”path”>
<service_fmri value=”file:///etc/apache2/apache2.conf”/>
</dependency>

<!– If there’s no network, then there’s no point in running –>
<dependency name=”loopback” grouping=”require_all” restart_on=”error” type=”service”>
<service_fmri value=”svc:/network/loopback:default”/>
</dependency>
<dependency name=”physical” grouping=”require_all” restart_on=”error” type=”service”>
<service_fmri value=”svc:/network/physical:default”/>
</dependency>
<dependency name=”fs-local” grouping=”require_all” restart_on=”none” type=”service”>
<service_fmri value=”svc:/system/filesystem/local”/>
</dependency>
<exec_method type=”method” name=”start” exec=”/lib/svc/method/apache2 start” timeout_seconds=”60″/>
<exec_method type=”method” name=”stop” exec=”/lib/svc/method/apache2 stop” timeout_seconds=”60″/>
<exec_method type=”method” name=”refresh” exec=”/lib/svc/method/apache2 refresh” timeout_seconds=”60″/>
<stability value=”Unstable”/>
<template>
<common_name>
<loctext xml:lang=”C”>Apache2 web server</loctext>
</common_name>
<documentation>
<manpage title=”apache2″ section=”7″/>
<doc_link name=”apache.org” uri=”http://httpd.apache.org/docs/2.0″/>
</documentation>
</template>
</service>
</service_bundle>

In the above file, we see that apache2 has defined the following dependencies:

• The file /etc/apache2/apache2.conf
• The network service (and the physical interface being up)
• The local filesystem

It also defines three methods start, stop and refresh, which are commands for this particular service (start the webserver, stop the webserver, reload the webserver). They call a script /lib/svc/method/apache2, which brings us to dh_installsmf

debian/rules

The service script /lib/svc/method/&lt;package&gt; is generated by dh-installsmf, so when you create an SMF manifest file for Nexenta, make sure you call that file. dh_installsmf will create a script /lib/svc/method/&lt;package&gt; which is a wrapper over the /etc/init.d/&lt;package&gt; file.

So in our case the debian/rules file in the apache2 package will be modified as

[snip]

rm -f debian/apache2-utils/usr/share/doc/apache2-utils/NEWS.Debian
dh_installinit -a –no-start -r –name=apache2 — defaults 91 09
dh_installsmf –service apache2 –manifest debian/apache2.xml -papache2.2-common
dh_installcron -a -r –name=apache2
if [ "$(LSB_RELEASE)" = "Ubuntu" ]; then \
dh_strip -a; \
else \
dh_strip -a –dbg-package=apache2-dbg -Napache2-mpm-worker -Napache2-mpm-event -Napache2-mpm-prefork -Napache2-dbg; \
fi
dh_link -a
dh_compress -a
[snip]

The –service argument gives the name of the service, the –manifest points to the manifest file and -p tells it which package the smf changes go into. This is basically the package that provides the /etc/init.d/&lt;package&gt; file.

And that is how we generate SMFed debian packages.

Had a wonderful short trip in Toulouse. It was a short trip and great to see the enthusiasm in the opensolaris community.

I’ll probably do a longer post a little later, for now heres a link to the presentation I made for you to peruse.

Goodbye, Toulouse!

I’ve been asked this question multiple times by mail and other forums, and it also pops up quite frequently on IRC. Does Nexenta have a GUI?

The Nexenta desktop

Short answer

Yes, but not by default.

Long answer

Nexenta started off as a clone of ubuntu with a Solaris kernel. These versions were quite popular and got a whole lot of users. However, maintaining and supporting a large selection of packages that go into a desktop was impossible for the developers.

This was when the focus of Nexenta changed from a desktop to a core platform. The idea is to build a solid core, and allow developers to build on top of this. Thus the Nexenta Core platform was born, also sortly and lovingly called NCP. NCP provides all the packages from Ubuntu’s main repository. The NCP cd builds do not include any WM.

However –theres always a however — older repositories do contain Gnome and KDE and XFCE packages. So to get a GUI environment, you’ll have to manually install those packages. This repositoy is the unstable elatte repository. The following line in your /etc/apt/sources.list should get you the packages.

deb http://www.nexenta.org/apt-gnusolaris elatte-unstable main contrib non-free

Going ahead..

All this talk is for the NCP1 (codename elatte) release. NCP2 is currently in alpha stage. Our plan for this release is to populate the repository completely with all GUI packages, so a desktop is only an apt-get install gnome-desktop away.

We cannot include this by default is because we wont be able to maintain it ourselves. We have fuzzy plans for a GUI version in the long term, but nothing concrete. We are looking for developers and enthusiasts to build distributions on top of NCP, and have various variants. So come join the community and be the Linux Mint to our Ubuntu. The nexenta-builder package is where you start.

Join us on IRC #nexenta@irc.freenode.net. You can also take a look at my 2 part development introduction to Nexenta here and here.

To put into use my newbie python skills, I wrote a tool that searches the output of apt-cache unmet -i

When populating a newly built repository, I often run into instances where I want to find all the unbuilt packages that depend on kdelibs, or all packages that have openoffice as a dependency. In general I want to know if what packages wont build without X.

So I wrote unmetlist. This will help debian/equivalent and apt developers list out packages that aren’t built, and required in the repository. It can look for (reverse) dependencies, suggests, etc. The difference between apt-cache’s inbuilt search and this tool is that apt-cache searches for the exact package name specified, where as unmetlist searches for the pattern (so kde will match kde-code, kde-libs, etc).

root@oahu:~/src/work3# ./unmetlist
Ver 0.1
Usage
./unmetlist <option>  <package> <unmet_listing_file>
Option can be one of the following
search : search for the <package> in listing and print full listing
depends : find dependencies of <package>
rdepends : find which packages depend on <package>
suggests : find suggested packages for <package>
rsuggests : find which packages suggest <package>
recommends : find recommended packages for <package>
rrecommends : find which packages recommend <package>

To run it first generate the unmet listing using

apt-cache unmet -i > /tmp/unmetfile

and use as follows

./unmetlist rdepends openoffice /tmp/unmetfile

Its still new had edges, but it works.

Get the tool here. (right click>save as..) and be sure to chmod +x the file. Let me know if you find this useful, have suggestions, or want to request a feature.

Oh, and python is a pleasure to work with. String processing has never been easier

I wrote an article laying out Nexenta’s advantage to Debian/Ubuntu developers.

OSNews has been reporting on the Debian/Ubuntu/GNU/Opensolaris hybrid for several years. But for those of you who’ve never looked more closely at this interesting OS, a Nexenta developer has laid out some of its more noteworthy features and advantages.

Read the whole article on osnews.com.

In this part we will take a look at the steps followed to port an application from Ubuntu and build a package.

I decided to do this via a screencast. You may not be able to fastforward or rewind, so I’ve also listed out the steps you will need to follow below the screencast. Click below to open the screencast. Or download it for offline viewing here.

Getting started with Nexenta screencast

Getting Access

We have a build machine available at gnusolaris.org where you can login to do your nexenta development (and not have to hassle with installing locally). You will use ssh to login, and can get the details by asking on IRC (#nexenta@freenode), or on the development mailing list gnusol-devel@sonic.net.

Logging in

You can login using your favourite shell and terminal program. The screencast uses gnome-terminal. If you are on Windows you can use the Putty client. Use

ssh username@gnusolaris.org

and enter the password when prompted.

Entering the Devzone

One of the core features of opensolaris is Zones (zfs is currently stealing all the thunder). Since giving root access to everyone is not an option, and building without it is not one either, the Nexenta project has innovated and created the so called development zones. These provide a method to quickly create a developer environment and give you root access to it. When no longer required, this zone can be painlessly removed.

The commands sure would be complex.. right? As simple as

• devzone_create  — creates a development zone
• devzone_enter — enter a devzone
• devzone_free — remove the zone when no longer required

Simple as that.

Once you have entered a newly created zone, use the following commands to get it up to date

apt­-get update; APT_CLONE_ENV=1 apt-­get dist­-upgrade

Also, you will have to create/import your gpg key for signing packages. Further instructions at http://www.nexenta.org/os/CreatePublicGPGKey

Building a package

So now that we have our environment ready, lets build a package. In the screencast we take an example of libid3tag, which is a C library that is used to parse the headers in mp3 files.

First check if the package is available in the repository (Well libid3tag is not, but you would need to check if the package you are trying to port is already in or not)

#apt-cache search package

Searched for the expression package in the repository and lists the matches if present.

If it doesnt exist, you get it from Ubuntu’s repostiory mirror using the apt-upstream-tool

#apt-upstream-tool -e -p package

-e : extract the package

-p : the package name

Once we have the sources.. move into the directory, there would be 2 copies.. one original, and one for us to edit and use.

Move into this directory (this is referred to as the “package root”) and make sure the changelog reflects the distro as Nexenta. You can edit the changelog entry using

#debchange

in the package root.

Next satisfy all the build dependencies of the package.. this is required if the package has to built. Do this via running the following command in the package root.

#apt-satisfydepends

The above tool will lookup all dependencies and install them from the repository if available. (If not, you would also have to port that package)

Once the above completes successfully, create the package using

#dpkg-buildpackage -sa

-sa : Bundle the source code with the package

If the above completes successfully, you will be presented with a prompt to enter your GPG passphrase to sign the package.

Once this is done your package is built and present in the parent directory to the package root. Install all .deb files to verify that they install correctly. Do this using

#dpkg -i packagename.deb

You can also take a look at the contents of the deb using

#dpkg-deb -c *.deb

Once the install goes smoothly and all looks good, upload it to the repository using

#dput *.changes

This will upload the files to the repository, and send out mails to the nexenta-changes mailing list.

Further information is availabe on the Nexenta wiki documentation section and at http://www.nexenta.org/os/BuildingPackages

So I’ve been working on Nexenta for a fortnight now, and have picked up a lot of things. Below is a short introduction/guide on how you can join the project.

Introduction

First a little bit of background. The debian project was started in the early nineties to be a one-stop package repository for Linux applications. It has been a fairly popular distribution. However, it was the Ubuntu distribution that made the most use of these packages. Ubuntu is a debian derivative, with focus on the desktop. It makes use of the debian’s packages, on top of the linux kernel.

The solaris kernel is very well regarded for its rock solid stability. When solaris was open sourced as opensolaris in 2005, many possibilities emerged.. what if we could combine Solaris’s famed stability with Ubuntu’s famed user experience. And this is the aim of the Nexenta project.

So that is the backstory. Nexenta = Ubuntu – Linux kernel + Solaris kernel. Nexenta is currently focusing on the Nexenta Core Platform, which is a starting point for any type of focussed distribution; ncp will allow for easy addition of packages, and you are free to develop any distribution on top of it, by adding any packages you want. Which brings us to how you contribute to the Nexenta project: Packages.

Ubuntu/Debian currently have 20000 packages. We need help from the community to port these to Nexenta. A sizable portion has already been done. And a sizable portion remains.

Getting Started

To get started.. these are the documentation a newbie would have to to pursue:

Nexenta uses the Debian/Ubuntu packaging model. This is thoroughly explained in

• Debian policy manual
• Debian Maintainers/developers guide
• Debian developers reference
• Apt packaging system

A majority of the open source applications use the GNU autotools to compile and build programs. This autotools tutorial should help you get started.

Platform

Locally: As NCP is a command-line only environment, the ideal setup for developing would be running NCP VMWare server on Ubuntu Hardy 8.04. NCP 2.0, the upcoming release is based off this Ubuntu release. You can however install this on baremetal or on Windows/ OpenSolaris.

Remotely: Another option is to use the build machines at gnusolaris.org remotely. get onto IRC, and request for a login details. We’ll setup a development zone, and give you the login details. Further details on the hackathon page in the “How to get started” section

Machine requirements

• If you are going to work remotely on the gnusolaris.org development machine, all you need is an ssh client like putty on Windows or your garden variety ssh on your favourite *nix distro.
• If working on NCP locally, the recommended specs are:
  • Processor: Pentium 4 2.0 Ghz or equivalent
  • Memory: 1Gb minimum, 2Gb recommended.. you’ll be using a VM.
  • Space :10Gb for the virtual hard disk.

Installing Nexenta Core Platform

A step by step guide with screenshots is available at http://www.nexenta.org/os/Getting_Started

If you run into issues setting up VMware with bridged networking, you can take a look at my previous entry.

Getting help

The nexenta project has a very helpful community, that answers your queries quickly.

• Drop in on IRC at #nexenta on freenode, or use the web-based interface at nexenta.org/chat.
• Signup for the developer mailing lists
• Web-based forum

[The next part will take a look at the important commands used in porting packages]

I had a little trouble setting up VMware to share the host’s (Ubuntu) nic interface eth0 (with static ip) with the guest ncp2.

Follow the instrucions here to setup a VMware install: http://www.nexenta.org/os/Getting_Started

When you need to pass network settings, for a static IP configuration on the host, follow the below steps.

Lets say the guest is setup as

IP:192.168.1.2, with netmask 255.255.255.0 and router 192.168.1.1

Then host needs to be

IP: 192.168.1.3 (or 4 or 5..), with netmask 255.255.255.0 and router 192.168.1.1