Anil Gulecha's Musings

I putback new packages into the NCP2 repository today, and you can now have SMF support for your favorite server packages in the repository. Before I list them, a little bit on SMF.

Service Management Framework (SMF)

SMF is the opensolaris replacement for the legacy services start unix framework (what /etc/init.d did until now, and continues to on Linux).

Services in Nexenta are handled using solaris SMF. This makes it very easy to start and stop services; you dont have to worry about it’s dependency on other services, which is taken care of by SMF automatically. This cheetsheet lists how easy it is to use SMF.

SMFed Packages

• apache2
• mysql 5
• postgresql-8.3
• exim4
• rsync

All of these are ports of their Ubuntu 8.04 counterparts.

If you want toadd SMF support to a Nexenta package, take a look at my guide on adding SMF support.

If you want to follow the latest in the nexenta codebase, bookmark this RSS feed. This is also visible on the planet homepage, if you frequent there. If you run into any issues, ping us on the -devel mailing list or on #nexenta@freenode

[Comments by clicking here]

Recently I had been hunting around the web for a very lightweight, simple to use, flat file using, php based wiki. And I managed to find one. It’s called Pawfaliki

Pawfaliki

This wiki package is under 50 kb in size, has an easy to use wiki syntax, is CSS based, and has a very cool dark look. It matched my needs almost perfectly.

Test it out at www.pawfal.org

Fancy URLs

One minor nit I had was how the URL looked.. for example the About page would look like

http://www.pawfal.org/index.php?page=About

rather than a nice wiki like

http://www.pawfal.org/About

So like any good FOSS denizen, I went ahead and implemented small changes for this. Get the modified sources: pawfaliki052-modified.tar.gz

If you’d like to see the changes, get the diff for the original package at this bug I raised. (Note, I’ve renamed pawfaliki.php to index.php, and added a .htaccess file). The changes would go in if theres another release (the last one was a while back).

So there. Give this very small wiki package a try.

Services in Nexenta are handled using solaris SMF. This makes it very easy to start and stop services; you dont have to worry about it’s dependency on other services, which is taken care of by SMF automatically. This cheetsheet lists how easy it is to use SMF.

dh_installsmf

dh_installsmf is one of Nexenta’s addition to Debian’s debhelper scripts. This makes it trivial to add SMF support in a debian package that installs a service at /etc/init.d/<service>

We’ll take a look at how we can add support to such a package.

XML Manifest file

The SMF framework requires a manifest file, which is an XML file, that provides information regarding the service. This information includes the services that this particular service depends on (or configurations files, etc).

It also lists the commands that need to be run to enable, disable the service, or to set various other parameters. The format of the manifest file is explained here.

The Blastwave package has a good collection of manifest files for various packages. Lets take a look at the manifest for apache2:

<?xml version=”1.0″?>
<!DOCTYPE service_bundle SYSTEM “/usr/share/lib/xml/dtd/service_bundle.dtd.1″>

<service_bundle type=”manifest” name=”apache2″>
<service name=”network/apache2″ type=”service” version=”4″>
<create_default_instance enabled=”false”/>
<single_instance/>

<!– First of all, if the config file is not present,
then we needn’t bother with anything else.  –>
<dependency name=”config-file” grouping=”require_all” restart_on=”none” type=”path”>
<service_fmri value=”file:///etc/apache2/apache2.conf”/>
</dependency>

<!– If there’s no network, then there’s no point in running –>
<dependency name=”loopback” grouping=”require_all” restart_on=”error” type=”service”>
<service_fmri value=”svc:/network/loopback:default”/>
</dependency>
<dependency name=”physical” grouping=”require_all” restart_on=”error” type=”service”>
<service_fmri value=”svc:/network/physical:default”/>
</dependency>
<dependency name=”fs-local” grouping=”require_all” restart_on=”none” type=”service”>
<service_fmri value=”svc:/system/filesystem/local”/>
</dependency>
<exec_method type=”method” name=”start” exec=”/lib/svc/method/apache2 start” timeout_seconds=”60″/>
<exec_method type=”method” name=”stop” exec=”/lib/svc/method/apache2 stop” timeout_seconds=”60″/>
<exec_method type=”method” name=”refresh” exec=”/lib/svc/method/apache2 refresh” timeout_seconds=”60″/>
<stability value=”Unstable”/>
<template>
<common_name>
<loctext xml:lang=”C”>Apache2 web server</loctext>
</common_name>
<documentation>
<manpage title=”apache2″ section=”7″/>
<doc_link name=”apache.org” uri=”http://httpd.apache.org/docs/2.0″/>
</documentation>
</template>
</service>
</service_bundle>

In the above file, we see that apache2 has defined the following dependencies:

• The file /etc/apache2/apache2.conf
• The network service (and the physical interface being up)
• The local filesystem

It also defines three methods start, stop and refresh, which are commands for this particular service (start the webserver, stop the webserver, reload the webserver). They call a script /lib/svc/method/apache2, which brings us to dh_installsmf

debian/rules

The service script /lib/svc/method/&lt;package&gt; is generated by dh-installsmf, so when you create an SMF manifest file for Nexenta, make sure you call that file. dh_installsmf will create a script /lib/svc/method/&lt;package&gt; which is a wrapper over the /etc/init.d/&lt;package&gt; file.

So in our case the debian/rules file in the apache2 package will be modified as

[snip]

rm -f debian/apache2-utils/usr/share/doc/apache2-utils/NEWS.Debian
dh_installinit -a –no-start -r –name=apache2 — defaults 91 09
dh_installsmf –service apache2 –manifest debian/apache2.xml -papache2.2-common
dh_installcron -a -r –name=apache2
if [ "$(LSB_RELEASE)" = "Ubuntu" ]; then \
dh_strip -a; \
else \
dh_strip -a –dbg-package=apache2-dbg -Napache2-mpm-worker -Napache2-mpm-event -Napache2-mpm-prefork -Napache2-dbg; \
fi
dh_link -a
dh_compress -a
[snip]

The –service argument gives the name of the service, the –manifest points to the manifest file and -p tells it which package the smf changes go into. This is basically the package that provides the /etc/init.d/&lt;package&gt; file.

And that is how we generate SMFed debian packages.

To put into use my newbie python skills, I wrote a tool that searches the output of apt-cache unmet -i

When populating a newly built repository, I often run into instances where I want to find all the unbuilt packages that depend on kdelibs, or all packages that have openoffice as a dependency. In general I want to know if what packages wont build without X.

So I wrote unmetlist. This will help debian/equivalent and apt developers list out packages that aren’t built, and required in the repository. It can look for (reverse) dependencies, suggests, etc. The difference between apt-cache’s inbuilt search and this tool is that apt-cache searches for the exact package name specified, where as unmetlist searches for the pattern (so kde will match kde-code, kde-libs, etc).

root@oahu:~/src/work3# ./unmetlist
Ver 0.1
Usage
./unmetlist <option>  <package> <unmet_listing_file>
Option can be one of the following
search : search for the <package> in listing and print full listing
depends : find dependencies of <package>
rdepends : find which packages depend on <package>
suggests : find suggested packages for <package>
rsuggests : find which packages suggest <package>
recommends : find recommended packages for <package>
rrecommends : find which packages recommend <package>

To run it first generate the unmet listing using

apt-cache unmet -i > /tmp/unmetfile

and use as follows

./unmetlist rdepends openoffice /tmp/unmetfile

Its still new had edges, but it works.

Get the tool here. (right click>save as..) and be sure to chmod +x the file. Let me know if you find this useful, have suggestions, or want to request a feature.

Oh, and python is a pleasure to work with. String processing has never been easier

I wrote an article laying out Nexenta’s advantage to Debian/Ubuntu developers.

OSNews has been reporting on the Debian/Ubuntu/GNU/Opensolaris hybrid for several years. But for those of you who’ve never looked more closely at this interesting OS, a Nexenta developer has laid out some of its more noteworthy features and advantages.

Read the whole article on osnews.com.

The comments are pure gold, in their insight and humour. Take a discussion on the oldest skeleton found on earth.

One comment starts off with a comment on the earth being 6000 years old (which is a popular loony creationist belief). Thereafter the comments are as follows

“The Earth *IS* only 6,000 years old. Give or take 4.54 billion years”

–”So…. Within that margin of error, the earth may not have been created yet?”

—-”Scary, but that DOES go along with how I feel some Monday mornings….”

—-”Don’t tell anyone, but we’re doing the public beta stress test, so the publisher can know how many players per server he can expect. There’ve been some bugs and balance problems found, though, so they might push back the actual release for another billion years. Although the publisher is calling it good enough and might shove it out the door as it is.”

——–”A billion year beta? I didn’t realize Google was the creator!”

————”Well, it is called Google Earth”

Follow this thread here.

And start reading Slashdot. IT may be a little US-centric at times, but the insight and rational debates that are a part of discussions always leave you better informed.

GNU screen is an extremely useful utility when used on remote machines. If you have a flaky net connection that disconnects often, it can be a boon.

What do mean: “tabs”?

GNU screen emulates a terminal, using your default shell. You start it by simply running

$screen

You get a welcome message and to a working shell. This is your open “tab” or screen. To start a new tab, enter the keys Ctrl-a + c (control-a followed by c). This opens a new tab with an empty shell. the first shell is “0″, second is “1″ and so on.

To move to shell X, use Ctrl-a + X.. so to go back to the first shell, i hit Ctrl-a + 0.

To scroll up in the tab, when text has run through the screen, press Ctrl+[ , and you are now in scroll mode: you can use the arrow keys, home, end, pgup and pgdown, etc to scroll on the screen. Esc closes the scroll mode.

Screens are persistent

So your network was reset and you connection to the remote server lost. Worry not, screen is still running up there. Login to your account again, and run

screen -x

This will attach to your old screen session, and your wordspace and opened tabs are all present

You may always want to start off with screen -x initially. If no session exist, this will report an error and you can start a new session using screen. If one does you will attach to it. If more than one does they will all be listed and you can attach to any one of those.

Multiplayer mode

This feature of attaching to open screen sessions isn't restricted to you alone. Another user logged into the system can also enter screen -x, and join you. You can both now collaborate on the screen! This is extrememly useful when you need the help of another person to get something done.. and you get to see live as they do their thing.

So, hope I've sold you on screen. Use it. The official page is on GNU.org, but its probably alreayd a part of your system or available in your distributions package repository.

Screenshots should be listed at google's result.

Default key bindings for various tasks are here and the manpage is here.

[update: To get a bar at the bottom of the console to list all windows + other details, take a look at the guide at redhatmagazine.com]

The Nexenta hackathon session starts in about a day. If you’ve been thinking of participating, join in at #nexenta@freenode and request your account.

More on Nexenta development in earlier posts here and here.

So, I’ve finally picked up Python (been meaning to do this for quite some time), and find it to be beautifully intuitive. The ternary operator that goes

result = x if y else z

is an example of this intuitiveness.

Expect many more Pythonic posts in the future. For now, I’m considering the problems from Project Euler

In this part we will take a look at the steps followed to port an application from Ubuntu and build a package.

I decided to do this via a screencast. You may not be able to fastforward or rewind, so I’ve also listed out the steps you will need to follow below the screencast. Click below to open the screencast. Or download it for offline viewing here.

Getting started with Nexenta screencast

Getting Access

We have a build machine available at gnusolaris.org where you can login to do your nexenta development (and not have to hassle with installing locally). You will use ssh to login, and can get the details by asking on IRC (#nexenta@freenode), or on the development mailing list gnusol-devel@sonic.net.

Logging in

You can login using your favourite shell and terminal program. The screencast uses gnome-terminal. If you are on Windows you can use the Putty client. Use

ssh username@gnusolaris.org

and enter the password when prompted.

Entering the Devzone

One of the core features of opensolaris is Zones (zfs is currently stealing all the thunder). Since giving root access to everyone is not an option, and building without it is not one either, the Nexenta project has innovated and created the so called development zones. These provide a method to quickly create a developer environment and give you root access to it. When no longer required, this zone can be painlessly removed.

The commands sure would be complex.. right? As simple as

• devzone_create  — creates a development zone
• devzone_enter — enter a devzone
• devzone_free — remove the zone when no longer required

Simple as that.

Once you have entered a newly created zone, use the following commands to get it up to date

apt­-get update; APT_CLONE_ENV=1 apt-­get dist­-upgrade

Also, you will have to create/import your gpg key for signing packages. Further instructions at http://www.nexenta.org/os/CreatePublicGPGKey

Building a package

So now that we have our environment ready, lets build a package. In the screencast we take an example of libid3tag, which is a C library that is used to parse the headers in mp3 files.

First check if the package is available in the repository (Well libid3tag is not, but you would need to check if the package you are trying to port is already in or not)

#apt-cache search package

Searched for the expression package in the repository and lists the matches if present.

If it doesnt exist, you get it from Ubuntu’s repostiory mirror using the apt-upstream-tool

#apt-upstream-tool -e -p package

-e : extract the package

-p : the package name

Once we have the sources.. move into the directory, there would be 2 copies.. one original, and one for us to edit and use.

Move into this directory (this is referred to as the “package root”) and make sure the changelog reflects the distro as Nexenta. You can edit the changelog entry using

#debchange

in the package root.

Next satisfy all the build dependencies of the package.. this is required if the package has to built. Do this via running the following command in the package root.

#apt-satisfydepends

The above tool will lookup all dependencies and install them from the repository if available. (If not, you would also have to port that package)

Once the above completes successfully, create the package using

#dpkg-buildpackage -sa

-sa : Bundle the source code with the package

If the above completes successfully, you will be presented with a prompt to enter your GPG passphrase to sign the package.

Once this is done your package is built and present in the parent directory to the package root. Install all .deb files to verify that they install correctly. Do this using

#dpkg -i packagename.deb

You can also take a look at the contents of the deb using

#dpkg-deb -c *.deb

Once the install goes smoothly and all looks good, upload it to the repository using

#dput *.changes

This will upload the files to the repository, and send out mails to the nexenta-changes mailing list.

Further information is availabe on the Nexenta wiki documentation section and at http://www.nexenta.org/os/BuildingPackages