WebDAV between Linux (Centos 5.5) and Windows 7

WebDAV (Web-based Distributed Authoring and Versioning) is, as Wikipedia says, “a set of methods based on the Hypertext Transfer Protocol (HTTP) that facilitates collaboration between users in editing and managing documents and files stored on World Wide Web servers”.  Popularly speaking, it makes it possible to access the files on your webserver as if they were on a local disk of your PC. Much more practical than FTP-ing files back and forth… It uses HTTP as transport protocol, so no need to open extra ports in your firewalls. No extra software needed on your web server if you run Apache: it’s included. You also don’t need any extra software or drivers on your Windows PC: it’s also already included.

Getting it to work was not really difficult, but it involved several steps that I didn’t find mentioned in a single place, so here I will summarize what I did. My configuration is:

  • Windows 7 Home Premium, 64 bit (behind ADSL router with NAT)
  • Centos 5.5 with direct internet connection + Apache 2.2.3 , configured for several virtual (name based) hosts

The short version for those who only need a few words:

  • Enable WebDAV for a directory on the web server, set desired permissions for apache group
  • Select “digest” authentication (not “basic”, as Windows doesn’t support that)
  • Create password file with “htdigest” (not “htpasswd!)
  • On Windows, disable “automatically detect settings” for Internet Explorer (essential for good performance)
  • Mount WebDAV folder on Windows
OK, now a bit more details:
My apache config file is located in /etc/httpd/conf/httpd.conf. I modified one of my virtual host sections into:
<VirtualHost *:80>
    ServerAdmin root@localhost
    DocumentRoot /var/www/html/yabu
    ServerName yabu.example.com

    Alias /webdav /var/www/html/yabu
    <Location /webdav>
        DAV on
        AuthType Digest
        AuthName "webdav"
        AuthDigestDomain /webdav http://yabu.example.com/webdav
        AuthUserFile /var/www/html/yabu/passwd.dav
        Require valid-user
    </Location>

</VirtualHost>

As you can see, I selected the “Digest” authentication method, not the “Basic” method that is used on most web pages I found. The reason is that Windows does not support the “Basic” method. Microsoft has a Knowledge Base article that tells you how to change that, but I didn’t try that myself.

I then created a password file with the following command:

htdigest -c /var/www/html/yabu/passwd.dav webdav yabu
chown root:apache /var/www/html/yabu/passwd.dav
chmod 640 /var/www/html/yabu/passwd.dav

And then restarted Apache by:

service httpd restart

If you wish, you can check the correct operation with the Linux tool “cadaver”. If you don’t have it:

yum install cadaver

Then try to connect:

cadaver http://yabu.example.com/webdav/

On Windows, it’s essential that you disable the “automatically detect settings” for Internet Explorer. If you don’t do that, WebDAV may be very slow. Open Internet Explorer (I use IE 8). Go to Tools, Internet Options, Connections, LAN settings and disable “Automatically detect settings”.

Now you’re ready to mount your WebDAV filder on your PC. Open Windows Explorer, right click on the “Computer” icon and select “Map network drive…”. Select a drive letter and in the “Folder” field, type the URL of you WebDAV folder. For instance:

http://yabu.example.com/webdav/

Enter the username and password as you have configured on your webserver (with the “htdigest” command) and after “OK”, you should see your new network folder…

For other versions of Windows, check out this link

Random photo on opensim login screen

I use the Diva D2 distribution of OpenSimulator, which comes bundled with the mini webserver “Wifi”. I use this for the login “splash” screen of my virtual world. In the directory “wifi/images”, I have a number of photos/snapshots, called “login-1.jpg”, “login-2.jpg”, “login-3.jpg” etc. I use a javascript random generator to select a random image from this set. Because I’m lazy, I hardcoded the maximum number in the javascript.

This is my (simplified) wifi/welcome.html:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<title><!-- #get var=GridName --></title>
<link rel="stylesheet" media="screen" type="text/css" href="/wifi/style.css" />
<meta http-equiv="Content-type" content="text/html;charset=UTF-8" />

<script type="text/javascript" language="JavaScript">
      NumberOfImagesToRotate = 12;
      FirstPart = '<img id="mainImage" src="/wifi/images/login-';
      LastPart = '.jpg"></p>';

      function printImage() {
          var r = Math.ceil(Math.random() * NumberOfImagesToRotate);
          document.write(FirstPart + r + LastPart);
      }
</script>
</head>
<body>

<script type="text/javascript" language="JavaScript">
   printImage();
</script>

<div id="container2" align="left" style="
        position:absolute;
        top:50px;
        right:50px;
        width:350px;
        font-family:Arial;
        font-size:10pt;
        color: #ffffff;
        link-color: #ffffff;
        background: #000000;
        opacity:0.7;
        ">
<p style="margin-left: 5px; margin-right: 5px">
        Users in World: <!-- #get var=UsersInworld --><br>
        Regions: <!-- #get var=RegionsTotal --><br>
        Total Users: <!-- #get var=UsersTotal --><br>
        Active Users (Last <!-- #get var=UsersActivePeriod --> Days): <!-- #get var=UsersActive --><br>
</div>
</body>
</html>

In addition, I added the following block to wifi/style.css:

img#mainImage {
  position: fixed;
  width: 100%;
  padding-left: 1%;
  left: 0px;
  top: 0px;
  height: 100%;
}

Remote access to MySql database

All communication with a MySQL database server takes place over a TCP connection on port 3306. This makes it easy to use a ssh tunnel to access it from a remote PC, without the need to open the MySQL port in the server’s firewall. Fairly standard, but if you haven’t done it before, here’s the recipe:

I use the following Linux script to connect to my MySQL server:

#!/bin/sh

echo "Setting up SSH tunnel to server.example.com..."
echo "mysql client settings: localhost:3300"

ssh -l loginname -g \
        -L 3300:server.example.com:3306 \
        server.example.com

On my Windows PC, I use a MySQL client (I use HeidiSQL) and configure the server host to be the local Linux machine that is running above script and port number as 3300.

Of course it is also possible to make the SSH connection on the Windows PC itself by means op putty.

flotsam groups

I recently installed code to implement group functionality. I used code from the flotsam project. I downloaded the group software from here (the file I got was called “mcortez-flotsam-963d99e.tar.gz”). This seems to work quite OK, although I didn’t test everything.

One problem I found: you can’t invite users to a group who are offline. The invitation will end up in the invitations table, but the user is not notified of this when he logs in. Haven’t found a solution yet. I checked, but the xmlrpc.php script is not even called when a user logs in.

updating email addresses in osprofiles

I use the “osprofile” software to implement persistent user profiles (later I will add a more exact reference). This works reasonably well, but it seems that the email field in the “usersettings” table is not updated properly. A made a quick fix by adding some code to my “cleanup” cron job that runs every 5 minutes. In my crontab:

*/5 * * * *  /var/www/html/cleanup.sh

And the cleanup script:

#!/bin/sh

mysql -u opensim -pmypassword opensim <<EOF
DELETE FROM Presence WHERE RegionID LIKE "00000000%";
EOF

mysql -u opensim -pmypassword <<EOF
UPDATE osprofile.usersettings,opensim.UserAccounts
SET osprofile.usersettings.email=opensim.UserAccounts.Email
WHERE osprofile.usersettings.useruuid=opensim.UserAccounts.PrincipalID;
EOF

Also included in the cleanup script is some command to remove not-logged-in users from the Presence table. Not sure if that is still needed.

Offline instant messages and email

I implemented a simple script for storing instant messages when the addressed user is offline. I believe I pulled the code from the wiredux project, but I’m not 100% sure, I will check that soon to give proper credits. Later, I made my own extension in order to forward the messages to the user’s email address (if enabled in his/her profile settings).

Sorry, the code looks horrible and I used some existing PHP module for accessing the opensim database, just because I was lazy 🙂 But it works ok so far.

In the root directory of my Apache web server, I made the directory “im”. There I placed the following script, named “offline.php”

<?php

/**************************************************************
 * Configuration settings
 **************************************************************/

define("HOST","localhost");
define("OPENSIM","opensim");
define("OSPROFILE","osprofile");
define("USER","opensim");
define("PASSWORD","mypassword");
define("OFFLINE_IM", "offline_msgs");
define("USERSETTINGS", "usersettings");
define("USERACCOUNTS", "UserAccounts");

include("mysql.php");

$DbLink = new DB(HOST,OPENSIM,USER,PASSWORD);

$method = $_SERVER["PATH_INFO"];

/**************************************************************
 * save message
 **************************************************************/

if ($method == "/SaveMessage/")
{
	$msg = $HTTP_RAW_POST_DATA;
	$start = strpos($msg, "?>");
	if ($start < 0) {
		echo "<?xml version=\"1.0\" encoding=\"utf-8\"?><boolean>false</boolean>";
		return;
	}
	$start += 2;
	$fullmsg = substr($msg, $start);

	try {
		$xml = new SimpleXMLElement($msg);

		$message = $xml->message;
		$toAgentID = $xml->toAgentID;
		$fromAgentName = $xml->fromAgentName;
		$fromAgentID = $xml->fromAgentID;
		$time = date('H:i',$xml->timestamp+0);

        	$DbLink->query("insert into ".OFFLINE_IM." (uuid, message) values ('" .
			mysql_escape_string($toAgentID) . "', '" .
			mysql_escape_string($fullmsg) . "')");

		send_email($toAgentID,$fromAgentName,$time,$message);

		echo "<?xml version=\"1.0\" encoding=\"utf-8\"?><boolean>true</boolean>";
	}
	catch(Exception $e) {
		echo "<?xml version=\"1.0\" encoding=\"utf-8\"?><boolean>false</boolean>";
	}

	$DbLink->close();

	exit;
}

/**************************************************************
 * retrieve messages
 **************************************************************/

if ($method == "/RetrieveMessages/")
{
	$parms = $HTTP_RAW_POST_DATA;
	$parts = split("[<>]", $parms);
	$agent_id = $parts[6];

	$DbLink->query("select message from ".OFFLINE_IM." where uuid='" .
            mysql_escape_string($agent_id) . "'");

	echo "<?xml version=\"1.0\" encoding=\"utf-8\"?><ArrayOfGridInstantMessage xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" xmlns:xsd=\"http://www.w3.org/2001/XMLSchema\">";

	while(list($message) = $DbLink->next_record()) {
	        echo $message;
	}

	echo "</ArrayOfGridInstantMessage>";

	$DbLink->query("delete from ".OFFLINE_IM." where uuid='" .
            mysql_escape_string($agent_id) . "'");
	exit;
}

/**************************************************************
 * email forwarding
 **************************************************************/

function send_email($toID,$fromName,$time,$msg)
{
	$Db = new DB(HOST,OSPROFILE,USER,PASSWORD);

	$Db->query("select imviaemail from " . USERSETTINGS .
	        " where useruuid='" .  mysql_escape_string($toID) . "'");

	$nrows = $Db->num_rows();
	if( $nrows < 1 ) { $Db->close(); return; }
	$row = $Db->next_record();
	$imviaemail = $row['imviaemail'];
	if( !$imviaemail ) { $Db->close(); return; }
	$email = get_email($toID);
	if( strpos($email,'@') === false ) { $Db->close(); return; }

	$body = "You received the following message on My Opensim Server:\n\n" .
        "[" . $time . "] " . $fromName . ": ". $msg . "\n\n" .
        "*** do not reply to this message by email! ***\n";

	$subject = "[My Opensim Server] Message from " . $fromName;
	$headers =
	    'From: ' . $fromName . " <no-reply@server.example.com>\r\n" .
	    'Reply-To: no-reply@server.example.com' . "\r\n" .
	    'X-Mailer: PHP/' . phpversion();

	mail($email, $subject, $body, $headers);

	$Db->close();
}

/**************************************************************
 * get email address
 **************************************************************/

function get_email($uuid)
{
	$Db = new DB(HOST,OPENSIM,USER,PASSWORD);

	$Db->query("select Email from " . USERACCOUNTS .
	        " where PrincipalID='" .  mysql_escape_string($uuid) . "'");

	$nrows = $Db->num_rows();
	if( $nrows < 1 ) { $Db->close(); return(""); }
	$row = $Db->next_record();
	$email = $row['Email'];
	$Db->close();
	return($email);
}

?>

Warning: the parsing of the XML message from opensimulator is very primitive and there are some hard-coded constants, like

$agent_id = $parts[6];

Such constants might have to be changed when using a different version of opensimulator. Actually, I should change that into some proper XML parsing, but no time for that yet and for the moment it works…

This script uses an extra table in the opensimulator database. I created this table as follows in the mysql command:

CREATE TABLE IF NOT EXISTS `offline_msgs` (
  `uuid` varchar(36) NOT NULL,
  `message` text NOT NULL,
  KEY `uuid` (`uuid`)
) ENGINE=MyISAM

The script also uses the profile database as created by the osprofile project (I believe it was osprofile, I will check that later), to get the preference of the user to receive offline IM’s by email or not. The profile database also includes the user’s email address, but this one doesn’t seem to updated reliably, so I pull the email address from the main opensim database instead.

The PHP module mysql.php:

<?php
/*
 * Copyright (c) 2007, 2008 Contributors, http://opensimulator.org/
 * See CONTRIBUTORS for a full list of copyright holders.
 *
 * See LICENSE for the full licensing terms of this file.
 *
*/

//This looks like its lifted from http://www.weberdev.com/get_example-4372.html  I'd contact the original developer for licensing info, but his website is broken.

class DB
{
	var $Host;					// Hostname of our MySQL server
	var $Database;					// Logical database name on that server
	var $User;					// Database user
	var $Password;					// Database user's password
	var $Link_ID = 0;				// Result of mysql_connect()
	var $Query_ID = 0;				// Result of most recent mysql_query()
	var $Record	= array();			// Current mysql_fetch_array()-result
	var $Row;					// Current row number
	var $Errno = 0;					// Error state of query
	var $Error = "";

	function __construct($host,$db,$user,$passwd)
	{
		$this->Host = $host;
		$this->Database = $db;
		$this->User = $user;
		$this->Password = $passwd;
	}

	function halt($msg)
	{
		echo("</TD></TR></TABLE><B>Database error:</B> $msg<BR>\n");
		echo("<B>MySQL error</B>: $this->Errno ($this->Error)<BR>\n");
		die("Session halted.");
	}

	function connect()
	{
		if($this->Link_ID == 0)
		{
			$this->Link_ID = mysql_connect($this->Host, $this->User, $this->Password);
			if (!$this->Link_ID)
			{
				$this->halt("Link_ID == false, connect failed");
			}
			$SelectResult = mysql_select_db($this->Database, $this->Link_ID);
			if(!$SelectResult)
			{
				$this->Errno = mysql_errno($this->Link_ID);
				$this->Error = mysql_error($this->Link_ID);
				$this->halt("cannot select database <I>".$this->Database."</I>");
			}
		}
	}

 	function escape($String)
 	{
 		return mysql_escape_string($String);
 	}

	function query($Query_String)
	{
		$this->connect();
		$this->Query_ID = mysql_query($Query_String,$this->Link_ID);
		$this->Row = 0;
		$this->Errno = mysql_errno();
		$this->Error = mysql_error();
		if (!$this->Query_ID)
		{
			$this->halt("Invalid SQL: ".$Query_String);
		}
		return $this->Query_ID;
	}

	function next_record()
	{
		$this->Record = @mysql_fetch_array($this->Query_ID);
		$this->Row += 1;
		$this->Errno = mysql_errno();
		$this->Error = mysql_error();
		$stat = is_array($this->Record);
		if (!$stat)
		{
			@mysql_free_result($this->Query_ID);
			$this->Query_ID = 0;
		}
		return $this->Record;
	}

	function num_rows()
	{
		return mysql_num_rows($this->Query_ID);
	}

	function affected_rows()
	{
		return mysql_affected_rows($this->Link_ID);
	}

	function optimize($tbl_name)
	{
		$this->connect();
		$this->Query_ID = @mysql_query("OPTIMIZE TABLE $tbl_name",$this->Link_ID);
	}

	function clean_results()
	{
		if($this->Query_ID != 0) mysql_freeresult($this->Query_ID);
	}

	function close()
	{
		if($this->Link_ID != 0) mysql_close($this->Link_ID);
	}
}
?>

Finally, I have the following in my OpenSim.ini (actually, in config-include/MyWorld.ini, since I use the Diva distribution):

[Messaging]
    ; Control which region module is used for instant messaging.
    ; Default is InstantMessageModule (this is the name of the core IM module a$
    InstantMessageModule = InstantMessageModule
    ; MessageTransferModule = MessageTransferModule
    OfflineMessageModule = OfflineMessageModule
    OfflineMessageURL = http://server.example.com/im/offline.php
    MuteListModule = MuteListModule
    MuteListURL = http://server.example.com/im/mute.php

    ; Control whether group messages are forwarded to offline users.  Default i$
    ; ForwardOfflineGroupMessages = true

The “mute.php” script doesn’t exist, but this config line still seems to be neccessary.

Monitoring opensimulator health

In order to keep track of the health of the opensimulator server process, I implemented some RRD/MRTG graphs on my Apache web server (on the same machine as opensimulator).  It shows graphs for CPU usage, memory usage, Sim fps and Phy fps. I have this script running continuously and it collects the data every 5 seconds:

#!/bin/sh

HOME=/var/www/html/RRD
cd $HOME

while true; do

  if [ -f /var/run/opensim.pid ]; then
    pid=`cat /var/run/opensim.pid`

    # get status of OpenSimulator process
    status=`top -b -p$pid -n 1 | grep mono`
    if [ "x$status" != "x" ]; then
      virt=`echo $status | cut -d ' ' -f 5 | sed 's/m//'`
      cpu=`echo $status | cut -d ' ' -f 9`
      time=`date +%s`
      rrdtool update virt.rrd $time:$virt
      rrdtool update cpu.rrd $time:$cpu
    fi

    # get simulator statistics
    #  Dilatn 1.0 SimFPS 55.0 PhyFPS 46.1022186279297 Prims 2351.0
    #  AtvScr 91.0 ScrLPS 5.0 Memory 290 Uptime 00 26 06.2927600
    #  Version OpenSim 0.7.1 Post_Fixes
    stats=`curl -s -m 1 http://localhost:9000/jsonSimStats/ | sed 's/[{},":]/ /g'`
    if [ "x$stats" != "x" ]; then
      simfps=`echo $stats | cut -d ' ' -f 16`
      phyfps=`echo $stats | cut -d ' ' -f 18`
      time=`date +%s`
      rrdtool update simfps.rrd $time:$simfps
      rrdtool update phyfps.rrd $time:$phyfps
    fi

    echo $time $cpu $virt $simfps $phyfps >>LOG

  fi

  # wait
  sleep 5

done

I created the RRD archive files with the following script:

#!/bin/sh

# 1: 360 x 5 sec = 30 min
# 12: 360 x 1 min = 6 hour
# 120: 720 x 10 min = 120 hour = 5 days

rrdtool create virt.rrd         \
        --start 920804400       \
        --step 5                \
        DS:virt:GAUGE:15:U:U    \
        RRA:AVERAGE:0.5:1:360   \
        RRA:AVERAGE:0.5:12:360  \
        RRA:AVERAGE:0.5:120:720 \

rrdtool create cpu.rrd          \
        --start 920804400       \
        --step 5                \
        DS:cpu:GAUGE:15:U:U     \
        RRA:AVERAGE:0.5:1:360   \
        RRA:AVERAGE:0.5:12:360  \
        RRA:AVERAGE:0.5:120:720 \

rrdtool create simfps.rrd               \
        --start 920804400       \
        --step 5                \
        DS:simfps:GAUGE:15:U:U  \
        RRA:AVERAGE:0.5:1:360   \
        RRA:AVERAGE:0.5:12:360  \
        RRA:AVERAGE:0.5:120:720 \

rrdtool create phyfps.rrd               \
        --start 920804400       \
        --step 5                \
        DS:phyfps:GAUGE:15:U:U  \
        RRA:AVERAGE:0.5:1:360   \
        RRA:AVERAGE:0.5:12:360  \
        RRA:AVERAGE:0.5:120:720 \

I enabled the jsonSimStats in my OpenSim.ini:

[Startup]
; Simulator Stats URI
; Enable JSON simulator data by setting a URI name (case sensitive)
Stats_URI = "jsonSimStats"

On my web server, I have the following PHP file (graphs.php):

<html>
<head>
<title>Server statistics</title>
</head>
<body>
<h1>Server statistics</h1>

<?php
  system("/var/www/html/RRD/makegraphs.sh >/dev/null");
?>

<h3>30 min (5 sec samples)</h3>
<img src="cpu30min.png"><br>
<img src="virt30min.png"><br>
<img src="simfps30min.png"><br>
<img src="phyfps30min.png"><br>

<h3>6 hours (1 min samples)</h3>
<img src="cpu6hour.png"><br>
<img src="virt6hour.png"><br>
<img src="simfps6hour.png"><br>
<img src="phyfps6hour.png"><br>

<h3>5 days (10 min samples)</h3>
<img src="cpu5day.png"><br>
<img src="virt5day.png"><br>
<img src="simfps5day.png"><br>
<img src="phyfps5day.png"><br>

</body>
</html>

As you can see, the PHP file starts by dynamically generating all the graph images by calling the shell script “makegraphs.sh”. This shell script is a bit long to include here, but I can’t find a way to include it here as an attachment. So here it is…

#!/bin/sh

#============================================================

# cpu 30 min

t2=`date +%s`
t1=`expr $t2 - 1800`

rrdtool graph cpu30min.png	\
	--start $t1	\
	--end $t2	\
	--lower-limit 0	\
	--upper-limit 100 \
	--rigid			\
	--title "cpu (%)"		\
	DEF:cpu=cpu.rrd:cpu:AVERAGE	\
	LINE1:cpu#FF0000	\

# cpu 6 hour

t2=`date +%s`
t1=`expr $t2 - 21600`

rrdtool graph cpu6hour.png	\
	--start $t1	\
	--end $t2	\
	--lower-limit 0	\
	--upper-limit 100 \
	--rigid			\
	--title "cpu (%)"		\
	DEF:cpu=cpu.rrd:cpu:AVERAGE	\
	LINE1:cpu#FF0000

# cpu 5 days

t2=`date +%s`
t1=`expr $t2 - 432000`

rrdtool graph cpu5day.png	\
	--start $t1	\
	--end $t2	\
	--lower-limit 0	\
	--upper-limit 100 \
	--rigid			\
	--title "cpu (%)"		\
	DEF:cpu=cpu.rrd:cpu:AVERAGE	\
	LINE1:cpu#FF0000

#============================================================

# virt 30 min

t2=`date +%s`
t1=`expr $t2 - 1800`

rrdtool graph virt30min.png	\
	--start $t1	\
	--end $t2	\
	--lower-limit 0	\
	--upper-limit 2000 \
	--rigid \
	--title "virtual memory (Mb)"		\
	DEF:virt=virt.rrd:virt:AVERAGE	\
	LINE1:virt#0000FF

# virt 6 hour

t2=`date +%s`
t1=`expr $t2 - 21600`

rrdtool graph virt6hour.png	\
	--start $t1	\
	--end $t2	\
	--lower-limit 0	\
	--upper-limit 2000 \
	--rigid \
	--title "virtual memory (Mb)"		\
	DEF:virt=virt.rrd:virt:AVERAGE	\
	LINE1:virt#0000FF

# virt 5 days

t2=`date +%s`
t1=`expr $t2 - 432000`

rrdtool graph virt5day.png	\
	--start $t1	\
	--end $t2	\
	--lower-limit 0	\
	--upper-limit 2000 \
	--rigid \
	--title "virtual memory (Mb)"		\
	DEF:virt=virt.rrd:virt:AVERAGE	\
	LINE1:virt#0000FF

#============================================================

# simfps 30 min

t2=`date +%s`
t1=`expr $t2 - 1800`

rrdtool graph simfps30min.png	\
	--start $t1	\
	--end $t2	\
	--title "simfps"		\
	DEF:simfps=simfps.rrd:simfps:AVERAGE	\
	LINE1:simfps#0000FF

# simfps 6 hour

t2=`date +%s`
t1=`expr $t2 - 21600`

rrdtool graph simfps6hour.png	\
	--start $t1	\
	--end $t2	\
	--title "simfps"		\
	DEF:simfps=simfps.rrd:simfps:AVERAGE	\
	LINE1:simfps#0000FF

# simfps 5 days

t2=`date +%s`
t1=`expr $t2 - 432000`

rrdtool graph simfps5day.png	\
	--start $t1	\
	--end $t2	\
	--title "simfps"		\
	DEF:simfps=simfps.rrd:simfps:AVERAGE	\
	LINE1:simfps#0000FF

#============================================================

# phyfps 30 min

t2=`date +%s`
t1=`expr $t2 - 1800`

rrdtool graph phyfps30min.png	\
	--start $t1	\
	--end $t2	\
	--title "phyfps"		\
	DEF:phyfps=phyfps.rrd:phyfps:AVERAGE	\
	LINE1:phyfps#0000FF

# phyfps 6 hour

t2=`date +%s`
t1=`expr $t2 - 21600`

rrdtool graph phyfps6hour.png	\
	--start $t1	\
	--end $t2	\
	--title "phyfps"		\
	DEF:phyfps=phyfps.rrd:phyfps:AVERAGE	\
	LINE1:phyfps#0000FF

# phyfps 5 days

t2=`date +%s`
t1=`expr $t2 - 432000`

rrdtool graph phyfps5day.png	\
	--start $t1	\
	--end $t2	\
	--title "phyfps"		\
	DEF:phyfps=phyfps.rrd:phyfps:AVERAGE	\
	LINE1:phyfps#0000FF

This is how my status page looks (click to see full size):

Scripts to start and stop opensimulator

My opensimulator setup is not stable enough to let it run indefinitely. Sometimes, CPU load increases for unclear reasons, or it just crashes or hangs. For this reason, I have a cron job that stops and starts opensimulator every night:

0 3 * * * /root/opensim/stop
5 3 * * * /root/opensim/start

the shell script “start”:

#!/bin/sh

echo starting opensimulator...

cd /root/opensim/current/bin
screen -S opensim -d -m ./run264.sh

As you can see, I run opensimulator in a screen command, so that I can connect to the opensim console whenever I want with “screen -r -d opensim”. The script “run264.sh”:

#!/bin/sh
export PATH=/opt/mono-2.6.4/bin:$PATH
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/mono-2.6.4/lib:/usr/local/lib
export MONO_PATH=/opt/mono-2.6.4/lib:/usr/local/lib
export MONO_THREADS_PER_CPU=400
export MONO_NO_SMP=1
export GC_NO_EXPLICIT=1
mono ./OpenSim.exe

The stop script is a bit more complicated. First I send a warning to any logged in users. Then I try a “soft shutdown” by invoking a shutdown command to the opensimulator server process. 3 minutes later, i do a hard kill, just in case the soft shutdown didn’t work.

The script “stop”:

#!/bin/sh

PIDFILE=/var/run/opensim.pid
DIR=/root/opensim

echo stopping opensimulator...
[ -e $PIDFILE ] || exit 0
PID=`cat $PIDFILE 2>/dev/null`
$DIR/softstop &
sleep 180
kill -KILL $PID 2>/dev/null

The “softstop” script:

#!/bin/sh

SERVER=http://server.example.com:8000
PASSWD=mypassword
DIR=/root/opensim

$DIR/broadcast -s $SERVER -p $PASSWD -m "The grid will restart in 1 minute! Please leave now!" &
sleep 30
$DIR/broadcast -s $SERVER -p $PASSWD -m "The grid will restart in 30 seconds! Please leave now!" &
sleep 30
$DIR/shutdown -s $SERVER -p $PASSWD &

The “broadcast” script is a python script that I found somewhere on the internet that accesses the opensimulator server process by means of XMLRPC. I forgot where I got it, but I will try to find it soon in order to credit the original author.

#!/usr/bin/python
# -*- encoding: utf-8 -*-
import ConfigParser
import xmlrpclib
import optparse
import os.path
if __name__ == '__main__':
        parser = optparse.OptionParser()
        parser.add_option('-s', '--server', dest = 'server', help = 'URI of the region server', metavar = 'SERVER')
        parser.add_option('-p', '--password', dest = 'password', help = 'password of the region server', metavar = 'PASSWD')
        parser.add_option('-m', '--message', dest = 'message', help = 'message to broadcast', metavar = 'MSG')
        (options, args) = parser.parse_args()
        server = options.server
        password = options.password
        message = options.message
        gridServer = xmlrpclib.Server(server)
        res = gridServer.admin_broadcast({'password': password, 'message': message})
        if res['success'] == True:
                print 'message was sent to %s' % server
        else:
                print 'sending message to %s failed' % server

The “shutdown” script is a similar python script:

#!/usr/bin/python
# -*- encoding: utf-8 -*-
import ConfigParser
import xmlrpclib
import optparse
import os.path
if __name__ == '__main__':
        parser = optparse.OptionParser()
        parser.add_option('-s', '--server', dest = 'server', help = 'URI of the region server', metavar = 'SERVER')
        parser.add_option('-p', '--password', dest = 'password', help = 'password of the region server', metavar = 'PASSWD')
        (options, args) = parser.parse_args()
        server = options.server
        password = options.password
        gridServer = xmlrpclib.Server(server)
        res = gridServer.admin_shutdown({'password': password})
        if res['success'] == True:
                print 'shutdown of %s initiated' % server
        else:
                print 'shutdown of %s failed' % server

Hacking the garbage collection in mono

When running opensimulator in older mono versions (2.4.2), I experienced huge spikes of 100% CPU load. Most of the times they last for a few seconds, but sometimes up to 5 or 10 minutes, during which everything freezes on my server. Also, shutting down opensimulator took a very long time (20 minutes). Most of these problems were solved by patching the garbage collector of mono. This trick used to be on the opensimulator wiki, but seems to have disappeared. The reason seems that the garbage collection goes through each and every allocated piece of memory and this can be time consuming…

I’m not 100% sure what this patch does, but it seems that it disables garbage collection in many cases (or disables the compacting of free memory?). This is potentially dangerous as it might cause unlimited growth of memory usage. But in the case of OpenSimulator, I hardly see the amount of memory grow over time.

This patch still proves useful for running opensimulator on mono 2.6.4. In source file mono/metadata/boehm-gc,c change the following around line 115:

void
mono_gc_collect (int generation)
{
        MONO_PROBE_GC_BEGIN (generation);

into:

void
mono_gc_collect (int generation)
{
        static int no_explicite_gc = 0;
        if (no_explicite_gc==0) {
          if (getenv("GC_NO_EXPLICIT")) {
            no_explicite_gc = 1;
            return;
          }
          else {
            no_explicite_gc = 2;
          }
        }
        else if (no_explicite_gc==1) {
          g_print("\n --------GC_NO_EXPLICIT \n");
          return;
        }

        MONO_PROBE_GC_BEGIN (generation);

To enable the partial disabling of garbage collection, include an environment variable as follows:

export GC_NO_EXPLICIT=1

With this, shutting down opensimulator is *much* faster and there are significantly less 100% CPU spikes. So far I haven’t noticed any memory leaks or something: memory usage doesn’t seem to grow over time (but OK, my server is not very busy).

Another trick I used to limit the CPU usage is to use this environment setting:

export MONO_NO_SMP=1

This limits mono to a single CPU, so even when it uses 100% CPU time, I can still login into the server and kill the process. Without this, I couldn’t even login. Clearly, when using this setting you’re wasting 50% of CPU power of a dual core machine, but so far, I can handle the load with a single CPU without problems.

Mono can’t find zlib

Since OpenSimulator 0.7.1 requires at least mono 2.4.3 (I had 2.4.2), I compiled a new version of mono. Since I didn’t have much success with binary Centos rpm’s, I compiled it from the source tar ball for version 2.6.4. In order not to mess up with the already installed mono and I possibly want to experiment with different mono versions, I installed it in /opt/mono-2.6.4.

Everything went well, but when I started opensimulator, I got:

APPLICATION EXCEPTION DETECTED: System.UnhandledExceptionEventArgs

Exception: System.EntryPointNotFoundException: CreateZStream
  at (wrapper managed-to-native) System.IO.Compression.DeflateStream:CreateZStream (System.IO.Compression.CompressionMode,bool,System.IO.Compression.DeflateStream/UnmanagedReadOrWrite,intptr)
  at System.IO.Compression.DeflateStream..ctor (System.IO.Stream compressedStream, CompressionMode mode, Boolean leaveOpen, Boolean gzip) [0x00000] in <filename unknown>:0
  at (wrapper remoting-invoke-with-check) System.IO.Compression.DeflateStream:.ctor (System.IO.Stream,System.IO.Compression.CompressionMode,bool,bool)

Apparently, mono can’t find the zlib library for decompression. I could solve this bu including the following in the startup script:

export PATH=/opt/mono-2.6.4/bin:$PATH
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/mono-2.6.4/lib:/usr/local/lib
export MONO_PATH=/opt/mono-2.6.4/lib:/usr/local/lib

This worked, but I still got this error messages when I run opensimulator in the “screen” command:

screen -S opensim -d -m mono ./OpenSim.exe &

It seems that the environment variables don’t propagate into the screen environment. This was easily solved by wrapping the mono command into a shell script:

screen -S opensim -d -m ./run264.sh

with the above mentioned environment variables in the shell script “run264.sh”