Saturday, October 31, 2009

Started looking at nagios on cumulus.

Created a host for cumulus in /etc/nagios3/conf.d/cumulus-host.cfg

Added user erl as having all access rights in /etc/nagios3/cgi.cfg

The nagios commands seem to be in /usr/lib/nagios/plugins

To check nagios configuration:
sudo nagios3 -v /etc/nagios3/nagios.cfg

Wednesday, October 28, 2009

Installed nagios (system watchdog) on cumulus with:
apt-get install nagios3
There are instructions here:

Need to setup https, auth against my user database, integrate into

Tuesday, October 27, 2009

Turned off ufw logging on cumulus:
ufw logging off
Because too much junk was ending up in the logs about blocket packets.

Sunday, October 25, 2009

Worked on e-mail web interface at

Now looking at making exim get its info from mysql.

Needed to install an exim version with mysql driver:
apt-get install exim4-daemon-heavy
Did the above for cirrus as well.

This seems to be the way to test exim:
sudo exim4 -C /etc/exim4/exim4.conf.template-db -bv -d
Actually, the above didn't quite work, I think some other config file must also be included.

Had to reset replication for common database. Did this:

flush tables with read lock;
mysqldump --all-databases --master-data --port=3306 --ssl --ssl-ca=mysql-ca-cert.pem -p >dbdump.db
unlock tables;

mysql1 -p
Wrote a backup script for the common mysql database, script & backups in /var/backup/mysql, hourly crontab action installed.

Thursday, October 22, 2009

Trying to fix internal DNS for Mårten, and help Magnus set up his domain.

Added Magnus as user to the common and Magnus' SQL databases, requiring SSL for non-local accesses.

Set up interfaces on cirrus like so (in /etc/network/interfaces):

# The primary network interface
auto eth0 eth0:1

iface eth0 inet static

#iface eth0 inet dhcp

# Have a second IP address on eth0, to give the intraweb DNS a separate
# IP from the internet DNS server.
iface eth0:1 inet static
After ifup eth0:1, I can ping the intranetaddress, but it does not show up in the routing table.

Created /etc/mydns-lan.conf. Updated both conf files to listen on different addresses.

Should set up cumulus to be a replication slave for cirrus database on port 3308.

Asked Mårten to forward ports 3306-3309 to cirrus.

Setting up my phpMyAdmin to access Mårten's master database.

Wednesday, October 21, 2009

Started working on an email system with the virtual addresses in a MySQL database.

Also, opened UDP ports 6112 and 30260 to be able to play Company of Heroes:

root@cumulus:/etc/exim4/virtualhosts# ufw allow proto udp from any to any port 6112
Rule added
root@cumulus:/etc/exim4/virtualhosts# ufw allow proto udp from any to any port 30260
Rule added
Preparing to plug in cumulus as router.

Updating MyDNS internet tables to change most references to sol to cumulus etc, activated all zones.

Enabled IP Masquerading (NAT) by instructions on:

Needed to add PIDFILE stuff to mydns in order for the daemon to start two versions of itself.

The new server is now router, and I think it works.


Tuesday, October 20, 2009

Installed smartmontools.
Printer works now after some fippling.

I should fix access control.
Started looking at the print server.

Modified /etc/cups/cupsd.conf to listen on intranet address.

Interrupted for dinner.
Started looking at dhcpd.

sudo apt-get install dhcp3-server
got error:
Setting up dhcp3-server (3.1.1-5ubuntu8.1) ...
Generating /etc/default/dhcp3-server...
* Reloading AppArmor profiles ... [ OK ]
* Starting DHCP server dhcpd3 * check syslog for diagnostics.
invoke-rc.d: initscript dhcp3-server, action "start" failed.
I think the only problem is that I have not created configuration files yet for it.

Added eth0 interface in /etc/default/dhcp3-server
Copied configuration file from sol to /etc/dhcpd.conf
still get error starting dhcpd.
Getting this error:
Oct 20 18:09:24 cumulus dhcpd: No subnet declaration for eth0 (
Oct 20 18:09:24 cumulus dhcpd: ** Ignoring requests on eth0. If this is not wh\
Oct 20 18:09:24 cumulus dhcpd: you want, please write a subnet declaration
Oct 20 18:09:24 cumulus dhcpd: in your dhcpd.conf file for the network segme\
Oct 20 18:09:24 cumulus dhcpd: to which interface eth0 is attached. **
Which is strange, because /etc/dhcpd.conf has:
subnet netmask
Ok, figured it out, I had put the conf file in /etc/dhcpd.conf, but under Ubuntu it is supposed to be under /etc/dhcp3/dhcpd.conf

It starts now. I stopped the dhcpd on sol.
Started looking at the firewall, docs at

First, did this so enabling the firewall wouldn't cut off my ssh connection(s):
sudo ufw allow proto tcp from any to any port 22
sudo ufw enable
Then (ports 3306-3309 are the mysql database servers, the ip number is Mårten's):
sudo ufw allow http
sudo ufw allow smtp
sudo ufw allow domain
sudo ufw allow proto tcp from to any port 3306
sudo ufw allow proto tcp from to any port 3307
sudo ufw allow proto tcp from to any port 3308
sudo ufw allow proto tcp from to any port 3309
Used "netstat -lp" to see what programs were listening on what ports on the new server. Noticed that I had an e-mail program called dovecot listening on imap and pop ports, so I did:

sudo apt-get remove dovecot-imapd dovecot-pop3d dovecot-common
sudo ufw allow proto tcp from to any port 9090

Strangely, after the last rule above, ufw status says:
erl@cumulus:/etc/apache2$ sudo ufw status
Status: active

To Action From
-- ------ ----
22/tcp ALLOW Anywhere
80/tcp ALLOW Anywhere
25/tcp ALLOW Anywhere
53 ALLOW Anywhere
3306/tcp ALLOW
3307/tcp ALLOW
3308/tcp ALLOW
3309/tcp ALLOW
9090/tcp ALLOW Anywhere
As though the 'from' restriction didn't work for port 9090.

Also did:
sudo ufw allow proto tcp from to port 9092
sudo ufw allow proto tcp from to port 9000
sudo ufw allow https
sudo ufw allow from to port 3483
sudo ufw allow from to any app Samba
sudo ufw allow from any to any proto udp port afs3-callback
Port 9000 is the SlimBoxServer's web interface. I will try to set up a proxy to forward web accesses from the regular web server to the SlimServer. Couldn't get it to work, will try more later.

Won't open ipp (print server) yet, havn't set it up.

Maybe we are ready to switch cables now? Perhaps I'll update the firewall rules to go by interface later.
All websites are set up, but the https sites don't seem to be working properly. All seem to be showing

Ok, fixed it. I had to add NameVirtualHost *:443 in the SSL section of /etc/apache2/ports.conf

I've commented out the Kerberos authentication in some places - I'll have to go back and fix that, and think about how to do authentication in the future.
Continuing with websites.

I need to install the apache connector to forward servlet calls to Tomcat.

Also enabled ssl.

Did this:
sudo apt-get install libapache2-mod-jk
sudo a2enmod ssl
sudo a2enmod rewrite
sudo a2enmod dav dav_fs
Copied SSL certificates from sol to /etc/apache2/ssl.key and /etc/apache2/ssl.crt.
Copied Certificate Authority and other certs from sol's /etc/mail/certs to newly created directory /etc/certs on cumulus.

To install subversion, and the webdav connection to subversion, I did this:
sudo apt-get install subversion libapache2-svn
Copied my svn repository from sol:/usr/local/src/svn to same dir on cumulus.

Hm, svn doesn't seem to work quite yet. Or at least something is strange about the virtual web hosts...

Monday, October 19, 2009

MyDNS strangeness:

erl@cumulus:~/src/mydns-1.2.8$ dig

; <<>> DiG 9.5.1-P2 <<>>
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 15541 ;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 0 ;; WARNING: Messages has 422 extra bytes at end ;; QUESTION SECTION: ; IN A ;; ANSWER SECTION: 66720 IN CNAME 62093 IN CNAME 62093 IN A ;; Query time: 15 msec ;; SERVER: erl@cumulus:~/src/mydns-1.2.8$ dig ; <<>> DiG 9.5.1-P2 <<>>
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 3984 ;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 1, ADDITIONAL: 1 ;; QUESTION SECTION: ; IN A ;; ANSWER SECTION: 86400 IN A ;; AUTHORITY SECTION: 86400 IN NS ;; ADDITIONAL SECTION: 86400 IN A ;; Query time: 5 msec ;; SERVER:
So, the first query tells me that is CNAME which is CNAME, while the second query says that is IN A How can the same server give me two different answers for

(while rebuilding mydns with debug enabled, installed package manpages-dev)

After debug-compiling mydns, it looks like the above query results in a recursive query, which in turn asks the bredbandsbolaget DNS server.

Found the problem - the domain was not marked as 'active' in mydns' database. We should de-activate it when the new internet DNS server is online.
I'll continue debugging this tomorrow.
Web sites.

It seems like the place for web sites under Ubuntu is /var/www.
Will copy my old /srv/www/htdocs subdirectories there.

Work in progress.
Started looking at E-mail.

Installed exim4. Specified server type as 'handling incoming mail, outgoing mail by smarthost.'.

Setup virtual domains by following instructions at:

I think things work, but it's getting hard to test without having the server in its final extranet configuration.

Will later work on reading the exim virtual domains from the MySQL database.

Sunday, October 18, 2009

Played around with Squeezeboxserver Music server on the new server.

Had some problems getting Swedish radio P1 to stream properly (it's in WMA format), worked after installing AlienBBC plugin, mplayer and lame, and specifying mms:// as the URL. Not sure what did it. Also added the following lines to /etc/squeezeboxserver/convert.conf:
wma wav * *
[mplayer] -really-quiet -vc null -vo null -cache 64 -af volume=0,resample=44100:0:1,channels=2 -ao pcm:nowaveheader:file=/dev/fd/4 $FILE$ 4>&1 1>/dev/null

wma mp3 * *
[mplayer] -really-quiet -vc null -vo null -cache 64 -af volume=0,resample=44100:0:1,channels=2 -ao pcm:nowaveheader:file=/dev/fd/4 $FILE$ 4>&1 1>/dev/null | [lame] --silent -q $QUALITY$ $RESAMPLE$ -v $BITRATE$ - -
Also downloaded Squeezeslave (from This is a command line software emulation of a Slim Devices music player. I intend to use it to play music in my living room, and control it from the web interface.

sudo cp squeezeslave /usr/local/bin
Created a startup script by modifying the one for mydns.

Installed it with:
update-rc.d squeezeslave defaults
Also made symbolic links from /media/music to my music library.

Saturday, October 17, 2009

Found a bug in MyDNS which causes it to ignore the port number in the database specification.

Attached a patch to fix the bug here:

Installed a patched version on Cumulus (but not Cirrus yet).

Will have to redo the MyDNS imports to make sure everything goes to the right database.
Noted the ubuntu command update-rc.d which can be used when installing init.d scripts, like:
update-rc.d mydns defaults
Started copying files from my afs server to the regular filesystem on cumulus.

eth0 will be LAN, and eth1 Internet interfaces.

Started setting up two MyDNS servers. Made the /etc/init.d/mydns into two files /etc/init.d/mydns-lan and /etc/init.d/mydns-internet. Made them use different conf files, /etc/mydns-lan.conf and /etc/mydns-internet.conf. The conf files access the shared database (port 3306) or my database (port 3307).

Next task is to import the external DNS info into the right MyDNS database. I'll try to pull the info from Mårten's DNS server so I don't get the intranet version.

Building mydns on Cirrus:
tar -xf mydns-
cd mydns-1.2.8
sudo apt-get install g++ gcc libmysqlclient15-dev
nice make -j4
sudo make install
Then finally ran this on cumulus:
sudo mydnsimport --conf=/etc/mydns-internet.conf --replace --verbose
Seems to have worked. Do the same for the other zones, but pulled some of them from Then import the intranet zones:
sudo mydnsimport --axfr=sol --conf=/etc/mydns-lan.conf --replace --verbose
In theory, all the DNS data should be in the databases, and replicated to cirrus. Checked it, looks good.

Next step is to try to serve intranet DNS from the new server.

Having problems specifying port for mydns server. Will continue with this later.

Friday, October 16, 2009

Trying to install openafs client on cumulus to copy data from my OpenAFS servers easily.


apt-get install openafs-client
module-assistant auto-install openafs
/etc/init.d/openafs-client force-start
apt-get install heimdal-clients

Finally think I got the replication working by adding:

max_allowed_packet = 200M

In exactly the right places in /etc/mysql/my.cnf (for every mysqld, for mysql, but not in the [client] section), on both the master and the slave.

The replication is still running, but I think it will work now.
mysqldump had stopped after 43 minutes with error:
mysqldump: Error 2020: Got packet bigger than 'max_allowed_packet' bytes when dumping table `Digitalization` at row: 3656
Updated max_allowed_packet in /etc/mysql/my.cnf to 200M, restarted copying at 15:15.
Setup phpMyAdmin on Cirrus to support the multiple databases.

Copied /etc/phpmyadmin/config-db.php to config-db1.php to config-db4.php.

Added fields so they look like:
## worry about manual changes being silently discarded. *however*,
## you'll probably also want to edit the configuration file mentioned
## above too.
$dbverbose='Erls Database';
Modified central part of /etc/phpmyadmin/ to look like:
* Server(s) configuration
for( $i = 1; $i < 5; $i++ )
// The $cfg['Servers'] array starts with $cfg['Servers'][1]. Do not use $cfg['Servers'][0].
// You can disable a server config entry by setting host to ''.
// $i++;

/* Read configuration from dbconfig-common */

/* Configure according to dbconfig-common if enabled */
if (!empty($dbname)) {
/* Authentication type */
$cfg['Servers'][$i]['auth_type'] = 'cookie';
/* Server parameters */
if (empty($dbserver)) $dbserver = 'localhost';
$cfg['Servers'][$i]['host'] = $dbserver;

if (!empty($dbport)) {
$cfg['Servers'][$i]['connect_type'] = 'tcp';
$cfg['Servers'][$i]['port'] = $dbport;
if (!empty($dbsocket)) {
$cfg['Servers'][$i]['connect_type'] = 'socket';
$cfg['Servers'][$i]['socket'] = $dbsocket;
# The text name of the server to be shown in the login menu
if (!empty( $dbverbose ) )
$cfg['Servers'][$i]['verbose'] = $dbverbose;
//$cfg['Servers'][$i]['compress'] = false;
/* Select mysqli if your server has it */
$cfg['Servers'][$i]['extension'] = 'mysqli';
/* Optional: User for advanced features */
$cfg['Servers'][$i]['controluser'] = $dbuser;
$cfg['Servers'][$i]['controlpass'] = $dbpass;
/* Optional: Advanced phpMyAdmin features */
$cfg['Servers'][$i]['pmadb'] = $dbname;
$cfg['Servers'][$i]['bookmarktable'] = 'pma_bookmark';
$cfg['Servers'][$i]['relation'] = 'pma_relation';
$cfg['Servers'][$i]['table_info'] = 'pma_table_info';
$cfg['Servers'][$i]['table_coords'] = 'pma_table_coords';
$cfg['Servers'][$i]['pdf_pages'] = 'pma_pdf_pages';
$cfg['Servers'][$i]['column_info'] = 'pma_column_info';
$cfg['Servers'][$i]['history'] = 'pma_history';
$cfg['Servers'][$i]['designer_coords'] = 'pma_designer_coords';

/* Advance to next server for rest of config */
/* Now done in for loop */
// $i++;
Seems to work!
Started copying my databases to mysql2, making sure it relays to cirrus.

There was some problem at start where I had to do "STOP SLAVE" and then "START SLAVE" again on cirrus for the data to start being relayed. Strange, we'll have to keep an eye on it.

I copy data from quetzal to cumulus like this:

erl@cumulus:/etc/mysql$ mysqldump -h -P 3307 --extended-insert --databases Languages --password=*** | mysql2 --password=***

(Multiple databases can be listed after 'Languages' with spaces in between.

2.7 GB used on cumulus after the first database server has been copied.

Will now copy the bigger database with the photos.

I observed the slave being at most 1114 seconds after the master.

We need a system for removing master log files after they have been replicated to all slaves. Replication logs seem to be trimmed automatically.

I should time transferring the biggest database, which is ~25GB (My photos), just for curiosity
mysqld process on cirrus is running at 95-108 % CPU (due to Hyperthreading) and computer is reported to be ~50% idle. Doing SSL encryption of all the data is what I guess is taking up the CPU time.

Turned off the regular logging, started the biggest transfer, timing it.

Sigh. Windows update rebooted the PC I was running the stuff from, and now I have to restart. I'm glad I have this blog with the commands I was running! Started at about 11:30.

Thursday, October 15, 2009

MySQL and SSL, continued

Barked up some wrong trees, but finally got it to work. I'd made some trivial errors specifying the wrong filename as --ssl-ca=... parameter to mysql, and also the permissions on the certificate file for the mysqld slave process on cirrus was wrong. But now it works for server 1!

Next step: enable slave for server 2, copy my data from quetzal to server 2 master on cumulus, make sure it ends up on cirrus.

Manually added port forwarding of ports 3307, 3308, 3309 from sol to cumulus, these settings won't survive a reboot.

Steps to enable replication:

On master:
mysql> flush tables with read lock;
Query OK, 0 rows affected (0.00 sec)

mysql> show master status;
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB |
| mysql2-bin.000005 | 98 | | |
1 row in set (0.00 sec)
mysql> flush tables with read lock;
Query OK, 0 rows affected (0.00 sec)

mysql> show master status;
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB |
| mysql2-bin.000005 | 98 | | |
1 row in set (0.00 sec)
On slave:
mysql> CHANGE MASTER TO MASTER_HOST='', MASTER_USER='replicationSlave', MASTER_PASSWORD='****', MASTER_LOG_FILE='mysql2-bin.000005', MASTER_LOG_POS=98, MASTER_SSL=1, MASTER_SSL_CA='/etc/mysql/mysql-ca-cert.pem', MASTER_PORT=3307;
Query OK, 0 rows affected (0.01 sec)
For some reason I got errors about mysql not being able to update the; I had to to 'RESET SLAVE', then CHANGE MASTER TO..., and then it would work.

Then do "unlock tables" on the master, and "start slave" on the slave.

do "SHOW SLAVE STATUS\G;" on the slave, the first line should read: "Waiting for master to send event".

Next step is to copy my date from quetzal to database 2, and make sure it is replicated to Mårten. It's tens of GB I think, so it will take a while.

Bedtime now!
SSL problem.

I now have problems with SSL and mysql.

I can connect from sol to cumulus with ssl:

erl@sol:~$ mysql -h --ssl --ssl-ca=/tmp/ca-cert.pem -P 3306 -p

But I can't do the same thing from cirrus via port 3306 forwarded from sol to cumulus:

erl@cirrus:~$ mysql --ssl --ssl-ca=ca-cert.pem -h -p -P 3306
Enter password:
ERROR 2026 (HY000): SSL connection error

Does the problem have to do with port forwarding?

From, I can:

[erl@symphony erl]$ mysql -h -p
Enter password:
ERROR 1251: Client does not support authentication protocol requested by server; consider upgrading MySQL client

Which indicates that the port forwarding works, but that symphony doesn't have an ssl enabled mysql client.

Doing the same thing on cirrus:

erl@cirrus:~$ mysql -h -p
Enter password:
ERROR 1045 (28000): Access denied for user 'erl'@'' (using password: YES)

Looks like the port forwarding works to me. How to debug the ssl?

Found this on debugging ssl connections:

So I try this:

erl@cirrus:~$ openssl s_client -ssl2 -connect
17539:error:1407F0E5:SSL routines:SSL2_WRITE:ssl handshake failure:s2_pkt.c:428:

Sigh. Stuck here.
New day - new tasks.

Now want to set up MySQL replication, at first from my server to Mårten's.
Also modifying MySQL setup to support multiple datbases.

Found this blog post which seems relevant:

Also found this page:

Followed the instructions on the second link. Got bit by one thing: apparently Ubuntu runs apparmor by default, I needed to modify /etc/apparmor.d/usr.sbin.mysqld to contain:

/var/lib/mysql?/ r,
/var/lib/mysql?/** rwk,
/var/log/mysql/ r,
/var/log/mysql/** rw,
/var/run/mysqld/mysqld?.pid w,
/var/run/mysqld/mysqld?.sock w,

I now have four mysql servers on cumulus.

Repeated instructions on cirrus. Only enabled logging on master servers.

Next step is to setup cirrus to replicate databases 1 and 2 from cumulus.

Wednesday, October 14, 2009

Continuing with new server.

Added port forwarding from old server to new server's mysql:

/sbin/iptables -t nat -A PREROUTING -p tcp -i eth-ext -d --dport 3306 -j DNAT --to
/sbin/iptables -A FORWARD -p tcp -i eth-ext --dport 3306 -j ACCEPT

Now I need to change the new server's mysql server to bind to all ip addresses. Comment out the line with bind-address in /etc/mysql/my.cnf

Followed instructions here to create SSL certificates for MySQL:

Copy the certificates mentioned in /etc/mysql/my.cnf to /etc/mysql

Uncomment the ssl lines in /etc/mysql/my.cnf

Restart mysql.

Added privileges to require SSL for external MySQL connections. For my user, added:


For the first one, in a text mysql session, do something like:

show grants for 'erl';

This will show something like:

| Grants for erl@% |

Modify the above and add REQUIRE SSL before WITH GRANT OPTION:


The MySQL ca-cert.pem file must then be copied to the client (!), in order for mysql to allow the connection. The mysql client program must be started like:

mysql --ssl --ssl-ca=ca-cert.pem -h -p

Next set is to set up replication, so that Mårten's server's main MySQL database is a slave to Erland's.

Installing MyDNS under Ubuntu 9.04

I want to try running MyDNS on my new server - a DNS server with its data in a MySQL database.

At first I started getting MyDNS, but it seems like the latest release is called MyDNS NG, at

There seems to be no package for Ubuntu 9.04.

I downloaded the source from
I needed to install gcc: apt-get install gcc
I needed to install a mysql client library: apt-get install mysql-client
Also needed: apt-get install make libmysqlclient15-dev
I unpacked the source, and could now run ./configure successfully

Basically followed instructions at:

(make, make install, create startup script, sudo update-rc.d mydns defaults)

downloaded MyDNSConfig from

sudo apt-get install php5-cli vlogger

Hm, mydnsconfig doesn't seem to work 100% (Doesn't show DNS rules), will not use it for now.
PhpMyAdmin installation on new server

I've started installing my new server 'cumulus' based on an Intel D945GSEJT motherboard with an Intel Atom N270 processor and a Western Digital Caviar Green WD15EADS 1.5TB hard drive. I'm using Ubuntu 9.04 Server.

I could easily mount the motherboard in an old PC case, the holes and spacers fit.

The IDE/PATA connector on the motherboard is a laptop standard, and will require an adaptor to connect a standard IDE connector.

Now installing phpmyadmin, did:

sudo apt-get install phpmyadmin

Include /etc/phpmyadmin/apache.conf

at the end of /etc/apache2/apache2.conf according to

Restarted apache with /etc/init.d/apache2 restart

I can now access phpMyAdmin at http://computer.../phpmyadmin