Cluster Kurrola

Installation

Cluster Kurrola desing targets: easy to manage ( webmin and LTSP are tested and compared ) , get parallel solvers work ( bind9,dhcp,nfs4 ) giganet whit IP4, stabilize system ( Twin Machine , "4G Cantenna mobile internet Antenna,.." server to work outside world at IPV6 and inside IPV4 / IPV6 (wwdial,hostapd,radvd,shoreline,shoreline6) basic servers ( mysql, postresql, apache2, mediawiki, mythweb, calibre ) and wlan services to workstations

To get parallel software like elmerfem and openfoam to work correctly you need full stack ( name services DNS and routing ) of network service and as learnt at practice at twin machine configuration. Twin machine configuration for DHCP ( both ipv4 and ipv6 ), BIND9 ( master and slave ). BIND9 namaserice is needed to have stable routing even connection to global internet breaks. DHCP is used to networkboot and network address managment. To get IPV6 and IPV4 filtered and routed correctly Shoreline and Shoreline6 are simple tool to configure firewall to Linux kernel. You need device level softaware like wvdial to get 4G modem/router to talk whit server routing and brdegeing software as well hostapd to open wlan services. To manage configuration of servers as well workstations webmin is good tool. To get network working well whit software you need common usernamre as well password base to get communication work between machines and services like NFS4 and rsh-redone.

For post prosessing you need Paraview as well support for 3d mouse. That's possible by building Praview 4.0 from sources whit vrpn and hidapi.

Cluster Kurrola ( MPI4YOU ) base parameters:

/mpi3 is directory shared via NFS4 to all nodes
/mpi3/B3 Build Script's for version 3
/mpi3/S3 Source Code version 3
/mpi3/C3 Code Verion 3

Each node has Ubuntu 12.04 LTS ( Long Term Suport version, no need to change OS for 7 years )

Connections

Installing NetworkManager is simple way to get all right for single machine network. I got big supprise when I understood how complex it come to setup STABILE beawulf parallel network to home, you need to setup all network components.

Ubuntu 12.04 have much automation at protocol stack, it finds devices when connected ([udev]) and most cases setups hardware drivers and creates device like eth and ttyUSB. At Cluster Kurrola Installation NetworkManager is totaly removed due instability and some server software incompatibility issues (calibre,mpich3,rsh-redone). What have to do is to glue kernel IPV4 and IPV6 part's together whit devices and give correct parameter's to devices for operation.

Physical: 4G mobile network Saunanlahti & ELISA, HSPA+DC

Modem  : Huawei 398

Protocol stack:

DC-HSPA ppp

   ipv4: rounting, bridge  Ubuntu kernel
         -shorewall, vwdial, isc-dhcp., bind9

   ipv6: System Six IPV6 tunnel over IPV4 network
         -shorewall6, isc-dhcp, bind9,aiccu

Server mpi2:

ppp0: wvdialD, wvdial, aiccu br0: eth0 shoreline shoreline6 aiccu raddvd, wlan0 (hostapd)

80 / http, drupal 443 / https, drupal 81 / nominatim-osm-MTK MAPs 8082 / owncloud

/ bind9
/ dhcp
/ dhcp ipv6

3142 / apt-cacher-ng 8080 / calibre 8081 / internal drupal site 7000-7200 / mpi4you mpich3

hostapd

/etc/hostapd/hostapd.conf :

interface=wlan0
bridge=br0
driver=nl80211
macaddr_acl=1
accept_mac_file=/etc/hostapd.accept
ssid=KURROAIR
hw_mode=g
channel=3 
wpa=2
wpa_passphrase=MyEncryptionKey
wpa_key_mgmt=WPA-PSK
wpa_pairwise=TKIP CCMP
wpa_ptk_rekey=600

wvdial

/etc/wvdial.conf :

[Dialer connect]
Modem Type = Analog Modem
Modem = /dev/ttyUSB0
Phone = *99#
Init1 = ATZ
Init2 = ATQ0 V1 E1 S0=0 &C1 &D2 +FCLASS=0
Init3 = AT^SYSCFGEX="0302",3fffffff,1,4,7fffffffffffffff,,
Init4 = AT+CGDCONT=1,"ip","internet",,0,0
ISDN = off
Ask Username = off
Username = no
Ask Password = off
Password = no
Baud = 9600
Stupid mode = on
Auto DNS = on
[Dialer signal]
Modem = /dev/ttyUSB1
Init1 = AT+CSQ
Init2 = AT^SYSINFO
Init3 = AT+CIND?

/etc/init.d/wvdialD :

#!/bin/sh
NAME="wvdialD"
DESC="wvdial keep alive and startup"
test -x $DAEMON || exit 0
case "$1" in
start)
date >> /var/log/syslog
echo "[wvdialD] *** Starting Mobile Broadband Connection. wvdial up" >> /var/log/syslog
date >> /var/log/3g4gsession 
while true; do
/home/joni/webon >> /var/log/3g4gsession &
wvdial connect >> /var/log/3g4gsession
echo "RESTART" >> /var/log/3g4gsession 
date >> /var/log/3g4gsession
sleep 5 
done
date >> /var/log/syslog
/home/joni/weboff
echo "[wvdialD] Mobile Broadband Connection closed *******." >> /var/log/syslog
echo "STOP-DROP" > /var/log/3g4gsession
date >> /var/log/3g4gsession
;;
stop)
echo "[wvdialD] Stopping wvdial." >> /var/log/syslog  
killall wvdial
echo "[wvdialD] wvdial down." >> /var/log/syslog
echo "STOPED BY COMMAND" > /var/log/3g4gsession
date >> /var/log/3g4gsession
killall wvdialD
;;
status)
# Check network status with nmcli 
cat /var/log/3g4gsession
;;
*)
echo "[wvdialD] Keeps wvdial up"
echo $"Usage: $0 {start|stop|status}"
exit 1
esac
exit 0

IPV4,IPV6

/etc/network/interface :

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).
# The loopback network interface
auto lo
iface lo inet loopback
#Primary internet connection
auto ppp0
iface ppp0 inet wvdial
# The primary network interface
auto eth0
iface eth0 inet manual
manual wlan0
#address 192.168.0.40
iface wlan0 inet manual
auto br0
iface br0 inet static
network 192.168.0.0
address 192.168.0.XX
broadcast 192.168.0.255
netmask 255.255.255.0
bridge_ports eth0 wlan0
dns-nameservers 192.168.0.40 192.168.0.XX 
dns-search X.kurrola.dy.fi X.kurrola.dy.fi
dns-domain X.kurrola.dy.fi X.kurrola.dy.fi
gateway 192.168.0.254
#ipv6
iface eth0 inet6 static
address 2001:XXXX:100:363::40
netmask 64

[postfix]

https://help.ubuntu.com/community/Postfix

Orginal: http://christian.skala.me/blog/gmail-why-are-you-doing-this-to-me/#.Unt2oEOJRwY

Forcing IPv4 when sending to Gmail

If we want to force postfix to use IPv4 instead IPv6 when sending to Gmail, we need to add a line in the main configuration file of postfix like so 1 2

$ sudo nano /etc/postfix/main.cf transport_maps = hash:/etc/postfix/transport

Let’s create this new file and add a new entry in the transport table. Basically, we fore mail for gmail.com to use smtp-ipv4 (which we have to define later, see below) 1 2

$ sudo nano /etc/postfix/transport gmail.com smtp-ipv4:

Here comes the key part. We create a new rule for our new entry and define to use IPv4 protocol explicitly when this rule applies: 1 2 3

$ sudo nano /etc/postfix/master.cf smtp-ipv4 unix .. .. .. .. smtp

-o inet_protocols=ipv4

We need to run the postmap command after the change and reload postfix 1 2

$ sudo postmap /etc/postfix/transport $ sudo postfix reload

Ltps

ACPI - SUSPEND - HIBERNATE START

/etc/udev/rules.d/

Install:

apt-get install usb-modeswitch

http://www.draisberghof.de/usb_modeswitch/#install http://www.siptune.net/tiki-index.php?page=udev+rules+mokkuloille http://sysadminnotebook.blogspot.fi/2012/03/vodafone-k5005-huawei-e389-4g-modem-on.html

/lib/udev/rules.d/40-usb_modeswitch.rules :

# Huawei 398 oma
ATTRS{idVendor}=="12d1", ATTRS{idProduct}=="1506", RUN+="usb_modeswitch '%b/%k'"

/etc/usb_modeswitch.d :

# Huawei E398
TargetVendor= 0x12d1
TargetProduct= 0x1506
MessageContent="55534243123456780000000000000011062000000100000000000000000000"

rsh-redone

apt-get install rsh-redone-client rsh-redone-server

/etc/hosts.equiv

1. command access to your system .

+ + ~

/etc/inetd.conf

1. BSD: Shell, login, exec and talk are BSD protocols.

shell stream tcp nowait root /usr/sbin/tcpd /usr/sbin/in.rshd login stream tcp nowait root /usr/sbin/tcpd /usr/sbin/in.rlogind

/home/joni/.rhosts

kalle
mika
hei
paakone
192.168.0.77
192.168.0.78
192.168.0.132
192.168.0.233

/etc/pam.d/rsh

#
# The PAM configuration file for the rsh (Remote Shell) service
#
# Due to limitations in the rsh protocol, modules depending on the conversation
# function to work cannot be used.  This includes authentication modules such
# as pam_unix.so.
#auth   required        pam_rhosts.so
auth    sufficient        pam_rhosts.so
#auth   required        pam_nologin.so
#auth   required        pam_env.so
#account        required        pam_unix.so
#session        required        pam_unix.so
auth    sufficient        pam_rhosts.so

Local Time keyboard

https://help.ubuntu.com/community/Howto%3A%20Custom%20keyboard%20layout%20definitions

sudo apt-get install loacles kde-l10n-fi language-selector-gnome language-selector-common language-pack-gnome-fi-base language-pack-gnome-en-base language-pack-gnome-de-base language-pack-gnome-fr-base language-pack-fi-base language-pack-de-base language-pack-fr-base language-pack-en-base language-pack-fi language-pack-fr language-pack-de language-pack-en language-pack-gnome-de language-pack-gnome-fr language-pack-gnome-en language-pack-gnome-fi language-pack-en-base language-pack-fi-base language-pack-de-base language-pack-fr-base firefox-locale-fi thunderbird-locale-fi voikko-fi 
#timezone
dpkg-reconfigure tzdata
#local languages

Files:

/etc/default/locale:

LANG=fi_FI.UTF-8

/etc/default/keyboard 
/etc/X11/xorg.conf
udevadm trigger --subsystem-match=input --action=change

THEN:

 locale-gen purge 

Debian not unbuntu:

dpkg-reconfigure localeconf
sudo dpkg-reconfigure console-data
sudo dpkg-reconfigure console-setup

sudo dpkg-reconfigure keyboard-configuration - work's after reboot
sudo dpkg-reconfigure kdb

setxkbmap fi - change keyboard command line

Monitoring

sudo apt-get install lm-sensors gkrellmd
sudo sensors dedect
sudo service module-init-tools start
sudo nano /etc/gkrell*.conf
sudo /etc/init.d/gkrellmd start

Nvidia most recent

http://www.nvidia.com/download/driverResults.aspx/69378/en-us

wget *

update-grub /dev/sda
update-grub /dev/sdb

MediaWiki

Running MediaWiki on Ubuntu [1]
PDF support [2]
Latex support sudo apt-get install ocml texlive and changes [3]

g++-4.7

sudo add-apt-repository ppa:ubuntu-toolchain-r/test
sudo apt-get update
sudo apt-get install g++-4.7

sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-4.6 60 --slave /usr/bin/g++ g++ /usr/bin/g++-4.6
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-4.7 40 --slave /usr/bin/g++ g++ /usr/bin/g++-4.7
sudo update-alternatives --config gcc

IPV6

Sixxs How To[4]
Ipv6 How to[5]
http://www.shorewall.net/IPv6Support.html
http://tldp.org/HOWTO/Linux+IPv6-HOWTO/x1083.html
http://www.shorewall.net/6to4.htm

ip -6 ro add 2001:14b8:100:363::/64 dev eth0
ip -6 route add default via 2001:14b8:100:363::40 dev eth0

Space Navigator

See vrpn how to get it work whit Paraview.

http://docs.salome-platform.org/salome_4_1_5/visu/user/navigation_in_gauss_viewer_page.html http://code.google.com/p/liquid-galaxy/wiki/LinuxSpaceNavigator

Running Server

MPI4YOU - elmer, netgen and openfoam whit mpich3
HYDRA
GIS - Installation openstreetmap, gdal, ogr2ogr, psql2osm
Library - Installation calibre, mendelay, recol
GPG - Finish

Server start & stop

/etc/init.d/networking restart
/etc/init.d/snmpd restart
service bind9 restart
sudo service isc-dhcp-server start
/etc/init.d/apache2 restart

BACULA

http://wiki.bacula.org/doku.php?id=faq

What to backup:
/etc/bacula - you need configuration
/var/lib/bacula/bacula.sql - you need backup from database

Crete index at MySql database:

   File.PathId
   File.FilenameId
   Job.FileSetId
   Job.ClientId

MYSQL to SSD

Mysql run on two SSD at server mpi1 and mpi2, asyncronouse mode mpi2 as master.

Innodb's can not be copyed palce to another, directory name must be same if so you can copy from system to other. So do mysqldump, setup new storage system and re-install databases.

Setup:

/etc/mysql/my.cnf datadir=<sdd dir>

datadir         = /media/dbs/mysql

/etc/apparmor.d/abstractions/mysql

  /var/lib/mysql/mysql.sock rw,
  /media/myjohn2/mysql/mysql.sock rw,

/etc/apparmor.d/usr.sbin.mysqld

 /media/dbs/mysql/ r,
 /media/dbs/mysql/** rwk,

/etc/passwd

mysql:x:129:139:MySQL Server,,,:/media/dbs/mysql:/bin/false

cd /media/dbs/mysql/
cp -Rp /var/lib/mysql ./
chown -R mysql:mysql /media/dbs/mysql/
/etc/init.d/apparmor restart
stop mysql
start mysql

Mysql Replication

http://dev.mysql.com/doc/refman/5.6/en/replication-howto.html http://www.howtoforge.com/how-to-set-up-database-replication-in-mysql-on-ubuntu-9.10-p2 http://dev.mysql.com/doc/refman/5.0/en/replication-howto-masterbaseconfig.html http://www.howtoforge.com/mysql_database_replication

master my.cnf

server-id               = 1
log_bin                 = /var/log/mysql/mysql-bin.log
expire_logs_days        = 10
max_binlog_size         = 100M
binlog_do_db            = bacula, mediawiki, kurrola

rights for slave to replicate:

GRANT REPLICATION SLAVE ON *.* TO 'slave_user'@'%' IDENTIFIED BY 'slave_password';
FLUSH PRIVILEGES;
quit;

LOCK & UNLOCK DATABASE FOR COPYING:

USE wanted; ### IF ONLY ONE
FLUSH TABLES WITH READ LOCK;
SHOW MASTER STATUS;

....COPY DATABASE

UNLOCK TABLES;
quit; 

slave my.cnf

server-id               =2
master-connect-retry    =60
#replicate-do-db         =bacula, mediawiki, kurrola

where to replicate and used user , password:

STOP SLAVE;
RESET SLAVE;
UNLOCK TABLES;
CHANGE MASTER TO MASTER_HOST='192.168.0.100', MASTER_USER='slave_user', MASTER_PASSWORD='slave_password', MASTER_LOG_FILE='mysql-bin.000001', MASTER_LOG_POS=19467;

I have also found that I needed to UNLOCK TABLES on the slave before starting the slave

In trouble do not forget:

 mysql_install_db  -  can fix missing files sometimes

 nano /etc/apparmor.d/local/usr.sbin.mysqld  -  add rights to access new locations for mysql
 service apparmor reload

Emergency Repair

DPKG , packages

http://www.thepowerbase.com/2012/04/how-to-fix-broken-packages-in-ubuntu-or-debian/

Copy Installation:

dpkg --get-selections > mypackages.txt
sudo dpkg --set-selections < mypackages.txt
sudo apt-get dselect-upgrade

or

Repair broken:

debsums -s -a 2> /tmp/broken.log
sed -n 's/^.*\(checksum mismatch\|changed file\) \([^ ]*\) file.*$/\1/p;s/^.*t open \([^ ]*\) file.*$/\1/p' < /tmp/broken.log |  sort -u > /tmp/broken.pkgs
cd /var/cache/apt/archives
rm  `cat /tmp/broken.pkgs`
apt-get --reinstall install `cat /tmp/broken.pkgs`

or

Try organize dpkg: https://help.ubuntu.com/community/PackageManagerTroubleshootingProcedure

ubuntu-support-status
sudo grep -R proxy /etc/apt/*
grep proxy  /etc/environment
echo $http_proxy
echo $ftp_proxy
grep proxy /etc/bash.bashrc
grep proxy ~/.bashrc
cat /etc/apt/apt.conf
sudo fuser -vvv /var/lib/dpkg/lock
cat /etc/lsb-release
uname -a
sudo rm /var/lib/apt/lists/lock 
sudo rm /var/lib/dpkg/lock
sudo cp -arf /var/lib/dpkg /var/lib/dpkg.backup
sudo mv /var/lib/dpkg/status /var/lib/dpkg/status-bad
sudo cp /var/lib/dpkg/status-old /var/lib/dpkg/status  ||  sudo cp /var/backups/apt.extended_states.0 /var/lib/dpkg/status
sudo mv /var/lib/dpkg/available /var/lib/dpkg/available-bad
sudo cp /var/lib/dpkg/available-old /var/lib/dpkg/available
sudo rm -rf /var/lib/dpkg/updates/*
sudo rm -rf /var/lib/apt/lists
sudo rm /var/cache/apt/*.bin
sudo mkdir /var/lib/apt/lists
sudo mkdir /var/lib/apt/lists/partial
LANG=C;sudo apt-get clean
LANG=C;sudo apt-get autoclean
LANG=C;sudo apt-get --purge autoremove
LANG=C;sudo apt-get --fix-missing update -o APT::Cache-Limit=100000000
sudo dpkg --configure -a
sudo dpkg --clear-avail
LANG=C;sudo apt-get -f install
LANG=C;sudo apt-get --fix-missing install
LANG=C;sudo apt-get update -o APT::Cache-Limit=100000000 && sudo apt-get dist-upgrade
find /etc/apt -name '*.list' -exec bash -c 'echo -e "\n$1\n"; cat -n "$1"' _ '{}' \;

Dpkg Packages re-install

If lot's debency, md5sum, pacpage not found,...

dpkg --get-selections > mypackages.txt

remove unwanted packages from *.txt so it looks list of packages to reinstall

Download all packages.

download:

#!/bin/bash
filecontent=( `cat $1 `)
echo $1
cd /var/cache/apt/archives
 for t in "${filecontent[@]}"
   do
    apt-get download $t
 done
echo "Download end!"

Use dpkg to install all packages.

install:

#!/bin/bash
filecontent=( `cat $1 `)
echo $1
cd /var/cache/apt/archives
 for t in "${filecontent[@]}"
   do
    dpkg -i $t*.deb
 done
echo "Install end!"

RESTORE whitout DATABASE

bscan -u bacula -n bacula -P bacula73 -s -S -V ST3POOL-0125 /media/KurrolaStorage3

bscan -u bacula -n <DB username> -P <DB password> -s -S -V <backup tape/file> <tabe/file directory>

MYSQL

backup bacula DB's:

 mysqldump -u steve -phuhaa bacula > bacula.sql

 mysqldump -u <DB's user> -p<DB's password> bacula > <backup file name>

Restore:

 mysql   -u yourname -phuhaa bacula < bacula.sql

 mysql -u <DB's user> -p<DB's password> bacula < <backup file name>

Status File Creation

http://www.solengtech.com/solengtech/opensource/howto-repair-ubuntu-dpkg-status-file

HowTo: Repair Ubuntu dpkg status file My Ubuntu 10.04.02 installs starting losing /var/lib/dpkg/status, and you can't update without it... Thanks to capink, but I ended up re-installing anyway so unfortunately losing this file you are near the end of the line.

http://ubuntuforums.org/archive/index.php/t-474587.html Using the script below will generate a new status file in your current directory (usually your home dir). Use this file in place of your status file after backing it up:

1. !/bin/bash

get_control_info () { for i in /var/lib/apt/lists/*_Packages do sed '/Package: '"$1"'$/,/^$/!d' $i done }

for i in /var/lib/dpkg/info/*.list do package_name=`basename $i | sed 's/.list$//'` get_control_info $package_name >> status-new done

sed -i -e '/^Filename: .*/d' -e '/^MD5sum:/d' -e '/^SHA1:/d' -e '/^SHA256:/d' status-new sed -i '/^Package: /a\Status: install ok installed' status-new