Difference between revisions of "MPI4YOU"
(→External Solvers & BASIC COMPONENTS) |
(→http://www.msg.ameslab.gov/graphics/other.html) |
||
| (5 intermediate revisions by 2 users not shown) | |||
| Line 1: | Line 1: | ||
=For Ubuntu 14.04= | =For Ubuntu 14.04= | ||
| + | |||
| + | ==Computtaional Chemistry paclages== | ||
| + | http://www1.bbiq.jp/zzzfelis/Facio.html<br> | ||
| + | http://classic.chem.msu.su/gran/gamess<br> | ||
| + | ===Databses=== | ||
| + | https://www.cfa.harvard.edu/hitran/<br> | ||
| + | http://www.nist.gov/srd/<br> | ||
| + | ====http://www.msg.ameslab.gov/graphics/other.html==== | ||
| + | gabedit<br> | ||
| + | https://sites.google.com/site/allouchear/Home/gabedit/gabedittutorialfiles/start<br> | ||
| + | https://sites.google.com/site/allouchear/Home/gabedit/installation-of-computational-chemistry-programs<br> | ||
| + | |||
| + | molpro<br> | ||
| + | facio<br> | ||
| + | gamess<br> | ||
| + | firfly<br> | ||
==Elmerfem== | ==Elmerfem== | ||
| + | https://github.com/elmercsc/elmerfem <br> | ||
===External Solvers & BASIC COMPONENTS=== | ===External Solvers & BASIC COMPONENTS=== | ||
| Line 7: | Line 24: | ||
LAPACK, BLAS from AMD | LAPACK, BLAS from AMD | ||
| − | + | http://graal.ens-lyon.fr/MUMPS/index.php MUMPS<br> | |
| + | |||
| + | http://acts.nersc.gov/hypre/ HYPRE <br> | ||
| − | + | http://www.pardiso-project.org/ PARDISO <br> | |
| − | + | http://trilinos.org/ TRILINOS <br> | |
| − | + | http SuperLU <br> | |
==basic software needed== | ==basic software needed== | ||
| Line 23: | Line 42: | ||
http://docs.adaptivecomputing.com/suite/8-0/basic/help.htm#topics/torque/1-installConfig/initializeConfigOnServer.htm <br> | http://docs.adaptivecomputing.com/suite/8-0/basic/help.htm#topics/torque/1-installConfig/initializeConfigOnServer.htm <br> | ||
| + | ==Netgen== | ||
| + | |||
| + | At 14.04 again must compile by own: | ||
| + | Togl1.7 ( no support 2.o at netgen ) | ||
| + | Tcl / Tk 8.6 | ||
| + | Tix | ||
| + | |||
| + | New occ dose not define PI so add: | ||
| + | |||
| + | Partition_Loop.cxx | ||
| + | #ifndef PI | ||
| + | # include <math.h> | ||
| + | # define PI M_PI | ||
| + | #endif | ||
=OLD= | =OLD= | ||
Latest revision as of 10:42, 15 April 2015
Contents
- 1 For Ubuntu 14.04
- 2 OLD
- 2.1 Running Installation
- 2.2 Parraler elmerfem
- 2.3 Netgen
- 2.4 New librrayes for elmerfem
- 2.5 Basic Server Installation tricks:
- 2.6 rsh-redone
- 2.7 BIND
- 2.8
- 2.9 Source Installation's
- 2.10 <a href="https://launchpad.net/mpi4you" rel="nofollow">https://launchpad.net/mpi4you</a> is under construction and you find there:
- 2.11 Basic "compile from sources" compatibility solutions
- 2.12 mpic2
- 2.13 CACTI
- 2.14 LTSP install Links:
- 2.15 DHCP, ISC DHCP
- 2.16
- 2.17 HPC
- 2.18 Paraview
- 2.19
- 2.20
- 2.21
- 2.22 Salomè
- 2.23 Elmer parraler for Ubuntu 12.4
- 2.24 Netgen parraler
For Ubuntu 14.04
Computtaional Chemistry paclages
http://www1.bbiq.jp/zzzfelis/Facio.html
http://classic.chem.msu.su/gran/gamess
Databses
https://www.cfa.harvard.edu/hitran/
http://www.nist.gov/srd/
http://www.msg.ameslab.gov/graphics/other.html
gabedit
https://sites.google.com/site/allouchear/Home/gabedit/gabedittutorialfiles/start
https://sites.google.com/site/allouchear/Home/gabedit/installation-of-computational-chemistry-programs
molpro
facio
gamess
firfly
Elmerfem
https://github.com/elmercsc/elmerfem
External Solvers & BASIC COMPONENTS
LAPACK, BLAS from AMD
http://graal.ens-lyon.fr/MUMPS/index.php MUMPS
http://acts.nersc.gov/hypre/ HYPRE
http://www.pardiso-project.org/ PARDISO
http://trilinos.org/ TRILINOS
http SuperLU
basic software needed
blcr
mpich3
PBS - Torque
http://en.wikipedia.org/wiki/Portable_Batch_System
http://docs.adaptivecomputing.com/suite/8-0/basic/help.htm#topics/torque/1-installConfig/initializeConfigOnServer.htm
Netgen
At 14.04 again must compile by own: Togl1.7 ( no support 2.o at netgen ) Tcl / Tk 8.6 Tix
New occ dose not define PI so add:
Partition_Loop.cxx
#ifndef PI # include <math.h> # define PI M_PI #endif
OLD
Running Installation
Parraler elmerfem
mpiexec -v -np <how many core> -host <hosts> -ckpointlib blcr -ckpoint-prefix <chekpoint files storage> -ckpoint-interval <chekponting interval seconds> /mpi<mpi4you version>/C/elmer_<elmer revision>/bin/ElmerSolver_mpi
mpiexec -v -np 3 -host ugh -ckpointlib blcr -ckpoint-prefix /mpi3/chekpoint -ckpoint-interval 10 /mpi3/C2/elmer_6003/bin/ElmerSolver_mpi
Netgen
/mpi3/C2/netgen_623/bin/netgen -geofile=Palkki_tila.brep -meshfile=Mesh_4 -meshsizefile=meshaize4.msz -meshfiletype="Elmer Format" -batchmode
/mpi3/C2/netgen_623/bin/netgen -geofile=<geometry> -meshfile=<result mesh file> -meshsizefile=<size parameters> -meshfiletype="Elmer Format" -batchmode
- .msz:
<how many points>
<X> <Y> <Z> <Mesh Size>
<how many lines>
<X1> <Y1> <Z1> <X2> <Y2> <Z2> <Mesh Size>
example mesh file:
1
0 0 0 0.001
7
0 0 0 0 0 0.208 0.001
0 0 0.208 0 0.100 0.208 0.001
0 0.100 0.208 0 0.100 0.200 0.001
0 0.100 0.200 0 0.008 0.200 0.001
0 0.008 0.200 0 0.008 0.008 0.001
0 0.008 0.008 0 0.100 0.008 0.001
0 0.100 0.008 0 0.100 0 0.001
0 0.100 0 0 0 0 0.001
New librrayes for elmerfem
-DHAVE_CHOLMOD http://faculty.cse.tamu.edu/davis/suitesparse.html
-Dhave_TRILINOS http://trilinos.org/oldsite/TrilinosBuildQuickRef.html
-DHAVE_SUPERLU http://crd-legacy.lbl.gov/~xiaoye/SuperLU/#superlu_dist
OLDER
-DHAVE_MUMPS
-DHAVE_HYPRE
-DHAVE_PARDISO
ALL SPARSE SOLVERS
Installation
THIS IS COPY, NOT DONE YET,... FROM https://sites.google.com/site/jpsdatareviewstheboy007/ubuntu-lts-12-4-companion-whit-ltsp-mpich2-elmer-openfoam
hi,
This page is under construction it has 'recepy' used to create easily manageable environment to run parallel solvers ( netgen,OpenFOAM and Elmer,... )
whit AMD Buldozer hardware at Ubuntu 12.4 LTS
This system is not fully tested, it's a like installation note's storage - it can help you.
I have done lot's experiments and to get all work together is not easy task, one software work's whit one and not all sofware's, compatibelity is real problem.
It can be Ubuntu is not good floor to build scientific calculations, based past experience but that's what I'm doing now.
It's worth to start from <a href="https://www.scientificlinux.org/" target="_blank">Scientific Linux Distribution</a> - but I can not recommend personally due no experience from that anyway it looks better!!
Files and future development of scripts, makefiles and changes can be done at <a href="https://launchpad.net/mpi4you">https://launchpad.net/mpi4you</a> . You are welcome !
Idea's:
-one installation, easy and fast to distribute to cluster/workstations ( that's <a href="http://www.ltsp.org/" target="_blank">ltsp</a> )
-parallel (mpich2), threads, OpenMP
-multi purpose use, all node's as well server can work as workstation or black box ( not recommended practice by Ubuntu )
-energy, resource efficiency, ecological solutions (hardware configuration, sleeping,..)
Tested:
-one installation and shareing it via netboot, it works whit ltsp but at moment ltsp closes out energy management options
-separately installed, more management work but energy saving options are working and that's money !
Generalideas, guide lines
Hardware in use:
old server:
- CPU AMD x 4
-ram 8G
-ASROCK
-0.5T raid 1
-nfs4 file server RAID1, syslog-nq, bacula
-bought from <a href="http://www.smarit.fi">http://www.smartit.fi</a>
new server:
-<a href="http://www.amd.com/us/products/desktop/processors/amdfx/Pages/amdfx.aspx" rel="nofollow" target="_blank">CPU AMD FX(tm)-8150 Eight-Core Processor × 8</a> it's <a href="http://en.wikipedia.org/wiki/Bulldozer_%28processor%29" target="_blank">Bulldozer CPU</a>
-ram 16G
-main board <a href="http://www.asrock.com/mb/AMD/990FX%20Extreme3/?cat=Specifications" target="_blank">AsROCK990FXextreme3</a>
-1T raid 1
-120G raid 0 , two 2x 300-500kt/s = 600-1000kt/s SATA3 Kingston 60G SSD's
-ltsp-server standalone,nfs4, file server RAID1, ssd RAID0
-bought from <a href="http://www.hardwareversand.de">http://www.hardwareversand.de</a>
Work Stations:
-<a href="http://www.amd.com/us/products/desktop/processors/amdfx/Pages/amdfx.aspx" target="_blank">CPU AMD FX(tm)-8150 Eight-Core Processor × 8</a> it's <a href="http://en.wikipedia.org/wiki/Bulldozer_%28processor%29" target="_blank">Bulldozer CPU</a>
-ram 16G
-main board AsROCK990FXextreme3
-bought from <a href="http://www.hardwareversand.de">http://www.hardwareversand.de</a> , yes price is ok, they deliver quite promptly whitin
2-4week's from order but if you have problem's whit product's it takes time - avoid them if you can!
System specifications:
Two identical servers, if another out of use other will take responsibility to run services.
- Ubuntu 12.4 LTS, kernel 3.2.0-36-generic
- DNS, bind9
- DHCP, ISC DHCP, <a href="http://consultancy.edvoncken.net/index.php/HOWTO_Configure_DHCP_failover" target="_blank">Fail-over primary and secondary servers </a>
- <a href="https://help.ubuntu.com/community/SettingUpNFSHowTo" target="_blank">NFS4</a>
- <a href="http://www.mcs.anl.gov/research/projects/mpich2" target="_blank">mpich2</a> for parraler calculation's
- <a href="http://www.ltsp.org/" target="_blank">ltsp</a> , ltsp-server-standalone
- syslog-ng, <a href="http://www.quietearth.us/articles/2006/09/27/Logging-remote-host-to-specific-logfile-with-syslogng" target="_blank">install ubuntu</a>
- bacula
- <a href="https://help.ubuntu.com/10.04/serverguide/openssh-server.html" target="_blank">openssh</a>
- apt-casher-ng ( you need to log )
- quata ( to avoid system crash due no room at hard disk )
- <a href="http://www.openfoam.com/" target="_blank">OpenFOAM</a>
- Elmer ( rev. 6033 ) whit <a href="http://www.pardiso-project.org" target="_blank">pardiso</a>
- <a href="http://www.code-aster.org/" target="_blank">Salomè meca</a> ( 6.5 )
- <a href="http://rene.bz/graphing-your-servers-motherboard-temperature-and-fan-rpm-speeds/" target="_blank">cacti</a> - to manage hardware, <a href="http://www.pitt-pladdy.com/blog/_20091031-160232_0000_LM_Sensors_stats_on_Cacti_via_SNMP_/" target="_blank">scripts </a>needed temp/fan/voltage
- netgen ( rev. 623 )
pre-build binaryes:
-rsh-redone
sources & library installation:
mpich2
Togl
Tcl
TK
parmetis
acml (blas,lapack),
MUMPS
PARDISO
Scalpack ( BLACS )
elmerfem ( rev 5912 )
netgen ( rev 596 )
Hierarchy:
<tbody>
</tbody>
Ubuntu DHCP. DNS, bind9
<tbody>
</tbody>
<tbody>
</tbody>
<tbody>
</tbody>
bash
RSH-redone / SSH
mipch2
<a href="https://help.ubuntu.com/community/SettingUpNFSHowTo" target="_blank">NFS4</a>
elmer
<a href="http://www.openfoam.com/" target="_blank">OpenFOAM</a>
Salomé
Netgen
parraler solvers
<a href="http://en.wikipedia.org/wiki/Network_block_device" target="_blank">nbd</a>
libraryes
<a href="http://en.wikipedia.org/wiki/Preboot_Execution_Environment" target="_blank">PXE</a>
core operateing system boot
<a href="http://www.ltsp.org/" target="_blank"> ltsp integration</a>
<a href="http://en.wikipedia.org/wiki/Simple_Network_Management_Protocol" target="_blank">SNMP</a> to see what happens at hardware
<a href="http://www.ltsp.org/" target="_blank">Ltsp</a> is set of script's and package to manage and create
configuration files, operating system distribution
over network (<a href="http://en.wikipedia.org/wiki/Preboot_Execution_Environment" target="_blank">netboot</a>). Orginaly it was used to supply software
to terminals (light computers) but why not use it to create cluster!!!
My Idea is to make one installation to sever and share it
to workstation's and nodes instead of having separate computers
installed separately. This cut downs installation and management duties
remarkably. Compared MAAS, EC2 & JUJU could cause unwanted overhead at parallel
computing at local network as well more management tasks.
Unsolved problem's whit installation ( tested kernel 3.2.0-35-generic ) :
1.a blcr supports kernel's older that 2.6.38, NO SUPPORT TO KERNEL - NO CHECKPOINTS <a href="https://code.launchpad.net/~ubuntu-branches/ubuntu/precise/blcr/precise-updates">https://code.launchpad.net/~ubuntu-branches/ubuntu/precise/blcr/precise-updates</a> , solution under work <a href="http://packages.debian.org/experimental/blcr-dkms">http://packages.debian.org/experimental/blcr-dkms currently it dose not work!!!</a>
1.b DMTCP dose not work eighter, it dose memory violation.
Change: blcr 0.5.0 compiles well whit 3.2.0-49-generic but not fully tested yet !
2. wakeonlan dose not work whit LTPS, <a href="https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1051325" target="_blank"> https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1051325</a>
Fixe's coming soon to precise: <a href="http://crd.lbl.gov/groups-depts/ftg/projects/current-projects/BLCR/berkeley-lab-checkpoint-restart-for-linux-blcr-downloads/" target="_blank">new blcr version</a>, <a href="http://wiki.xbmc.org/index.php?title=How-to:Enable_Wake-On-Device_for_Ubuntu" target="_blank">ACPI corrections to kernel</a>.
Installationnotes
<img src="https://www.google.com/chart?chc=sites&cht=d&chdp=sites&chl=%5B%5BTable+of+contents%27%3D20%27f%5Cv%27a%5C%3D0%2710%27%3D249%270%27dim%27%5Cbox1%27b%5CF6F6F6%27fC%5CF6F6F6%27eC%5C0%27sk%27%5C%5B%27%5D%27a%5CV%5C%3D12%27f%5C%5DV%5Cta%5C%3D10%27%3D0%27%3D250%27%3D297%27dim%27%5C%3D10%27%3D10%27%3D250%27%3D297%27vdim%27%5Cbox1%27b%5Cva%5CF6F6F6%27fC%5CC8C8C8%27eC%5C%27a%5C%5Do%5CLauto%27f%5C&sig=TzxATGSyGo59uvdEcnpk0pMmCmI" data-type="toc" data-props="align:left;maxDepth:6;width:250;" style="display:block;text-align:left;margin-right:auto;" height="300" width="250">
Server's have IP 192.168.0.40 and 192.168.0.41.
ltsp client's are useing IP's starting 192.168.0.42
Basic Server Installation tricks:
Compatible Usage Rights
At moment best solution have been:
nsh drives: group = nogroup
user must belong nogroup
/etc/passwd must be same ltsp and Ubuntu !!!
Both server and ltps nodes have same /home/user/.ssh/ssh-keys
To Install
-install basic configuration from CD-ROM only. This way you avoid re-installation due broken packages that can be tricky to find
-install server version so you can configure RAID disks, ext,... then
apt-get install Ubuntu-Desktop
-last step is graphics card drivers.If you have nvidia based graphics card first:
apt-get kernel-headers kernel-sources
and after that
apt-get nvidia-current nvidia-common
Install system related applications:
apt-get install apt-cacher-ng
apt-get install nfs-common nfs-kernel
apt-get install quota
apt-get install ltsp-server-standalone
apt-get install syslog-ng apt-cacher-ng bacula ntp
apt-get install ubuntu-desktop
apt-get install linux-headers kernel-source
apt-get install nvidia-current nvidia-common
nvidia-xconfigure
- ADD/USE FIREWALL /NAT TO ROUTER THE DISABLE ALL LINUX FIREWALL'S
apt-get remove ufw
apt-install firestarter
- for fun
apt-get install samba
apt-get install wine jstest-gtk wine
playonlinux ubuntu-restricted-extras mythplugins
apt-get install openjdk-7-jre icedtea-7-plugin
Network
/etc/network/interfaces
======================================================
- This file describes the network interfaces available on your system
- and how to activate them. For more information, see interfaces(5).
- The loopback network interface
auto lo
iface lo inet loopback
- The primary network interface
auto eth0
iface eth0 inet static
address 192.168.0.41
netmask 255.255.255.0
gateway 192.168.0.254
=======================================================
ipv6:
<a href="http://www.shorewall.net/IPv6Support.html#idp67096" target="_blank">IPV6, Shorewall, Webmin</a>
<a href="http://codeghar.wordpress.com/2012/08/03/ipv6-in-ubuntu-12-04/" target="_blank">Ubuntu IPv6 </a>- by HS
<a href="https://wiki.ubuntu.com/IPv6#Configure_your_Ubuntu_box_as_a_IPv6_router" target="_blank">Ubuntu IPV6</a> - Ubuntu Wiki
<a href="http://www.sixxs.net" target="_blank">Sixx - tunnel from IPV4</a>
<a href="http://www.csc.fi/funet/status/tools/funet_ipv6_status" target="_blank">Real Use IPV6</a>
ssh
Links:
<a href="http://www.open-mpi.org/faq/?category=rsh" target="_blank">Ssh OpenMPI</a>
<a href="https://help.ubuntu.com/10.04/serverguide/openssh-server.html" target="_blank">openssh</a>
- install
apt-get install openssh-client openssh-server
gedit /etc/ssh/sshd_config
======================================================================
- Package generated configuration file
- See the sshd_config(5) manpage for details
- What ports, IPs and protocols we listen for
Port 22
- Use these options to restrict which interfaces/protocols sshd will bind to
- ListenAddress ::
- ListenAddress 0.0.0.0
Protocol 2
- HostKeys for protocol version 2
HostKey /etc/ssh/ssh_host_rsa_key
HostKey /etc/ssh/ssh_host_dsa_key
HostKey /etc/ssh/ssh_host_ecdsa_key
- Privilege Separation is turned on for security
UsePrivilegeSeparation yes
- Lifetime and size of ephemeral version 1 server key
KeyRegenerationInterval 3600
ServerKeyBits 768
- Logging
SyslogFacility AUTH
LogLevel INFO
- Authentication:
LoginGraceTime 120
PermitRootLogin yes
StrictModes yes
RSAAuthentication yes
PubkeyAuthentication yes
- AuthorizedKeysFile %h/.ssh/authorized_keys
- Don't read the user's ~/.rhosts and ~/.shosts files
IgnoreRhosts yes
- For this to work you will also need host keys in /etc/ssh_known_hosts
RhostsRSAAuthentication no
- similar for protocol version 2
HostbasedAuthentication no
- Uncomment if you don't trust ~/.ssh/known_hosts for RhostsRSAAuthentication
- IgnoreUserKnownHosts yes
- To enable empty passwords, change to yes (NOT RECOMMENDED)
PermitEmptyPasswords no
- Change to yes to enable challenge-response passwords (beware issues with
- some PAM modules and threads)
ChallengeResponseAuthentication no
- Change to no to disable tunnelled clear text passwords
- PasswordAuthentication yes
- Kerberos options
- KerberosAuthentication no
- KerberosGetAFSToken no
- KerberosOrLocalPasswd yes
- KerberosTicketCleanup yes
- GSSAPI options
- GSSAPIAuthentication no
- GSSAPICleanupCredentials yes
X11Forwarding yes
X11DisplayOffset 10
PrintMotd no
PrintLastLog yes
TCPKeepAlive yes
- UseLogin no
- MaxStartups 10:30:60
- Banner /etc/issue.net
- Allow client to pass locale environment variables
AcceptEnv LANG LC_*
Subsystem sftp /usr/lib/openssh/sftp-server
- Set this to 'yes' to enable PAM authentication, account processing,
- and session processing. If this is enabled, PAM authentication will
- be allowed through the ChallengeResponseAuthentication and
- PasswordAuthentication. Depending on your PAM configuration,
- PAM authentication via ChallengeResponseAuthentication may bypass
- the setting of "PermitRootLogin without-password".
- If you just want the PAM account and session checks to run without
- PAM authentication, then enable this but set PasswordAuthentication
- and ChallengeResponseAuthentication to 'no'.
UsePAM yes
======================================================================
gedit /etc/ssh/ssh_config
======================================================================
- This is the ssh client system-wide configuration file. See
- ssh_config(5) for more information. This file provides defaults for
- users, and the values can be changed in per-user configuration files
- or on the command line.
- Configuration data is parsed as follows:
- 1. command line options
- 2. user-specific file
- 3. system-wide file
- Any configuration value is only changed the first time it is set.
- Thus, host-specific definitions should be at the beginning of the
- configuration file, and defaults at the end.
- Site-wide defaults for some commonly used options. For a comprehensive
- list of available options, their meanings and defaults, please see the
- ssh_config(5) man page.
Host *
- ForwardAgent no
- ForwardX11 no
- ForwardX11Trusted yes
- RhostsRSAAuthentication no
- RSAAuthentication yes
- PasswordAuthentication yes
- HostbasedAuthentication no
- GSSAPIAuthentication no
- GSSAPIDelegateCredentials no
- GSSAPIKeyExchange no
- GSSAPITrustDNS no
- BatchMode no
- CheckHostIP yes
- AddressFamily any
- ConnectTimeout 0
- StrictHostKeyChecking ask
- IdentityFile ~/.ssh/identity
- IdentityFile ~/.ssh/id_rsa
- IdentityFile ~/.ssh/id_dsa
- Port 22
- Protocol 2,1
- Cipher 3des
- Ciphers aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc
- MACs hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-ripemd160
- EscapeChar ~
- Tunnel no
- TunnelDevice any:any
- PermitLocalCommand no
- VisualHostKey no
- ProxyCommand ssh -q -W %h:%p gateway.example.com
SendEnv LANG LC_*
HashKnownHosts yes
GSSAPIAuthentication yes
GSSAPIDelegateCredentials no
===========================================================================
Then generate public/private rsa for user mpi and copy public one to authoritzed one's.
sudo adduser mpi
sudo login mpi
<span><b>cd .ssh<br>ssh-keygen -t rsa</b></span>
cat id_rsa.pub >> authorized_keys
Later at LTSP install mpi user's ~/.ssh directory will copyed to ltsp-client chroot /home/mpi/.ssh
That's due when client is has finished network boot it has image based chroot and after user at
client login's ltsp-client will link user's home directory. While useing mpi we are takeing connection
remotely trough ssh and thus sshserver at client side needs ssh sertificate's and those should be
in ltsp chroot image!!!
To get rid off keypasphere enter
ssh-add $HOME/.ssh/id_rsa and give passphere once
before launching applications.
rsh-redone
Ubuntu default rsh is fake rsh, just link's to ssh package.
sudo apt-get remove rsh
sudo apt-get install rsh-redone-server rsh-redone-client
apt-cacher-ng
sudo apt-get install apt-cacher-ng
- fill cache from CD-ROM
- After install http://localhost:3142 you find instructions from there
- create and add following file to server /etc/apt/apt.conf.d/ as well to ltsp image root /opt/ltsp/amd64/etc/apt
sudo gedit /etc/apt/apt.conf.d/02proxy
=================
Acquire::http { Proxy "http://192.168.0.40:3142"; };
=================
BIND
<a href="https://help.ubuntu.com/community/BIND9ServerHowto" target="_blank">Bind9 Installation</a>
<a href="http://www.sixxs.net/faq/dns/?faq=reverse" target="_blank">IPv6 Sixx'x Reverse Domain</a>
Source Installation's
<a href="https://launchpad.net/mpi4you" rel="nofollow">https://launchpad.net/mpi4you</a> is under construction and you find there:
-build cripts, dir /BX
-general flag's, dir /Bx/flag
-download code script, dir /Bx/download
-changed source file files's, dir /Sx
Basic "compile from sources" compatibility solutions
To get sources to work each other we must standardize communication between libraries and I have done it:
<a href="http://www.unix.org/version2/whatsnew/lp64_wp.html" target="_blank">Memory model LP64</a>
int 32bit real 64bit pointers/float 64bit and to get it work we use
-m64 whit fortran and metis -DIDXSIZE32, fortran -fdefault-real-8
-fdefault-double-8 dose the work.
Calling convetion, actual name
of function at Linker Level should be same and flag -DAdd_ adds one
underscore to fort name's and dose the job.
Unfortunately the way
gcc treats linking has changed, to get it remember all libraries to
bare end of linking sometime's you need to add -Wl,--no-as-needed
as well <a href="http://gcc.gnu.org/onlinedocs/gfortran/Code-Gen-Options.html" target="_blank">-fno-align-commons</a> gives some help.
Ubuntu 12.4 come whit GCC 4.6.3 and GCC 4.6 dose not support Buldozer architecture optimisation it's possible whit GCC 4.7.
mpic2
Got elmer working first time but only whit mpich2
compiled from <a href="http://www.mcs.anl.gov/research/projects/mpich2" target="_blank">sources</a> and ALL mpi realted distribution file's removed!
apt-get install mpich2 mpich2-doc libmpich2-dev libmpich2-3
CACTI
To find botlenecks you need some statistics from system. CACTI can collect needed information.
sudo apt-get snmpd
sudo apt-get install snmp-mibs-downloader
sudo apt-get cacti
<a href="http://rene.bz/graphing-your-servers-motherboard-temperature-and-fan-rpm-speeds/" rel="nofollow" target="_blank">cacti</a> - to manage hardware, <a href="http://www.pitt-pladdy.com/blog/_20091031-160232_0000_LM_Sensors_stats_on_Cacti_via_SNMP_/" rel="nofollow" target="_blank">scripts </a>needed temp/fan/voltage
LTSP install
Links:
<a href="https://help.ubuntu.com/community/UbuntuLTSP/FatClients" target="_blank">ltsp-fat client</a>
sudo gedit /etc/ltsp/ltsp-build-client.conf
==================================================
# The chroot architecture.
ARCH=amd64
LOCALE="fi_FI.UTF-8"
# ubuntu-desktop and edubuntu-desktop
are tested.
# If you test with [k|x]ubuntu-desktop,
edit this page and mention if it worked OK.
# kubuntu lucid (10.10) working okay.
FAT_CLIENT_DESKTOPS="ubuntu-desktop"
# Space separated list of programs to
install.
# The java plugin installation
contained in ubuntu-restricted-extras
# needs some special care, so let's use
it as an example.
LATE_PACKAGES="
kde-l10n-fi
language-selector-gnome
language-selector-common
language-pack-gnome-fi-base
language-pack-gnome-en-base
language-pack-gnome-de-base
language-pack-gnome-fr-base
language-pack-fi-base
language-pack-de-base
language-pack-fr-base
language-pack-fen-base
language-pack-fi
language-pack-fr
language-pack-de
language-pack-en
language-pack-gnome-de
language-pack-gnome-fr
language-pack-gnome-en
language-pack-gnome-fi
language-pack-en-base
language-pack-fi-base
language-pack-de-base
language-pack-fr-base
firefox-locale-fi
thunderbird-locale-fi
voikko-fi
libreoffice-l10n-fr
libreoffice-l10n-de
libreoffice-l10n-sv
libreoffice-l10n-fi
libreoffice-help-fi
hyphen-de
hyphen-fr
mythes-de
mythes-de-ch
mythes-fr
mythes-en-us
hyphen-en-us
"
# This is needed to answer "yes"
to the Java EULA.
# We'll create that file in the next
step.
DEBCONF_SEEDS="/etc/ltsp/debconf.seeds"
# This uses the server apt cache to
speed up downloading.
# This locks the servers dpkg, so you
can't use apt on
# the server while building the chroot.
MOUNT_PACKAGE_DIR="/var/cache/apt/archives/"
===============================================================
# created to /opt/ltsp
sudo ltsp-build-client
sudo ltsp-chroot -c -p
#timezone
dpkg-reconfigure tzdata
#local languages
dpkg-reconfigure localeconf
- system softwareapt-get
#apt-get install openmpi-common
openmpi-bin
#apt-get install libscotch-dev
libopenmpi-dev
apt-get install nfs-client nfs-common
samba cifs-utils
apt-get install playonlinux
#powerwake powernap
apt-get install mythtv-frontend
apt-get install gimp darktable
apt-get install mytharchive mythgallery
mythgame mythmusic mythnews mythweather
#IMAGE UPDATE
sudo ltsp-chroot -c -p
adduser root
adduser mpi
passwd root
passwd mpi
apt-get update
exit
sudo mkdir /opt/ltsp/amd64/mpi
sudo mkdir
/opt/ltsp/amd64/home/mpi
sudo cp -R /home/mpi/.ssh
/opt/ltsp/amd64/home/mpi #enviroment
sudo cp /home/mpi/.bashrc /opt/ltsp/amd64/home/mpi
sudo cp /home/mpi/.bash_login /opt/ltsp/amd64/home/mpi
#sudo cp -R /root/.ssh
/opt/ltsp/amd64/root
sudo ltsp-chroot -c -p
cd /home
chown -R mpi:mpi mpi
cd /home/mpi
chmod 700 .ssh
chmod 600 .ssh/id*
chmod 644 .ssh/known_hosts
chmod 644 .ssh/authorized_keys
cd /root
chmod 700 .ssh
chmod 600 .ssh/id*
chmod 644 .ssh/known_hosts
chmod 644 .ssh/authorized_keys
mkdir /mpi
mkdir /mpi2
mkdir /mp3
apt-get install quota
apt-get install wine jstest-gtk wine
playonlinux ubuntu-restricted-extras mythplugins
apt-get install ltsp-server-standalone
apt-get syslog-ng bacula-fd ntp
# COMMAND'S TO UPDATE IMAGE
sudo ltsp-update-sshkeys
sudo ltsp-update-image -n
sudo ltsp-update-kernels
Tip's:
-diagnostics, apt-get wireshark
# NFS filesystem
#SERVER
/etc/fstab :
/mpi2 /export/mpi2 none bind 0 0
/mpi3 /export/mpi3 none bind 0 0
/etc/exports :
/export/mpi2 192.168.0.0/24(rw,nohide,insecure,no_subtree_check,sync)
/export/mpi3 192.168.0.0/24(rw,nohide,insecure,no_subtree_check,sync)
service idmapd restart
#CLIENT info at server
/var/lib/tftpbootp/ltsp/amd64/lts.conf :
FSTAB_1="192.168.0.41:/mpi3 /mpi3 nfs4 _netdev,auto 0 0"
FSTAB_2="192.168.0.40:/ /mpi nfs4 _netdev,auto 0 0"
FSTAB_3="//192.168.0.51/OpenShare /media/ST1 cifs guest,rw,iocharset=utf8,file_mode=0777,dir_mode=0777 0 0"
FSTAB_4="192.168.0.41:/mpi2 /mpi2 nfs4 _netdev,auto 0 0"
=========================================================================================
# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
#
# <file system> <mount point> <type> <options> <dump> <pass>
proc /proc proc nodev,noexec,nosuid 0 0
# / was on /dev/md0 during installation
UUID=24d52ebb-9a49-4949-831a-0702ca032406 / ext4 errors=remount-ro 0 1
# /boot/efi was on /dev/sda1 during installation
UUID=9AD9-4C15 /boot/efi vfat defaults 0 1
# /home was on /dev/md1 during installation
UUID=a9091c8f-31e2-43af-8167-88d8191a4f2c /home ext4 usrquota,grpquota 0 2
# /var was on /dev/md2 during installation
UUID=3dae13d4-6310-417a-bc67-e987638dbe54 /var ext4 defaults 0 2
# swap was on /dev/sda5 during installation
UUID=c5de965b-1adc-40e4-ab8b-cddb8bd153c6 none swap sw 0 0
# swap was on /dev/sdb5 during installation
UUID=bc109a85-d598-41e4-8805-a8bce1c68aa2 none swap sw 0 0
/dev/md127 /mpi3 ext4 rw,nosuid,nodev,uhelper=udisks 0 0
192.168.0.40:/ /mpi nfs4 _netdev,auto 0 0
//192.168.0.51/OpenShare /media/ST1 cifs guest,rw,iocharset=utf8,file_mode=0777,dir_mode=0777 0 0
#/mpi2 /export/mpi2 none bind 0 0
/dev/sr0 /media/cdrom iso9660,udf user,noauto,exec,utf8,uid=1000,mode=0777 0 0
#//path to your windows share/ /your_mount_path cifs username=your_username,password=your_password,workgroup=your_workgroup,users,auto,user_xattr 0 0
<a>file:///etc/fstab</a>
===================================================================================================
DHCP, ISC DHCP
Links:
<a href="http://consultancy.edvoncken.net/index.php/HOWTO_Configure_DHCP_failover" target="_blank">Failover primary and secondary servers </a>
<a href="http://mirrors.bieringer.de/Linux+IPv6-HOWTO/hints-daemons-isc-dhcp.html" target="_blank">FOR IPv6</a> you need to run second ISC DHCP deamon whit own configuration for IPv6 <a href="http://www.buntschu.ch/blog/?p=344" target="_blank">more</a>.
Other solution is radvd deamon but I did not get it work whit ISC DHCP serving IPv4 clients.
# check out that isc-dhcp is only dhcp server installed at server
gedit /etc/ltsp/dhcpd.conf
=======================================
#dhcp
- Default LTSP dhcpd.conf config file.
INTERFACES="eth0";
authoritative;
failover peer "dhcp-failover" {
primary; # declare this to be the primary server
address 192.168.0.41;
port 647;
peer address 192.168.0.40;
peer port 647;
max-response-delay 30;
max-unacked-updates 10;
split 128;
mclt 3600;
load balance max seconds 3;
}
subnet 192.168.0.0 netmask 255.255.255.0 {
pool {
failover peer "dhcp-failover";
deny dynamic bootp clients;
max-lease-time 1800; # 30 minutes
range 192.168.0.100 192.168.0.200;
}
pool {
max-lease-time 1800; # 30 minutes
range 192.168.0.201 192.168.0.210;
}
option domain-name "kurrola.dy.fi";
- option domain-name-servers 193.229.0.40, 193.229.0.42;
option domain-name-servers 192.168.0.40,192.168.0.41;
option broadcast-address 192.168.0.255;
option routers 192.168.0.254;
- get-lease-hostnames true;
option subnet-mask 255.255.255.0;
option root-path "/opt/ltsp/amd64";
if substring( option vendor-class-identifier, 0, 9 ) = "PXEClient" {
filename "/ltsp/amd64/pxelinux.0";
} else {
filename "/ltsp/amd64/nbi.img";
}
next-server 192.168.0.41;
allow booting;
allow bootp;
}
- subnet ends
- staic lease
host ugh {
hardware ethernet xxxxxxxxxxxxxxxx;
fixed-address 192.168.0.42;
}
host kaak {
hardware ethernet xxxxxxxxxxxxxxxx;
fixed-address 192.168.0.43;
}
host kurrola.dy.fi {
hardware ethernet xxxxxxxxxxxxxxxx;
fixed-address 192.168.0.40;
}
host JuuliaHP {
hardware ethernet xxxxxxxxxxxxxxxx;
fixed-address 192.168.0.43;
}
host JoniXperiaActiv {
hardware ethernet xxxxxxxxxxxxxxxx;
fixed-address 192.168.0.90;
}
host ipadHanna {
hardware ethernet xxxxxxxxxxxxxxxx;
fixed-address 192.168.0.91;
}
host KINDLEJoni {
hardware ethernet xxxxxxxxxxxxxxxx;
fixed-address 192.168.0.92;
}
host KurrolaStorage1 {
hardware ethernet xxxxxxxxxxxxxxxx;
fixed-address 192.168.0.51;
}
host paula {
hardware ethernet xxxxxxxxxxxxxxxx;
fixed-address 192.168.0.81;
}
host EPSONSX420W {
hardware ethernet xxxxxxxxxxxxxxxx;
fixed-address 192.168.0.50;
}
host mpi1 {
hardware ethernet xxxxxxxxxxxxxxxx;
fixed-address 192.168.0.41;
}
=======================================
HPC
More About <a href="http://en.wikipedia.org/wiki/Supercomputer" target="_blank">HPC</a>. Most work done to compiled at make script distribution <a href="https://launchpad.net/mpi4you">https://launchpad.net/mpi4you</a> .
Needed Calculation libraries shortlist:-
<a href="http://www.cmake.org/cmake/help/install.html" target="_blank">CMAKE</a>
<a href="http://www.vtk.org/Wiki/VTK/Building/Linux" target="_blank">VTK</a>
<a href="http://developer.amd.com/membership/licenseagreement.aspx?id=4c8a0be8-f366-418e-b8b6-85ffdebca990&ReferUrl=" target="_blank">AMCL AMD ( BLAS, LAPACK )</a>
<a href="http://graal.ens-lyon.fr/MUMPS/index.php?page=dwnld" target="_blank">MUMPS</a>
<a href="http://acts.nersc.gov/hypre/" target="_blank">HYPRE</a>
<a href="http://www.pardiso-project.org/branch" target="_blank">PARDISO</a>
Before production use, one way to to check basic installation is to take part <a href="http://icl.cs.utk.edu/hpcc/" target="_blank">HPCC Speed Chalange</a>
and benchmark your system against others, here is my very first trial out at <a href="http://icl.cs.utk.edu/hpcc/hpcc_results.cgi?vid=051M03bTda&ea=the@boy007.dy.fi" target="_blank">cluster.kurroila.dy.fi - mpi4you</a> !
That way you recognize bottle neck's in your system and have realistic understanding what it can do for you.
Compile methods:
a) CMAKE is used whit VTK, HDF5. , paraview, parmetis, metis
b) make / Makefile.inc ARPACK, MUMPS 4.1, scalapck ( scalapac_installer ), scotch_esmumps
c) ./configure hypre, mpich, netgen, elmerfem
Paraview
Links: <a href="http://www.paraview.org/paraview/resources/software.php" target="_blank">download and compile,</a> <a href="http://www.vtk.org/Wiki/ParaView:Build_And_Install" target="_blank">Instructions VTK-Wiki</a>
<a href="http://www.vtk.org/Wiki/ParaView:Build_And_Install" target="_blank"></a>
To get space ball working install vrpn config it for spaceball (uncomment vrpn_3DConnexion_Navigator device0 and comment activated).
#hidapi
sudo apt-get install libudev-dev libusb-dev libfox-dev
sudo apt-get install autotools-dev
git clone git://github.com/signal11/hidapi.git
cd /mpi3/S2
./bootstrap
./configure --prefix=/mpi3/C2/hidapi
make
sudo make install
cd ..
- download vrpn
- mkdir vrpn-BUILD
- cmake -i ../vrpn
cd vrpn-BUILD
make
sudo make install
Download Paraview
mkdir /mpi3/S2/ParaView6
cd /mpi3/S2/ParaView6
git clone git://paraview.org/ParaView.git /mpi3/S2/ParaView6
git checkout master
git pull origin master
git submodule update --init
Run SpaceBall <a href="http://review.source.kitware.com/#/c/10740/" target="_blank">patch</a> ( Thanks for Cory from this patch ) before compilationt of paraview
git fetch http://review.source.kitware.com/p/ParaView refs/changes/40/10740/3 && git checkout FETCH_HEAD -b add_space_navigator_grab_world_style
cd /mpi3/S2
mkdir ParaView6-BUILD
cd ParaView6-BUILD
cmake -i ../ParaView6
- gedit cmake
make
make install
Salomè
<a href="http://www.code-aster.org/V2/spip.php?article303" target="_blank">Links: Salomé Meca code Aster </a>work's but crashes time to time and
GUI dose not work correctly.
<a href="http://www.salome-platform.org/" target="_blank">Salomé Source: Debian 6.0 Squeeze 64bit</a> can be compiled as
binary. Can not build from SOURCES at moment.
Just download InstallWizard, extract it and run install.
Read: VTK & SpaceBall
download <a href="http://www.salome-platform.org/" target="_blank">Salomé Source: Debian 6.0 Squeeze 64bit</a>
cd /mpi3/S2/InstallWizard_6.6.0_Debian_6.0_64bit
./runInstall
<a href="http://www.salome-platform.org/" target="_blank">Salomé Source: Debian 6.5 Squeeze 64bit</a> can be compiled as
binary. Can not build from SOURCES at moment.
Prerequisites for binary build:
sudo apt-get install freeglut3
sudo ln -s /usr/lib/libicuuc.so.48 /usr/lib/libicuuc.so.44
sudo ln -s /usr/lib/libicui18n.so.48 /usr/lib/icui18n.so.44
sudo ln -s /usr/lib/libicudata.so.48 /usr/lib/libicudata.so.44
....
Elmer parraler for Ubuntu 12.4
Links:
<a href="http://www.elmerfem.org/elmerwiki/index.php/Compiling_Elmer_parallel_version_%28Linux%29">Compling Elmer parallel</a>
<a href="https://launchpad.net/%7Etormodvolden/+archive/ppa" target="_blank">Update ELMER binary for ubuntu 12.4 ppa by Tormod Volden</a>
At system Elmer is installed RAID0 SDD disk,
elmer will be under /mpi3 directory.
You should install elmer as user:mpi to get it fully
work at cluster.
Elmer binaryes will after installation at /mpi3/elmer/bin and source /mpi3/elmerfem.
Trunk source will be used at directory /mpi3/elmerfem/trunk
To build binaryes:
a) first download all code, libraryes commented out #sudo #svn
at first section of buildelmer.sh once.
b) do changes to ElmerGUI.pri
c) After that run ./buildelmer.sh. I use buildelmer.sh
as storage of infromation to download and all libraryes,...
To start gui use elmerG.sh
If you need to compile again do ./clean.sh first.
- Load Elmer sources:
mkdir /mpi3
cd /mpi3
mkdir elmer
svn co https://elmerfem.svn.sourceforge.net/svnroot/elmerfem elmerfem
- pardiso library and licence file to directory elmer/lib
- pardiso available by separate licence from <a href="http://www.pardiso-project.org/download/bb6ex6hze7cvjhuz15" target="_blank">http://www.pardiso-project.org</a>
- ==============================
- Do changes /mpi3/elmerfem/trunk/ElmerGUI/ElmerGUI.pri
- cd /usr/include
- sudo ln -s vtk-5.8/* ./*
- =DO CHANGES BEFORE COMPILE =======
gedit /mpi3/elmerfem/trunk/ElmerGUI/ElmerGUI.pri
- ===================================
- ==============================================================================
- ElmerGUI: configuration file
- ==============================================================================
- ------------------------------------------------------------------------------
- Optional components (undefine or comment out to exclude from compilation):
- ------------------------------------------------------------------------------
DEFINES += EG_QWT # Use QWT for convergence monitor?
DEFINES += EG_VTK # Use VTK for postprocessing?
DEFINES += EG_MATC # Use MATC for internal operations in postprocessing?
DEFINES += EG_OCC # Use OpenCASCADE 6.3 for importing CAD files? Needs VTK.
DEFINES -= EG_PYTHONQT # Use PythonQt for scripting in post processor?
- ------------------------------------------------------------------------------
- 64 bit system?
- ------------------------------------------------------------------------------
BITS = 64
- ------------------------------------------------------------------------------
- Python library:
- ------------------------------------------------------------------------------
unix {
PY_INCLUDEPATH = /usr/include/python3.2
PY_LIBPATH = /usr/lib
PY_LIBS = -lpython3.2
}
- ------------------------------------------------------------------------------
- VTK library:libQVTK.so.5.8.0
- ------------------------------------------------------------------------------
unix {
VTK_INCLUDEPATH = /usr/include/vtk-5.8
VTK_LIBPATH = /usr/lib
VTK_LIBS = -lQVTK \
-lvtkCommon \
-lvtkDICOMParser \
-lvtkFiltering \
-lvtkGenericFiltering \
-lvtkGraphics \
-lvtkHybrid \
-lvtkIO \
-lvtkImaging \
-lvtkInfovis \
-lnetcdf \
-lvtkRendering \
-lvtkViews \
-lvtkVolumeRendering \
-lvtkWidgets
}
=== TO WIPE OUT COMPILE FILE'S ======
run at install directory
cd /mpi3
===================================
===================================
build script - at first time do uncommented
parts by copy paste at terminal
===================================
/mpi3/buildelmer.sh
==============================
- !/bin/sh -f
export ElmerMPI_DIR="$( cd "$( dirname "$0" )" && pwd )"
- svn co https://elmerfem.svn.sourceforge.net/svnroot/elmerfem elmerfem
- sudo apt-get install libvtk5-qt4-dev
cd elmerfem/trunk
- Choose the installation directory and set up the environment:
export ELMER_HOME=$ElmerMPI_DIR/elmer
export ELMER_POST_HOME=$ELMER_HOME/share/elmerpost
export ELMER_FRONT_HOME=$ELMER_HOME/share/elmerfront
export ELMERGUI_HOME=$ELMER_HOME/bin
export PATH=$PATH:$ELMER_HOME/bin
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$ELMER_HOME/lib
- the compilers
export CC=mpicc
export CXX=mpic++
export FC=mpif90
export F77=mpif90
- the compiler flags
export CFLAGS="-I/usr/include/tcl8.5"
export CXXFLAGS="-I/usr/include/tcl8.5"
export FCFLAGS=""
export F77FLAGS=""
export FFLAGS=""
- linking
- export LDFLAGS=""
- if PARDISO available
- export FCPPFLAGS="-DHAVE_PARDISO"
- export LDFLAGS="-L/mpi4/elmer/lib/ -lpardiso412-GNU443-MPI-X86-64 -lgfortran -fopenmp -lpthread -lm"
- -Ofast -ipa -lacml -lfortran -lffio -lacml mv -lmv
export OMP_NUM_THREADS=3
export PARDISO_LIC_PATH=$ELMER_HOME/lib
export MKL_SERIAL=YES
- paths
- Modules definition and configure options setting:
modules="matc umfpack mathlibs elmergrid meshgen2d eio hutiter fem"
for m in $modules; do
cd $m
./configure --prefix=$ELMER_HOME --with-64bits=yes --with-mpi=yes
make
make install
cd ..
done
cd post
export CXXFLAGS="-I/usr/include/FTGL -I/usr/include/tcl8.5"
./configure --prefix=$ELMER_HOME
make
make install
cd ..
cd front
./configure --prefix=$ELMER_HOME
make
make install
cd ..
cd ElmerGUI
echo "now in ElmerGUI"
qmake ElmerGUI.pro
make
cp ./Application/ElmerGUI $ELMERGUI_HOME/ElmerGUI
cp -r ./Application/edf $ELMERGUI_HOME
cd ..
======== START UP ElmerGUI ==========
/mpi3/elmerG.sh
====================================
- !/bin/sh -f
export ElmerMPI_DIR="$( cd "$( dirname "$0" )" && pwd )"
export ELMER_HOME=$ElmerMPI_DIR/elmer
export PATH=$PATH:$ELMER_HOME/bin
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$ELMER_HOME/lib
export ELMERGUI_HOME=$ELMER_HOME/bin
export ELMER_POST_HOME=$ELMER_HOME/bin
export OMP_NUM_THREADS=3
export PARDISO_LIC_PATH=$ELMER_HOME/lib
export MKL_SERIAL=YES
$ELMERGUI_HOME/ElmerGUI
====================================
===================================
/mpi3/clean.sh
===================================
- !/bin/sh -f
cd elmerfem/trunk
- Modules definition and configure options setting:
modules="matc umfpack mathlibs elmergrid meshgen2d eio hutiter fem"
for m in $modules; do
cd $m
make clean
cd ..
done
cd front
make clean
cd ..
cd post
make clean
cd ..
cd ElmerGUI
make clean
cd Application
make clean
cd..
cd ..
==============
- All ok?
- go:
cd elmer/bin
ElmerSolver
MAIN: =============================================================
MAIN: ElmerSolver finite element software, Welcome!
MAIN: This program is free software licensed under (L)GPL
MAIN: Copyright 1st April 1995 - , CSC - IT Center for Science Ltd.
MAIN: Webpage http://www.csc.fi/elmer, Email elmeradm@csc.fi
MAIN: Library version: 7.0 (Rev: 5825)
MAIN: Running in parallel using 18 tasks.
MAIN: HYPRE library linked in.
MAIN: MUMPS library linked in.
MAIN: PARDISO library linked in.
MAIN: =============================================================
==============
- TEST
cd /mpi3/elmerfem/trunk/fem/tests
.runtest
joni@mpi1:/mpi3/elmerfem/trunk/fem/tests$ ./runtests
$ELMER_HOME undefined, setting it to ../src
test 1 : 1dtests [PASSED], CPU time=0.06
test 2 : 1sttime [PASSED], CPU time=0.43
test 3 : 2ndtime [PASSED], CPU time=0.74
test 4 : adaptivity1 [PASSED], CPU time=0.98
test 5 : adaptivity2 [PASSED], CPU time=1.19
test 6 : adaptivity3 [PASSED], CPU time=1.74
test 7 : adaptivity4 [PASSED], CPU time=2
test 8 : adaptivity5 [PASSED], CPU time=2.22
test 9 : adv_diff1 [PASSED], CPU time=2.82
test 10 : adv_diff2 [PASSED], CPU time=3.46
test 11 : adv_diff3 [PASSED], CPU time=5.68
test 12 : adv_diff4 [PASSED], CPU time=9.11
test 13 : AdvDiffHandles [PASSED], CPU time=9.33
test 14 : AdvDiffHandles2 [PASSED], CPU time=10.25
test 15 : AdvDiffHandles3 [PASSED], CPU time=10.46
test 16 : AdvReactDG [PASSED], CPU time=10.81
test 17 : AdvReactDG_P [PASSED], CPU time=11.24
test 18 : amultg [PASSED], CPU time=12.36
test 19 : amultg2 [PASSED], CPU time=12.79
test 20 : beam-springs [PASSED], CPU time=13.64
test 21 : bentonite [PASSED], CPU time=13.72
test 22 : BlockLinElast1 [PASSED], CPU time=14.04
test 23 : BlockLinElast2 [PASSED], CPU time=14.18
test 24 : BlockLinElast2b [PASSED], CPU time=14.31
test 25 : BlockLinElast3 [PASSED], CPU time=15.79
test 26 : BlockPoisson1 [PASSED], CPU time=15.85
test 27 : BlockPoisson2 [PASSED], CPU time=15.99
test 28 : BlockPoisson3 [PASSED], CPU time=16.28
test 29 : bodydir [PASSED], CPU time=16.37
test 30 : bodydir2 [PASSED], CPU time=20.36
test 31 : bodyload [PASSED], CPU time=20.44
test 32 : buckling [PASSED], CPU time=24.36
test 33 : CapacitanceMatrix [PASSED], CPU time=24.54
test 34 : CavityLid [PASSED], CPU time=25.04
test 35 : CavityLid2 [PASSED], CPU time=28.68
test 36 : channel_v2f [PASSED], CPU time=34.9
test 37 : cmultg [PASSED], CPU time=38.89
test 38 : coating [PASSED], CPU time=42.06
test 39 : CoordinateScaling [PASSED], CPU time=42.14
test 40 : CoupledPoisson1 [PASSED], CPU time=42.21
test 41 : CoupledPoisson1b [PASSED], CPU time=42.29
test 42 : CoupledPoisson2 [PASSED], CPU time=43.58
test 43 : CoupledPoisson2b [PASSED], CPU time=45
test 44 : CoupledPoisson3 [PASSED], CPU time=45.08
test 45 : CoupledPoisson4 [PASSED], CPU time=45.15
test 46 : CoupledPoisson5 [PASSED], CPU time=45.25
test 47 : CoupledPoisson6 [PASSED], CPU time=45.32
test 48 : CoupledPoisson7 [PASSED], CPU time=45.4
test 49 : CoupledPoisson8 [PASSED], CPU time=45.48
test 50 : CoupledPoisson9 [PASSED], CPU time=45.56
test 51 : current [PASSED], CPU time=45.76
test 52 : current_heat_control [PASSED], CPU time=46.2
test 53 : CurvedBndryPFEM [PASSED], CPU time=46.26
test 54 : dft-water [PASSED], CPU time=46.26
test 55 : diffuser_sa [PASSED], CPU time=54.64
test 56 : diffuser_sst [PASSED], CPU time=60.6
test 57 : diffuser_v2f [PASSED], CPU time=75.99
test 58 : DivergenceAnalytic2D [PASSED], CPU time=76.87
test 59 : el_adaptivity [PASSED], CPU time=77.01
test 60 : ElastElstat1DBeam [PASSED], CPU time=77.16
test 61 : ElastElstatBeam [PASSED], CPU time=77.46
test 62 : elasticity [PASSED], CPU time=77.99
test 63 : ElasticLubrication [PASSED], CPU time=80.55
test 64 : ElastPelem [PASSED], CPU time=85.2
test 65 : Electrokinetics [PASSED], CPU time=97.87
test 66 : elstat [PASSED], CPU time=102.1
test 67 : elstat_infty [PASSED], CPU time=105.89
test 68 : elstat_source [PASSED], CPU time=106.03
test 69 : ExtrusionStructured2D [PASSED], CPU time=108.68
test 70 : FlowResNoslip [PASSED], CPU time=117.48
test 71 : FlowResSlip [PASSED], CPU time=126.22
test 72 : fluxsolver [PASSED], CPU time=126.31
test 73 : fluxsolver2 [PASSED], CPU time=126.39
test 74 : fluxsolver3 [PASSED], CPU time=126.69
test 75 : freesurf [PASSED], CPU time=127.4
test 76 : freesurf_axi [PASSED], CPU time=128.71
test 77 : freesurf_int [PASSED], CPU time=129.73
test 78 : freesurf_ltd [PASSED], CPU time=138.1
test 79 : fsi_beam [PASSED], CPU time=139.49
test 80 : fsi_beam_optimize [PASSED], CPU time=146.08
test 81 : fsi_box [PASSED], CPU time=148.21
test 82 : fsi_box2 [PASSED], CPU time=149.63
test 83 : geomstiff [PASSED], CPU time=149.87
test 84 : gmultg [PASSED], CPU time=150.16
test 85 : HeatControl [PASSED], CPU time=150.23
test 86 : HeatControl2 [PASSED], CPU time=150.37
test 87 : heateq [PASSED], CPU time=150.74
test 88 : heateq_bdf2 [PASSED], CPU time=150.97
test 89 : heateq_bdf3 [PASSED], CPU time=151.19
test 90 : heateq_newmark [PASSED], CPU time=151.35
test 91 : heateq_newmark_global [PASSED], CPU time=151.5
test 92 : heateq-par [PASSED], CPU time=151.59
test 93 : HeatGap [PASSED], CPU time=151.69
test 94 : HelmholtzBEM [PASSED], CPU time=152.35
test 95 : HelmholtzEdge [PASSED], CPU time=152.64
test 96 : HelmholtzFace [PASSED], CPU time=152.94
test 97 : HelmholtzFEM [PASSED], CPU time=153.15
test 98 : HelmholtzPlaneWaves [PASSED], CPU time=153.66
test 99 : HelmholtzPlaneWavesAxis [PASSED], CPU time=154.2
test 100 : HelmholtzStructure [PASSED], CPU time=154.85
test 101 : HelmholtzStructure2 [PASSED], CPU time=155.33
test 102 : HelmholtzStructure3 [PASSED], CPU time=156.88
test 103 : InductionHeating [PASSED], CPU time=157.36
test 104 : InductionHeating2 [PASSED], CPU time=158.15
test 105 : L2norm [PASSED], CPU time=160.29
test 106 : levelset1 [PASSED], CPU time=165.11
test 107 : levelset2 [PASSED], CPU time=167.31
test 108 : levelset3 [PASSED], CPU time=174.16
test 109 : levelset3b [PASSED], CPU time=180.73
test 110 : LimitDisplacement [PASSED], CPU time=184.43
test 111 : LimitTemperature [PASSED], CPU time=189.43
test 112 : LimitTemperature2 [PASSED], CPU time=192.22
test 113 : linearsolvers [PASSED], CPU time=192.45
test 114 : linearsolvers_cmplx [PASSED], CPU time=193.49
test 115 : LubricationTunedForce [PASSED], CPU time=194.21
test 116 : marangoni [PASSED], CPU time=194.34
test 117 : MeshRefineGrading [PASSED], CPU time=197.73
test 118 : mgdyn_3phase [PASSED], CPU time=201.81
test 119 : mgdyn_bh [PASSED], CPU time=232.21
test 120 : mgdyn_harmonic [PASSED], CPU time=246.3
test 121 : mgdyn_steady [PASSED], CPU time=256.66
test 122 : mgdyn_torus [PASSED], CPU time=262.11
test 123 : mgdyn_torus_harmonic [PASSED], CPU time=269.4
test 124 : mgdyn_transient [PASSED], CPU time=291.99
test 125 : mhd [PASSED], CPU time=294.85
test 126 : mhd2 [PASSED], CPU time=298.2
test 127 : multimesh [PASSED], CPU time=298.52
test 128 : NaturalConvection [PASSED], CPU time=304.29
test 129 : NonnewtonianChannelFlow [PASSED], CPU time=305
test 130 : normals [PASSED], CPU time=305.07
test 131 : NormalTangentialBC [PASSED], CPU time=307.2
test 132 : OptimizeSimplexFourHeaters [PASSED], CPU time=311.06
test 133 : ParticleAdvector [PASSED], CPU time=312.96
test 134 : ParticleAdvector2 [PASSED], CPU time=316.34
test 135 : ParticleAdvector3 [PASSED], CPU time=323.75
test 136 : ParticleAdvectorZalesak [PASSED], CPU time=328.92
test 137 : ParticleFalling [PASSED], CPU time=329.07
test 138 : ParticleHeating [PASSED], CPU time=339.2
test 139 : passive [PASSED], CPU time=339.92
test 140 : periodic1 [PASSED], CPU time=340.04
test 141 : periodic2 [PASSED], CPU time=340.17
test 142 : periodic3 [PASSED], CPU time=340.25
test 143 : periodic_explicit [PASSED], CPU time=341.49
test 144 : periodic_nonconforming [PASSED], CPU time=341.59
test 145 : periodic_offset [PASSED], CPU time=341.7
test 146 : periodic_offset2 [PASSED], CPU time=342.42
test 147 : periodic_rot [PASSED], CPU time=342.53
test 148 : PhaseChange [PASSED], CPU time=346.09
test 149 : PhaseChange2 [PASSED], CPU time=346.94
test 150 : PhaseChange3 [PASSED], CPU time=347.83
test 151 : piezo [PASSED], CPU time=347.97
test 152 : plates [PASSED], CPU time=348.07
test 153 : pmultg [PASSED], CPU time=349.36
test 154 : pointdir [PASSED], CPU time=349.43
test 155 : pointload [PASSED], CPU time=349.51
test 156 : pointload2 [PASSED], CPU time=351.69
test 157 : PoissonBEM [PASSED], CPU time=351.79
test 158 : PoissonBoltzmann [PASSED], CPU time=356.47
test 159 : PoissonDG [PASSED], CPU time=356.66
test 160 : PoissonPFEM [PASSED], CPU time=356.74
test 161 : PorousPipe [PASSED], CPU time=357.54
test 162 : Q1Q0 [PASSED], CPU time=357.7
test 163 : radiation [PASSED], CPU time=357.9
test 164 : radiation2 [PASSED], CPU time=358.09
test 165 : radiation2d [PASSED], CPU time=360.01
test 166 : radiation3d [PASSED], CPU time=364.17
test 167 : reload [PASSED], CPU time=364.97
test 168 : reynolds1 [PASSED], CPU time=367.86
test 169 : reynolds2 [PASSED], CPU time=368.55
test 170 : reynolds3 [PASSED], CPU time=373.73
test 171 : reynolds3b [PASSED], CPU time=378.24
test 172 : rgdblock [PASSED], CPU time=380.04
test 173 : RichardsDyke [PASSED], CPU time=384.67
test 174 : RichardsDyke2 [PASSED], CPU time=394.69
test 175 : RigidMeshMapper1 [PASSED], CPU time=395.36
test 176 : RigidMeshMapper2 [PASSED], CPU time=395.73
test 177 : rot_aniso [PASSED], CPU time=399.02
test 178 : RotatingFlow [PASSED], CPU time=399.8
test 179 : rotflow [PASSED], CPU time=400.55
test 180 : savescalars [PASSED], CPU time=400.84
test 181 : savescalars_boundary [PASSED], CPU time=400.96
test 182 : savescalars_flux [PASSED], CPU time=401.05
test 183 : ShallowWaterNS [PASSED], CPU time=403.49
test 184 : shell [PASSED], CPU time=403.71
test 185 : shell2 [PASSED], CPU time=404.31
test 186 : staged_sim [PASSED], CPU time=405.64
test 187 : Step_ke [PASSED], CPU time=407.57
test 188 : Step_ns [PASSED], CPU time=408
test 189 : Step_sa [PASSED], CPU time=416.5
test 190 : Step_sst-kw-wf [PASSED], CPU time=434.12
test 191 : Step_stokes [PASSED], CPU time=434.24
test 192 : Step_stokes_block [PASSED], CPU time=434.45
test 193 : Step_v2f [PASSED], CPU time=458.62
test 194 : StokesPFEM [PASSED], CPU time=458.75
test 195 : StokesProj [PASSED], CPU time=459.18
test 196 : StrainCalculation01 [PASSED], CPU time=459.46
test 197 : StrainCalculation02 [PASSED], CPU time=465.41
test 198 : StrainCalculation03 [PASSED], CPU time=472.1
test 199 : streamlines [PASSED], CPU time=472.44
test 200 : stress [PASSED], CPU time=472.74
test 201 : structmap [PASSED], CPU time=472.85
test 202 : structmap2 [PASSED], CPU time=474.1
test 203 : structmap3 [PASSED], CPU time=486.62
test 204 : ThermalActuator [PASSED], CPU time=551.68
test 205 : ThermalBiMetal [PASSED], CPU time=551.84
test 206 : ThermalBiMetal2 [PASSED], CPU time=552.04
test 207 : ThermalCompress [PASSED], CPU time=552.89
test 208 : ThermoElectric [PASSED], CPU time=553.02
test 209 : TimeAdapt [PASSED], CPU time=555.13
test 210 : TimeFunc [PASSED], CPU time=557.59
test 211 : tresca [PASSED], CPU time=557.76
test 212 : vortex2d [PASSED], CPU time=558.9
test 213 : vortex3d [PASSED], CPU time=560.19
test 214 : WaveEqu [PASSED], CPU time=560.33
test 215 : WeightComputation [PASSED], CPU time=560.41
Tests completed, passed: 215 out of total 215 tests
Cumulative CPU time used in test: 560.41 s
joni@mpi1:/mpi3/elmerfem/trunk/fem/tests$
Netgen parraler
NOT TESTED
Source:<a href="http://code.google.com/p/parallel-netgen/" target="_blank"> netgen-paraler</a>
UNDER TEST:
apt-get libtogl1 libtogl-dev "tcl" "tix" "occ" "jpeglib" "ffmpeglib"
http://sourceforge.net/projects/netgen-mesher/files/latest/download?source=files
./configure --enable-parallel --with-tcl=/usr/lib/tcl8.5 --with-tk=/usr/lib/tk8.5 --with-occ=/usr/lib/ --with-togl=/usr/lib --enable-jpeglib --enable-ffmpeg
build instruction:
http://sourceforge.net/apps/mediawiki/netgen-mesher/index.php?title=Installing_on_Ubuntu
http://glaros.dtc.umn.edu/gkhome/metis/parmetis/download
<img src="https://www.google.com/chart?chc=sites&cht=d&chdp=sites&chl=%5B%5BGoogle+Gadget%27%3D20%27f%5Cv%27a%5C%3D0%2710%27%3D499%270%27dim%27%5Cbox1%27b%5CF6F6F6%27fC%5CF6F6F6%27eC%5C0%27sk%27%5C%5B%22Google+Group%22%27%5D%27a%5CV%5C%3D12%27f%5C%5DV%5Cta%5C%3D10%27%3D0%27%3D500%27%3D597%27dim%27%5C%3D10%27%3D10%27%3D500%27%3D597%27vdim%27%5Cbox1%27b%5Cva%5CF6F6F6%27fC%5CC8C8C8%27eC%5C%27a%5C%5Do%5CLauto%27f%5C&sig=DUR0sa0cgFwYKZ2Gld2wIgZ7yrE" data-igsrc="http://153.gmodules.com/ig/ifr?mid=153&synd=trogedit&url=http%3A%2F%2Fwww.gstatic.com%2Fsites-gadgets%2Fforum%2Fsingle_group.xml&up_GroupUrl=https%3A%2F%2Fgroups.google.com%2Fforum%2F%23!forum%2Fubuntuparraler&up_ShowSearch=true&up_ShowPopout=true&up_Transparency=false&h=600&w=100%25" data-type="ggs-gadget" data-props="align:left;borderTitle:Google Group Ubuntu Parraler;height:600;igsrc:http#58//153.gmodules.com/ig/ifr?mid=153&synd=trogedit&url=http%3A%2F%2Fwww.gstatic.com%2Fsites-gadgets%2Fforum%2Fsingle_group.xml&up_GroupUrl=https%3A%2F%2Fgroups.google.com%2Fforum%2F%23!forum%2Fubuntuparraler&up_ShowSearch=true&up_ShowPopout=true&up_Transparency=false&h=600&w=100%25;mid:153;scrolling:no;showBorder:true;showBorderTitle:true;spec:http#58//www.gstatic.com/sites-gadgets/forum/single_group.xml;up_GroupUrl:https#58//groups.google.com/forum/#35!forum/ubuntuparraler;up_ShowPopout:true;up_ShowSearch:true;up_Transparency:false;width:100%;" style="display:block;text-align:left;margin-right:auto;" class="igm" height="600" width="500">
<img src="https://www.google.com/chart?chc=sites&cht=d&chdp=sites&chl=%5B%5BGoogle+Gadget%27%3D20%27f%5Cv%27a%5C%3D0%2710%27%3D119%270%27dim%27%5Cbox1%27b%5CF6F6F6%27fC%5CF6F6F6%27eC%5C0%27sk%27%5C%5B%22Include+gadget+(iframe)%22%27%5D%27a%5CV%5C%3D12%27f%5C%5DV%5Cta%5C%3D10%27%3D0%27%3D120%27%3D237%27dim%27%5C%3D10%27%3D10%27%3D120%27%3D237%27vdim%27%5Cbox1%27b%5Cva%5CF6F6F6%27fC%5CC8C8C8%27eC%5C%27a%5C%5Do%5CLauto%27f%5C&sig=gUycQn5kgb7bG9PkOC0UpjtwoTI" data-igsrc="http://0.gmodules.com/ig/ifr?mid=0&synd=trogedit&url=http%3A%2F%2Fwww.gstatic.com%2Fsites-gadgets%2Fiframe%2Fiframe.xml&up_iframeURL=http%3A%2F%2Frcm.amazon.com%2Fe%2Fcm%3Ft%3Dprodurevie049-20%26o%3D1%26p%3D8%26l%3Das1%26asins%3DB000LB7G00%26ref%3Dqf_sp_asin_til%26fc1%3D000000%26IS2%3D1%26lt1%3D_blank%26m%3Damazon%26lc1%3D0000FF%26bc1%3D000000%26bg1%3DFFFFFF%26f%3Difr&up_scroll=no&w=120px&h=240px" data-type="ggs-gadget" data-props="align:left;height:240px;igsrc:http#58//0.gmodules.com/ig/ifr?mid=0&synd=trogedit&url=http%3A%2F%2Fwww.gstatic.com%2Fsites-gadgets%2Fiframe%2Fiframe.xml&up_iframeURL=http%3A%2F%2Frcm.amazon.com%2Fe%2Fcm%3Ft%3Dprodurevie049-20%26o%3D1%26p%3D8%26l%3Das1%26asins%3DB000LB7G00%26ref%3Dqf_sp_asin_til%26fc1%3D000000%26IS2%3D1%26lt1%3D_blank%26m%3Damazon%26lc1%3D0000FF%26bc1%3D000000%26bg1%3DFFFFFF%26f%3Difr&up_scroll=no&w=120px&h=240px;mid:0;spec:http#58//www.gstatic.com/sites-gadgets/iframe/iframe.xml;up_iframeURL:http#58//rcm.amazon.com/e/cm?t=produrevie049-20&o=1&p=8&l=as1&asins=B000LB7G00&ref=qf_sp_asin_til&fc1=000000&IS2=1<1=_blank&m=amazon&lc1=0000FF&bc1=000000&bg1=FFFFFF&f=ifr;up_scroll:no;width:120px;wrap:true;" style="display:inline;float:left;margin:5px 20px 5px 0;" class="igm" height="240" width="120">
<img src="https://www.google.com/chart?chc=sites&cht=d&chdp=sites&chl=%5B%5BGoogle+Gadget%27%3D20%27f%5Cv%27a%5C%3D0%2710%27%3D119%270%27dim%27%5Cbox1%27b%5CF6F6F6%27fC%5CF6F6F6%27eC%5C0%27sk%27%5C%5B%22Include+gadget+(iframe)%22%27%5D%27a%5CV%5C%3D12%27f%5C%5DV%5Cta%5C%3D10%27%3D0%27%3D120%27%3D237%27dim%27%5C%3D10%27%3D10%27%3D120%27%3D237%27vdim%27%5Cbox1%27b%5Cva%5CF6F6F6%27fC%5CC8C8C8%27eC%5C%27a%5C%5Do%5CLauto%27f%5C&sig=gUycQn5kgb7bG9PkOC0UpjtwoTI" data-igsrc="http://0.gmodules.com/ig/ifr?mid=0&synd=trogedit&url=http%3A%2F%2Fwww.gstatic.com%2Fsites-gadgets%2Fiframe%2Fiframe.xml&up_iframeURL=http%3A%2F%2Frcm.amazon.com%2Fe%2Fcm%3Ft%3Dprodurevie049-20%26o%3D1%26p%3D8%26l%3Das1%26asins%3DB006H2RP12%26ref%3Dqf_sp_asin_til%26fc1%3D000000%26IS2%3D1%26lt1%3D_blank%26m%3Damazon%26lc1%3D0000FF%26bc1%3D000000%26bg1%3DFFFFFF%26f%3Difr&up_scroll=no&w=120px&h=240px" data-type="ggs-gadget" data-props="align:left;height:240px;igsrc:http#58//0.gmodules.com/ig/ifr?mid=0&synd=trogedit&url=http%3A%2F%2Fwww.gstatic.com%2Fsites-gadgets%2Fiframe%2Fiframe.xml&up_iframeURL=http%3A%2F%2Frcm.amazon.com%2Fe%2Fcm%3Ft%3Dprodurevie049-20%26o%3D1%26p%3D8%26l%3Das1%26asins%3DB006H2RP12%26ref%3Dqf_sp_asin_til%26fc1%3D000000%26IS2%3D1%26lt1%3D_blank%26m%3Damazon%26lc1%3D0000FF%26bc1%3D000000%26bg1%3DFFFFFF%26f%3Difr&up_scroll=no&w=120px&h=240px;mid:0;spec:http#58//www.gstatic.com/sites-gadgets/iframe/iframe.xml;up_iframeURL:http#58//rcm.amazon.com/e/cm?t=produrevie049-20&o=1&p=8&l=as1&asins=B006H2RP12&ref=qf_sp_asin_til&fc1=000000&IS2=1<1=_blank&m=amazon&lc1=0000FF&bc1=000000&bg1=FFFFFF&f=ifr;up_scroll:no;width:120px;wrap:true;" style="display:inline;float:left;margin:5px 20px 5px 0;" class="igm" height="240" width="120"><img src="https://www.google.com/chart?chc=sites&cht=d&chdp=sites&chl=%5B%5BGoogle+Gadget%27%3D20%27f%5Cv%27a%5C%3D0%2710%27%3D119%270%27dim%27%5Cbox1%27b%5CF6F6F6%27fC%5CF6F6F6%27eC%5C0%27sk%27%5C%5B%22Include+gadget+(iframe)%22%27%5D%27a%5CV%5C%3D12%27f%5C%5DV%5Cta%5C%3D10%27%3D0%27%3D120%27%3D237%27dim%27%5C%3D10%27%3D10%27%3D120%27%3D237%27vdim%27%5Cbox1%27b%5Cva%5CF6F6F6%27fC%5CC8C8C8%27eC%5C%27a%5C%5Do%5CLauto%27f%5C&sig=gUycQn5kgb7bG9PkOC0UpjtwoTI" data-igsrc="http://0.gmodules.com/ig/ifr?mid=0&synd=trogedit&url=http%3A%2F%2Fwww.gstatic.com%2Fsites-gadgets%2Fiframe%2Fiframe.xml&up_iframeURL=http%3A%2F%2Frcm.amazon.com%2Fe%2Fcm%3Ft%3Dprodurevie049-20%26o%3D1%26p%3D8%26l%3Das1%26asins%3DB0016OOS16%26ref%3Dqf_sp_asin_til%26fc1%3D000000%26IS2%3D1%26lt1%3D_blank%26m%3Damazon%26lc1%3D0000FF%26bc1%3D000000%26bg1%3DFFFFFF%26f%3Difr&up_scroll=no&w=120px&h=240px" data-type="ggs-gadget" data-props="align:left;height:240px;igsrc:http#58//0.gmodules.com/ig/ifr?mid=0&synd=trogedit&url=http%3A%2F%2Fwww.gstatic.com%2Fsites-gadgets%2Fiframe%2Fiframe.xml&up_iframeURL=http%3A%2F%2Frcm.amazon.com%2Fe%2Fcm%3Ft%3Dprodurevie049-20%26o%3D1%26p%3D8%26l%3Das1%26asins%3DB0016OOS16%26ref%3Dqf_sp_asin_til%26fc1%3D000000%26IS2%3D1%26lt1%3D_blank%26m%3Damazon%26lc1%3D0000FF%26bc1%3D000000%26bg1%3DFFFFFF%26f%3Difr&up_scroll=no&w=120px&h=240px;mid:0;spec:http#58//www.gstatic.com/sites-gadgets/iframe/iframe.xml;up_iframeURL:http#58//rcm.amazon.com/e/cm?t=produrevie049-20&o=1&p=8&l=as1&asins=B0016OOS16&ref=qf_sp_asin_til&fc1=000000&IS2=1<1=_blank&m=amazon&lc1=0000FF&bc1=000000&bg1=FFFFFF&f=ifr;up_scroll:no;width:120px;wrap:true;" style="display:inline;float:left;margin:5px 20px 5px 0;" class="igm" height="240" width="120">
