This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
hardware:flexbuff [2018/07/02 00:32] Warren Hankey [2018 Version] |
hardware:flexbuff [2018/07/26 01:48] Warren Hankey [2018 Version] |
||
---|---|---|---|
Line 8: | Line 8: | ||
* When installing debian (stretch), if you need to create a new partition table for the OS disks do not use GPT - this (when combined with RAID) is incompatible with setting the bootable flag and/or installing grub. Use a DOS partition table, mark as for linux RAID & bootable, install the OS to the software RAID and install grub to one or both SSDs. | * When installing debian (stretch), if you need to create a new partition table for the OS disks do not use GPT - this (when combined with RAID) is incompatible with setting the bootable flag and/or installing grub. Use a DOS partition table, mark as for linux RAID & bootable, install the OS to the software RAID and install grub to one or both SSDs. | ||
* In the main BIOS, make sure that the HDD boot priority is set to use the SSDs rather than the megaRAID drives. | * In the main BIOS, make sure that the HDD boot priority is set to use the SSDs rather than the megaRAID drives. | ||
- | * After installing debian, edit /etc/apt/sources.list to use "main contrib non-free" sources throughout. Run ''apt-get update'' and then ''apt-get install emacs nfs-kernel-server tightvncserver libfftw3-dev python iftop iotop wireshark make automake build-essential gfortran x11vnc nfs-kernel-server nfs-common gparted pkg-config g++ gfortran bison flex libexpat1-dev fftw3-dev autoconf automake libtool build-essential pgplot5 subversion libopenmpi-dev openmpi-bin rpcbind ethtool libpng-dev'' | + | * After installing debian, edit /etc/apt/sources.list to use "main contrib non-free" sources throughout. Run ''apt-get update'' and then ''apt-get install libfuse-dev emacs nfs-kernel-server tightvncserver libfftw3-dev python iftop iotop wireshark make automake build-essential gfortran x11vnc nfs-kernel-server nfs-common gparted ntp ntpdate pkg-config g++ gfortran bison flex libexpat1-dev fftw3-dev autoconf automake libtool build-essential pgplot5 subversion libopenmpi-dev openmpi-bin rpcbind ethtool libpng-dev libx11-dev mlocate bc dnsutils rsync xfsprogs'' |
* This should build in the pre-requisites for DiFX, fourfit, jive5ab, etc. | * This should build in the pre-requisites for DiFX, fourfit, jive5ab, etc. | ||
+ | |||
+ | * Edit the /etc/exports file to make any relevant mount points NFS-accessible | ||
+ | * Edit /etc/network/interfaces to set up the networking appropriately. Make a note of how the physical ports map to the names & use ifconfig to get the MAC addresses (as these are needed by the DBBC3) | ||
+ | * Download the latest jive version (e.g ''wget http://www.jive.nl/~verkout/evlbi/jive5ab-2.9.0.tar.gz'', untar and run ''make B2B=64'' and ''make B2B=64 install'' (the last as root) | ||
* Edit the /etc/sysctl.conf file to add in the following lines | * Edit the /etc/sysctl.conf file to add in the following lines | ||
- | <code> net.core.wmem_max=201326592 | + | <code>net.core.wmem_max=201326592 |
net.core.rmem_max=1073741824 | net.core.rmem_max=1073741824 | ||
net.ipv4.conf.all.arp_filter = 1 | net.ipv4.conf.all.arp_filter = 1 | ||
Line 22: | Line 26: | ||
vm.min_free_kbytes = 1048576 | vm.min_free_kbytes = 1048576 | ||
kernel.perf_event_max_sample_rate=10000 | kernel.perf_event_max_sample_rate=10000 | ||
- | <\code> | + | </code> |
+ | |||
+ | * Use gparted or the like to install a GPT partition table on each of the VBS drives and add an XFS partition on each. | ||
+ | * Run ''for i in $(seq 0 35); do mkdir /mnt/disk$i ;chown -R observer /mnt/disk$i ;done'' to create the mount points. For the HDDs in VBS mode, the mapping between physical location and disk ID is not simple. Copy across the MountAllFlexBuff.sh script from flexbuffhb to see the recommended method, using the ''by-path'' characteristics to identify the drives. | ||
+ | * As root, download and install the latest vbsfs file (e.g ''wget http://www.jive.nl/~verkout/flexbuff/vbsfs-r57.deb ; dpkg -i vbsfs-r57.deb'' | ||
+ | * The Intel IPP libraries are now freely available. Early 2017 vintages are available on flexbuffhb, a mid-2018 version is on ops1hb (~/Downloads/l_ipp_2018.3.222.tgz). Installation should be as simple as untarring the file and running install.sh | ||
+ | * Install DiFX via ''svn co https://svn.atnf.csiro.au/difx/master_tags/DiFX-2.5.2'', edit setup.bash to provide the correct paths (see flexbuffhb for an example) and then run ''./install-difx --withpolconvert'' | ||
+ | * As observer, create the HOPS subdirectory and download the latest tarball to this directory e.g ''cd ; mkdir HOPS; cd HOPS; wget ftp://gemini.haystack.mit.edu/pub/hops/hops-3.18-2251.tar.gz; tar zxvf hops-3.18-2251.tar.gz; cd hops-3.18/ ;'' | ||
+ | * NB - in Debian 9, the libpng12-dev package has been removed while HOPS explicitly links against it. You can work around this by editing the configure script and changing all references to ''-lpng12'' to ''-lpng''. Then you should be able to configure & build the software with ''./configure ; make'' | ||
+ | * If all goes well, then use ''make install'' as su and then run ''make check'' | ||
+ | |||
+ | * Lastly, edit the ''~/.bashrc'' file to fix some issues and to link the newly installed programs. | ||
+ | * Comment out the "If not running interactively" logic - this interferes with openmpi | ||
+ | * export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib/ | ||
+ | * . ~/DiFX-2.5.2/setup.bash | ||
+ | * export CALCDB=/home/observer/DiFX-2.5.2/applications/calcserver/trunk/calc9.1 | ||
+ | * export PATH=${DIFXROOT}/bin:${PATH} | ||
+ | * export HOPS_QUIET=1 | ||
+ | * . HOPS/hops-3.18/hops.bash | ||
+ | * export PGPLOT_DEV="/xs" | ||
+ | * export HOPS_STATION_CODE=/home/observer/hops_station_code.txt | ||
+ | * export DEF_CONTROL=~/cf_def |