Configuring the Flexbuff

• Main difference is that the OS system disk is provided by two SSDs attached to the mainboard SATA interface rather than through the RAID card.
• To install, boot the flexbuff and after the megaRAID configuration is complete, enter the “INTEL STORAGE TECHNOLOGY” bios with “ctrl-i”. From there, unselect the drives from their currrent RAID-1 configuration (as this is not visible to the debian installer). Then reboot
• For VBS recordings, please ensure that the JBOD configuration is selected in the megaRAID controller as the default setting for unconfigured good drives, and that any existing RAID configurations are removed.
• When installing debian (stretch), if you need to create a new partition table for the OS disks do not use GPT - this (when combined with RAID) is incompatible with setting the bootable flag and/or installing grub. Use a DOS partition table, mark as for linux RAID & bootable, install the OS to the software RAID and install grub to one or both SSDs.
• In the main BIOS, make sure that the HDD boot priority is set to use the SSDs rather than the megaRAID drives.
• After installing debian, edit /etc/apt/sources.list to use “main contrib non-free” sources throughout. Run apt-get update and then apt-get install libfuse-dev emacs nfs-kernel-server tightvncserver libfftw3-dev python iftop iotop wireshark make automake build-essential gfortran x11vnc nfs-kernel-server nfs-common gparted ntp ntpdate pkg-config g++ gfortran bison flex libexpat1-dev fftw3-dev autoconf automake libtool build-essential pgplot5 subversion libopenmpi-dev gv openmpi-bin rpcbind ethtool libpng-dev libx11-dev mlocate bc dnsutils rsync xfsprogs ghostscript-x libgsl0-dev
• This should build in the pre-requisites for DiFX, fourfit, jive5ab, etc.

• Edit the /etc/exports file to make any relevant mount points NFS-accessible
• Edit /etc/network/interfaces to set up the networking appropriately. Make a note of how the physical ports map to the names & use ifconfig to get the MAC addresses (as these are needed by the DBBC3)
• Download the latest jive version (e.g wget http://www.jive.nl/~verkout/evlbi/jive5ab-2.9.0.tar.gz, untar and run make B2B=64 and make B2B=64 install (the last as root)

• Edit the /etc/sysctl.conf file to add in the following lines

net.core.wmem_max=201326592
net.core.rmem_max=1073741824
net.ipv4.conf.all.arp_filter = 1
net.core.netdev_max_backlog = 20971752
net.ipv4.udp_mem = 65536 131072 262144
net.core.optmem_max = 50331648 
vfs_cache_pressure = 50
vm.min_free_kbytes = 1048576
kernel.perf_event_max_sample_rate=10000

• Use gparted or the like to install a GPT partition table on each of the VBS drives and add an XFS partition on each.
• Run for i in $(seq 0 35); do mkdir /mnt/disk$i ;chown -R observer /mnt/disk$i ;done to create the mount points. For the HDDs in VBS mode, the mapping between physical location and disk ID is not simple. Copy across the MountAllFlexBuff.sh script from flexbuffhb to see the recommended method, using the by-path characteristics to identify the drives.
• As root, download and install the latest vbsfs file (e.g wget http://www.jive.nl/~verkout/flexbuff/vbsfs-r57.deb ; dpkg -i vbsfs-r57.deb
• The Intel IPP libraries are now freely available. Early 2017 vintages are available on flexbuffhb, a mid-2018 version is on ops1hb (~/Downloads/l_ipp_2018.3.222.tgz). Installation should be as simple as untarring the file and running install.sh
• Install DiFX via svn co https://svn.atnf.csiro.au/difx/master_tags/DiFX-2.5.2, edit setup.bash to provide the correct paths (see flexbuffhb for an example) and then run ./install-difx –withpolconvert
• As observer, create the HOPS subdirectory and download the latest tarball to this directory e.g cd ; mkdir HOPS; cd HOPS; wget ftp://gemini.haystack.mit.edu/pub/hops/hops-3.18-2251.tar.gz; tar zxvf hops-3.18-2251.tar.gz; cd hops-3.18/ ;
• NB - in Debian 9, the libpng12-dev package has been removed while HOPS explicitly links against it. You can work around this by editing the configure script and changing all references to -lpng12 to -lpng. Then you should be able to configure & build the software with ./configure ; make
• If all goes well, then use make install as su and then run make check

• Lastly, edit the ~/.bashrc file to fix some issues and to link the newly installed programs.
  • Comment out the “If not running interactively” logic - this interferes with openmpi
  • export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib/
  • . ~/DiFX-2.5.2/setup.bash
  • export CALCDB=/home/observer/DiFX-2.5.2/applications/calcserver/trunk/calc9.1
  • export PATH=${DIFXROOT}/bin:${PATH}
  • export HOPS_QUIET=1
  • . HOPS/hops-3.18/hops.bash
  • export PGPLOT_DEV=“/xs”
  • export HOPS_STATION_CODE=/home/observer/hops_station_code.txt
  • export DEF_CONTROL=~/cf_def