We usually have a weather station connected to pcfsyg. However, if it is unavailable or broken, there is another weather sensor (used for the SLR) and it is possible to extract the data and merge it with the PCFS logs before they are submitted. Below is the procedure for doing this (a detailed description as well as a summary). Note that the external logs are updated every 30 minutes, so the Yg logs can only be processed 30 mins or more after the experiment finishes.

On ops2 in /home/observer/ask/yglogs lives a script called yg_wx_logs.py which can be used to merge the logs with the weather data. To run the script type in

 python yg_wx_logs.py 

The script will prompt the user for

• Experiment name,
• Date of the experiment,
• Start time,
• End time.

With the given information, the script will

• Download the required mets files containing the weather data using sftp,
• Download the log file from pcfyg,
• Merge the log from pcfs with the downloaded weather data,
• Backup the original log on pcfsyg with a _original2.log,
• Copy the modified log back to the pcfsyg.

Once all that is done you can run the flogit script to transfer that log to the IVS servers. As usual please eyeball the log to make sure the script has done the right thing.

It's probably best to modify the PCFS log on pcfsyg, then follow the usual end-of-experiment procedure. This way the log file will be the same on pcfsyg, ops2, and on the IVS servers. The procedure below uses the experiment r4750 as an example:

1. Identify the start date of the experiment. There's a separate log of weather for every UT day, so you'll need to retrieve the log for the start day as well as the following day. In the case of R4750, the start date was July 28 (day 210) at 18:30 UT, so we need logs from July 28 and 29.
2. Once the experiment is complete (and it's more than 30 minutes from the end), open a terminal on pcfsyg. Do this in the pcfsyg VNC session, especially if you want to plot the data, which is an option later.

3. cd /usr2/log/wx2

4. The weather data are on an sftp server at Yg. Use the usual observer password. The file naming format is mets_<yyyy>_<mm>_<dd>.dat so we want mets_2016_07_28.dat and mets_2016_07_29.dat

   sftp -oPort=26 observer@59.167.111.242
   cd metdata
   dir
   get mets_2016_07_28.dat
   get mets_2016_07_29.dat
   quit

5. The weather data files now need to be converted to the correct format for a PCFS log. There's a script to do that called metdat_format.sh. By default it writes the output log to the screen, so send it to a temporary file. e.g.

   metdat_format.sh mets_2016_07_28.dat mets_2016_07_29.dat > wx_tmp.log

   Specify the input data files in chronological order.

6. Now you need to edit the new log file and delete any weather measurements before or after the experiment. There's a script for that too called metdat_startstop.sh. Usage is

   metdat_startstop.sh <input_file_name> <start_time> <end_time>
   Where the time format is ddd.hh:mm. The output from this should be ready to be merged into the original log file, so give it a sensible output name and put it in the log directory (''/usr2/log''). 

   For example for R4750:

   metdat_startstop.sh wx_tmp.log 210.18:30 211.18:30 > /usr2/log/wx_r4750yg.log

7. You can now merge the original log file and the weather data. Change to the /usr2/log directory

   cd /usr2/log

8. Check that the weather data start and stop at the right time. Show the first 5 lines

   head -5 wx_r4750yg.log

   Show the last 5 lines

   tail -5 wx_r4750yg.log

   If something is amiss, you may need to go back to the start and work out what happened. You can also plot the weather data quite easily:

   metdat_plot_weather.sh wx_r4750yg.log

9. It's a good idea to make a backup of the original log first

   cp r4750yg.log r4750yg_original.log

10. Processing the log is a two step process.
    1. First, you want to remove any existing weather data from the log as it will be blank or contain errors

       grep -v /wx/ r4750yg.log > r4750yg_no_wx.log

    2. Next merge the field system and weather logs to produce a final version:

       sort -s -m -k 1,1.20 -o r4750yg.log r4750yg_no_wx.log wx_r4750yg.log

11. Finally, make a few spot checks of the final log by eye to see if it looks OK. Hunt around for the wx lines and see if they look right and are in the right place.
12. You can now go back to ops2 and process the log file and submit a end email as usual. Please add a note to the email to say that weather data were merged into the log after the experiment.

This assumes R4750 is being processed.

1. Identify the start date of the experiment.
2. After the experiment AND 0 UT open a terminal in the pcfsyg VNC session.

3. cd /usr2/log/wx2
   sftp -oPort=26 observer@59.167.111.242
   cd metdata
   dir
   get mets_2016_07_28.dat
   get mets_2016_07_29.dat
   quit
   metdat_format.sh mets_2016_07_28.dat mets_2016_07_29.dat > wx_tmp.log
   metdat_startstop.sh wx_tmp.log 210.18:30 211.18:30 > /usr2/log/wx_r4750yg.log
   cd /usr2/log
   head -5 wx_r4750yg.log
   tail -5 wx_r4750yg.log
   metdat_plot_weather.sh wx_r4750yg.log
   cp r4750yg.log r4750yg_original.log
   grep -v /wx/ r4750yg.log > r4750yg_no_wx.log
   sort -s -m -k 1,1.20 -o r4750yg.log r4750yg_no_wx.log wx_r4750yg.log

4. Make a few spot checks of the final log by eye
5. You can now go back to ops2 and process the log file and submit a end email as usual. Please add a note to the email to say that weather data were merged into the log after the experiment.