User Tools
operations:documentation:yg_manual_weather
This wiki is not maintained! Do not use this when setting up AuScope experiments!
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision Last revision Both sides next revision | ||
|
operations:documentation:yg_manual_weather [2016/08/11 00:24] Jim Lovell [Manual extraction and logging of Yarragadee weather data] |
operations:documentation:yg_manual_weather [2016/11/19 02:26] Ross Turner [Using a script] |
||
|---|---|---|---|
| Line 9: | Line 9: | ||
| - Identify the start date of the experiment. There's a separate log of weather for every UT day, so you'll need to retrieve the log for the start day as well as the following day. In the case of R4750, the start date was July 28 (day 210) at 18:30 UT, so we need logs from July 28 and 29. | - Identify the start date of the experiment. There's a separate log of weather for every UT day, so you'll need to retrieve the log for the start day as well as the following day. In the case of R4750, the start date was July 28 (day 210) at 18:30 UT, so we need logs from July 28 and 29. | ||
| - | - Once the experiment is complete and the time is past 0 UT, open a terminal on pcfsyg. Do this in the pcfsyg VNC session, especially if you want to plot the data, which is an option later. | + | - Once the experiment is complete (and it's more than 30 minutes from the end), open a terminal on pcfsyg. Do this in the pcfsyg VNC session, especially if you want to plot the data, which is an option later. |
| - <code>cd /usr2/log/wx2</code> | - <code>cd /usr2/log/wx2</code> | ||
| - The weather data are on an sftp server at Yg. Use the usual observer password. The file naming format is ''mets_<yyyy>_<mm>_<dd>.dat'' so we want ''mets_2016_07_28.dat'' and ''mets_2016_07_29.dat'' <code>sftp -oPort=26 observer@59.167.111.242 | - The weather data are on an sftp server at Yg. Use the usual observer password. The file naming format is ''mets_<yyyy>_<mm>_<dd>.dat'' so we want ''mets_2016_07_28.dat'' and ''mets_2016_07_29.dat'' <code>sftp -oPort=26 observer@59.167.111.242 | ||
| Line 18: | Line 18: | ||
| quit | quit | ||
| </code> | </code> | ||
| - | - The weather data files now need to be converted to the correct format for a PCFS log. There's a script to do that called ''metadat_format.sh''. By default it writes the output log to the screen, so send it to a temporary file. e.g. <code>metadat_format.sh mets_2016_07_28.dat mets_2016_07_29.dat > wx_tmp.log</code>Specify the input data files in chronological order. | + | - The weather data files now need to be converted to the correct format for a PCFS log. There's a script to do that called ''metdat_format.sh''. By default it writes the output log to the screen, so send it to a temporary file. e.g. <code>metdat_format.sh mets_2016_07_28.dat mets_2016_07_29.dat > wx_tmp.log</code>Specify the input data files in chronological order. |
| - Now you need to edit the new log file and delete any weather measurements before or after the experiment. There's a script for that too called ''metdat_startstop.sh''. Usage is <code>metdat_startstop.sh <input_file_name> <start_time> <end_time> | - Now you need to edit the new log file and delete any weather measurements before or after the experiment. There's a script for that too called ''metdat_startstop.sh''. Usage is <code>metdat_startstop.sh <input_file_name> <start_time> <end_time> | ||
| Where the time format is ddd.hh:mm. The output from this should be ready to be merged into the original log file, so give it a sensible output name and put it in the log directory (''/usr2/log''). </code> For example for R4750: <code>metdat_startstop.sh wx_tmp.log 210.18:30 211.18:30 > /usr2/log/wx_r4750yg.log</code> | Where the time format is ddd.hh:mm. The output from this should be ready to be merged into the original log file, so give it a sensible output name and put it in the log directory (''/usr2/log''). </code> For example for R4750: <code>metdat_startstop.sh wx_tmp.log 210.18:30 211.18:30 > /usr2/log/wx_r4750yg.log</code> | ||
| Line 42: | Line 42: | ||
| get mets_2016_07_29.dat | get mets_2016_07_29.dat | ||
| quit | quit | ||
| - | metadat_format.sh mets_2016_07_28.dat mets_2016_07_29.dat > wx_tmp.log | + | metdat_format.sh mets_2016_07_28.dat mets_2016_07_29.dat > wx_tmp.log |
| metdat_startstop.sh wx_tmp.log 210.18:30 211.18:30 > /usr2/log/wx_r4750yg.log | metdat_startstop.sh wx_tmp.log 210.18:30 211.18:30 > /usr2/log/wx_r4750yg.log | ||
| cd /usr2/log | cd /usr2/log | ||
| Line 54: | Line 54: | ||
| - You can now go back to ops2 and [[operations:documentation.ivs.logs|process the log file and submit a end email as usual]]. Please add a note to the email to say that weather data were merged into the log after the experiment. | - You can now go back to ops2 and [[operations:documentation.ivs.logs|process the log file and submit a end email as usual]]. Please add a note to the email to say that weather data were merged into the log after the experiment. | ||
| + | ===== Using a script ===== | ||
| + | On ops2 in ''/home/observer/ask/yglogs'' lives a script called ''yg_wx_logs.py'' which can be used to merge the logs with the weather data. To run the script type in | ||
| + | <code> python yg_wx_logs.py </code> | ||
| + | |||
| + | The script will prompt the user for | ||
| + | * Experiment name, | ||
| + | * Date of the experiment, | ||
| + | * Start time, | ||
| + | * End time. | ||
| + | |||
| + | With the given information, the script will | ||
| + | |||
| + | * Download the required mets files containing the weather data using sftp, | ||
| + | * Download the log file from pcfyg, | ||
| + | * Merge the log from pcfs with the downloaded weather data, | ||
| + | * Backup the original log on pcfsyg with a ''_original2.log'', | ||
| + | * Copy the modified log back to the pcfsyg. | ||
| + | |||
| + | Once all that is done you can run the flogit script to transfer that log to the IVS servers. As usual please eyeball the log to make sure the script has done the right thing. | ||
/home/www/auscope/opswiki/data/pages/operations/documentation/yg_manual_weather.txt · Last modified: 2016/11/23 21:31 by Bryn Emptage
Page Tools
