User Tools

Site Tools


analysis:fourfit

This wiki is not maintained! Do not use this when setting up AuScope experiments!

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
Next revision Both sides next revision
analysis:fourfit [2014/01/19 23:53]
Jamie McCallum
analysis:fourfit [2015/06/03 00:32]
Jamie McCallum [Phase Cal estimation]
Line 5: Line 5:
 Make a directory and download the tarball from CURTIN into this directory [''​mkdir AUST10 ; cd AUST10; wget -c ftp://​cira.ivec.org/​aust10/​AUST10.MARK4.tar''​] Make a directory and download the tarball from CURTIN into this directory [''​mkdir AUST10 ; cd AUST10; wget -c ftp://​cira.ivec.org/​aust10/​AUST10.MARK4.tar''​]
  
-Untar the AUST10.MARK4.tar file to create the 1234 directory. Copy the ~/cf_aust template file into the 1234 directoey ​as cf_1234 [''​cp ~/cf_aust ~/​AUST10/​1234/''​]+Untar the AUST10.MARK4.tar file to create the 1234 directory. Copy the ~/cf_aust template file into the 1234 directory ​as cf_1234 [''​cp ~/cf_aust ~/​AUST10/​1234/''​]
  
 Change to the 1234 directory and create a list of scans with ''​cd ~/​AUST10/​1234/​ ; ls -d 219-* 220* > /​tmp/​scans''​ (Replacing the 219 and 220 with the appropriate DOYs for the experiment. ​ Change to the 1234 directory and create a list of scans with ''​cd ~/​AUST10/​1234/​ ; ls -d 219-* 220* > /​tmp/​scans''​ (Replacing the 219 and 220 with the appropriate DOYs for the experiment. ​
Line 11: Line 11:
 Now perform an initial fourfit on the data. You need to do this scanwise as there are often corrupted scans in the data set which will crash fourfit otherwise. The script below gives verbose output for each scan: Now perform an initial fourfit on the data. You need to do this scanwise as there are often corrupted scans in the data set which will crash fourfit otherwise. The script below gives verbose output for each scan:
  
-   ''​for i in $(cat /tmp/scans) ; do fourfit -m 1 -c cf_1234 $i ; done''​+   for i in $(cat /tmp/scans) ; do fourfit -m 1 -c cf_1234 $i ; done
  
 A more concise version is included below: A more concise version is included below:
  
-   ''​for i in $(cat /tmp/scans) ; do echo $i ; fourfit -c cf_1234 $i ; done''​+   for i in $(cat /tmp/scans) ; do echo $i ; fourfit -c cf_1234 $i ; done
  
 Once the fourfit processes are complete (15-30 minutes?), run alist with ''​alist * ''​ and then edit the file with the following commands Once the fourfit processes are complete (15-30 minutes?), run alist with ''​alist * ''​ and then edit the file with the following commands
  
    sed -i 's/ 16383 / 1234 /g' alist.out    sed -i 's/ 16383 / 1234 /g' alist.out
-   grep -v " LL " alist.out | grep -v " ii " | grep -v " xx " | grep -v " gg " | grep -v " WW " > alist.ed.out+   grep -v " LL " alist.out | grep -v " ii " | grep -v " xx " | grep -v " ee " | grep -v " HH " | grep -v " gg " | grep -v " WW " > alist.ed.out
        
 Download the skd file and reprocess it into an ovex file. NB - ''​nskd_ovex.pl''​ needs to have internet access - you'll need to define the proxy settings for most machines. ​ Download the skd file and reprocess it into an ovex file. NB - ''​nskd_ovex.pl''​ needs to have internet access - you'll need to define the proxy settings for most machines. ​
Line 28: Line 28:
 Edit the ovex file & change the site IDs to match the Curtin ones Edit the ovex file & change the site IDs to match the Curtin ones
  
-eg Warkworth = +   Warkworth = 
    ​Hobart12 = L    ​Hobart12 = L
    ​Hart15 = g    ​Hart15 = g
    ​Katherine = i    ​Katherine = i
-   ​Yarragadee = x+   ​Yarragadee ​= e 
 +   ​Hobart26 = H 
 +   ​Hart26m ​= x 
  
 Define these two variables as the directories that contain the 1234 directory. ​ Define these two variables as the directories that contain the 1234 directory. ​
Line 42: Line 44:
    aedit -f alist.ed.out    aedit -f alist.ed.out
          edi dup snr           edi dup snr 
-      ​ 
-====== Manual Phasecal ====== 
- 
-If you have a station with no [or weak] phase-cal, you will need to do a manual pcal. To do this, pick a scan with good SNR (50+) in both bands - you can check this within aedit with  
- 
-   axis snr  
-   ​baselines LW 
-   ​device /xs 
-   plot 
- 
-[L == Hobart12 is being used as the reference station here, and W == Warkworth is the station lacking phasecal] 
- 
-Once you've identified your scan, exit aedit  
- 
-Modify your cf_1234 file to include these sections ​ 
- 
-   if station W and f_group X 
-      pc_mode manual 
-      pc_phases ghijklmn 0 0 0 0 0 0 0 0  
- 
-   if station W and f_group S 
-      pc_mode manual 
-      pc_phases abcdef 0 0 0 0 0 0  
- 
-With the modified the cf_1234 file, run fourfit on your selected scan [219-1239 in this example] 
- 
-fourfit -pt -m 1 -c ../cf_1234 -bLW 219-1239 
- 
-In he debugging output of each plot will be the estimated pcal phases for that scan. If the baseline is reference->​manual pcal target, you'll need to reverse the sign on these after pasting them into your control file.  
- 
-Once the cf_1234 file is updated, re-run fourfit and check that the phases are aligned on zero in all the individual IF channels for that scan. If not, check for a sign error? Also, check that other scans also show the IF phases aligned (although not necessarily on zero). ​ If the manual pcal stops working after a certain time, you can re-fringe fit and specify valid time ranges for the varrious pcal values with  
- 
-   if station O and f_group S and scan < 256-165000 
-     ​pc_phases abcdef -4.0    2.3    2.8   ​-1.7 ​   1.0   -0.4 
- 
-   if station O and f_group S and scan  256-170000 to 258-165000 
-     ​pc_phases abcdef ​  -4.0 290.3 146.8  -1.7  73.0  -0. 
- 
-   if station O and f_group S and scan > 258-165000 
-     ​pc_phases abcdef -4.0   ​2.3 ​  ​2.8 ​ -1.7   ​1.0 ​ -0.4  
- 
 ===== Phase Cal estimation ===== ===== Phase Cal estimation =====
  
Line 116: Line 77:
    alist *    alist *
    sed -i 's/ 16383 / 1234 /g' alist.out    sed -i 's/ 16383 / 1234 /g' alist.out
-   grep -v " LL " alist.out | grep -v " ii " | grep -v " xx " | grep -v " gg " | grep -v " WW " > alist.ed.out+   grep -v " LL " alist.out | grep -v " ii " | grep -v " HH " | grep -v " ee " | grep -v " xx " | grep -v " gg " | grep -v " TT " | grep -v " TT " | grep -v " WW " > alist.ed.out
  
  
Line 126: Line 87:
       freqs g h i j k l m n+       freqs g h i j k l m n+
       ​       ​
-This will flag out the X8L channel [dubbed n- in HOPS] in the Warkworth data from the fitting process. Re-run fourfit on the affected scans and quickly check the results [ e,g, remove previous fourfit results with the find command above & then run ''​fourfit -pt -m 1 -c cf_1234 ddd-hhmm''​. Check that the G-code error is gone & that the can is good. If so, reprocess the entire data set with the updated cf file and resume. ​+This will flag out the X8L channel [dubbed n- in HOPS] in the Warkworth data from the fitting process. Re-run fourfit on the affected scans and quickly check the results [ e,g, remove previous fourfit results with the find command above & then run ''​fourfit -pt -m 1 -c cf_1234 ddd-hhmm''​. Check that the G-code error is gone & that the scan is good. If so, reprocess the entire data set with the updated cf file and resume. ​ 
 + 
 +       
 +====== Manual Phasecal ====== 
 + 
 +If you have a station with no [or weak] phase-cal, you will need to do a manual pcal. To do this, pick a scan with good SNR (50+) in both bands - you can check this within aedit with  
 + 
 +   axis snr  
 +   ​baselines LW 
 +   ​device /xs 
 +   ​plot 
 + 
 +[L == Hobart12 is being used as the reference station here, and W == Warkworth is the station lacking phasecal] 
 + 
 +Once you've identified your scan, exit aedit  
 + 
 +Modify your cf_1234 file to include these sections  
 + 
 +   if station W and f_group X 
 +      pc_mode manual 
 +      pc_phases ghijklmn 0 0 0 0 0 0 0 0  
 + 
 +   if station W and f_group S 
 +      pc_mode manual 
 +      pc_phases abcdef 0 0 0 0 0 0  
 + 
 +With the modified the cf_1234 file, run fourfit on your selected scan [219-1239 in this example] 
 + 
 +fourfit -pt -m 1 -c ../cf_1234 -bLW 219-1239 
 + 
 +In he debugging output of each plot will be the estimated pcal phases for that scan. If the baseline is reference->​manual pcal target, you'll need to reverse the sign on these after pasting them into your control file.  
 + 
 +Once the cf_1234 file is updated, re-run fourfit and check that the phases are aligned on zero in all the individual IF channels for that scan. If not, check for a sign error? Also, check that other scans also show the IF phases aligned (although not necessarily on zero). ​ If the manual pcal stops working after a certain time, you can re-fringe fit and specify valid time ranges for the varrious pcal values with  
 + 
 +   if station O and f_group S and scan < 256-165000 
 +     ​pc_phases abcdef -4.0    2.3    2.8   ​-1.7 ​   1.0   ​-0.4 
 + 
 +   if station O and f_group S and scan  256-170000 to 258-165000 
 +     ​pc_phases abcdef ​  -4.0 290.3 146.8  -1.7  73.0  -0. 
 + 
 +   if station O and f_group S and scan > 258-165000 
 +     ​pc_phases abcdef -4.0   ​2.3 ​  ​2.8 ​ -1.7   ​1.0 ​ -0.4  
 + 
 + 
 +===== =====
  
 Recreate the alist [with sed & grep -v as before], load it in aedit and check the "sum 2" or psplot outputs to make sure it's improved things. ​ Recreate the alist [with sed & grep -v as before], load it in aedit and check the "sum 2" or psplot outputs to make sure it's improved things. ​
/home/www/auscope/opswiki/data/pages/analysis/fourfit.txt · Last modified: 2017/06/06 01:26 by Jamie McCallum