Following a report from the Washington correlator that the Auscope data was suffering from a large number of dropped frames due to Sync errors (David Hall, 26/1/12 - R4515), we investigated the recorded mark5b data. The results are as follows.

Mark5b data format

The recorded data consists of frames, each with a length of 10016 bytes. Each frame consists of a 16-byte header, followed by 10000 bytes of data. The header starts with a sync word (represented as abaddeed in hexadecimal), followed by a frame counter (represented as bead0000, etc). This frame counter should start at bead 0000 the count should increment by one at each header. The counter should loop at a count of 3200 (for a data rate of 256 Mbps - this scales as (Data rate)/256 so 512 Mbps loops at a count of 6400).

*Checking recorded data*

Once you have exported a mark5b data file, you can check the data quality on a byte-level use this recipe.

od -Ad -tx4 test_034_011402.mk5b | grep abaddeed | cut -d' ' -f1 > test.byteindex od -Ad -tx4 test_034_011402.mk5b | grep abaddeed | cut -d' ' -f3 > test.counter

The output files consist of the byteindex file - this is the list of bytes at which the sync word was found. If the data was exported from the start of a scan, the first byte should be 0 and the index should increment by 10016. You can check this in octave with this recipe.

load test.byteindex;find(diff(test)~=10016)

It should return ans = [](0x1) if it is incrementing correctly. Please note that abaddeed may occur naturally in the recorded data (although rarely). An inspection of the data around any apparent jump should make this clear.

You can check that the counter is correctly incrementing using that recipe.

FID=fopen('test.counter');clear C; A=fread(FID,'uint8');B=reshape(A, 9, length(A)/9)';for i=1:length(A)/9; C(i)=sum(hex2dec(char(B(i,1:8))));end;fclose(FID);find(diff(C)~=1 & diff(C)~=-3199)

This recipe should return ans = [](0x1) if the counter is correctly incrementing. Again, if abaddeed has occurred in the data the counter will apparently jump.