Contents:

• 24oct13 (last run?) .. 382 processed (or attempted  to process) sessions.
• At 10Tb/session.. 3.2 petabytes..
  

• When data was taken
• Filename ranges recorded for each day
• Cal start phase for scans

• List of all files recorded (with various parameters).
• data prior to first full run
  
• nov08:
  
• may09 commensal with igalfa
• nov09: full 6 hour runs
• mar11:
• may11:
• oct11:
• may12:
• jul12
• dec12   (for data close to 18deg dec. not taken at transit)
• jan13
  

• Plot caldeflection total power vs time and za
• jun11: look at adc,pfb overflows vs za for Ndays
  

• 20may11: beam1b caldeflection jump occurs during 20K stage temp diode jump
  

• testReadDec.tar.gz - test programs that read compressed a2130_dtm and a2130_dfrq files (decimated in time and decimated in frequencyt). To download right click on line and save link location (at least on firefox)

Data taking daily summary: (top)

Disk Usage:

Column descriptions:

• Col1:Day/date:tm(hrs ast)
• Day number of run for this session (counts from 1 start of each session)
• Starting date from schedule (ast)
• Hour range data taken (ast)
• Col2:Filenames
  
• File date and  sequence number range for files taken for this run.
• If more than 1 scan then there can be multiple entries.
• Col4:numScans
• Number of scans taken for this run. I've only marke the days with a single galfacts scan (i'm not counting the galfa scan at the beginning).
  
• Col5: calphase
• Phase=0 means that the winking cal started on the correct phase., phase=1 means it started 1 integration early.
• Individual sequence number ranges are listed for those scans with calphase=1
  
• Col6: status
• Compressed --> that the files have been compressed
• inpDeleted --> that the raw input files have been deleted
• Col7:disc label
• A2130_AmmN  The A discs are the Arecibo discs. The C disks are the calgary copy. N=1..7 for the 7 beams
  
• These discs can be found on galfas1 under /share/a2130m/A2130_AmmN
• onlineN: These are the online discs that are still being filled.
•  They can be accessed from galfas1 as /share/a2130m/onlineN/
  

History: (top)

• 07sep14: morning sessions run ok. evening session stopped with 2hour power fail then pnt cpu failure.
  
•  15aug13: 0..262 looks like alfa amps were off. 
  
• 14mar13: short run. rolling surface on dome came loose.
  
• 13mar13: the data  files 000-380 where taken with  the alfa amps off. The compressed data was deleted. the 000-069 data taken on 14mar13 is ok
  
• 03sep12: network switch port pdevs6 ->switch-> spectrometer failed. ran at 100 mb its. switched to a new port
  
• 25aug12: cima hung up 1:32 am 26aug12. up za strip probably finished ok at 1:41am. file 298-310 probably taken with telescope sitting still.l
  
• 21aug12: data taken but if/lo not set up correctly (control computer had crashed). No signal. compressed deleted.
  
• 01dec11: trouble with mock pdevs5 (bm4) i/o was not keeping up.
• the files 500 - 699 had non printable chars in the observer keyword field of the fits file.
• ran svn/pdev/pdev/fixhdr_observerdec11/fixfiles.sc to fix the files. some of the file modification times get changed when this was done.
  
• 23oct11: problem with azimuth amplifier...
  
• 19oct11:: beam1a high tsys (150K). Problem with fiber optic cable.
  
• oct11: beam 6b intermittently bad. 4a working but bias adjusted low so no oscillation.
• 4a Tsys about 55, then 11oct11 move 12db after rfFb to 16 dbamp after FB. tsys about 43K.
  
• 14jul11: 20jul11 found bad files from 14jul11 data. since raw files already deleted. probably lost files.
  
• a2130_dfrq.20110714.b1s1g0.00105.fits
• a2130_dfrq.20110714.b1s1g0.00117.fits
• a2130_dfrq.20110714.b1s1g0.00129.fits
• a2130_dfrq.20110714.b1s0g0.00373.fits
• a2130_dtm.20110714.b1s0g0.00161.fits
• a2130_dtm.20110714.b1s1g0.00152.fits
• a2130_dtm.20110714.b1s0g0.00173.fits
• a2130_dtm.20110714.b1s0g0.00185.fits
• a2130_dtm.20110714.b1s0g0.00197.fits
• 29jun11: forgot to delete previous days files. disc filled up partway through run.
  
• 24may11: alfa rotation trouble gave late start.
  
• 23may11: bias voltages bm1 readjusted prior to run. Thebeam1b jumps (see mar,apr,may11) fixed by this change.
  
• 30apr11: pdevs1 pnetg0:b0s1g0: FPGA1 FIFO overflow  around  00:48:20  01may11
• 24apr11: pdevs1 pnetg0:b0s1g0: FPGA1: DMA buffer overflow:  around 22:54:00
  
• 110329: dome failed (final limit cable broke) 2:48 am.
  
• 091217: started calibration scans after the normal 6 hour run.
• 090906: b1sXg0 spectrometer box failed before run started. Replace the group 1 box..
  
• started late so only got about 76 files.
• There is not a2144 data for this night since i stole the box from group 1. Spectrometer has to run with group 0, group 1 is optional.
• There is a large dc offset in b1sXg0 since i didn't update the a/d  converter offset file prior to running.
  
• 090905: testReadDec.tar.gz. downloadable test program.
• Before 090905: contained
• : testReadDec.c to read compressed time domain files.
• After 090905:  contains
  
• testReadTmDec.c (same as old test program to read decimated time domain files).
• testReadFrqDec.c . new program to read  decimated in freq files.
• 090902: punta salinas ran in all frequency modes till 21:17. They then ran left 1367,1382 on till 23:00.
  
• 090830: bm 2, band 0 lost 600millisecs of data between file .00000 and file 00001.
• Found first 12000 bytes of file 2130.20090830.b2s0g0.00001.fits to be bad (not a fits header).
• copied first 12000 bytes of 2130.20090830.b2s1g0.00001.fits to this bad file (since hdrs are identical).
• Found that there is 1 missing row (600 millisecs of data) between end of file .00000 and first row in .00001 for b2s0g0.00001
• The header info is correct for the data in b2s0g0.00001. This means that the problem occurred on the open of file .00001. The sequence was probably:
  
• write header for .00001
• write first row of data.. this ended up overwriting part of the header
• write 2nd row of data (this went in the location for row 1..)
  
• So all data files .00001 -> end of run for b2s0g0 is 1 row (600 Milliseconds) later than all the other corresponding beams/bands). The headers should be correct so if you use the headers then things should be ok.
  
• 090823: pdevs6 halted at 23:00. Lost last 40 minutes of run.
• 081209: finished filling /export/a2130 on 1 Tbyte drives. 9 days with 18 Gb left over.
• 081205:
  
• prior to start of run disc #5 on pdevs1 raid array was replaced.
  
• During the run there was an azimuth failure. This split the data into 2 scans. There were no pdev i/o problems.
  
• 081201:
• data taking stopped after 3 hours with fifo buffer overflow.
•  Look at what was happening during the abort.
  
• 081117:. Had to renumber sequence numbers for a days files.
  
• 17nov08 20:00. the files taken were 00000-000423.
•  The previous run (day 4 20081116) also had files named 20081117 00000-00404. They had been reomved from the online disk. When day 5 started it re-used sequence numbers starting at 0.
• I took all of the day 5 files  and added 500 to the sequence number. So the sequence numbers in the cima logs and the disc files will differ by 500.
• 081114:Scans sometimes start with calOff phase off by 1 ( see winking cal phase offset).
• There should be 20 1 millisecond cal offs followed by 20 1 millisecond cal ons.
• The cal should transition 500 usecs into the 20th calOff and the 20th cal on.
• The accumulation excludes the last 1 millisecond of the 20 cal offs and the 20 calons.
• The scans with calphase=1 start with:
  
• 18  1millisecond caloffs, 1 transition, 19 calOns, 1 transition, 19 caloffs, 1 transition, 19 calons, 1 transition. So the first cal off is 1 millisecond short.
• The accumulation throwing out the last 1 millisecond of the each 20 milliseconds leaves the caloff, calOn accumulations with the cal transition.
• The calcyle length remains at 20 milliseconds off, 20 milliseconds on, it's just been shifted by 1 millsecond.
• Whatever the cal starting phase is, it remains fixed for the entire duration of the scan.
•  It can only change when the spectrometer is told to start running.
• If there are no problems the spectrometer does 1 scan for the entire 6 hour observation session.
  
• The day, filesequence numbers listed with calphase=1 have this problem.
• The problem is occurring at the spectrometer ttl output that is synchonized with the fft accumulations.
• It looks like occasionally the fpga is starting its cal off down count off by 1.
  
• When galfacts turns around, the winking cal is disabled (via cima).
  
• When this occurs you see calOff or calOn for long stretches of data.
• The transitions are not synchronized with the 40 millisecond cal cycle
• When we return to the winking cal, the phase remains as it was before cima's intervention.
• 081113. beam 0. files a2130.20081113.b0sNg0.00210 are a few rows short, files a2130.20081113.b0sNg0.000211 are missing.
• This is when the disk drive for pdevs1 (beam0) crashed.
• the missing file 000211 is only 1 row long on the other 6 beams (a2130.20081113.bNsNg0.00211). They probabably stopped taking data a little later than pdevs1.
  

List of files  (top)

• raw data files (list isn't inclusive until  20081119..)
  
• decimated in time files
  
• decimated in freq files
  

• scan
• The scan number for the data in this file.
• The scan number remains constant for 1 spectrometer run. For a1230 this is usually 4 hours long
• specNum
• The number of the specta within the scan for the first spectrum in this file (count from 1).
  
• There are usually 600 1millisecond spectra per row and 109 rows per file (6540 spectra per file.. except maybe the last file of a scan).
• filename.. name of file. b0=beam0, s0=subband 0 (1450 cfr), s1=subband 1 (1300 cfr).
• starttime: seconds since 1970 (unix time) for start of first spectra in the file (crval5 of first row).
• nrow: number of rows in the file (usually 109).
• on19_on1 on19_on1(con):
• This averages the calOn stat bit for the 15 cal cycles (1 row of 600 spectra)
• The calcycle is 20 spectra calOff, 20 spectra cal on  (each spectra 1 ms).
  
• The 4 numbers are:
• average number of calOns for spc1-19 (start of calOff).
  
• average number of calOns for spc20 (end of caloff)  
  
• average number of calOns for spc21-39 (start of calOn)
• average number of calOns for spc40 (end of calOn)
• The correct values are:
• 00.00 01.00 19.00 1.00
• When calphase=1 and starts 1 early then the values are:
• 01.00 01.00 19.00 1.00
• The calon transition in spc 19 rather than spc20.

Processing the data: (top)

• it's probably easiest to do it as phil since the script pdevctlall uses my sshkeys.
• For the instructions below I'm using the data taken on 15dec08 starting at 22:00 as the example.
  
• See if the data taking aborted and gave filenames that crossed into the next day:
  
• cd /share/pdata1/pdev
• ls -lt a2130*
• look to see if this days data spanned 2 dates. This could happen if the program aborted after midnight.
• If the run has filenames with two dates then you will have to process each date separately.
• Each scan is a continuous set of file numbers. A new scan always starts on  a file number that is the next multiple of 100.
• For 15dec08 there are only two scans
• 00000->00004. This data was taken during the togs inititialization (before za scanning). You process this data, but ignore the cal phase.
  
•  00100->00425. this is the galfacts data with za scanning. 6 hours usually gives about 326 files.
  
• Find the starting cal phase for all of the galfacts za scans taken for the day. This also appends to the log file.
• Go to a relatively fast computer (galfas1,2, or 3)
• cd /share/megs/phil/usr/a2130/logfiles
• makelog 20081215  (for our example)
  
• This will read the first row all of the files on 20081215.
• For each file read, it will append a summary line to /share/megs/phil/usr/a2130/logfiles/a2130_raw.log.
• (makelog -h gives the other options for makelog)
  
• In the log file examine the first entry for each galfacts scan of the day (Once the scan starts the cal phase is constant for the scan).
• #   scan     specNum               FILENAME                            StartTime    nrow  on19_on1 on19_on1(con)
  
• 835000124        1      a2130.20081215.b0s0g0.00100.fits 1229385732  109  0.00  1.00 19.00  1.00
  
• The last 4 numbers has the number of calOn milliseconds  found in the first 20 milliseconds (caloff) and the last 20 milliseconds calon)
• If the cal is at phase 0 (normal), then it should have  0.00 1.00   19.00 1.00
  
• 0.00: 19 milliseconds no cal
  
• 1.00 cal transition off to on in the middle of the 20th millisecond
• 19.00   19 milliseconds of cal on
• 1.00   cal transition on to off in the middle of the 20th millisecond.
• In this case you will delete 1 millisecond (sample) at the end of each 20 milliseconds.
• If the cal phase is off by 1 you will see:
• 834100275        1      a2130.20081206.b0s0g0.00300.fits 1228610462  109  1.00  1.00 19.00  0.00
  
• In this case you will have to delete two samples at the end of each 20 milliseconds of data.
• If you see all zeros during file numbers 00000 - 00004 just ignore them. During the togs init, there is no cal.
• You need to check each galfacts scan for the run. If you find 1 scan with phase 0 and 1 scan with phase 1:
• If the scans are on different days then process each day with a different number of dropped samples
• If the scans are on the same day then you need to process the entire days with 2 dropped samples.
  
• Compress a runs data:
• Check that there is enough space on the disks:
• /share/megs/phil/svn/pdev/pdev/scripts/pdevdisk -d
• You need about 118Gb on /export/a2130 for each pdev s1-s7 server for a full days compression.
  
• cd /share/megs/phil/svn/pdev/pdev/galfacts
• edit compressday
• set numToDrop to either 1 or 2 depending on the cal phase you found above.
  
• set yymmdd= to the date you want to process
• compressday  .. this will compress the entire days files.
• It runs 7 processes in the background so the script returns immediately.
• You can monitor the 7 pdevs with top via: ../scripts/monpdev
• It take about 3 hrs 20 minutes to compress a full day if the discs are not busy copying other things.
  
• Delete the raw datafiles:
• In /share/megs/phil/svn/pdev/pdev/galfacts..
  
• You can dry run the delete with: deletescript 20081215 
  
• To actually delete the files use: deletescript -d 20081215
• It will only delete non - zero input files that have a _dtm and _dfrq compressed file. I
• It outputs a message of the files it deletes (or intends to delete with no -d). It also outputs a message of any files it won't delete.
• Deleted files are just set to 0 size with the original time stamp.
• It is ok to run deletescript over again. It tries hard not to make a mistake...
  
• Add the compressed files to the compressed file logs:
• cd /share/megs/phil/usr/a2130/logfiles
• edit junk changing the dates to 20081215
• junk
• this will run:
• makelog -t 20081215  (runs on the decimated in time files)
• makelog -f 20081215 (runs on the decimated in frequency files)
• The logfiles are
  
• a2130_dtm.log and a2130_dfrq.log
• Ignore the cal phase for the a2130_dtm.log files. It has removed the cal transition millisecond.
• Update the a2130 html  daily info:
• Look at http://www.naic.edu/~phil/usrproj/a2130.html in a browser to see what the daily info table looks like.
  
• Edit ~phil/public_html/usrproj/a2130/a2130_entry.html
• insert the date, file number ranges, phase, etc in each line.
• edit ~phil/public_html/usrproj/a2130/a2130.htl
• before the line:
  
• <!--  insert new table entry before this line (from a2130_entry.html after editing)..   -->
• read in  a2130_entry.html that you just edited..
• Or just use an html editor (like seamonkey..).