intro

18oct15 - 21oct15: satellite data

 Setups used:

• gregorian dome tracks object, ch used for rfi monitoring
• All data taken with topocentric frequencies, no doppler correction done.
  
• gregorian dome
• 327 mhz receiver:
• 53.3 Mhz bw, 128 frequency channels
  
• 9.6 usecond spectral averages
• center frequency: 327 Mhz.
• 430 Mhz receiver
• 26.6667 Mhz bw, 64 frequency channels
• 9.6 usecond spectral averages
• center frequency 432 Mhz.
• lbw receiver
• 80 Mhz bw, 4096 channels
  
• 999.9872 ms spectral averages
• center frequency 1590 Mhz.
• carriage house receiver (for rfi monitoring)
• 47 Mhz  receiver
• 2 Mhz bw, .5 usecs complex sampling of voltage data
• center frequency 46.8 Mhz
• 430 Mhz carriage housee receiver
• 2 Mhz bw , .5 usec complex sampling of voltage data
• center frequency 430 Mhz.
• The carriage house remained at the stow position (8.834 deg) while the gregorian dome tracked the satellite (or pulsar).
  

Description of daily observations

• 151019(UTC) (151018 AST): lbw, 327 , 47 Mhz

• start time hh:mm:ss utc	object	mock fnum	mock rcvr	ri File nums	ri rcvr	notes
  00:51:12	gps prn 22	000	lbw	-	-	wait for satellite to enter beam
  01:39:02	gps prn 22	100	lbw	-	-	tracking satellite.. front end saturated.. ignore data.
  02:46:52	gps prn 22	200- 208	327	000-001	47	ri file 0 rec 28749 bad hdr. lost 311 records don't process ri file.
  03:04:49	J0006+1834	300- 315	327	-	-
  03:27:27	J0030+0451	400- 402	327	-	-	millisec psr. dm smearing 50ms, period:4ms probably junk data..

•  
• 151020 (UTC) (151019 AST): 327 with dome, 430 with carriage house
  
•  

  start time hh:mm:ss utc	object	mock fnum	mock rcvr	ri File nums	ri rcvr	notes
  01:48:52	gps prn 22	000- 025	327	.002-.005	430ch	ri ok 227742 recs
  02:21:02	prn22 +.75deg offset	100- 108	327	.006-.007	430ch	3 bm offset ri  rec 19022 bad hdr ,skip
  02:31:52	gps prn 22	200- 217	327	.008-010	430ch	ri ok 157461 recs
  02:58:09	B2315+21	300- 313	327	-	-	tracked pulsar

  
  
•  
  
• 151020/21  (UTC) (151020 AST) 430 with dome, 430 with carriage house (afternoon)
  

• start time hh:mm:ss utc	object	mock fnum	mock rcvr	ri File nums	ri rcvr	notes
  						mock beam 0
  17:44:24	rbspb	000- 018	430	.011-.015	430ch	did not hit new file for ridata between this scan and the next. So ri file .015 has end of 1st, start of 2nd file. ri started 2 minutes after mock..
  18:29:13	rbspb 1bm offset	100- 104		.015-.016		ri file started middle of file .015 merged ri file ok
  18:39:43	rbspb	200- 204		.017-.018		ri file rec 12283 file 17 bad hdr.. lost 311 ri recs..
  18:49:41	rbspb 1 bm offset	300- 304		.019-.020		ri 68803 recs ok
  18:59:41	rbspb	400- 404		.021-.022		ri 68999 recs ok
  19:09:35	rbspb 1 bm offset	500- 504		.023-.024		ri 68666 recs ok
  19:19:35	rbspb	600- 604		.025-.026		ri 68875 recs ok
  19:29:38	rbspb 1 bm offset	700- 704		.027-.028		ri 68703 recs ok
  19:39:40	rbspb	800- 805		.029-.030		remained on satellite till it set.
  19:53:26	1 bm offset	900- 901		.031		offset at least 1 bm. increased as satellite continued out of the beam.
  151021 UTC - mock beam 4
  01:42:27	gps prn 22	100- 104	430	.032-.033	430ch	raw data written to mock b4s1g0 (/share/pdata5/pdev)
  01:52:35	1 bm offset	200- 204		.034-.035
  02:02:37	on sat	300- 304		.036-.037
  02:12:36	1 bm offset	400- 404		.038-.039
  02:22:35	on sat	500- 504		.040-.041
  02:32:32	1 bm offset	600- 604		.042-.043
  02:42:35	on sat	700- 702		.044
  02:52:46	off	800-803		.000 .045		raw file .000, after processing filenum is .045 offset by at least 1 bm as went out of the beam.

  
  
•  
  
• 151022 UTC (151021 AST)  mock bm 4
  

• start time hh:mm:ss utc	object	mock fnum	mock rcvr	ri File nums	ri rcvr	notes
  01:38:42	gps prn 22	000-012	327	001-002	430	on satellite.. too much rfi, switch to 430 on next scan ri ok. 109874 recs
  01:56:00	off sat	100-117	430	003-007	430	sitting at az,za the entire time. entire scans is an off ri ok   nrec 289950
  02:40:10	on sat	200-202		008		on satellite ri ok. nrecs=37541
  02:51:59	B2351+21	300-302		-		track pulsar B2351+21 till set

  
  

 Acquiring the data:

• satellite beam:
• this used the gregorian dome. The receivers used were 430, 327, or lband.
• The satellite position was tracked or a position 1 beam offset from the satellite.
  
•  The two receivers used were:
• 430 receiver: 26.6667 Mhz bandwidth (160Mhz/6) 64 frequency channels
• 327 receiver: 53.3333 Mhz bandwidth (160Mhz/3.) 128 frequency channels.
• the data was piped to the mock spectrometer where the spectral density function was computed  and integrated for 9.6 micro seconds
• the spectra were output to fits files:
• A continuous (in time) data set (called a scan)  consisted of multiple fits files (each about 2.2 gbytes).
• each file contained:
• 430 receiver:
• 139 rows, 104165 spectra each with 64 freq channels.. (about 1 second of data) per row.
• 327 receiver:
• 97 rows, 79435 spectra each with 128 freq channels..(about .8 seconds of data) per row.
  
• rfi beam:
• This used the carriage house receivers:
• the postion was 180 Degrees from the satellite. The za was fixed at the stow position (8.8343 deg).
• The data was piped to the ri a/d converters.
• complex voltage samples were taken at a .5 usec sampling rate (giving a 2 Mhz bw), 8 bit samples, and then written to disc.
• The two receivers used were:
• 430 Mhz line feed
• 47 Mhz line feed (reciever bandwidth is less then 2Mhz).
  

Processing the data:

• deltaTsys/Tsys = 1/sqrt(channelBandwidth*time)
• (Since we've added polarizations, the channel bandwidth has been doubled..)
• So computing the rms by channel and then normalizing to the average channel value will give sigma values that should be determined by the bandwidth and integration time.. if the signal is noise like.. Any intermittent rfi (narrow channel frequeny broadcasts) will increase the rms values in these channels..
  
• broad band bursts from the satellite that lasts for 10's of usecs will not affect this  (since they are too short in time).
  

Satellite beam:

• each row of data (1 sec 430, .8 sec 327) was processed separately:
• compute rms/mean for each channel (over the 104165 or 79435 spectra)
• do a linear fit to the rms over the 64 (or 128) freq channels (excluding the edge channels where the analog/digital filter) response fell off.
• Create a freq channel mask that includes all freq channels whose fit is within 3 sigma of the fit.
  
• This mask will be used to compute the total power for each 9.6 usec spectra .. for this 1 sec of data.
• for each of the 9.6 usec spectra in the row (104165 or 79435 specra) compute the total power using the above row
• output the data, and record which frequency channels were used for this row.
• repeat the above for each file of the scan
• and continue for each scan of the day.
• There will be 1 output file of floating point total power data (sampled at 9.6 usecs) for each scan.
• repeat the above for each day of data taking.

RFI BEAM:

• this was 2 Mhz of complex voltage samples, centered at 430, 327, or 46.8 Mhz.
• an entire file (2 Gbytes of 8bit complex voltages was read in.
• the mean was subtracted from the i, q channels (to get rid of DC offsets)
• the power was then computed for each .5 usec sample , and then averaged to 10 usecs.
• the mean an rms was then computed for the entire data set.
• the output values were the tpOut[i]= (tpAvg[i] - mean)/rms
• Comparing the two beams.
• This is use when the on source an rfi beams are both 430 Mhz...
  
• If we find a large sigma in the on source beam, we should check the same time (and maybe 10 usecs on each side) to see if there is a corresponding large sigma in the rfi beam..
  
• If this is true, then the sigma in the on source beam is not from the satellite.
• Passing this test does not guarantee that the on source large sigma is from the satellite..
  
• the rfi bandwidth is only 2 Mhz.. there could be some frequency channels outside this 2 mhz that has terrestrial rfi.

Decide which broad band spikes to keep.

• The satellite beam total power array was converted to sigmas.
• sigSat[i]=(tpSat[i]- mean(tpSat))/rms(tpSat)
• The rfi beam was already in sigmas
  
• to be a valid  broad band spike, the point had to meet:
• the sigSat[i]  had to be larger than the sigma threshold (say 5 sigma)
• the sigRfi[j]  had to be less then the sigma threshold
  
• For every sigSat[i] point, 3 points were checked in the sigRfi[j] array (the one closest, 1 above, and 1below in time)
• If any of the 3 points in the sigRfi array were > than the sigma threshold, then the sigSat[i] point was ignored.
• The satellite spikes should be narrow in time.. about 1 time sample (9.6 usecs).
• i tried excluding all > 5sigma spikes that lasted for more than 3 consecutive time samples.. this reduced the number of spikes by about 10% (this last one has not yet been implemented..)
  

 Looking at the data

• Does it make a difference how many spectra we used when computing the rms?
• by default, 1 row of data was used to compute the rms by channel (about 1 second)
• for 430 Mhz this was
  
• for 430 Mhz that was 104165
  
• I looked at a file with lots of spikes to see if the rfi rejection changed with the # spectra used for rms computation:
• 14,467,824 spectra in the file
  
• date:20151020 scan start: 17:44:24 (utc), file number 00005 has lots of spikes
• the table below shows the number of 5 sigma spikes i got while varying the number spectra per rms

• # of spectra per rms	TmDuration for rms (secs)	# spikes>5sigma
  104165	.999984	6315
  10416	.1	6291
  1041	.01	6283

  
  
• So the number of spikes decreased a little, but no appreciably.  So i stayed with 1 sec of data for the rms by chan computation
  

Exporting files:

• A scan is a contiguous set of data.
• For each scan:
• there is are two data sets:
  
• there is a data set from the gregorian dome, mock spectrometer
• this is the beam that pointed at or close to the satellite
• the files start with mtp_ (mock Total Power)
• There is a data set from the carriage house, ri a/d interface
• This is the beam that pointed 180 degrees from the satellite (rfi beam)
  
• the files start with rtp_ 
• Each data set has:
• file of total power data (4 byte floating point, little endian)
• Equally spaced in time
• ends with .dat
• an ascii  header file that has some info on the scan
  
• Filenames:
• mtp_yyyymmdd_hhmmss.dat / .hdr
  
• The date, time is the utc start time for the data in the file
  
• rtp_yyyymmdd_hhmmss.dat/hdr
• The date, time is the utc start time for the data in the file
• Note: the mtp and rtp files did not start at the same second
• Headers:
• mtp_20151020_174424.hdr
• start1970 :                         1445363064    start time secs 1970
  timeStep  :                        0.000009600    time step seconds
  nsamples  :                          265099925    number of samples in file
  cfr       :                        432.0000000    center freq Mhz
  bandwid   :                         26.6666667    bandwidth Mhz before bad channels
  mockNmSt  :   s3043.20151020.b0s1g0.00000.fits    fname for first mock raw file
  nmockFiles:                                 19    number of mock raw files this scan
  rifname   :                rtp_20151020_174620    basename (or none) for ri tp (rfi beam) file
  
• rifname specified the ri  file that corresponds to the  mock total power file
• rtp_20151020_174620.hdr
• start1970 :                         1445363180    start time secs 1970
  timeStep  :                        0.000010000    time step seconds
  nsamples  :                          243258982    number of samples in file
  cfr       :                        430.0000000    center freq Mhz
  bandwid   :                          2.0000000    bandwidth Mhz
  riNmSt    :                 s3043.20151020.011    fname for first ri raw file
  mfname    :                mtp_20151020_174424    basename for mock tp (on beam) file
• mfname is the mock file that goes with this ri file.