Pdev diagnostics

05aug10

Terminology
Intro
The diagnostics
Running the tests
Diagnostics results
  • 170620: tests failed till power cycled on all spectrometers
FPGA test signals. PNcode and noise src (with amplitudes), pncode

Terminology

  • configuration files:
    • Used to configure the registers in the spectrometer/fpga. look in /dump/pdev/files on pdevs1.
  • DLPF:
    • Digital low pass filter. This reduces the bandwidth of the input data.
  • File server:
    • The spectrometer connects to the file server via Ethernet.
  • pfb
    • polyPhase filter bank
  • sp
    • spectral code in fpga. include complex mixing, dlpf,pfb,ff

  • spectrometer
    • The box containing the A/D, fpga, and power pc. connects to file server via Ethernet

Intro.

   The pnet spectrometer control program written by jeff mock contains options to perform diagnostics on the spectrometer boxes and attached file servers. The diagnostics make use of a pncode generator built into the fpga as well as noise and cw tone sources built into the spectrometer fpga. The tests include:

    A debugging method would be:

The tests:


Running the tests:

    Config files to run the tests are on pdevs1.

Some things that can cause the tests to fail:


fpga test signals:

    The fpga contains a pncode generator, noise generator, as well as cw generator.

Pncode generator:

    Used by --ltest,--rtest, --rcomp. It does not pass through the fpga sp code. I'm not sure what the levels are.

Noise source generator:

    The noise source is enabled with the   TS_NOISE_A,TS_NOISE_B keywords. The value determines the strength of the noise source (0 to 65535). The table below shows the mean value of the output spectrum (average channels 2000-6000 in an 8K spectrum) for various settings of the noise generator and spectrometer configuration:

TS_NOISE_X
 value
hr_shft
decimation
Fcnt
time Rec
secs
spc Mean
counts
spcMean
1 spc
65535
12
2
21000
2
2.4e8
11428
65535 12
2
10500
1
1.2e8
11428
65535
11
2
10500
1
3.1e7
2952
32767
12
2
10500
1
3.1e7
2952
32767
11
4
10500
2
1.33e7
1266
32767
11
4
5200
1
6.6e6
1269
32767
-
full bandwidth
21000
1
6.5e7
3095


Diagnostics results:



Dates:


ltest table

   
bsg
pdevNum
maxI/o
PMAX
MAXREAD
DATE
b0s0g0
110
110
40880
327680
100806
b0s1g0
110
110
41072


group 0 both bands
b0s0g0
b0s1g0
b1s0g0
b1s1g0
b2s0g0
b2s1g0
b3s0g0
b3s1g0
b4s0g0
b4s1g0
b5s0g0
b5s1g0
b6s0g0
b6s1g0
110

103

107

118

116

104

113
55
4496
39728
4560
39952
4544
40192
5040
40512
5088
40224
4768
40080
4640
40224

131072
131072
100806
group 1 both bands
b0s0g1
b0s1g1
b1s0g1
b1s1g1
b2s0g1
b2s1g1
b3s0g1
b3s1g1
b4s0g1
b4s1g1
b5s0g1
b5s1g1
b6s0g1
b6s1g1
119

108

122

106

115

117

120
55
same as
above

100806
21jun17
  b0s0g0: 100 5479.596 55.05 6000 196608
   b0s1g0: 100 5479.596 55.05 40176 131072
   b1s0g0: 100 5499.388 55.05 7744 196608
   b1s1g0: 100 5499.388 55.05 40208 196608
   b2s0g0: 100 5492.244 55.05 4752 196608
   b2s1g0: 100 5492.244 55.05 40192 131072
   b3s0g0: 100 5497.881 55.05 7760 262144
   b3s1g0: 100 5497.815 55.05 40160 131072
   b4s0g0: 100 5482.742 55.05 4640 196608
   b4s1g0: 100 5482.742 55.05 39968 131072
   b5s0g0: 100 5478.810 55.05 4512 196608
   b5s1g0: 100 5478.810 55.05 40208 131072
   b6s0g0: 100 5461.377 55.05 4960 196608
   b6s1g0: 100 5461.377 55.05 40512 196608

110

103

107

118

116

112

111

170621
   b0s0g1: 103 5668.209 55.05 5840 196608
   b0s1g1: 103 5668.143 55.05 40016 131072
   b1s0g1: 103 5628.494 55.05 5680 196608
   b1s1g1: 103 5628.494 55.05 39952 131072
   b2s0g1: 103 5639.176 55.05 4464 327680
   b2s1g1: 103 5639.111 55.05 41232 196608
   b3s0g1: 103 5627.838 55.05 4496 262144
   b3s1g1: 103 5627.773 55.05 40112 196608
   b4s0g1: 103 5636.293 55.05 4704 196608
   b4s1g1: 103 5636.227 55.05 39968 131072
   b5s0g1: 103 5644.091 55.05 4512 131072
   b5s1g1: 103 5644.026 55.05 39776 131072
   b6s0g1: 103 5665.653 55.05 4432 196608
   b6s1g1: 103 5665.587 55.05 40656 196608

119

101

122

106

115

117

120

171114: group 0 both bands

   b0s0g0: 147 8018.395 54.89 6800 262144
   b0s1g0: 147 8018.395 54.89 40160 196608
   b1s0g0: 147 8034.320 54.89 7664 196608
   b1s1g0: 147 8034.320 54.89 40064 196608
   b2s0g0: 147 8043.626 54.89 7744 262144
   b2s1g0: 147 8043.561 54.89 40048 196608
   b3s0g0: 147 8023.245 54.89 7696 262144
   b3s1g0: 147 8023.179 54.89 40048 196608
   b4s0g0: 147 8028.029 54.89 4688 196608
   b4s1g0: 147 8027.963 54.89 39136 131072
   b5s0g0: 147 8055.423 54.89 7424 262144
   b5s1g0: 147 8055.423 54.89 40160 196608
   b6s0g0: 147 8039.301 54.89 6784 196608
   b6s1g0: 147 8039.236 54.89 39168 131072
110

103

107

118

116

112

111



171114
group 1 both bands
   b0s0g1: 102 5579.342 54.89 6608 327680
   b0s1g1: 102 5579.276 54.89 40832 196608
   b1s0g1: 102 5592.580 54.89 5904 131072
   b1s1g1: 102 5592.515 54.89 39184 131072
   b2s0g1: 102 5573.116 54.89 6128 196608
   b2s1g1: 102 5573.050 54.89 39664 131072
   b3s0g1: 102 5567.676 54.89 4496 131072
   b3s1g1: 102 5567.676 54.89 40160 196608
   b4s0g1: 102 5559.419 54.89 6048 327680
   b4s1g1: 102 5559.353 54.89 40160 131072
   b5s0g1: 102 5592.515 54.89 7040 262144
   b5s1g1: 102 5592.515 54.89 39648 131072
   b6s0g1: 102 5595.529 54.89 5392 196608
   b6s1g1: 102 5595.529 54.89 39648 131072
119

101

122

106

115

117

120

171114












   
   

rtest table

    Pmax

bsg
devnum
MaxI/O(perfpga)
Mb/Sec
Pmax
maxRead
date
b0s0g0
110
50
512
62390272
100805
group 0 two bands
b0s0g0
b0s1g0
b1s0g0
b1s1g0
b2s0g0
b2s1g0
b3s0g0
b3s1g0
b4s0g0
b4s1g0
b5s0g0
b5s1g0
b6s0g0
b6s1g0
110

103

107

118

116

104

113
35

40

40

40

45

40

40
5248
16896
44544
112768
15360
458752
41344
102528
50816
192384
44928
114304
44928
97536
19857408
12976128
20054016
15990784
786432
589824
589824
655360
7733248
1114112
655360
655360
655360
720896
100805
group 1 two bands
b0s0g1
b0s1g1
b1s0g1
b1s1g1
b2s0g1
b2s1g1
b3s0g1
b3s1g1
b4s0g1
b4s1g1
b5s0g1
b5s1g1
b6s0g1
b6s1g1
119

108

122

106

115

117

120
40

40

40

40

40

25

40
25984
83072
34176
45312
28288
51328
41728
106240
35968
112128
128
128
38528
59776
589824
589824
589824
786432
589824
589824
524288
720896
655360
655360
57212928
39976960
589824
655360
100805
b5s0g1
117
35
128
43384832
100805
b5s1g1 117
30
0
23986176
100805
171114  group 0 two bands
  b0s0g0: 6126.436 34.97 10240 11337728
   b0s1g0: 6126.436 34.97 42368 11272192
   b1s0g0: 6136.463 34.97 24704 11075584
   b1s1g0: 6136.431 34.97 52736 11403264
   b2s0g0: 6142.362 34.97 38784 524288
   b2s1g0: 6142.362 34.97 70528 589824
   b3s0g0: 6129.648 34.97 39424 589824
   b3s1g0: 6129.615 34.97 58240 655360
   b4s0g0: 6132.531 34.97 39552 589824
   b4s1g0: 6132.466 34.97 70400 589824
   b5s0g0: 6149.571 34.97 22656 589824
   b5s1g0: 6149.538 34.97 56832 589824
   b6s0g0: 6139.085 34.97 22784 655360
   b6s1g0: 6139.052 34.97 68992 458752
110

103

107

118

116

112

111

171114
group 1 two bands
   b0s0g1: 6267.732 34.97 34048 589824
   b0s1g1: 6267.666 34.97 55296 589824
   b1s0g1: 6276.153 34.97 33408 589824
   b1s1g1: 6276.121 34.97 54912 524288
   b2s0g1: 6263.538 34.97 12160 12386304
   b2s1g1: 6263.538 34.97 22528 11468800
   b3s0g1: 6260.392 34.97 35328 589824
   b3s1g1: 6260.392 34.97 69248 589824
   b4s0g1: 6290.211 34.97 16896 655360
   b4s1g1: 6290.211 34.97 44544 458752
   b5s0g1: 6275.924 34.97 30208 21954560
   b5s1g1: 6275.891 34.97 69760 28901376
   b6s0g1: 6278.218 34.97 34944 589824
   b6s1g1: 6278.218 34.97 69248 720896

119

101

122

106

115

117

120

171114








rcomp table

    using pn code generator, merge all sp streams to pdevs1. Let psrv then compare each value from each sp to make sure they are the same.

bsg
devNum
i/orate
Mb/sec (each sp)
Pmax
maxRead
date
group 0
b0s0g0
b0s1g0
b1s0g0
b1s1g0
b2s0g0
b2s1g0
b3s0g0
b3s1g0
b4s0g0
b4s1g0
b5s0g0
b5s1g0
b6s0g0
b6s1g0
110

103

107

118

116

104

113
5
0
3997696
11796480
5439488
10747904
4587520
6094848
5570560
12124160
5177344
15532032
8192000
21889024
12845056
10813440

100805
group 1
b0s0g1
b0s1g1
b1s0g1
b1s1g1
b2s0g1
b2s1g1
b3s0g1
b3s1g1
b4s0g1
b4s1g1
b5s0g1
b5s1g1
b6s0g1
b6s1g1
119

108

122

106

115

117

120
5
0




Rdcomp table:


date
clk
dec
Duration(Mb)
grp
errors
17apr12

164.6592
2

19.268
0
none
15.0
1
none
1

21.2
0
none
22.6
1
none





172.032
2
15
0
  • immediate failure:
    • b5s0g0,b0s0g0,b1s1g0
15.
1
  • b5s1g1 .. aborts after a few secs
  • b1s1g1 ..intermittant errs. aborted after 11 mb.
  • b3s0g1 .. aborts after 1.5 mb.
  • b6s1g1 .. intermittant failures. ran till 16mb.
  • b2s0g1, b4s0g1 aborts after 2.5 mb
1
15.35
0
none
17
1
none
01oct12
172.032
2

0
  • b0s0g0 fails immediately
  • b0s1g9 fails immediately
  • rest run vor 6mb no errro
2
6mb
1
  • b4s0g1 1 error
  • b0s0g1 2 errors
163.84
2

0
  • b0s0g0 failed after 2mb
  • commented out b0s0g0 . ran for 6 mb no errors
  • then put all back in and ran for 6bm no errors.
05nov12
165.00
2
0
1
  • b1s1g1 polA failed at most clocks. pdev108 was bad
20mb
1
  • ran for 20 mb without errors with pdev-101 replacing pdev=108 at b1sxg1
  • prior to 20mb run there were a few errors at b4s?g1
28mar13
172.032
2
abortedd
13mb
0
  • b5s0g0 failed immediately.
  • After removing this one, it ran for 17.8 mbytes with only occasional errors from b6s?g0
15mb
1
  • occasional b6s1g1 error but ran until i stopped it at 15.7 Mbytes
170.0
aborted
15.5mb
aborted
0
  • ran for 3.9 mbytes then faild b1s1g0 and a few b6s1g0 errors
  • I then reran, and it ran for 15.5 Mbytes no errors
  • later reran. b6s1g0 occasional error, b1s1g0 errs. quit after 5 mbytes.
15.2mb
1
  • occasional err b6s1g1.. but ran till i aborted it.
165.0
aborted
0
  • failed after 3.5 mbytes. b5s1g0
160
33mb
0
  • ran with no errors

1
  • ran with no errors,
03oct14









172.032
2

aborted
0
b5s0g0 fails immediately
170.00
aborted
b5s0g0 fails immediately
168
19mb
runs ok
170
24mb
b5s0g0 has about 10 errors
172.032
aborted
b5s0g0 fails immediately
172.032
24mb
no errors!
172.032
aborts
1
b3s0g1
172.032
aborts
b4s0g1
170.00
15mb
b4s0g1 had a few errors
168
36mb
runs no errors
21jun17
  b0s0g0: 5.177 0.07 54656 131072
   b0s1g0: 5.243 0.07 65408 131072
   b1s0g0: 5.177 0.07 54656 131072
   b1s1g0: 5.177 0.07 65408 131072
   b2s0g0: 5.177 0.07 54656 131072
   b2s1g0: 5.177 0.07 65408 131072
   b3s0g0: 5.177 0.07 65408 131072
   b3s1g0: 5.177 0.07 65408 131072
   b4s0g0: 5.243 0.07 59776 131072
   b4s1g0: 5.243 0.07 65408 131072
   b5s0g0: 5.243 0.07 54656 131072
   b5s1g0: 5.243 0.07 65408 131072
   b6s0g0: 5.243 0.07 65408 131072
   b6s1g0: 5.243 0.07 65408 131072

  • first 3 attempts at running, got errors from b6s1g0, and then others.
  • after a few restarts, ran ok, with results on left.
  b0s0g1: 5.374 0.07 65408 131072
   b0s1g1: 5.439 0.07 65408 131072
   b1s0g1: 5.439 0.07 65408 131072
   b1s1g1: 5.439 0.07 65408 131072
   b2s0g1: 5.439 0.07 65408 131072
   b2s1g1: 5.439 0.07 65408 131072
   b3s0g1: 5.439 0.07 65408 131072
   b3s1g1: 5.439 0.07 65408 131072
   b4s0g1: 5.439 0.07 65408 131072
   b4s1g1: 5.439 0.07 65408 131072
   b5s0g1: 5.407 0.07 65408 131072
   b5s1g1: 5.439 0.07 65408 131072
   b6s0g1: 5.439 0.07 65408 131072
   b6s1g1: 5.439 0.07 65408 131072

  • first time run, errors b5s1g1,b6s1g1
  • 2nd run gave no errors, and results on left.
date clk dec results
notes

171114

   170




2
   b0s0g0: 10.945 0.07 65408 131072
   b0s1g0: 11.010 0.07 65408 131072
   b1s0g0: 10.945 0.07 65408 131072
   b1s1g0: 10.945 0.07 65408 131072
   b2s0g0: 10.945 0.07 54528 131072
   b2s1g0: 10.945 0.07 65408 131072
   b3s0g0: 10.945 0.07 54656 131072
   b3s1g0: 10.945 0.07 65408 131072
   b4s0g0: 10.945 0.07 54528 131072
   b4s1g0: 10.945 0.07 65408 131072
   b5s0g0: 10.945 0.07 65408 131072
   b5s1g0: 10.945 0.07 65408 131072
   b6s0g0: 10.945 0.07 54528 131072
   b6s1g0: 10.945 0.07 65408 131072
  • got occasional errors b0s1g0.. but not enough to stop diag.
    • 5d exp dd
    • 19 exp 99
    • 00 exp 01
    • 42 exp c2
    • ...
    • This is the same board that is causing overflow errors in regular datataking
  • At  160 Mhz got no errors.

170
2
 b0s0g0: 33.489 0.07 65408 131072
   b0s1g0: 33.489 0.07 65408 131072
   b1s0g0: 33.489 0.07 65408 131072
   b1s1g0: 33.489 0.07 65408 131072
   b2s0g0: 33.554 0.07 54528 131072
   b2s1g0: 33.554 0.07 65408 131072
   b3s0g0: 33.489 0.07 54656 131072
   b3s1g0: 33.489 0.07 65408 131072
   b4s0g0: 33.489 0.07 54528 131072
   b4s1g0: 33.489 0.07 65408 131072
   b5s0g0: 33.554 0.07 65408 131072
   b5s1g0: 33.554 0.07 65408 131072
   b6s0g0: 33.489 0.07 59776 131072
   b6s1g0: 33.489 0.07 65408 131072

  • 2nd run.. no errors? 33mb..
170
2
b0s0g1: 10.813 0.07 65408 131072
   b0s1g1: 10.879 0.07 65408 131072
   b1s0g1: 10.879 0.07 65408 131072
   b1s1g1: 10.879 0.07 65408 131072
   b2s0g1: 10.879 0.07 65408 131072
   b2s1g1: 10.879 0.07 65408 131072
   b3s0g1: 10.879 0.07 65408 131072
   b3s1g1: 10.879 0.07 65408 131072
   b4s0g1: 10.879 0.07 65408 131072
   b4s1g1: 10.879 0.07 65408 131072
   b5s0g1: 10.879 0.07 65408 131072
   b5s1g1: 10.879 0.07 65408 131072
   b6s0g1: 10.879 0.07 65408 131072
   b6s1g1: 10.879 0.07 65408 131072

  • no errors







rdcomp Notes:


<- page up
home_~phil