Filtering

The primary purpose of the filter is to eliminate irrelevant frequency bands from the raw data. This band-limits the data stream, which also allows some decimation to be applied without loss of signal. By performing the filter and decimation stages on the data before the numerically intensive analysis, the processor load is minimized so that a lower price computer chassis can be used.

The secondary purpose of the filter module is decoupling, through use of independent input and output fifo structures. The demanding real time operation of the serial port concludes by dumping the raw data into the input fifos.

A separate thread, at a lower priority, retrieves blocks of data $V$ from the input fifo and applies a sequence of FIR filters and decimations before writing smaller blocks of data $B$ into the output fifos. The FIR calculation simply involves multiplying the most recent $i = 0 \ldots n-1$ readings $V(T-i,c)$ on each channel $c$ with a constant array of $n$ coefficents $C(i)$ that were loaded from file. The decimation is implicit by simply neglecting to perform that calculation for the $2 \ldots d$ output readings that will subsequently be discarded.

$$B(j,c) = \sum _{ i = 0 \ldots n-1 } C(i) V ( d \times j-i, c )$$ (4)

Clearly, the filter calculation takes linear CPU time for the volume of data and is completely predictable. The real time needed for execution depends on the system cache (or worse, paging) performance. The filter thread is important, since it reduces the amount of memory in use and thus improves cache hits. It is also using large amounts of memory to store the historical raw data and so is best batched.

When the filter batch thread doesn't have anything to do, the output thread builds magnetic data records in the format expected by the analysis software and deposits them into a socket for delivery. This will trigger the computational load that was discussed above.

The software infrastructure to implement this scheme was written in St Paul MN and tested on the hardware being developed and located in the San Diego CA labs. After a few weeks, the San Diego people learned not to unplug hardware without checking for remote logins first, and the St Paul people learned to phone and find out whether someone had removed the hardware before blaming their code.

The source code can be compiled and run, without changes, on full x86 computers with linux and also on the Dragonball microcontroller with uClinux on the UCSIMM from (now) Lineo.