HIGH SPEED DOCUMENT CLUSTERING IN RECONFIGURABLE HARDWARE

High-performance document clustering systems enable similar documents to automatically self-organize into groups. In the past, the large amount of computational time needed to cluster documents prevented practical use of such systems with a large number of documents. A full hardware implementation of K-means clustering has been designed and implemented in reconfigurable hardware that clusters 512k documents rapidly. This implementation, uses a cosine distance metric to cluster document vectors that each have 4000 dimensions. The system was synthesized on a Xilinx XC4VLX200 with a clock frequency of 250 MHz. With this FPGA the hardware accelerated algorithm runs up to 328 times faster than the software version running on an Intel 3.6 GHz Xeon. Experiments were also performed using the Field Programmable Port Extender (FPX) platform. It is shown that a fully pipelined architecture running on a Xilinx XCV2000E-8 FPGA (with a clock frequency of 80 Mhz) can outperform software implementations running on an Intel 3.60 GHz PC by a factor of twenty-six. 1