A parallel and pipelined architecture for accelerating fingerprint computation in high throughput data storages

  • Li D
  • Yang Q
  • Wang Q
 et al. 
  • 4

    Readers

    Mendeley users who have this article in their library.
  • 1

    Citations

    Citations of this article.

Abstract

Rabin fingerprints are short tags for large objects that can be used in a wide range of applications, such as data deduplication, web querying, packet routing, and caching. We present a pipelined hardware architecture for computing Rabin fingerprints on data being transferred on a high throughput bus. The design conducts real-time fingerprinting with short latencies, and can be tuned for optimized clock rate with "split fresh" technique. A pipelined sampling logic selects fingerprints based on the Minwise theory and adds only a few clock cycles of latency before returning the final results. The design can be replicated to work in parallel for higher throughput data traffic. This architecture is implemented on a Xilinx Virtex-6 FPGA, and is tested on a storage prototyping platform. The implementation shows that the design can achieve clock rates above 300 MHz with an order of magnitude improvement in latency over prior software implementations, while consuming little hardware resource. The scheme is extensible to other types of fingerprints and CRC computations, and is readily applicable to primary storages and caches in hybrid storage systems.

Author-supplied keywords

  • Caching
  • Data deduplication
  • NVMe
  • PCIe
  • Parallel and pipeline architecture
  • Rabin fingerprint
  • Storage

Get free article suggestions today

Mendeley saves you time finding and organizing research

Sign up here
Already have an account ?Sign in

Find this document

Authors

  • Dongyang Li

  • Qing Yang

  • Qingbo Wang

  • Cyril Guyot

  • Ashwin Narasimha

  • Dejan Vucinic

Cite this document

Choose a citation style from the tabs below

Save time finding and organizing research with Mendeley

Sign up for free