Open Access

Dataflow-Based Mapping of Computer Vision Algorithms onto FPGAs

  • Mainak Sen1Email author,
  • Ivan Corretjer1,
  • Fiorella Haim1,
  • Sankalita Saha1,
  • Jason Schlessman2,
  • Tiehan Lv2,
  • Shuvra S Bhattacharyya1 and
  • Wayne Wolf2
EURASIP Journal on Embedded Systems20072007:049236

DOI: 10.1155/2007/49236

Received: 1 May 2006

Accepted: 9 October 2006

Published: 28 January 2007


We develop a design methodology for mapping computer vision algorithms onto an FPGA through the use of coarse-grain reconfigurable dataflow graphs as a representation to guide the designer. We first describe a new dataflow modeling technique called homogeneous parameterized dataflow (HPDF), which effectively captures the structure of an important class of computer vision applications. This form of dynamic dataflow takes advantage of the property that in a large number of image processing applications, data production and consumption rates can vary, but are equal across dataflow graph edges for any particular application iteration. After motivating and defining the HPDF model of computation, we develop an HPDF-based design methodology that offers useful properties in terms of verifying correctness and exposing performance-enhancing transformations; we discuss and address various challenges in efficiently mapping an HPDF-based application representation into target-specific HDL code; and we present experimental results pertaining to the mapping of a gesture recognition application onto the Xilinx Virtex II FPGA.

[1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24]

Authors’ Affiliations

Department of Electrical and Computer Engineering, University of Maryland
Department of Electrical Engineering, Princeton University


  1. Sriram S, Bhattacharyya SS: Embedded Multiprocessors: Scheduling and Synchronization. Marcel Dekker, New York, NY, USA; 2000.
  2. Sen M, Bhattacharyya SS, Lv T, Wolf W: Modeling image processing systems with homogeneous parameterized dataflow graphs. Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP '05), March 2005, Philadelphia, Pa, USA 5: 133-136.
  3. Bhattacharyya SS, Leupers R, Marwedel P: Software synthesis and code generation for signal processing systems. IEEE Transactions on Circuits and Systems II: Analog and Digital Signal Processing 2000,47(9):849-875. 10.1109/82.868454View Article
  4. Lee E, Messerschmitt D: Synchronous data flow. Proceedings of the IEEE 1987,75(9):55-64. 10.1109/PROC.1987.13876View Article
  5. Bilsen G, Engels M, Lauwereins R, Peperstraete J: Cyclo-static dataflow. IEEE Transactions on Signal Processing 1996,44(2):397-408. 10.1109/78.485935View Article
  6. Buck JT: A dynamic dataflow model suitable for efficient mixed hardware and software implementations of DSP applications. Proceedings of the 3rd International Workshop on Hardware/Software Codesign (CODES '94), September 1994, Grenoble, France 165-172.View Article
  7. Bhattacharya B, Bhattacharyya SS: Parameterized dataflow modeling for DSP systems. IEEE Transactions on Signal Processing 2001,49(10):2408-2421. 10.1109/78.950795MathSciNetView Article
  8. Gokhale M, Stone J, Arnold J, Kalinowski M: Stream-oriented FPGA computing in the Streams-C high level language. Proceedings of IEEE Symposium on Field-Programmable Custom Computing Machines, April 2000, Napa Valley, Calif, USA 49-56.
  9. Hoare CAR: Communicating Sequential Processes. Prentice-Hall, New York, NY, USA; 1985.MATH
  10. Chappell S, Sullivan C: Handel-C for co-processing & co-design of Field Programmable System on Chip. In White Paper. Celoxica, Oxford, UK; 2002.
  11. Banerjee P, Bagchi D, Haldar M, Nayak A, Kim V, Uribe R: Automatic conversion of floating-point MATLAB programs into fixed-point FPGA based hardware design. Proceedings of the 41st Annual Design Automation Conference (DAC '04), June 2004, San Diego, Calif, USA 484-487.
  12. Kienhuis B, Rijpkema E, Deprettere E: Compaan: deriving process networks from Matlab for embedded signal processing architectures. Proceedings of the 18th International Workshop on Hardware/Software Codesign (CODES '00), May 2000, San Diego, Calif, USA 13-17.
  13. Kahn G: The semantics of simple language for parallel programming. Proceedings of IFIP Congress, August 1974, Stockholm, Sweden 471-475.
  14. Lee EA: Multidimensional streams rooted in dataflow. Proceedings of the IFIP Working Conference on Architectures and Compilation Techniques for Fine and Medium Grain Parallelism, January 1993, Orlando, Fla, USA (23):295-306.
  15. Wolf W, Ozer B, Lv T: Smart cameras as embedded systems. Computer 2002,35(9):48-53. 10.1109/MC.2002.1033027View Article
  16. Eker J, Janneck JW, Lee EA, et al.: Taming heterogeneity—the ptolemy approach. Proceedings of the IEEE 2003,91(1):127-144. 10.1109/JPROC.2002.805829View Article
  17. Haim F, Sen M, Ko D-I, Bhattacharyya SS, Wolf W: Mapping multimedia applications onto configurable hardware with parameterized cyclo-static dataflow graphs. Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP '06), May 2006, Toulouse, France 3: 1052-1055.
  18. Horstmannshoff J, Grötker T, Meyr H: Mapping multirate dataflow to complex RT level hardware models. Proceedings of the International Conference on Application-Specific Systems, Architectures and Processors (ASAP '97), July 1997, Zurich, Switzerland 283-292.View Article
  19. Sen M, Bhattacharyya SS: Systematic exploitation of data parallelism in hardware synthesis of DSP applications. Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP '04), May 2004, Montreal, Quebec, Canada 5: 229-232.
  20. Williamson MC, Lee EA: Synthesis of parallel hardware implementations from synchronous dataflow graph specifications. Proceedings of the 30th Asilomar Conference on Signals, Systems and Computers, November 1996, Grove, Calif, USA 2: 1340-1343.
  21. Data-sheet for ZBT memory, http://​www.​samsung.​com/​Products/​
  22. OpenCores Organization : WISHBONE System-on-Chip (SoC) Interconnection Architecture for Portable IP Cores. revision B.3, September 2002, http://​www.​opencores.​org
  23. Jain AK: Fundamentals of Digital Image Processing. Prentice-Hall, New York, NY, USA; 1989.MATH
  24. IEEE Working Group http://​www.​eda.​org/​vhdl-200x/​vhdl-200x-ft/​packages/​files.​html


© Mainak Sen et al. 2007

This article is published under license to BioMed Central Ltd. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.