Dataflow-Based Mapping of Computer Vision Algorithms onto FPGAs
© Mainak Sen et al. 2007
Received: 1 May 2006
Accepted: 9 October 2006
Published: 28 January 2007
We develop a design methodology for mapping computer vision algorithms onto an FPGA through the use of coarse-grain reconfigurable dataflow graphs as a representation to guide the designer. We first describe a new dataflow modeling technique called homogeneous parameterized dataflow (HPDF), which effectively captures the structure of an important class of computer vision applications. This form of dynamic dataflow takes advantage of the property that in a large number of image processing applications, data production and consumption rates can vary, but are equal across dataflow graph edges for any particular application iteration. After motivating and defining the HPDF model of computation, we develop an HPDF-based design methodology that offers useful properties in terms of verifying correctness and exposing performance-enhancing transformations; we discuss and address various challenges in efficiently mapping an HPDF-based application representation into target-specific HDL code; and we present experimental results pertaining to the mapping of a gesture recognition application onto the Xilinx Virtex II FPGA.
- Sriram S, Bhattacharyya SS: Embedded Multiprocessors: Scheduling and Synchronization. Marcel Dekker, New York, NY, USA; 2000.
- Sen M, Bhattacharyya SS, Lv T, Wolf W: Modeling image processing systems with homogeneous parameterized dataflow graphs. Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP '05), March 2005, Philadelphia, Pa, USA 5: 133-136.
- Bhattacharyya SS, Leupers R, Marwedel P: Software synthesis and code generation for signal processing systems. IEEE Transactions on Circuits and Systems II: Analog and Digital Signal Processing 2000,47(9):849-875. 10.1109/82.868454View Article
- Lee E, Messerschmitt D: Synchronous data flow. Proceedings of the IEEE 1987,75(9):55-64. 10.1109/PROC.1987.13876View Article
- Bilsen G, Engels M, Lauwereins R, Peperstraete J: Cyclo-static dataflow. IEEE Transactions on Signal Processing 1996,44(2):397-408. 10.1109/78.485935View Article
- Buck JT: A dynamic dataflow model suitable for efficient mixed hardware and software implementations of DSP applications. Proceedings of the 3rd International Workshop on Hardware/Software Codesign (CODES '94), September 1994, Grenoble, France 165-172.View Article
- Bhattacharya B, Bhattacharyya SS: Parameterized dataflow modeling for DSP systems. IEEE Transactions on Signal Processing 2001,49(10):2408-2421. 10.1109/78.950795MathSciNetView Article
- Gokhale M, Stone J, Arnold J, Kalinowski M: Stream-oriented FPGA computing in the Streams-C high level language. Proceedings of IEEE Symposium on Field-Programmable Custom Computing Machines, April 2000, Napa Valley, Calif, USA 49-56.
- Hoare CAR: Communicating Sequential Processes. Prentice-Hall, New York, NY, USA; 1985.MATH
- Chappell S, Sullivan C: Handel-C for co-processing & co-design of Field Programmable System on Chip. In White Paper. Celoxica, Oxford, UK; 2002.
- Banerjee P, Bagchi D, Haldar M, Nayak A, Kim V, Uribe R: Automatic conversion of floating-point MATLAB programs into fixed-point FPGA based hardware design. Proceedings of the 41st Annual Design Automation Conference (DAC '04), June 2004, San Diego, Calif, USA 484-487.
- Kienhuis B, Rijpkema E, Deprettere E: Compaan: deriving process networks from Matlab for embedded signal processing architectures. Proceedings of the 18th International Workshop on Hardware/Software Codesign (CODES '00), May 2000, San Diego, Calif, USA 13-17.
- Kahn G: The semantics of simple language for parallel programming. Proceedings of IFIP Congress, August 1974, Stockholm, Sweden 471-475.
- Lee EA: Multidimensional streams rooted in dataflow. Proceedings of the IFIP Working Conference on Architectures and Compilation Techniques for Fine and Medium Grain Parallelism, January 1993, Orlando, Fla, USA (23):295-306.
- Wolf W, Ozer B, Lv T: Smart cameras as embedded systems. Computer 2002,35(9):48-53. 10.1109/MC.2002.1033027View Article
- Eker J, Janneck JW, Lee EA, et al.: Taming heterogeneity—the ptolemy approach. Proceedings of the IEEE 2003,91(1):127-144. 10.1109/JPROC.2002.805829View Article
- Haim F, Sen M, Ko D-I, Bhattacharyya SS, Wolf W: Mapping multimedia applications onto configurable hardware with parameterized cyclo-static dataflow graphs. Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP '06), May 2006, Toulouse, France 3: 1052-1055.
- Horstmannshoff J, Grötker T, Meyr H: Mapping multirate dataflow to complex RT level hardware models. Proceedings of the International Conference on Application-Specific Systems, Architectures and Processors (ASAP '97), July 1997, Zurich, Switzerland 283-292.View Article
- Sen M, Bhattacharyya SS: Systematic exploitation of data parallelism in hardware synthesis of DSP applications. Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP '04), May 2004, Montreal, Quebec, Canada 5: 229-232.
- Williamson MC, Lee EA: Synthesis of parallel hardware implementations from synchronous dataflow graph specifications. Proceedings of the 30th Asilomar Conference on Signals, Systems and Computers, November 1996, Grove, Calif, USA 2: 1340-1343.
- Data-sheet for ZBT memory, http://www.samsung.com/Products/
- OpenCores Organization : WISHBONE System-on-Chip (SoC) Interconnection Architecture for Portable IP Cores. revision B.3, September 2002, http://www.opencores.org
- Jain AK: Fundamentals of Digital Image Processing. Prentice-Hall, New York, NY, USA; 1989.MATH
- IEEE Working Group http://www.eda.org/vhdl-200x/vhdl-200x-ft/packages/files.html
This article is published under license to BioMed Central Ltd. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.