Real Time Image Processing

This work presents a real-time video-based fire and smoke detection using YOLOv2 Convolutional Neural Network (CNN) in antifire surveillance systems. YOLOv2 is designed with light-weight neural network architecture to account the requirements of embedded platforms. The training stage is processed off-line with indoor and outdoor fire and smoke image sets in different indoor and outdoor scenarios. Ground truth labeler app is used to generate the ground truth data from the training set. The trained model was tested and compared to the other state-of-the-art methods. We used a large scale of fire/smoke and negative videos in different environments, both indoor (e.g., a railway carriage, container, bus wagon, or home/office) or outdoor (e.g., storage or parking area). YOLOv2 is a better option compared to the other approaches for real-time fire/smoke detection. This work has been deployed in a low-cost embedded device (Jetson Nano), which is composed of a single, fixed camera per scene,...

In computer vision and video encoding applications, one of the first and most important steps is to establish a pixel-to-pixel correspondence between two images of the same scene obtained at slightly different times or points of view. One of the most popular methods to find these correspondences, known as Area Matching, consists in performing a computationally intensive search for each pixel in the first image, around a neighborhood of the same pixel in the second image. In this work we propose a method which significantly reduces the search space to only a few candidates, and permits the implementation of real-time vision and video encoding algorithms which do not require specialized hardware such as GPU's or FPGA's. Theoretical and experimental support for this method is provided. Specifically, we present results from the application of the method to the realtime estimation of dense optical flow and stereo disparity maps, where a basic implementation achieves up to 100 fps in a typical dual-core PC. We also discuss the application of our method in realtime video compression and transmission. Keywords phase correlation • realtime vision • optical flow • stereo vision • video encoding

COVID-19 is a disease caused by a severe respiratory syndrome coronavirus. It was identified in December 2019 in Wuhan, China. It has resulted in an ongoing pandemic that caused infected cases including many deaths. Coronavirus is primarily spread between people during close contact. Motivating to this notion, this research proposes an artificial intelligence system for social distancing classification of persons using thermal images. By exploiting YOLOv2 (you look at once) approach, a novel deep learning detection technique is developed for detecting and tracking people in indoor and outdoor scenarios. An algorithm is also implemented for measuring and classifying the distance between persons and to automatically check if social distancing rules are respected or not. Hence, this work aims at minimizing the spread of the COVID-19 virus by evaluating if and how persons comply with social distancing rules. The proposed approach is applied to images acquired through thermal cameras, to...

The Oxford English Dictionary defines a robot as ''a machine capable of carrying out a complex series of actions automatically […]''. So the key point of a robot is that it can act. However, to be able to act, a robot must first perceive: What is around me? Where am I? And where are things I want to act on? Since the early days of robotics, the prime perception channel to answer these questions is vision-robot vision. Nowadays, this is even more true than before, since RGBD cameras provide direct geometrical perception. Remarkably, these devices provide ''vision'' in two senses. Seen from the outside, they extend the classical RGB images, where geometry must be indirectly inferred from with a direct sense of depth of scene geometry. Seen from the inside, they are actually vision sensors based on a conventional camera and structured light. As in robotics the objective is to make robots act, there are two other points related (at least to some degree) to robot vision worth stating here: first, the final output is usually metric 3D information, which is needed as input for robot motion. And second, since motion involves time, robot perception and robot vision need to be accomplished in real-time, which is the theme of the JRTIP journal. This special issue on Robot Vision aims at reporting on recent progress made to use real-time image processing towards addressing the above three questions of robotic perception. The call for papers of this special issue

Recent flight tests of the Airborne Reconfigurable Imaging System (ARIS) developed by BAE Systems have demonstrated several key capabilities for day-night multiple target detection and tracking of maritime surface vessels. The flight test system includes the integration of BAE Systems-developed real-time image processing algorithms with multispectral band electro-optical (EO) (RGB)/mid-wave infrared (MWIR) sensors and a COTS turreted system. Presented here are significant flight test results that demonstrate real-time multispectral sensor image co-registration, geo-registration, fusion, target detection and tracking. High performance geo-pointing and line-of-sight stabilization capabilities enable the airborne system to provide maritime domain awareness objectives for search and rescue, persistent surveillance of moving and stationary targets, contraband traffic control through detection and tracking of concealed vessels, and autonomous tracking of both moving and stationary vessels. Solar glint and cloud coverage false alarms are minimized while multiple target detections and tracks are maintained. The fundamental real-time processing methodologies used for ARIS are applied to a COTS multiple field of view system with high-resolution RGB and midwave infrared (MWIR) video rate imaging. The techniques discussed for this four spectral band system can be applied to both extended multispectral systems (greater than four spectral bands) and hyperspectral systems to further enhance system capabilities for analogous terrestrial applications.

The paper presents a novel technique for robust motion analysis in real automotive scenarios based on integrated Retinex-like pre-processing algorithm with block matching video motion estimator. Both algorithmic and real-time hardware design issues are discussed. The benefits of the proposed technique are manifold: the entire system is more robust; the estimated motion vectors are more reliable and less dependent on critical ambient conditions like shadows or flashes; the proposed algorithm may allow to perform motion estimation using very few bits and running as a 2-or 1-bit transform, still maintaining good performances. Real-time hardware implementation is achieved by design and synthesis in 65 nm CMOS standard-cells technology of an Application Specific Instruction-set Processor. Design optimizations for both the processing core and the memory organization are presented. With respect to the state of the art the proposed hardware implementation ensures bounded circuit complexity, low power consumption and reprogrammability of the technique.

Abstract. We introduced a real time Image Processing technique using modern programmable Graphic Processing Units (GPU) in this paper. GPU is a SIMD (Single Instruction, Multiple Data) device that is inherently data-parallel. By utilizing NVIDIA's new GPU Programming ...

Currently, the design of aesthetic products is a process that requires a set of activities where digital models and physical mockups play a key role. Typically, these are modified (and built) several times before reaching the desired design, increasing the development time and, consequently, the final product cost. In this paper, we present an innovative design environment for computer-aided design (CAD) surface analysis. Our system relies on a direct visuo-haptic display system, which enables users to visualize models using a stereoscopic view, and allows the evaluation of sectional curves using touch. Profile curves are rendered using an haptic device that deforms a plastic strip, thanks to a set of actuators, to reproduce the curvature of the shape co-located with the virtual model. By touching the strip, users are able to evaluate shape characteristics, such as curvature or discontinuities (rendered using sound), and to assess the surface quality. We believe that future computer-aided systems (CAS)/CAD systems based on our approach will contribute in improving the design process at industrial level. Moreover, these will allow companies to reduce the product development time by reducing the number of physical mockups necessary for the product design evaluation and by increasing the quality of the final product, allowing a wider exploration and comparative evaluation of alternatives in the given time.

In recent years, there has been a growing interest in realizing methodologies to integrate more and more computation at the level of the image sensor. The rising trend has seen an increased research interest in developing novel event cameras that can facilitate CNN computation directly in the sensor. However, event-based cameras are not generally available in the market, limiting performance exploration on high-level models and algorithms. This paper presents an event camera simulator that can be a potent tool for hardware design prototyping, parameter optimization, attention-based innovative algorithm development, and benchmarking. The proposed simulator implements a distributed computation model to identify relevant regions in an image frame. Our simulator’s relevance computation model is realized as a collection of modules and performs computations in parallel. The distributed computation model is configurable, making it highly useful for design space exploration. The Rendering e...

Human face and hand detection, recognition and tracking are important research areas for many computer interaction applications. Face and hand are considered as human skin blobs, which fall in a compact region of colour spaces. Limitations arise from the fact that human skin has common properties and can be defined in various colour spaces after applying colour normalization. The model therefore, has to accept a wide range of colours, making it more susceptible to noise. We have addressed this problem and propose that ...

This paper deals with real-time image processing of crowded outdoor scenes with the objective of creating an effective traffic management system that monitors urban settings (urban intersections, streets after athletic events, etc.). The proposed system can detect, track, and monitor both pedestrians (crowds) and vehicles. We describe the characteristics of the tracker that is based on a new detection method. Initially, we produce a motion estimation map. This map is then segmented and analyzed in order to remove inherent noise and focus on particular regions. Moreover, tracking of these regions is obtained in two steps: fusion and measurement of the current position and velocity, and then estimation of the next position based on a simple model. The instability of tracking is addressed by a multiple-level approach to the problem. The computed data is then analyzed to produce motion statistics. Experimental results from various sites in the Twin Cities area are presented. The final step is to provide this information to an urban traffic management center that monitors crowds and vehicles in the streets.

Recently, real-time image data generated is increasing not only in resolution but also in amount. This large-scale image originates from a large number of camera channels. There is a way to use GPU for high-speed processing of images, but it cannot be done efficiently by using single GPU for large-scale image processing. In this paper, we provide a new method for constructing a distributed environment using open source called Apache Kafka for real-time processing of large-scale images. This method provides an opportunity to gather related data into single node for high-speed processing using GPGPU or Xeon-Phi processing.

High-Efficiency Video Coding (HEVC) was developed to improve its predecessor standard, H264/AVC, by doubling its compression efficiency. As in previous standards, Motion Estimation (ME) is one of the encoder critical blocks to achieve significant compression gains. However, it demands an overwhelming complexity cost to accurately remove video temporal redundancy, especially when encoding very high-resolution video sequences. To reduce the overall video encoding time, we propose the implementation of the HEVC ME block in hardware. The proposed architecture is based on (a) a new memory scan order, and (b) a new adder tree structure, which supports asymmetric partitioning modes in a fast and efficient way. The proposed system has been designed in VHDL (VHSIC Hardware Description Language), synthesized and implemented by means of the Xilinx FPGA, Virtex-7 XC7VX550T-3FFG1158. Our design achieves encoding frame rates up to 116 and 30 fps at 2 and 4K video formats, respectively.

This paper presents a study of an important improvement in an already installed monochrome vision system for detection of defects during coated board manufacture. This improvement allows the inspection system to detect targeted defect in subsampling conditions (i.e. Nyquist theorem is not satisfied). This improvement is accomplished entirely in the image processing software without the need for additional changes in the vision hardware. Processing software is optimized for SIMD instruction set requiring no improvement in the processing hardware. Results obtained by the use of this improved algorithm provide real time detection of the targeted defect that in turns enables elimination of the waist coated board.

Real-time image processing requires high computational and I͞O throughputs obtained by use of optoelectronic system solutions. A novel architecture that uses focal-plane optoelectronic-area I͞O with a fine-grain, low-memory, single-instruction-multiple-data ͑SIMD͒ processor array is presented as an efficient computational solution for real-time hyperspectral image processing. The architecture is evaluated by use of realistic workloads to determine data throughputs, processing demands, and storage requirements. We show that traditional store-and-process system performance is inadequate for this application domain, whereas the focal-plane SIMD architecture is capable of supporting real-time performances with sustained operation throughputs of 500-1500 gigaoperations͞s. The focal-plane architecture exploits the direct coupling between sensor and parallel-processor arrays to alleviate databandwidth requirements, allowing computation to be performed in a stream-parallel computation model, while data arrive from the sensors.

My research is focused on intelligent systems and human-machine interfaces. In intelligent systems, My focus is on designing algorithms capable of mimicking visual perception. In human-machine interfaces, my focus is designing highly ergonomic systems capable of interacting with humans through natural channels such as natural languages, bodial and facial gestures.

This paper presents a computer vision system that successfully discriminates between weed patches and crop rows under uncontrolled lighting in real-time. The system consists of two independent subsystems, a fast image processing delivering results in real-time (Fast Image Processing, FIP), and a slower and more accurate processing (Robust Crop Row Detection, RCRD) that is used to correct the first subsystem's mistakes. This combination produces a system that achieves very good results under a wide variety of conditions. Tested on several maize videos taken of different fields and during different years, the system successfully detects an average of 95% of weeds and 80% of crops under different illumination, soil humidity and weed/crop growth conditions. Moreover, the system has been shown to produce acceptable results even under very difficult conditions, such as in the presence of dramatic sowing errors or abrupt camera movements. The computer vision system has been developed for integration into a treatment system because the ideal setup for any weed sprayer system would include a tool that could provide information on the weeds and crops present at each point in real-time, while the tractor mounting the spraying bar is moving.

A multiplierless pruned approximate 8-point discrete cosine transform (DCT) requiring only 10 additions is introduced. The proposed algorithm was assessed in image and video compression, showing competitive performance with state-of-the-art methods. Digital synthesis in 45 nm CMOS technology up to place-and-route level indicates clock speed of 288 MHz at a 1.1 V supply. The 8 × 8 block rate is 36 MHz. The DCT approximation was embedded into HEVC reference software; resulting video frames, at up to 327 Hz for 8-bit RGB HEVC, presented negligible image degradation.

Along with the development of powerful processing platforms, heterogeneous architectures are nowadays permitting new design space explorations. In this paper we propose a novel heterogeneous architecture for reliable pedestrian detection applications. It deploys an efficient Histogram of Oriented Gradient pipeline tightly coupled with a neuro-inspired spatio-temporal filter. By relying on hardware-software co-design principles, our architecture is capable of processing video sequences from real-word dynamic environments in real-time. The paper presents the implemented algorithm and details the proposed architecture for executing it, exposing in particular the partitioning decisions made to meet the required performance. A prototype implementation is described and the results obtained are discussed with respect to other state of the art solutions.

the improvement of Internet's network conditions helped realtime applications to flourish. The demand for Ultra High Definition video is constantly increasing. Apart from video and sound a new kind of real-time data is making its appearance, haptic data. The efficient synchronization of video, audio and haptic data is a rather challenging effort. The new High Efficiency Video Coding is quite promising for realtime ultra-high definition video transferring through the Internet. This paper presents related work on High Efficiency Video Coding. It points out the challenges and the synchronization techniques that have being proposed for synchronizing video and haptic data. Comparative tests between H.264 and HEVC are undertaken. Measurements for the network conditions of the Internet are carried out. The equations for the transferring delay of all the inter prediction configurations of the HEVC are defined. Finally, it proposes a new efficient algorithm for transferring a real-time High Efficient Video Coding stream with haptic data through the Internet.

Non Destructive Evaluation (NDE) uses many data processing techniques to transform raw data or information from a specific NDE technique into useful NDE information from which decisions can be made. Historically, the data processing systems associated with each NDE technique were tuned or optimized to that technique . In today's environment it is desirable to quickly evaluate promising techniques while

Large datasets, such as pixels and voxels in 2D and 3D images can usually be reduced during their processing to smaller subsets with less datapoints. Such subsets can be the objects in the image, features-edges or corners-or more general, regions of interest. For instance, the transformation from a set of datapoints representing an image, to one or more subsets of datapoints representing objects in the image, is due to a segmentation algorithm and may involve both the selection of datapoints as well as a change in datastructure. The massive number of pixels in the original image, points to a data parallel approach, whereas the processing of the various objects in the image is more suitable for task parallelism. In this paper we introduce a framework for parallel image processing and we focus on an array of buckets that can be distributed over a number of processors and that contains pointers to the data from the dataset. The benefit of this approach is that the processor activity remains focussed on the datapoints that need processing and, moreover, that the load can be distributed over many processors, even in a heterogeneous computer architecture. Although the method is generally applicable in the processing of sets, in this paper we obtain our examples from the domain of image processing. As this method yields speedups that are data dependent, we derived a run-time evaluation that is able to determine if the use of distributed buckets is beneficial.

In this paper, an implementation of an educational platform for real-time linear and morphological image filtering using a FPGA NexysII, Xilinx ® , Spartan 3E, is described. The system is connected to a USB port of a personal computer, which in that way form a powerful and low-cost design station for educational purposes. The FPGA-based system is accessed through a MATLAB graphical user interface, which handles the communication setup and data transfer. The system allows the students to perform comparisons between results obtained from MATLAB simulations and FPGA-based real-time processing. Concluding remarks derived from course evaluations and lab reports are presented.

Cellular neural networks (CNNs) have locally connected neurons. This characteristic makes CNNs adequate for hardware implementation and, consequently, for their employment on a variety of applications as real-time image processing and construction of efficient associative memories. Adjustments of CNN parameters is a complex problem involved in the configuration of CNN for associative memories. This paper reviews methods of associative memory design based on CNNs, and provides comparative performance analysis of these approaches.

Fourier Descriptors can be used as feature vector components in various applications, such as real-time color object recognition or image retrieval. The full process is composed of the feature extraction followed by a classification step performed using Support Vector Machine (SVM). In order to accelerate the computation of Fourier Descriptors, a hardware implementation using FPGA technology is presented in this paper. We evaluated classification performance with respect to lighting variations and noise sensibility. Several experiments were carried out on three databases. Then an efficient architecture for FD computation on FPGAs is proposed and designed as accelerator. The WildCard is used to prototype this implementation. This design can have an operation speed up of approximately 10 compared to the standard software PC implementation.

In this paper we propose two architectures of cellular neural network (CNN) for edge detection and segmentation of noisy image based on FitzHugh-Nagumo reaction-diffusion equation. These networks give better results compare to other methods and are capable to real-time applications due to parallel processing nature of the CNN. The mathematical description and nonlinear phenomena analysis of the FitzHugh-Nagumo reaction-diffusion equation

In this paper we present the design of an ASIC chip for real-time image processing in industrial applications. The chip is a module of a system for the automatic surface inspection of mechanical parts: it implements the feed-forward phase of a neural network model (multi-layer perceptron with local connections) tuned to the specic application. The design has been performed i n 0. 7 m CMOS technology using an approach based on high level transformations on the VHDL specications. Special emphasis was given to achieve real-time speed. As a result, the architecture i s b ased o n a d e ep pipeline and the performance i s b eyond the real-time specications (1).

There is an increasing demand for lightweight multi-person pose estimation for many emerging smart IoT applications. However, the existing algorithms tend to have large model sizes and intense computational requirements, making them illsuited for real-time applications and deployment on resource-constrained hardware. Lightweight and real-time approaches are exceedingly rare and come at the cost of inferior accuracy. In this paper, we present EfficientHRNet, a family of lightweight multi-person human pose estimators that are able to perform in real-time on resource-constrained devices. By unifying recent advances in model scaling with high-resolution feature representations, EfficientHRNet creates highly accurate models while reducing computation enough to achieve real-time performance. The largest model is able to come within 4.4% accuracy of the current state-of-the-art, while having 1/3 the model size and 1/6 the computation, achieving 23 FPS on Nvidia Jetson Xavier. Compared to the top real-time approach, EfficientHRNet increases accuracy by 22% while achieving similar FPS with 1 3 the power. At every level, EfficientHRNet proves to be more computationally efficient than other bottom-up 2D human pose estimation approaches, while achieving highly competitive accuracy.

This paper deals with real-time image processing of crowded outdoor scenes with the objective of creating an effective traffic management system that monitors urban settings (urban intersections, streets after athletic events, etc.). The proposed system can detect, track and monitor both pedestrians (crowds) and vehicles. We describe the characteristics of the tracker that is based on a new detection method. Initially, we produce a motion estimation map. This map is then segmented and analyzed in order to remove inherent noise and focus on particular regions. Moreover, tracking of these regions is obtained in two steps: fusion and measurement of the current position and velocity, and then estimation of the next position based on a simple model. The instability of tracking is addressed by a multiple-level approach to the problem. The computed data is then analyzed to produce motion statistics. Experimental results from various sites in the Twin Cities area are presented. The final st...

Multi-image super-resolution (MISR) usually outperforms single-image super-resolution (SISR) under a proper inter-image alignment by explicitly exploiting the inter-image correlation. However, the large computational demand encumbers the deployment of MISR in practice. In this work, we propose a distributed optimization framework based on data parallelism for fast large-scale MISR using multi-GPU acceleration named FL-MISR. The scaled conjugate gradient (SCG) algorithm is applied to the distributed subfunctions and the local SCG variables are communicated to synchronize the convergence rate over multi-GPU systems towards a consistent convergence. Furthermore, an inner-outer border exchange scheme is performed to obviate the border effect between neighboring GPUs. The proposed FL-MISR is applied to the computed tomography (CT) system by super-resolving the projections acquired by subpixel detector shift. The SR reconstruction is performed on the fly during the CT acquisition such tha...

Arvand is the name of robots specially designed and constructed by sharif CE team for playing soccer according to RoboCup rules and regulations for the middle size robots. Two different types of robots are made, players and the goal keeper. A player robot consists of ...

Computer vision applications are becoming more and more popular in embedded systems such as drones, robots, tablets, and mobile devices. These applications are both compute and memory intensive, with memory bound stalls (MBS) making a significant part of their execution time. For maximum reduction in memory stalls, compilers need to consider architectural details of a platform and utilize its hardware components efficiently. In this paper, we propose a compiler optimization for a vision-processing system through classification of memory references to reduce MBS. As the proposed optimization is based on the architectural features of a specific platform, i.e., Myriad 2, it can only be applied to other platforms having similar architectural features. The optimization consists of two steps: affinity analysis and affinity-aware instruction scheduling. We suggest two different approaches for affinity analysis, i.e., source code annotation and automated analysis. We use LLVM compiler infra...

The level-set method, a technique for the computation of evolving interfaces, is a solution commonly used to segment images and volumes in medical applications. GPUs have become a commodity hardware with hundreds of cores that can execute thousands of threads in parallel, and they are nowadays ideal platforms to execute computational intensive tasks, such as the 3D level-set-based segmentation, in real time. In this paper we propose two GPU implementations of the level-set-based segmentation method called Fast Two-Cycle. Our proposals perform computations in independent domains called tiles and modify the structure of the original algorithm to better exploit the features of the GPU. The implementations were tested with real images of brain vessels and a synthetic MRI image of the brain. Results show that they execute faster than a CPU-sequential implementation of the same method, without any significant loss of the segmentation quality and without requiring distributed parallel computer infrastructures.

Two crucial aspects of general-purpose embedded visual point tracking are addressed in this paper. First, the algorithm should reliably track as many points as possible. Second, the computation should achieve real-time video processing, which is challenging on low power embedded platforms. We propose a new multi-scale semidense point tracker called Video Extruder, whose purpose is to fill the gap between short-term, dense motion estimation (optical flow) and long-term, sparse salient point tracking. This paper presents a new detector, including a new salience function with low computational complexity and a new selection strategy that allows to obtain a large number of keypoints. Its density and reliability in mobile video scenarios are compared with those of the FAST detector. Then, a multi-scale matching strategy is presented, based on hybrid regional coarse-to-fine and temporal prediction, which provides robustness to large camera and object accelerations. Filtering and merging strategies are then used to eliminate most of the wrong or useless trajectories. Thanks to its high degree of parallelism, the proposed algorithm extracts beams of trajectories from the video very efficiently. We compare it with the state-of-the-art pyramidal Lucas-Kanade point tracker and show that, in short range mobile video scenarios, it yields similar quality results, while being up to one order of magnitude faster. Three different parallel implementations of this tracker are presented, on multi-core CPU, GPU and ARM SoCs. On a commodity 2010 CPU, it can track 8,500 points in a 640 9 480 video at 150 Hz.

The Optical Character Recognition (OCR) is a process that converts characters within images into text documents. In paperless applications, OCR systems have to ensure a better accuracy as well as a high speed. One of the most important steps in OCR is binarization. In this context, we proposed recently the Hybrid Binarization based-Kmeans method (HBK) [1]. HBK offers a satisfying recognition rate while scoring 91% accuracy. In the other hand, running on an Intel Core i3 CPU processor, the HBK requires at least 1.9 seconds to process one A4 300 dpi document. However, binarization step should not exceed 460 ms in our real time OCR system. For this, we propose in this paper a parallel implementation of the HBK method on the NVIDIA GTX 660 Graphic Processing Unit (GPU). Our implementation, combines finegrained and coarse-grained parallelism strategies for the best GPU use. In addition, the costly CPU-GPU communication overhead is avoided and an efficient memory management is ensured. The effectiveness of our implementation is validated through extensive experiments, which demonstrate that the proposed HBK parallelization accelerates the studied process. Indeed, we ensure the binarization of one document in just 425 ms. Consequently, the implemented design is able to meet the targeted real time OCR system in paperless application. Keywords OCR • Scanned documents • Binarization • HBK • Kmeans • CUDA • GPU 1 Introduction Optical Character Recognition (OCR) [2] is the process that recognizes numerical characters on printed pages

In this paper, we introduce several new schemes for calculation of discrete wavelet transforms of images. These schemes reduce the number of steps and, as a consequence, allow to reduce the number of synchronizations on parallel architectures. As an additional useful property, the proposed schemes can reduce also the number of arithmetic operations. The schemes are primarily demonstrated on CDF 5/3 and CDF 9/7 wavelets employed in JPEG 2000 image compression standard. However, the presented method is general, and it can be applied on any wavelet transform. As a result, our scheme requires only two memory barriers for 2-D CDF 5/3 transform compared to four barriers in the original separable form or three barriers in the non-separable scheme recently published. Our reasoning is supported by exhaustive experiments on high-end graphics cards.

This paper proposes a low-complexity convolutional neural network (CNN) for super-resolution (SR). The proposed deep-learning model for SR has two layers to deal with horizontal, vertical, and diagonal visual information. The front-end layer extracts the horizontal and vertical high-frequency signals using a CNN with one-dimensional (1D) filters. In the high-resolution image-restoration layer, the high-frequency signals in the diagonal directions are processed by additional two-dimensional (2D) filters. The proposed model consists of 1D and 2D filters, and as a result, we can reduce the computational complexity of the existing SR algorithms, with negligible visual loss. The computational complexity of the proposed algorithm is 71.37%, 61.82%, and 50.78% lower in CPU, TPU, and GPU than the very-deep SR (VDSR) algorithm, with a peak signal-to-noise ratio loss of 0.49 dB.

Object detection and tracking are main tasks in video surveillance systems. Extracting the background is an intensive task with high computational cost. This work proposes a hardware computing engine to perform background subtraction on low cost FPGAs, focused on resource-limited environments. Our approach is based on the codebook algorithm and offers very low degradation. We have analyzed resource consumption and performance trade-offs in Spartan-3 FPGAs by Xilinx. In addition, an accuracy evaluation with standard benchmark sequences has been performed, obtaining better results than other hardware approaches. The implementation is able to segment objects in sequences with resolution 768x576 at 50 fps using a robust and accurate approach, and an estimated power consumption of 5.13 W.