A Computer Vision System to Read Meter Displays

16th IMEKO TC4 Symposium Exploring New Frontiers of Instrumentation and Methods for Electrical and Electronic Measurements Sept. 22-24, 2008, Florence, Italy A COMPUTER VISION SYSTEM TO READ METER DISPLAYS Danilo Alves de Lima1, Guilherme Augusto Silva Pereira2, Flávio Henrique de Vasconcelos3 Department of Electric Engineering, School of Engineering, Av. Antônio Carlos, 6627 – Campus UFMG Pampulha, Escola de Engenharia Bloco-1, sala 2116 Universidade Federal de Minas Gerais, Belo Horizonte, Brasil. C.E.P. 31.270-010 1 danetteduklan@yahoo.com.br, 2gpereira@dee.ufmg.br, 3fvasc@dee.ufmg.br Topics of interest: Automated Test and Measurement Systems Abstract: Since the beginning of the movement for quality management programs based on ISO 9000 or/and ISO/IEC 17025 that were adopted by many type of industries, calibration laboratories have been under pressure to increase productivity. This work is about a computational system to automate the entire instrument calibration process. The computer vision system proposed is to be used to read the display of both analogue as well as digital instruments that do not have communication interfaces to computers such as GPIB or RS-232. In order to figure out instrument indication, the system employs an optical character recognition technique in digital displays and Canny’s method for edge detection and the Hough’s transform method for line localization in analogue displays, accelerating the data acquisition process and makes it less prone to errors. Consequently, it contributes to improve calibration and reduce costs, increasing the number of instruments with quality assured measurements. Keywords: computer vision, automation, instrument’s calibration. 1. Introduction The human work in repetitive tasks is very prone to errors. In the calibration of measurement instruments, for example, a technician is in charge of writing down all the data, one by one, on a spreadsheet. If by some reason like fatigue he or she makes a mistake in one or more values, the calibration process and its credibility can be compromised. In order to avoid this sort of problems and to increase productivity in calibration laboratories, the greater the level of automation the better. In many modern instruments the calibration process is very easy to be automated, since the acquired data can be transferred to a host computer through a communication interface. However, for some digital or classic analogue meters, the absence of a computer interface may prevent the automation of the calibration process. In order to provide an interface between the instrument and a host computer, a computer vision system seems to be a nice solution. As an important tool, computer vision has been widely used in the automation of many processes. Examples such as adjustment or verification of car speedometers in the automotive industry [1] are becoming commons. Also, this has been associated to computational intelligence systems for the automatic recognition of vehicles plates for traffic control. Some previous work on using computer vision to calibrate measurement instrument can be found in the literature. Foiatto et al. [2], for example, presented a system that is able to read digital instrument displays. The reading of analogue displays was not contemplated by the authors. Another similar work was done by Andres [3], whom have proposed a system for reading digital and analogue instrument displays, but that relies on user intervention for many tasks. In this paper a computer vision system for reading the display of digital and analogue instruments is presented. Some interesting metrological questions arise from the fact that the computer vision seems to contribute to increase reading resolution for analogue displays and therefore, improving the original instrument accuracy. The system presented here is built to read a large variety of displays with good performance and efficiency, despite the fact of employ ordinary computer vision devices. This article was divided in four sections. The structure of the algorithm for reading digital and analogue instruments display is presented and the way it works is explained in Section 2. The 16th IMEKO TC4 Symposium Exploring New Frontiers of Instrumentation and Methods for Electrical and Electronic Measurements Sept. 22-24, 2008, Florence, Italy effectiveness of the system is discussed and experimental results are presented in section 3. Finally, in section 4 considerations and perspectives is presented. 2. The computer vision system In the next subsections, it is proposed a computer vision system to read digital and analogue instrument displays. A. The solution structure Figure 1 shows the block diagram that represent the process steps for digit recognition of digital displays and cursors recognition of analogue instrument displays. To follow this diagram, every block will be detailed for each type of meter. Im a g e Ac q uisitio n Disp la y De limita tio Ed g e De te c tio n Ho ug h Tra nsfo rm fo r Line s De te c tio n Po inte r re c o g nitio n a nd re turn o f the re a d Bina ry Ima g e Tra nsfo rma tio Dig it Re c o g nitio n Re turn o f the Dig its Va lue Figure 1 Steps for image capture and meter indication recognition for digital (right) and analogue (left) instrument displays. B. Digital instruments reading Following the diagram presented, first there are the image caption and the delimitation of the viewfinder blocks. Although these stages are common for both types of instruments, the case for the analogue instruments will be presented in Section 2.3. Once an initial image of the digital instrument has been acquired, the user must employ the computer mouse to delimit the region of interest of the display, what will result in a smaller picture, with the numbers in prominence, facilitating the processing for the subsequent images. Although this step could be done automatically, since it is very critical to the rest of the process, it was chosen to perform it manually. If the system is connected to a data base, the display position for each type of instrument could be stored to be used in later calibration section. In the delimited image the upper, down, left and right limits of each digit must be recognized, so that each one of them can be treated separately. In this step a method based on binary images was used (two levels of color, black and white) [6]. Images of this type reduce the time spent in the processing, improving the performance. After the image is binaryzed, vertical and horizontal cross projection are created. Cross projection are the normalized sum of the black pixels for each line and column. These process returns zero values or very near to it for columns separations and/or the beginning and the end of lines, where there is a little incidence of black pixels. Figure 2 has a sample of this method. Values 16th IMEKO TC4 Symposium Exploring New Frontiers of Instrumentation and Methods for Electrical and Electronic Measurements Sept. 22-24, 2008, Florence, Italy down to 0.02 (maximum tolerance for noises) has been considered as points of division between the digits. After this step, each digit of the viewfinder is delimited. The next step is the classification of each digit, according to Figure 1. It is proposed a method based on active segments, which relies on the comparison between a standard digit represented with seven segments and a display digit. Figure 3 presents a standard number with its 7 segments. Even though some instruments do not use 7 segments displays, this method has been shown to be useful to identify digits from several instruments. If a particular instrument has a digit that prevents this method to work, it is still possible to store different digits in a data base system and use a specific standard digit for each sort of instrument. To be robust to digit segmentation, the segments regions were delimited so that they are bigger than the actual segment. The percentage of black pixels inside the regions is them used to decide whether a segment is active or not. At the end, the segments information is concatenated to form the digit. Next section will discuss how the proposed system reads analogue instruments. Figure 2. Original image, horizontal and vertical projections and their lower limits. Fig. 3. Segments in the standard number. C. Analogue instruments reading For analogue measuring, the recognition task is related to the angle position (inclination) of the needle. Following the same procedure used for digital meters, the area of the display image that encloses the needle has to be bounded. This region is the largest space where the needle moves with less interference of the background, such as scale and other characters. The example of a good quality area of interest is shown in figure 4. Figure 4 also presents the result of the Canny’s method [6] for edge detection, which is important to get a more distinct needle image. Subsequently, the Hough’s transform method for line localization [7] is applied. To improve the result, it is possible to control some characteristic of the straight line as the minimum size and thickness. These parameters are available for real time adjustment by the user and it may also be stored, for each instrument, creating a data base for the calibration system. 16th IMEKO TC4 Symposium Exploring New Frontiers of Instrumentation and Methods for Electrical and Electronic Measurements Sept. 22-24, 2008, Florence, Italy With the pointer straight line recognized, it is necessary to apply algebraic relationships that associate the straight line and its inclination with the pointed value. These relations depend of the construction of the measurement instrument and can be linear and non-linear. Considering a linear scale and two given values with their respective angles, it is possible to get all the other values of the instrument scale. In some cases, the construction of the instrument is symmetric in some axis, making the process easier because only the angle of the 0 value and the final scale value need to be provided. The angle of the final value can be founded with this symmetry as shown in Figure 5. The recognized pointer straight line is also showed in this figure. It is important to mention that the method presented in Figure 5 needs the instrument perfectly horizontally lined up with the camera. Therefore, the system needs a physical guide to control the disposal of the instrument under the camera. Fig. 4. Binary image of the display obtained after the Canny’s method and its corresponding region of interest (ROI) selected by the user. Fig. 5. Symmetry of some analogue meters Experimental results obtained for both analogue and digital instruments will be presented in the next section. 3. Results This section presents results that illustrate the performance of the implemented system. A. Digital meters The method employed presented a quite acceptable result, considering the resources available. The image capture system is comprised of a webcam located in a room without luminosity control and a computer. The percentage level of right readings was 100% when the display of the instrument was well visualized and all the digits defined. These conditions can be easily obtained. The effectiveness of the system was expected since when a segment is inactive the percentage of black pixels (or possibility to be active) is very low in a low noise image. Most of the time, the system is able to correctly detect the digits when the human user can do too. Thus, when noises in the images occur they can easily be treated by changing the conditions of acquisition. In a calibration environment it is supposed that these conditions can be controlled. B. Analogue meters Simple tests were performed in the system for analogue display meters. Despite the poor quality of the camera used and the absence of controlled illumination, the needle image is blurred only to a small amount of up to ±1,66°, accounting for image resolution. As shown in figure 6, when the applied voltage is zero the instrument needle is in its initial position and the angle with the horizontal line was found to be 45º. Considering that the scale spans from 0 to 150 volts, each measured value determined 16th IMEKO TC4 Symposium Exploring New Frontiers of Instrumentation and Methods for Electrical and Electronic Measurements Sept. 22-24, 2008, Florence, Italy with this system presents an uncertainty due to image resolution which is approximately equal to ±1 volt. This value matches with that due to the meter scale resolution, when evaluating the uncertainty budget. The measurement model for uncertainty determination with a conventional method based on user’s reading has to include a compensation factor related to instrument scale resolution, besides measurement expected value, a compensation factor due to the standard, among others. In the case of computational vision system with an improved vision system resolution, a more precise reading may lead to a mistaken uncertainty evaluation if the uncertainty due to image resolution is taken instead of the uncertainty due to instrument scale resolution, like in conventional calibration. Figure 6. Result for analogue meter. Recognized angle=58.24° and meter indication=22 volts. 4. Conclusion This paper described the development of a system for reading the display of digital and analogue measurement instruments using computational vision. For instruments with a digital display, the system presented robustness and 100% of accuracy. For analogue instruments the results presented an additional uncertainty component related to the imperfections of the image acquisition system. For a more definitive calibration system, a better camera has to be used in order to improve the system. The implemented system is algorithmically simple. Because of this, it is not completely automatic, being necessary a little user intervention at the beginning of the process. On the other hand, real time performance is obtained. In parallel to the development of the system, a friendly user interface was developed. This allows the visualization of the results for digital as well as for analogue instruments that can easily be fitted in a more complex system for automated calibration of instrument meters. References [1] A. WILSON, “System tests and calibrates instruments”. Vision Systems Design Magazine. December 2004. Available in: <http://vsd.pennnet.com/Articles/Article_Display.cfm?Section=Archives&Subsection= Display&ARTICLE_ID=218277&KEYWORD=automotive>. Acessed in: 5 ago. 2005>. [2] N. Foiatto, J. Roeche, M. Castro, (in portuguese) “Sistema automatizado de calibração para medidores digitais a partir da captura de imagens e interface de comunicação GPIB (IEEE 488)”, In Seminário Internacional de Metrologia Elétrica, 6, RJ. 21-23 set. 2005. Anais. Rio de Janeiro, 2005. 332 p. [3] F. Alegria, (in portuguese) “Calibração automática de aparelhos de medida”.M.Sc thesis. Universidade Técnica de Lisboa. Out. 1997. [4] OpenCV (2007). http://www.intel.com/technology /computing/opencv/. [5] J. Parker, Algorithms for image processing and computer vision. Canadá: John Wiley & Sons, 1997. p. 417. [6] R. O. Duda, P.E. Hart, "Use of the Hough transform to detect lines and curves in pictures", Com. of ACM, Vol. 18(9), pp. 509-517, 1975.