Skip to main content
Springer Nature Link
Log in

Building a user-centered semantic hierarchy in image databases

  • Regular Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

Recent development in the field of digital media technology has resulted in the generation of a huge number of images. Consequently, content-based image retrieval has emerged as an important area in multimedia computing. Research in human perception of image content suggests that the semantic cues play an important role in image retrieval. In this paper, we present a new paradigm to establish the semantics in image databases based on multi-user relevance feedback. Relevance feedback mechanism is one way to incorporate the users’ perception during image retrieval. By treating each feedback as a weak classifier and combining them together, we are able to capture the categories in the users’ mind and build a user-centered semantic hierarchy in the database to support semantic browsing and searching. We present an image retrieval system based on a city-landscape image database comprising of 3,009 images. We also compare our approach with other typical methods to organize an image database. Superior results have been achieved by the proposed framework.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
€32.70 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (France)

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Smeulders A.W.M., Worring M., Santini S., Gupta A., Jain R. (2000). Content-based image retrieval at the end of the early years. IEEE Trans. Pattern Anal. Mach. Intell. 22(12):1349–1380

    Article  Google Scholar 

  2. Mehrotra, S., Rui, R., Ortega, M., Huang, T.S.: Supporting content-based queries over images in mars. In: Proceedings of IEEE International Conference on Multimedia Computing Systems, pp. 632–633 (1997)

  3. Ma, W.Y., Manjunath, B.: Netra: A toolbox for navigating large image databases. In: Proceedings of IEEE ICIP, pp. 568–571 (1997)

  4. Gupta A., Jain R. (1997). Visual information retrieval. Commun. ACM 40(5):70–79

    Article  Google Scholar 

  5. Rogowitz B., Frese T., Smith J., Bouman C., Kalin E. (1998). Perceptual image smilarity experiments. Proc. SPIE 8:576–590

    Article  Google Scholar 

  6. Wang J., Li J., Wiederhold G. (2002). Simplicity: semantics-sensitive integrated matching for picture libraries. IEEE Trans. PAMI 23(9):947–963

    Google Scholar 

  7. Vailaya A., Figueiredo M.A.T., Jain A.K., Zhang H.J. (2001). Image classification for content-based image retrieval. IEEE Trans. Image Process. 10(1):117–129

    Article  MATH  Google Scholar 

  8. Chen J., Bouman C.A., Dalton J.C. (2000). Hierarchical browsing and search of large image databases. IEEE Trans. Image Process. 9:442–455

    Article  Google Scholar 

  9. Ji C., Ma S. (1997). Combinations of weak classifiers. IEEE Trans. Neural Netw. 8:32–42

    Article  Google Scholar 

  10. Bishop C.M. (1995). Neural Networks for Pattern Recognition. University Press, New York

    Google Scholar 

  11. Kuncheva L. (2002). A theoretical study on six classifier fusion strategies. IEEE Trans. Pattern Anal. Mach. Intell. 24(2):281–286

    Article  Google Scholar 

  12. Ponnusamy, A., Breen, C., Khan, L., Wang, L.: Ontology-based image classification using neural networks. In: SPIE: The International Society for Optical Engineering, Boston (2002)

  13. Guo G.-D., Jain A.K., Ma W.-Y., Zhang J.-J. (2002). Learning similarity measure for natural image retrieval with relevance feedback. IEEE Trans. Neural Netw. 13(4):811–820

    Article  Google Scholar 

  14. Schapire, R.: Theoretical views of boosting and applications. In: 10th International Conference on Algorithmic Learning Theory, pp. 13–25 (1999)

  15. Li J., Wang J. (2003). Automatic linguistic indexing of pictures by a statistical modeling approach. IEEE Trans. Pattern Anal. Mach. Intell. 25(9):1075–1088

    Article  Google Scholar 

  16. Mori, Y., Takahashia, H., Oka, R.: Image-to-word transformation based on dividing and vector quantizing images with words. In: Proceedings of MISRM (1999)

  17. Duygulu, P., Barnard, K., de Freitas N., Forsyth, D.: Object recognition as machine translation: learning a lexicon for a fixed image vocabulary. In: Proceedings of ECCV (2002)

  18. Jeon, J., Lavrenko, V., Manmatha, R.: Automatic image annotation and retrieval using cross-media relevance models. In: Proceedings of ACM SIGIR (2003)

  19. Barnard K., Duygulu P., de Fretias N., Forsyth D., Blei D., Jordan M.I. (2003). Matching words and pictures. J. Mach. Learn. Res. 3:1107–1135

    Article  MATH  Google Scholar 

  20. Lu, Y., Hu, C., Zhu, X., Zhang, H.J., Yang, Q.: A unified framework for semantics and feature based relevance feedback in image retrieval systems. In: Proceedings of ACM Multimedia (2000)

  21. Lee, C.S., Ma, W.-Y., Zhang, H.J.: Information embedding based on users relevance feedback for image retrieval. In: Proceedings of SPIE International Conference on Multimedia Storage and Archiving Systems (1999)

  22. Fournier, J., Cord, M.: Long-term similarity learning in content-based image retrieval. In: Proceedings of IEEE International Conference on Image Processing (2002)

  23. Yang, J., Li, Q., Zhuang, Y.: Image retrieval and relevance feedback using peer indexing. In: Proceedings of IEEE International Conference on Multimedia and Expo (2002)

  24. Li M., Chen Z., Zhang H. (2002). Statistical correlation analysis in image retrieval. Pattern Recogn. 35:2687–2693

    Article  MathSciNet  MATH  Google Scholar 

  25. Zhuang, Y., Yang, J., Li, Q., Pan, Y.: A graphic-theoretic model for incremental relevance feedback in image retrieval. In: Proceedings of IEEE International Conference on Image Processing (2002)

  26. Heisterkamp, D.R.: Building a latent semantic index of an image database from patterns of relevance feedback. In: Proceedings of 16th International Conference on Pattern Recognition (2002)

  27. He X., King O., Ma W.-Y., Li M., Zhang H.-J. (2003). Learning a semantic space from user’s relevance feedback for image retrieval. IEEE Trans. Circuits Syst. Video Technol. 13(1):39–48

    Article  Google Scholar 

  28. Hoi S.C.H., Lyu M.R., Jin R. (2006). A unified log-based relevance feedback scheme for image retrieval. IEEE Trans. Knowl. Data Eng. 18(4):509–524

    Article  Google Scholar 

  29. Kohrs, A., Merialdo, B.: Improving collaborative filtering with multimedia indexing techniques to create user-adapting web sites. In: Proceedings of ACM International Conference on Multimedia (1999)

  30. Zhang, H.J., Zhong, D.: A scheme for visual feature based image indexing. In: Proceedings of SPIE Conference on Storage Retrieval Image Video Databases, San Jose, pp. 36–46 (1995)

  31. Zhong, D., Zhang, H.J., Chang, S.F.: Clustering methods for video browsing and annotation. In: Proceedings of SPIE Conference on Storage Retrieval Image Video Databases IV, San Jose, pp. 239–246 (1996)

  32. Tieu K., Viola P. (2000). Boosting image retrieval. Proc. IEEE CVPR 1:228–235

    Google Scholar 

  33. Santini S., Gupta A., Jain R. (2001). Emergent semantics through interaction in image databases. IEEE Trans. Knowl. Data Eng. 13(3):337–351

    Article  Google Scholar 

  34. Picard R., Minka T., Szummer M. (1996). Modeling subjectivity in image libraries. Proc. IEEE ICIP. 1:777–780

    Article  Google Scholar 

  35. Rui Y., Huang T., Ortega M., Mehrotra S. (2002). Relevance feedback: A power tool in interactive content-based image . IEEE Trans. Circuits Syst. Video Tech. 13(4):811–820

    Google Scholar 

  36. Chawla N., Moore T., Hall L., Bowyer K., Kegelmeyer W., Springer C. (2003). Distributed learning with bagging-like performance. Pattern Recogn. Lett. 24:455–471

    Article  Google Scholar 

  37. Vailaya A., Jain A.K., Zhang H.J. (1998). On image classification: City vs. landscape. Pattern Recogn. 31:1921–1936

    Article  Google Scholar 

  38. Vailaya A., Zhang H., Liu F., Jain A.K. (2002). Automatic image orientation detection. IEEE Trans. Image Process. 11:746–755

    Article  Google Scholar 

  39. Canny J. (1986). A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. 8:679–698

    Article  Google Scholar 

  40. Jain A., Vailaya A. (1996). Image retrieval using color and shape. Pattern Recogn. 29:1233–1244

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Machine Vision and Pattern Recognition Lab at Department of Computer Science, Wayne State University, Detroit, MI, 48202, USA

    Manjeet Rege & Ming Dong

  2. Database and Multimedia Systems Group at Department of Computer Science, Wayne State University, Detroit, MI, 48202, USA

    Farshad Fotouhi

Authors
  1. Manjeet Rege
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ming Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Farshad Fotouhi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Manjeet Rege.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Rege, M., Dong, M. & Fotouhi, F. Building a user-centered semantic hierarchy in image databases. Multimedia Systems 12, 325–338 (2007). https://doi.org/10.1007/s00530-006-0049-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-006-0049-6

Keywords

Search

Navigation