Abstract
We investigate maximum likelihood parameter learning in Conditional Random Fields (CRF) and present an empirical study of pseudo-likelihood (PL) based approximations of the parameter likelihood gradient. We show, as opposed to [1][2], that these parameter learning methods can be improved and evaluate the resulting performance employing different inference techniques. We show that the approximation based on penalized pseudo-likelihood (PPL) in combination with the Maximum A Posteriori (MAP) inference yields results comparable to other state of the art approaches, while providing advantages to formulating parameter learning as a convex optimization problem. Eventually, we demonstrate applicability on the task of detecting man-made structures in natural images.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Kumar, S., Hebert, M.: Discriminative random fields: A discriminative framework for contextual interaction in classification. In: Proc. of the 9th IEEE International Conference on Computer Vision, vol. 2, pp. 1150–1157 (2003)
Kumar, S., August, J., Hebert, M.: Exploiting inference for approximate parameter learning in discriminative fields: An empirical study. In: 5th International Workshop on Energy Minimization Methods in Computer Vision and Pattern Recognition (2005)
Lafferty, J., McCallum, A., Pereira, F.: Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In: Proc. 18th International Conf. on Machine Learning, pp. 282–289 (2001)
Korč, F., Förstner, W.: Interpreting terrestrial images of urban scenes using Discriminative Random Fields. In: Proc. of the 21st Congress of the International Society for Photogrammetry and Remote Sensing (ISPRS) (July 2008)
Kumar, S., Hebert, M.: Discriminative random fields. International Journal of Computer Vision 68(2), 179–201 (2006)
Vishwanathan, S.V.N., Schraudolph, N.N., Schmidt, M.W., Murphy, K.: Accelerated training of conditional random fields with stochastic gradient methods. In: Cohen, W.W., Moore, A. (eds.) Proc. of the 24th International Conf. on Machine Learning. ACM International Conference Proceeding Series, vol. 148, pp. 969–976. ACM Press, New York (2006)
Sutton, C., McCallum, A.: Piecewise pseudolikelihood for efficient training of conditional random fields. In: Ghahramani, Z. (ed.) International Conference on Machine learning (ICML). ACM International Conference Proceeding Series, vol. 227, pp. 863–870 (2007)
Lee, C.-H., Wang, S., Jiao, F., Schuurmans, D., Greiner, R.: Learning to model spatial dependency: Semi-supervised discriminative random fields. In: Schölkopf, B., Platt, J., Hoffman, T. (eds.) Advances in Neural Information Processing Systems 19, pp. 793–800. MIT Press, Cambridge (2007)
Kumar, S., Hebert, M.: Multiclass discriminative fields for parts-based object detection. In: Snowbird Learning Workshop (2004)
Besag, J.: Statistical analysis of non-lattice data. The Statistician 24(3), 179–195 (1975)
Besag, J.: Efficiency of pseudo-likelihood estimation for simple gaussian fields. Biometrika 64, 616–618 (1977)
Geiger, D., Girosi, F.: Parallel and deterministic algorithms from mrfs: Surface reconstruction. IEEE Trans. on Pattern Analysis and Machine Intelligence 13(5), 401–412 (1991)
McCallum, A., Rohanimanesh, K., Sutton, C.: Dynamic conditional random fields for jointly labeling multiple sequences. In: NIPS Workshop on Syntax, Semantics, and Statistics (December 2003)
Hinton, G.E.: Training products of experts by minimizing contrastive divergence. Neural Computation 14(8), 1771–1800 (2002)
Greig, D.M., Porteous, B.T., Seheult, A.H.: Exact maximum a posteriori estimation for binary images. Journal of the Royal Statistical Society 51(2), 271–279 (1989)
Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. IEEE Trans. on Pattern Analysis and Machine Intelligence 23(11), 1222–1239 (2001)
Besag, J.: On the statistical analysis of dirty pictures. Journal of the Royal Statistical Society 48(3), 259–302 (1986)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Korč, F., Förstner, W. (2008). Approximate Parameter Learning in Conditional Random Fields: An Empirical Investigation. In: Rigoll, G. (eds) Pattern Recognition. DAGM 2008. Lecture Notes in Computer Science, vol 5096. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-69321-5_2
Download citation
DOI: https://doi.org/10.1007/978-3-540-69321-5_2
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-69320-8
Online ISBN: 978-3-540-69321-5
eBook Packages: Computer ScienceComputer Science (R0)