Advanced Machine Learning

The prerequisite is the completion of the course Data Analysis with Machine Learning or a similar course covering the fundamentals of machine learning.

Meta learning: no-free lunch theorem for machine learning, learning about learning, attribute representation of data sets, parametrization of learning algorithms, optimizing the parameter settings of learning algorithms, surrogate models.

Handling background knowledge in machine learning: equation discovery from data and knowledge, relational learning and surrogate models, hierarchically structured background knowledge (taxonomies), background knowledge and (deep) artificial neural networks.

Deep learning and neural networks: handling different objective functions and back propagation, multi-layer feed-forward neural networks, neural networks for images and sequential data (recurrent neural networks, attention mechanisms, and architectures encoder, decoder, and autoencoder), embedding of complex data into vector spaces (graph and recursive neural networks), generative neural netwroks and large language models.

1. S. J. D. Prince: Understanding deep learning. MIT Press, Cambridge, Massachusetts, 2023.
2. T. Hastie, R. Tibshirani, J. Friedman: The elements of statistical learning : data mining, inference, and prediction, 2nd ed., New York : Springer, 2017.

Predavatelj poleg tega izbere tudi primerne novejše raziskovalne članke.

Students master advanced machine learning methods, such as deep learning and generative neural networks, meta learning and automatic configuration of learning algorithms, knowledge-intensive learning and learning models of dynamical systems. Students through seminars and homework apply the mastered knowledge on various tasks of upgrading existing algorithms and building predictive models from data and formalized knowledge.

Knowledge and understanding: Understanding concepts and components of the machine learning algorithms.

Application: Applying existing algorithms and tailoring/upgrading algorithms for solving practical problems in various scientific and engineering fields.

Reflection: Critical insight into the inner workings of the machine learning algorithms and identifying opportunities for their improvement, formal representation of practical problems that allow for solutions based on machine learning.

Transferable skills: Ability to identify, formulate and solve practical problems. Ability to design predictive models with machine learning algorithms. Critical assessment of scientific literature.

lectures, seminars, excercises, homework and consultations

Homework

Oral exam

grading: 5 (fail), 6-10 (pass) (according to the Statute of UL)

Ljupčo Todorovski:
– MEŽNAR, Sebastian, DŽEROSKI, Sašo, TODOROVSKI, Ljupčo. Efficient generator of mathematical expressions for symbolic regression. Machine Learning, 2023, vol. 112 (str. 4563–4596)
– BRENCE, Jure, DŽEROSKI, Sašo, TODOROVSKI, Ljupčo. Dimensionally-consistent equation discovery through probabilistic attribute grammars. Information Sciences, 2023, vol. 632 (str. 742–756)
– BAUER, Andrej, PETKOVIĆ Matej, TODOROVSKI, Ljupčo. MLFMF: data sets for machine learning for mathematical formalization. NIPS-23: Proceedings of the 37th International Conference on Neural Information Processing Systems, 2023 (str. 50730–50741)

Matija Pretnar:
– PLOTKIN, Gordon, PRETNAR, Matija. Handling algebraic effects. Logical methods in computer science, ISSN 1860-5974, 2013, vol. 9, iss. 4, paper 23 (str. 1-36) [COBISS-SI-ID 16816729]
– PRETNAR, Matija. Inferring algebraic effects. Logical methods in computer science, ISSN 1860-5974, 2014, vol. 10, iss. 3, paper 21 (str. 1-43) [COBISS-SI-ID 17190745]
– BAUER, Andrej, PRETNAR, Matija. An effect system for algebraic effects and handlers. Logical methods in computer science, ISSN 1860-5974, 2014, vol. 10, iss. 4, paper 9 (str. 1-29). http://arxiv.org/pdf/1306.6316 [COBISS-SI-ID 17191001]