Ateneo - UPO

The content traces the evolution of artificial intelligence, with a strong focus on LLMs and their practical applications. It briefly covers machine learning and neural network types and then delves into advanced LLM techniques, such as tokenization, embeddings, transformers, prompts engineering, and agents. Particular emphasis is placed on the practical applications of LLMs, in particular in two areas:
- I use them as application writing assistants
- infusion of intelligence into applications.

"Hands-On Large Language Models" by Jay Alammar, Maarten Grootendorst - O’Reilly Media, 2024
"Build a Large Language Model (From Scratch)" by Sebastian Raschka - Manning, 2024

Develop skills in designing and implementing AI-based applications, with a particular focus on LLMs and their architectures.

Acquire the ability to critically evaluate the performance and limitations of AI models, distinguishing between different deployment solutions.

Explore advanced tools and techniques for optimizing and integrating LLMs into applications, such as Prompt Engineering, Retrieval Augmented Generation, Workflows, and Agents.

Basic knowledge of the Python programming language.
Basic knowledge of linear algebra.

The course is divided into approximately 36 hours of didactic teaching and 12 hours of interactive teaching to be carried out in the laboratory.

Students with physical disabilities, Learning Disabilities or Special Education Needs can request specific services and tools via the "Staff Sviluppo e Coordinamento Carriere e Servizi alle Studentesse e agli Studenti", consulting the University webpage: https://www.uniupo.it/en/services/services-students-physical-or-learning-disabilities. Students with disabilities, learning disabilities or special education needs, once they have contacted the University Staff, can refer to the tutor in charge of the course to define the examination modalities, concerning academic aspects.

The exam of the course on LLMs consists of two parts:
- Practical project (2/3 of the final grade), carried out individually or in groups (maximum 3 people).
- Written exam (1/3 of the final grade), taken individually.

The submission and discussion of the project is required to access the written exam. To pass the exam, both parts must achieve at least 18/30.

Knowledge and understanding
- The written exam verifies theoretical and methodological understanding of LLMs, including architectures, training techniques, tools, and applications.
- During submission/discussion, the instructor assesses knowledge of the concepts underlying the project.
- The project report must clearly document architectural choices, tools, techniques, and design principles.

Applying knowledge and understanding
- The project evaluates the ability to apply AI and LLM techniques to solve concrete cases, ensuring functionality and correctness of results.
- Students may propose their own project, subject to instructor approval, encouraging creativity and autonomy.
- The submission and discussion verify the actual functioning of the system and mastery of the tools used.

Communication skills
- The project report must clearly present objectives, choices, results, and limitations using appropriate technical language.
- The discussion verifies the ability to explain system functioning and implementation choices.
- The written exam includes questions requiring understanding and concept synthesis, possibly with short open-ended items.

Autonomy of judgment
- Students must show autonomy and awareness in project decisions, justifying key aspects in the report and discussion.
- The possibility of a self-proposed project promotes critical thinking and independent decision-making.
- The written exam verifies the ability to recognize and evaluate techniques and methodologies covered in the course.

Learning skills
- The written exam assesses the ability to learn, integrate, and autonomously re-elaborate the course content.
- The project strengthens skills through practical application and end-to-end development of an LLM-based system.
- Honors require the maximum score in both parts and excellent quality in the report and the developed system.

Brief history of artificial intelligence from its origins to the present day

Introduction to Machine Learning
- Supervised learning
- Unsupervised learning
- Reinforcement learning

Introduction to Neural Networks
- Feedforward networks
- Learning (Backpropagation)
- Generalization / Overfitting
- Universal Approximation Theorem
- Introduction to PyTorch: implementation and training of a simple neural network

Large Language Models
- Transforming text into numerical vectors (Tokenization / Embeddings)
- Introduction to the Transformers architecture
- Characteristics, capabilities and limitations of Large Language Models
- Life cycle of a Large Language Model
- Prompt Engineering

AI-assisted programming

Retrieval Augmented Generation (RAG)

Agents

Knowledge and Understanding: Students will acquire a solid understanding of the methodologies and architectures underlying AI-based applications, with a particular focus on LLMs and their practical implications.

Ability to Apply Knowledge and Understanding: Students will be able to develop and implement medium-complexity AI solutions, integrating pre-trained models into real-world applications. They will be capable of applying optimization techniques, such as Prompt Engineering and Retrieval Augmented Generation, to enhance model effectiveness.

Communication Skills: Students will develop the ability to document and present their work in a clear and structured manner, using appropriate technical terminology.

Independent Judgment: Students will acquire the ability to critically assess the effectiveness of different AI models, identifying their strengths and weaknesses. They will be able to select the most suitable model for a given application context, considering technical, ethical, and economic aspects.

Learning Skills: Students will develop an autonomous approach to updating their skills, knowing how to identify the most appropriate resources to explore emerging topics in AI and LLMs.