Lecture notes and videos

Vector and Matrices
- Linear combination of vectors, inner product, length, angle between vectors; Different views of Ax=b.
  [Video-Lecture] [Lecture Notes]
Solving Ax=b, Linear Independence, Rank
- When can I solve Ax=b? Basis and Linear Independence; Matrix operations, symmetric, skew, tringular and diagonal matrices; Rank of a matrix.
  [Video-Lecture] [Lecture Notes]
Rank, Column, Row and Null Spaces
- Rank of a matrix, rank and linear independence; Vector spaces and subspaces, basis, span; A first look at the column space, nullspace and row space; Inner product spaces, Cauchy-Scwartz inequality.
  [Video-Lecture] [Lecture Notes]
Matrix Operations and Elementary Matrices
- Review of the column and row views of Ax=b; Matrix Multiplication, Elementary Matrices, Permutation Matrices, Exchanging Columns, Multiplication as a Sum of Rank 1 Matrices, Matrix Block Multiplication; Matrix Inversion, Cancellation Laws of Matrix Multiplication, Inverse and Solution of Ax=b; Summary of Matrix Operations
  [Video-Lecture] [Lecture Notes]
Matrix Elimination
- Matrix Elimination, the augmented matrix, elimination example, elimination matrices and their inverses, elimination with row exchange; A=LU factorization, A=LDU factorization, forward elimination and back substitution, Cost of factorization, Permutation matrices and row exchanges, row equivalent systems, Schur Complement.
  [Video-Lecture] [Lecture Notes]
Row Echelon Form, Matrix Inversion, Gauss-Jordan Form, Computing the nullspace, Complete Solution to Ax=b
- Row Echelon form and when does elimination fail? Inverse of a matrix, Gauss – Jordan form; Computing the solutions to the nullspace Ax=0, pivot columns, pivot and free variables, nullspace basis; The complete solution to Ax=b.
  [Video-Lecture] [Lecture Notes]
Vector Spaces and Subspaces, The Column Space, The Nullspace, Computing Ax=0
- Spaces of vectors, the space ℝⁿ, vector space, examples of vector spaces, the zero vector; Subspace, union of subspaces, subspaces of ℝ³, examples; The column space of A, Subspace spanned by a set of vectors, examples; The null space of A, solving Ax=0 and Rx=0, examples, pivot columns and free columns, the reduced row echelon form, solving Ax=0 and special solutions, computing the nullspace; The rank of a matrix, rank 1 matrices, rank as the dimension of C(A); Matrix elimination summary.
  [Video-Lecture] [Lecture Notes]
The Complete Solution of Ax=b
- The complete solution to Ax=b, particular solution, full column rank, full row rank, not full rank, summary of the four possibilities.
  [Video-Lecture] [Lecture Notes]
Independence, Basis, and Dimension
- Independence, basis and dimension, vectors that span a subspace, row and column spaces, bases for the column and row spaces, dimension of a vector space, dimensions of C(A) and N(A), bases for matrix and function spaces; Summary – independence, basis and dimension.
  [Video-Lecture] [Lecture Notes]
Dimensions of the Four Fundamental Subspaces
- The four subspaces of A, C(A), N(A), C(A^T), N(A^T); The dimensions of the column, row, nullspace and left null spaces; Applications.
  [Video-Lecture] [Lecture Notes]
The Four Fundamental Subspaces: Applications to Graphs and Orthogonality
- An example of the fundamental subspaces from graphs, networks and electrical circuits; Example in rank 1 matrices and in rank 2 matrices; Summary of dimensions and bases of the four fundamental subspaces; Inner Product Spaces, Orthogonality, Orthogonality of the four fundamental subspaces, Orthogonal complements, Combining spaces.
  [Video-Lecture] [Lecture Notes]
Projecting on a Subspace
- Projections matrices, Projection onto a line, Projection onto a subspace, Normal equations of least squares.
  [Video-Lecture] [Lecture Notes]
Least Squares, Orthogonal Matrices, Gram-Schmidt and A=QR Factorization
- Reviewing projection onto a subspace, Least Squares approximation, Minimizing the error, Linear regression; Orthogonal basis and Gram-Schmidt orthogonalization, Projections using an orthonormal basis, The A=QR factorization, Summary.
  [Video-Lecture] [Lecture Notes]
Least Squares, Orthogonal Matrices, Gram-Schmidt and A=QR Factorization (Continued)
- Reviewing projection onto a subspace, Least Squares approximation, Minimizing the error, Linear regression; Orthogonal basis and Gram-Schmidt orthogonalization, Projections using an orthonormal basis, The A=QR factorization, Summary.
  [Video-Lecture] [Lecture Notes]
The Properties of Determinants
- detI=1, sign reversal and linearity in each row; Elimination and determinant; Invertable matrices and determinant; The determinant of AB and of A^T; Permutations and cofactors; The Pivot Formula.
  [Video-Lecture] [Lecture Notes]
Determinant Formulas, Cofactors, Cramer's Rule, Inverse Matrix, Area and Volume
- Formulas for evaluating the determinant; Cofactor formula; Evaluating A^-1; Cramer's rule for x=A^-1b; Area and Volume as Determinants.
  [Video-Lecture] [Lecture Notes]
Introduction to Eigenvalues and Eigenvectors
- Introduction to Eigenvalues and Eigenvectors; Determining eigenvalues and eigenvectors; Applications to Symmetric and Orthogonal Matrices; The Eigenvalue Problem for Projection and Reflection Matrices; Introduction to Diagonalization of a Matrix.
  [Video-Lecture] [Lecture Notes]
Diagonalization of a Matrix, Matrix Exponential
- Distinct Eigenvalues, Independent Eigenvectors and Diagonalization of a General Matrix; Examples of Diagonalizable and Non-Diagonalizable Matrices; Markov Matrices; Similar Matrices; Example of Fibonacci Numbers; Geometric and Algebraic Multiplicity.
  [Video-Lecture] [Lecture Notes]
Differential Equations and the Eigenvalue Problem, Exponential of a Matrix, Diagonalization of Symmetric Matrices
- System of Differential Equations, Difference Equations, Stability; Exponential of a Matrix, Examples; Symmetric and Anti-Symmetric Matrices, Diagonalization and Spectral Theorem; Repeated eigenvalues and Diagonalization; Schnur Theorem.
  [Video-Lecture] [Lecture Notes]
Positive Definite Matrices and the Jordan Form
- Positive Definite Matrices; Tests for positive definite matrices - eigenvalues, pivots, sub-determinants, energy functional, A^TA form; Positive Semi-Definite Matrices; Test for a Minimum of Energy; Quadratic form, transformation to principal axes; Similar matrices and Jordan form; Similarity Transformation.
  [Video-Lecture] [Lecture Notes]
Singular Value Decomposition
- Low rank image representation; Eigenvectors for the SVD; Singular Values; Proof of the SVD; Examples of the SVD; Accounting for the nullspaces; Singular Value Stability versus Eigenvalue Stability.
  [Video-Lecture] [Lecture Notes]
Linear Transformations
- Linear Transformations, Matrix Representation; Basis; Examples of linear transformations; Range and Kernel; Product of transformations; Choosing the best bases; Jordan form; Bases for Function spaces.
  [Video-Lecture] [Lecture Notes]
Introduction to 1st Order Ordinary Differential Equations
- Differential Equation Models; Ordinary and Partial Differential Equations; Classification of ODEs; Direction fields and Equilibrium solutions; Solutions of simple ODEs; Initial Value Problems; Method of Integrating factors; Integrating factors for general 1st Order ODEs.
  [Video-Lecture] [Lecture Notes]

Homework

September 1, Homework 1
- Linear Independence, Elimination, LU Factorization, Gauss-Jordan and the Inverse of a Matrix, Nullspace and Column Space
  [Homework] [Solution]
September 16, Homework 2
- The four fundamental subspaces
  [Homework] [Solution]
September 24, Homework 3
- Least Squares, Orthogonality, Gram-Schmidt and QR Factorization
  [Homework] [Solution]
October 2, Homework 4
- Properties of Determinants, Orthogonality, Eigenvalues and Eigenvectors, Diagonalization
  [Homework] [Solution]
October 8, Homework 5
- Eigenvalues and Eigenvectors, Positive Definite Matrices, Quadratic Forms, SVD
  [Homework] [Solution]
October 15, Homework 6
- Jordan Matrices, Similar Matrices, Linear Transformations, 1st Order ODEs
  [Homework] [Solution]

Exams

September 27, Exam 1
- Linear Algebra and Applications
  [Homework] [Solution]

Course info and references

Credit: 3 Units

Lectures: MWF, 10:30 -- 11:20 am, DeBartolo Hall 215.

Professor: Nicholas Zabaras, 311 I Cushing Hall, nzabaras@gmail.com

Teaching Assistants: Zhuogang Peng, zpeng5@nd.edu, Samaresh Midya, smidya@nd.edu

Office hours: Teaching Assistants, Mondays and Wednesdays 5:15 -- 6:15 p.m. (DeBartolo 203); N. Zabaras, Tuesdays 5:00-6:00 pm (Cushing 311 I).

Course description: This course provides a rigorous review of linear algebra including vector spaces and subspaces, eigenvalues and eigenvectors, orthogonality, singular value decomposition, linear transformations; Applications of linear algebra to networks, structures, estimation, Lagrange multipliers and optimization will be discussed. Also covered are: differential equations of equilibrium; Laplace's equation and potential flow; boundary-value problems and Sturm-Liuville Theory; system of linear ODEs, non-linear differential equations and stability; Fourier series; discrete Fourier transform; convolution; and applications.

Goals for the Course: The purpose of the course is to provide you useful insights and understanding of mathematical methods that will be relevant to your research work. While the complexity of calculations will be kept to a minimum, we will put emphasis on integrating concepts from linear algebra to understand the nature of the solutions of systems of linear and non-Linear differential equations. Our goal is to identify the underlying patterns in many applications and allow clear understanding of the nature of the solutions of equations and development of efficient methods to compute them.

Intended audience: Graduate Students in Engineering and the Sciences. Qualified undergraduates can also be enrolled.

References of General Interest: The course lecture slides will become available on the course web site. Please note that the lecture slides are to be read on the internet - and not to be printed. For in depth study, readings from the recommended textbooks will be highlighted. While there is no required text for this course, most of the lectures will follow closely the books by G. Strang and W.E. Boyce and R.C. DiPrima. The books by E. Kreysig and M. Greenberg provide comprehensive material for all lectures (and beyond) and can be useful for subsequent engineering Mathematics courses.

Recommended Textbooks:

Gilbert Strang, Introduction to Linear Algebra, Fifth Edition 2007.
William E. Boyce and Richard C. DiPrima, Elementary Differential Equations and Boundary Value Problems, 10th Edition.
Erwin Kreysig, Advanced Engineering Mathematics, 10th Edition.
Michael Greenberg, Advanced Engineering Mathematics, 2nd Edition.

Homework: Assigned every week. We anticipate approximately ten homework sets. While ideally all homeworks should be submitted typed to allow easy readability, as a minimum you will need to scan your work and submit it electronically. Your homework solutions should be accompanied by any computer programs (e.g. MatLab scripts, data files, Readme files, etc.) and mailed by midnight of the due date to this Email address. All attachments should arrive on an appropriately named zipped directory (e.g. HW1_Submission_YourName.rar). We will be asking for volunteers weekly to prepare Latex solutions to each homework set. We much appreciate your help.

Exams: There will be three midterm take-home exams (no final). The designated days for the exam are as follows: September 27th, November 2nd and December 6th. The exams are two hrs long but you will be provided freedom within 24 hrs of selecting your start. Managing of downloading the exams and their solution within 2 hrs will be provided through Sakai. Note that we reserve the right to change the format of the exams if it becomes clear that we cannot guarantee the university's honor code.

Grading: Homework 10% and each exam 30%.

Prerequisites: Elementaty Linear Algebra, Differential Calculus, Previous exposure to differential equations. Some programming background in MatLab (or if you desire in Mathematica, Python, etc.) is required for some of the homework. Also it will be helpful to acquire Latex skills in assisting with the homework solution preparation.

Honor Code: Students are expected to understand and abide by the principles and procedures set forth in the University of Notre Dame Academic Code of Honor and uphold the pledge that "As a member of the Notre Dame community, I acknowledge that it is my responsibility to learn and abide by principles of intellectual honesty and academic integrity, and therefore I will not participate in or tolerate academic dishonesty". Students may collaboratively discuss course assignments, but are expected to write and complete their own assignments independently. Downloading solutions from the web or other sources is not allowed. Finally, there should be absolutely no interactions between students regarding the takehome exams.

Syllabus

Introduction to Vectors and Matrices
- Lengths and Dot Products
- Linear Combinations of Vectors
Linear Equations
- Elimination using Matrices
- Inverse Matrices, Transposes and Permutations
- A=LU Factorization
Vector Spaces and Subspaces
- Spaces of Vectors
- The nullspace of A
- Independence, Basis, Dimension
- The Four Fundamental Subspaces
- Orthogonality
- Projections and Least Squares
- Orthonormal Bases and Gram-Schmidt
Eigenvalues and Eigenvectors
- Properties of Determinants, Cramer's Rule and Inverses
- Eigenvalue Problem
- Diagonalizing a Matrix
- Symmetric and Positive Definite Matrices
- Applications to Systems of ODEs
Singular Value Decomposition
- Fundamentals of SVD
- Principal Component Analysis
- Geometry of SVD
Linear Transformations
- Matrix Representation
- Optimal Basis Representation
Applications of Linear Algebra
- Hermitian and Unitary Matrices, Fast Fourier Transform
- Linear Algebra for Functions
- Graphs and Networks
- Markov Matrices
- Linear Programming
First-Order Differential Equations
- Integrating Factors
- Numerical Approximations
- Existence and Uniqueness of Solution
- First Order Difference Equations
Second-Order Linear Equations
- Solutions of Linear Homogeneous Equations
- Reduction of Order
- Method of Undetermined Coefficient
- Variation of Parameters
- Forced Vibrations
Series Solutions of 2nd Order Linear Equations
- Power Series Fundamentals
- Series Solutions Near an Ordinary Point
- Euler Equations
- Series Solutions Near a Regular Singular Point, Bessel's Equation
The Laplace Transform
- Solution of Initial Value Problems
- Differential Equations, Forcing Functions, Impulse Functions
- Convolution
Systems of First Order Linear Equations
- Homogeneous Linear Systems with Constant Coefficients
- Nonhomogeneous Linear Systems
Nonlinear Differential Equations and Stability
- The Phase Plane, Linear Systems and Stability
- Locally Linear Systems
- Predator-Prey Equations
- Liapunov's Second Method
- Limit Cycles
- Chaos and Strange Attractors, The Lorenz Equations
Introduction to Partial Differential Equations
- Fourier Series and The Fourier Convergence Theorem
- Separation of Variables; Heat Conduction in a Rod
- The Wave Equation: Vibrations of an Elastic String
- Laplace’s Equation
Boundary Value Problems and Sturm-Liuville Theory
- Sturm–Liouville Boundary Value Problems
- Nonhomogeneous Boundary Value Problems
- Singular Sturm–Liouville Problems
- Bessel Series Expansion
- Series of Orthogonal Functions: Mean Convergence