Lecture notes and videos
- Vector and Matrices
- Linear combination of vectors, inner product, length, angle between vectors; Different
views of Ax=b.
[Video-Lecture]
[Lecture Notes]
- Solving Ax=b, Linear Independence, Rank
- When can I solve Ax=b? Basis and Linear Independence; Matrix operations, symmetric, skew, tringular and
diagonal matrices; Rank of a matrix.
[Video-Lecture]
[Lecture Notes]
- Rank, Column, Row and Null Spaces
- Rank of a matrix, rank and linear independence; Vector spaces and subspaces, basis, span;
A first look at the column space, nullspace and row space; Inner product spaces, Cauchy-Scwartz
inequality.
[Video-Lecture]
[Lecture Notes]
- Matrix Operations and Elementary Matrices
- Review of the column and row views of Ax=b;
Matrix Multiplication, Elementary Matrices, Permutation Matrices, Exchanging Columns,
Multiplication as a Sum of Rank 1 Matrices, Matrix Block Multiplication;
Matrix Inversion, Cancellation Laws of Matrix Multiplication, Inverse and Solution of Ax=b;
Summary of Matrix Operations
[Video-Lecture]
[Lecture Notes]
- Matrix Elimination
- Matrix Elimination, the augmented matrix, elimination example, elimination matrices and their inverses,
elimination with row exchange; A=LU factorization, A=LDU factorization, forward elimination and back substitution,
Cost of factorization, Permutation matrices and row exchanges, row equivalent systems, Schur Complement.
[Video-Lecture]
[Lecture Notes]
- Row Echelon Form, Matrix Inversion, Gauss-Jordan Form, Computing the nullspace, Complete Solution to Ax=b
- Row Echelon form and when does elimination fail?
Inverse of a matrix, Gauss – Jordan form; Computing the solutions to the nullspace Ax=0, pivot columns,
pivot and free variables, nullspace basis; The complete
solution to Ax=b.
[Video-Lecture]
[Lecture Notes]
- Vector Spaces and Subspaces, The Column Space, The Nullspace, Computing Ax=0
- Spaces of vectors, the space ℝn, vector space, examples of vector spaces,
the zero vector; Subspace, union of subspaces, subspaces of ℝ3, examples;
The column space of A, Subspace spanned by a set of vectors, examples;
The null space of A, solving Ax=0 and Rx=0, examples, pivot columns and free columns,
the reduced row echelon form, solving Ax=0 and special solutions, computing the nullspace;
The rank of a matrix, rank 1 matrices, rank as the dimension of C(A); Matrix elimination summary.
[Video-Lecture]
[Lecture Notes]
- The Complete Solution of Ax=b
- The complete solution to Ax=b, particular solution, full column rank,
full row rank, not full rank, summary of the four possibilities.
[Video-Lecture]
[Lecture Notes]
- Independence, Basis, and Dimension
- Independence, basis and dimension, vectors that span a subspace, row and column spaces,
bases for the column and row spaces, dimension of a vector space, dimensions of
C(A) and N(A), bases for matrix and function spaces; Summary – independence, basis and dimension.
[Video-Lecture]
[Lecture Notes]
- Dimensions of the Four Fundamental Subspaces
- The four subspaces of A, C(A), N(A), C(AT), N(AT); The dimensions of
the column, row, nullspace and left null spaces; Applications.
[Video-Lecture]
[Lecture Notes]
- The Four Fundamental Subspaces: Applications to Graphs and Orthogonality
- An example of the fundamental subspaces from graphs, networks and electrical circuits;
Example in rank 1 matrices and in rank 2 matrices; Summary of dimensions and bases of the four fundamental subspaces;
Inner Product Spaces, Orthogonality, Orthogonality of the four fundamental subspaces,
Orthogonal complements, Combining spaces.
[Video-Lecture]
[Lecture Notes]
- Projecting on a Subspace
- Projections matrices, Projection onto a line, Projection onto a subspace, Normal
equations of least squares.
[Video-Lecture]
[Lecture Notes]
- Least Squares, Orthogonal Matrices, Gram-Schmidt and A=QR Factorization
- Reviewing projection onto a subspace,
Least Squares approximation, Minimizing the error, Linear regression;
Orthogonal basis and Gram-Schmidt orthogonalization,
Projections using an orthonormal basis, The A=QR factorization, Summary.
[Video-Lecture]
[Lecture Notes]
- Least Squares, Orthogonal Matrices, Gram-Schmidt and A=QR Factorization (Continued)
- Reviewing projection onto a subspace,
Least Squares approximation, Minimizing the error, Linear regression;
Orthogonal basis and Gram-Schmidt orthogonalization,
Projections using an orthonormal basis, The A=QR factorization, Summary.
[Video-Lecture]
[Lecture Notes]
- The Properties of Determinants
- detI=1, sign reversal and linearity in each row; Elimination and determinant;
Invertable matrices and determinant; The determinant of AB and of AT; Permutations
and cofactors; The Pivot Formula.
[Video-Lecture]
[Lecture Notes]
- Determinant Formulas, Cofactors, Cramer's Rule, Inverse Matrix, Area and Volume
- Formulas for evaluating the determinant; Cofactor formula; Evaluating A-1;
Cramer's rule for x=A-1b; Area and Volume as Determinants.
[Video-Lecture]
[Lecture Notes]
- Introduction to Eigenvalues and Eigenvectors
- Introduction to Eigenvalues and Eigenvectors; Determining eigenvalues and eigenvectors;
Applications to Symmetric and Orthogonal Matrices; The Eigenvalue Problem for Projection and Reflection Matrices;
Introduction to Diagonalization of a Matrix.
[Video-Lecture]
[Lecture Notes]
- Diagonalization of a Matrix, Matrix Exponential
- Distinct Eigenvalues, Independent Eigenvectors and Diagonalization of a General Matrix;
Examples of Diagonalizable and Non-Diagonalizable Matrices; Markov Matrices; Similar Matrices;
Example of Fibonacci Numbers; Geometric and Algebraic Multiplicity.
[Video-Lecture]
[Lecture Notes]
- Differential Equations and the Eigenvalue Problem, Exponential of a Matrix, Diagonalization of Symmetric Matrices
- System of Differential Equations, Difference Equations, Stability; Exponential of a Matrix, Examples;
Symmetric and Anti-Symmetric Matrices, Diagonalization and Spectral Theorem; Repeated eigenvalues
and Diagonalization; Schnur Theorem.
[Video-Lecture]
[Lecture Notes]
- Positive Definite Matrices and the Jordan Form
- Positive Definite Matrices;
Tests for positive definite matrices - eigenvalues, pivots, sub-determinants, energy functional, ATA form;
Positive Semi-Definite Matrices; Test for a Minimum of Energy; Quadratic form, transformation to principal axes;
Similar matrices and Jordan form; Similarity Transformation.
[Video-Lecture]
[Lecture Notes]
- Singular Value Decomposition
- Low rank image representation; Eigenvectors for the SVD; Singular Values; Proof of the SVD;
Examples of the SVD; Accounting for the nullspaces; Singular Value Stability versus Eigenvalue
Stability.
[Video-Lecture]
[Lecture Notes]
- Linear Transformations
- Linear Transformations, Matrix Representation; Basis; Examples of linear transformations;
Range and Kernel; Product of transformations; Choosing the best bases; Jordan form; Bases for
Function spaces.
[Video-Lecture]
[Lecture Notes]
- Introduction to 1st Order Ordinary Differential Equations
- Differential Equation Models; Ordinary and Partial Differential Equations; Classification
of ODEs; Direction fields and
Equilibrium solutions; Solutions of simple ODEs; Initial Value Problems; Method of Integrating factors;
Integrating factors for general 1st Order ODEs.
[Video-Lecture]
[Lecture Notes]
- 1st Order Ordinary Differential Equations
- Direction fields and
Equilibrium solutions; Solutions of ODEs; Separable ODEs, Method of Integrating Factors;
Exact ODEs.
[Video-Lecture]
[Lecture Notes]
- Linear Vs. Nonlinear ODEs, Autonomous Eqs and Population Dynamics, Existence and Uniqueness of Solutions
- Difference between linear and nonlinear Equations; Autonomous Equations and Population
Dynamics; Numerical Approximations, Euler Method; Existence and Uniqueness of Solutions
for linear and nonlinear 1st Order ODEs.
[Video-Lecture]
[Lecture Notes]
- 2nd Order Differential Equations
- Second-Order Linear Homogeneous Equations with Constant Coefficients,
Solutions of Linear Homogeneous Equations; The Wronskian;
Complex Roots of the Characteristic Equation,
Repeated Roots; Reduction of Order,
Nonhomogeneous Equations; Method of Undetermined Coefficients,
Variation of Parameters.
[Video-Lecture]
[Lecture Notes]
- Mechanical Vibrations
-
Variation of Parameters;
Undamped free vibrations; Damped free vibrations; Critical
damping value; Forced vibrations with damping; Transient and Steady-State
Solutions; Maximum amplitude and resonance; Undamped Equatiion and Beats.
[Video-Lecture]
[Lecture Notes]
- Power Series Methods
-
Review of Power Series; Series Solutions near ordinary points; Euler equations and regular
singular points.
[Video-Lecture]
[Lecture Notes]
- Series Solutions of ODEs, Bessel Functions
- Laplace Transform and ODEs
Homework
- September 1, Homework 1
- Linear Independence, Elimination, LU Factorization, Gauss-Jordan and the Inverse of a Matrix, Nullspace and Column Space
[Homework]
[Solution]
- September 16, Homework 2
- September 24, Homework 3
- Least Squares, Orthogonality, Gram-Schmidt and QR Factorization
[Homework]
[Solution]
- October 2, Homework 4
- Properties of Determinants, Orthogonality, Eigenvalues and Eigenvectors, Diagonalization
[Homework]
[Solution]
- October 8, Homework 5
- Eigenvalues and Eigenvectors, Positive Definite Matrices, Quadratic Forms, SVD
[Homework]
[Solution]
- October 15, Homework 6
- Jordan Matrices, Similar Matrices, Linear Transformations, 1st Order ODEs
[Homework]
[Solution]
- October 29, Homework 7
- November 8, Homework 8
Exams
- September 27, Exam 1
- November 1, Exam 2
Course info and references
Credit: 3 Units
Lectures: MWF, 10:30 -- 11:20 am, DeBartolo Hall 215.
Professor: Nicholas
Zabaras, 311 I Cushing Hall, nzabaras@gmail.com
Homework/Exam Grading Assistants: Zhuogang Peng, zpeng5@nd.edu, Samaresh Midya, smidya@nd.edu
Office hours: N. Zabaras, Tuesdays 5:00-6:00 pm (Cushing 311 I).
Course description: This course provides a rigorous review of linear algebra including
vector spaces and subspaces,
eigenvalues and eigenvectors, orthogonality, singular value decomposition, linear transformations; Applications of
linear algebra to networks, structures, estimation, Lagrange multipliers and optimization will be discussed.
Also covered are: differential equations of equilibrium; Laplace's equation and potential flow;
boundary-value problems and Sturm-Liuville Theory; system of linear ODEs, non-linear differential equations and stability;
Fourier series; discrete Fourier transform; convolution; and applications.
Goals for the Course: The purpose of the course is to provide you useful insights and
understanding of mathematical
methods that will be relevant to your research work.
While the complexity of calculations will be kept
to a minimum, we will put emphasis on integrating concepts from linear algebra to understand the nature of the solutions
of systems of linear and non-Linear
differential equations. Our goal is to identify the underlying patterns in many applications and allow clear understanding of the
nature of the solutions of equations and development of efficient methods to compute them.
Intended audience: Graduate Students in Engineering and the Sciences. Qualified undergraduates can also
be enrolled.
References of General Interest: The course lecture slides will become available on the course web site. Please note that
the lecture slides are to be read on the internet - and not to be printed. For in depth study,
readings from the recommended textbooks will be highlighted. While there is
no required text for this course, most of the lectures will follow closely the books by G. Strang and W.E. Boyce and R.C. DiPrima.
The books by E. Kreysig and M. Greenberg provide comprehensive material for all lectures (and beyond) and can be useful for subsequent engineering Mathematics
courses.
Homework: Assigned every week. We anticipate approximately ten homework sets. While ideally all homeworks should be submitted typed
to allow easy readability, as a minimum you will need to scan your work and submit it electronically. Your homework solutions
should be accompanied by any computer programs (e.g. MatLab scripts, data files, Readme files, etc.) and mailed by midnight of the due date to
this Email address. All attachments should
arrive on an appropriately named zipped directory (e.g. HW1_Submission_YourName.rar). We will be asking
for volunteers weekly to prepare Latex solutions to each homework set. We much appreciate your help.
Exams: There will be three midterm take-home exams (no final). The designated days for the exam
are as follows: September 27th, November 1st and December 6th. The exams are two hrs long but you will be provided
freedom within 24 hrs of selecting your start. Managing of downloading the exams and their solution
within 2 hrs will be provided through Sakai. Note that we reserve the right to change the format
of the exams if it becomes clear that we cannot guarantee the university's honor code.
Grading: Homework 10% and each exam 30%.
Prerequisites: Elementaty Linear Algebra, Differential Calculus, Previous exposure to
differential equations. Some programming background in MatLab (or if you desire in
Mathematica, Python, etc.) is required
for some of the homework. Also it will be helpful to acquire Latex skills in assisting with the homework
solution preparation.
Honor Code: Students are expected to understand and abide by the principles and procedures set forth in
the
University of Notre Dame Academic Code of Honor and uphold the pledge that
"As a member of the Notre Dame community, I acknowledge that it is my responsibility to learn and abide
by principles of intellectual honesty and academic integrity, and therefore I will not participate in or tolerate
academic dishonesty". Students may collaboratively discuss course assignments, but are expected to
write and complete their own assignments independently. Downloading solutions from the web or other sources
is not allowed. Finally, there should be absolutely no interactions between students regarding
the takehome exams.
Syllabus
- Introduction to Vectors and Matrices
- Lengths and Dot Products
- Linear Combinations of Vectors
- Linear Equations
- Elimination using Matrices
- Inverse Matrices, Transposes and Permutations
- A=LU Factorization
- Vector Spaces and Subspaces
- Spaces of Vectors
- The nullspace of A
- Independence, Basis, Dimension
- The Four Fundamental Subspaces
- Orthogonality
- Projections and Least Squares
- Orthonormal Bases and Gram-Schmidt
- Eigenvalues and Eigenvectors
- Properties of Determinants, Cramer's Rule and Inverses
- Eigenvalue Problem
- Diagonalizing a Matrix
- Symmetric and Positive Definite Matrices
- Applications to Systems of ODEs
- Singular Value Decomposition
- Fundamentals of SVD
- Principal Component Analysis
- Geometry of SVD
- Linear Transformations
- Matrix Representation
- Optimal Basis Representation
- Applications of Linear Algebra
- Hermitian and Unitary Matrices, Fast Fourier Transform
- Linear Algebra for Functions
- Graphs and Networks
- Markov Matrices
- Linear Programming
- First-Order Differential Equations
- Integrating Factors
- Numerical Approximations
- Existence and Uniqueness of Solution
- First Order Difference Equations
- Second-Order Linear Equations
- Solutions of Linear Homogeneous Equations
- Reduction of Order
- Method of Undetermined Coefficient
- Variation of Parameters
- Forced Vibrations
- Series Solutions of 2nd Order Linear Equations
- Power Series Fundamentals
- Series Solutions Near an Ordinary Point
- Euler Equations
- Series Solutions Near a Regular Singular Point, Bessel's Equation
- The Laplace Transform
- Solution of Initial Value Problems
- Differential Equations, Forcing Functions, Impulse Functions
- Convolution
- Systems of First Order Linear Equations
- Homogeneous Linear Systems with Constant Coefficients
- Nonhomogeneous Linear Systems
- Nonlinear Differential Equations and Stability
- The Phase Plane, Linear Systems and Stability
- Locally Linear Systems
- Predator-Prey Equations
- Liapunov's Second Method
- Limit Cycles
- Chaos and Strange Attractors, The Lorenz Equations
- Introduction to Partial Differential Equations
- Fourier Series and The Fourier Convergence Theorem
- Separation of Variables; Heat Conduction in a Rod
- The Wave Equation: Vibrations of an Elastic String
- Laplace’s Equation
- Boundary Value Problems and Sturm-Liuville Theory
- Sturm–Liouville Boundary Value Problems
- Nonhomogeneous Boundary Value Problems
- Singular Sturm–Liouville Problems
- Bessel
Series Expansion
- Series of Orthogonal Functions: Mean Convergence