ME 470: Syllabus

a. In the context of open-ended assignments, students will:

• Identify sources of uncertainty
• Determine what – if any – conclusions can be drawn, given uncertainty
• Determine what additional data would be necessary to improve the analysis

Topics:

1. Risk, model validation, decision-making under uncertainty
2. Probability
3. Aleatory vs epistemic uncertainty

a. In the context of analyzing scientific and engineering models, students will:

• Identify potential sources of model discrepancy
• Given data, determine the level of discrepancy with the model
• Use global sensitivity analysis to rank input uncertainties
• Survey additional techniques, including local sensitivity approaches

Topics:

1. Model-form uncertainty, model validation
2. Sensitivity analysis
3. Bayesian inference/calibration
4. Nonlinear regression
5. Model checking

a. In the context of general uncertainty propagation, students will:

• Identify cases where dimensionality is an issue
• Employ dimension reduction techniques to make problems tractable

Topics:

1. Exponential cost of dimensionality
2. Sobol' indices
3. Active subspaces

a. In the context of data, students will:

• Explore and understand data using point and interval estimates
• Practice data skepticism and criticism
• Model uncertainty

Topics:

1. Summaries
2. Variability
3. Regression for visualization
4. Random variable modeling

a. In the context of forward propagation, students will:

• Use the delta, Monte Carlo, and quadrature methods
• Select a reasonable technique for the problem at hand
• Gain familiarity with other methods, e.g. intrusive PCE, GPR
• Diagnose problems in propagation

Topics:

1. Delta method
2. Monte Carlo
3. Polynomial chaos
   • Intrusive
   • Non-intrusive

b. In the context of inverse propagation, students will:

• Use regression and Bayesian inference
• Build models for inference
• Diagnose problems in inference, and criticize inferential models

Topics:

1. Regression
   • Linear
   • Nonlinear
2. Bayesian inference/calibration
3. Model checking

Content will be delivered through a number of different formats, described below. All in-class delivery strategies will be adjusted to use modern, research-backed principles and strategies to promote learning outcomes – active learning.

The majority of assignments are problem sets, with a small number of extensive problems intended to reinforce concepts learned in class. Each problem set will include at least one `open-ended' question that will stretch your executive reasoning, and require you to make and justify analysis choices. The project is essentially a large, student-defined assignment. It includes additional components above a normal problem set, described above.

Personnel and Office Hours:

Assignment late policy:

Assignment due-dates are listed below in the 'Content Sequence' subsection. All assignments are due at the beginning of class on the specified date. Assignments are considered late if not submitted at the specified time. Late assignments may be submitted for partial credit, though they will not be graded in detail, and will receive a maximum partial credit depending on the number of late days:

Of course, in extenuating circumstances, exceptions to this policy will be made. Examples of reasonable exemptions include issues of health, family emergencies, and other unforeseeable circumstances. If you have planned events that may impede an assignment deadline (e.g. conference travel), please come discuss your situation with me, and we'll try to work something out.

Assignment re-grade policy:

Re-grades must be requested within one week of the assignment's return date; no exceptions will be made. If you think a grading decision is genuinely in error, please feel free to contact the teaching team (in a timely manner).

Assignment re-submission:

Generally, I would prefer that you take assignment feedback and incorporate it into future assignments. In extenuating circumstances, I may consider a re-submission of an assignment. In this case, the maximum grade that may be earned is the previous grade plus half the deficit; for example, if one scored a 50% on an assignment, the maximum re-submission grade will be 75%. Re-submissions will only be accepted until a week after the relevant assignment has been returned.

I have started a textbook to support teaching this content; however, it will not be complete in time to support this offering of the course. I hope to have a few chapters finished in time – particularly those that do not have a published equivalent.

The following book (Smith 2014) should be considered the primary resource for technical details, particularly when it comes to understanding and implementing algorithms. Copies are available in the Stanford library.

Ralph Smith, Uncertainty quantification: Theory, implementation, and applications (2014) SIAM

As Stanford students, we all have access to digital copies of ["A Certain Uncertainty"](https://www.cambridge.org/core/books/certain-uncertainty/65FB78B0DFC097572F2F9746A3A51C8F), which is a (rather detailed) treatment of relevant material. Consider this to be optional reading.

I will also point to technical papers when referencing material that lies outside the texbook.

Both formative and summative feedback will be deployed in this class, though the emphasis will be on feedback to help students develop (formative), and less on assigning grades (summative).

Formative feedback will be provided through in-class case studies, as I will be able to gauge student understanding in real-time. Additional feedback will be provided through peer review, as students will both get commentary on their work, and be afforded to see how other students conceptualize the material. Finally, all assignments will include some amount of open-ended problem solving that will require executive thought – students are welcome and encouraged to attend office hours to discuss these not-straightforward problems.

Summative feedback will be deployed in grading the assignments and final project. More is stated below in the Grading Philosophy.

This is an upper-level graduate course. My assumption is that students taking this class are primarily engaged in research, and are now taking courses to support that activity. Thus, my expectation is that this class is of secondary importance in your current workload, and my grading philosophy is formulated with this assumption in mind.

There will be four homeworks and a final project, in order to minimize the external workload for the class. I would like to grade the in-class assignments, but I realize that upper-level graduates will need to miss classes for conferences and the like. The class has been designed around reinforcing concepts through these in-class activities, so please do not take my lack of point assignment as an indication of the relative importance of these activities.

The four homeworks will compose half of your grade, with the final project weighted equally. The final project is further broken down into the final report (delivered both through presentation and in writing), and the peer review stage. The relative weight of Project components is intentional.

It is common in upper-level graduate courses to be lenient in grading, and I have every intent to do the same in this class. You are researchers at this stage – we should be beyond ``playing the game of school'' by now.

Since this is an upper-level graduate course, I will assume a fair amount of background knowledge on your part. However, I realize some topics are often not well-covered in a `traditional' engineering curriculum, and you may feel uncomfortable with some of the necessary topics. I have sequenced the content with this fact in mind, and make specific my expectations for background knowledge below.

This course is meant to start a shift in your mindset and toolkit. Like any kind of learning, grappling with uncertainty is a lifelong journey. I recommend the following 'post-requisites' to build upon the ideas introduced in this course. Note that these are current as of the 2018-19 academic year:

STATS 305A: Applied Statistics I

• All about (multivariate) regression, with a focus on practical concerns. I found this course very helpful for building intuition in statistics, but also found it rather challenging with only an engineer's background in statistics.

STATS 362: Topic: Monte Carlo

• Art Owen infrequently teaches this class on the Monte Carlo method. There are a lot of practical considerations and subtleties – I would take it if you find the chance!

CS 228: Probabilistic Graphical Models

• Specialty class on using probabilistic graphical models for (machine) learning. These form a very flexible, powerful framework for Bayesian inference, and are an exciting area of current research.

CS 238: Decision Making under Uncertainty

• I hear wonderful things about Mykel Kochenderfer's Decision making under uncertainty course. He has great skill in explaining complex topics in a lucid way.

ENGR 150: Data Challenge Lab

• Very unique course; by application only. Intensive course on data science – teaches visualization techniques and exploratory data analysis.