Bayesian Item Response Modeling in PyMC

Uncover the power of Bayesian Item Response Theory in PyMC. Learn how it revolutionizes data analysis and opens up new possibilities for personality modeling.


Thomas Wiecki




In this panel discussion, we discuss IRT (Item Response Theory), GRM (Graded Response Model) and the advantages to using the Bayesian approach at Alva Labs.

Item response theory, also known as the latent response theory, refers to a family of mathematical models that attempt to explain the relationship between latent traits (unobservable characteristic or attribute) and their manifestations (i.e. observed outcomes, responses or performance). Graded response model (or Ordered Categorical Responses Model) is a family of mathematical models for grading responses.


00:00 Introduction to PyMC Labs

02:48 Panelists introduction

06:05 Outline of the talk by Morgan

06:51 Alva Labs

08:33 Alva Labs personality test

12:14 Item Response Theory and its advantages

16:12 Question: Won't people fake answers to the personality questionnaire if they know what the company is looking for?

18:09 Question: What algorithm is used for combining data points?

19:36 Graded Response Model

20:50 Bayesian Inference

23:19 ALva Labs workflow

25:08 Is the person trait supposed to be a measure of performance and how is it quantified for training?On which data is the model trained?

27:37 Emerging challenges over the years

30:45 How PyMC helped Alva Labs improve their personality model

32:33 Understanding the problem at Alva Labs

34:34 Bayesian workflow

35:32 Simulate the data generating process

38:12 Develop the model

40:25 Evaluate alternative parameterizations

43:40 Test different inference engines

46:05 Use the mode4 with real data

47:52 The final deliverable

49:48 Results of the new model compared to the original Alva Labs model

51:12 Question: Could you comment on how much faster the sampler becomes and why do you care about memory?

53:40 Question: How is the trained model validated and how do you know the person trait is useful and how is the usefulness measured?

55:02 How do you often rerun the model to update the parameters

56:02 Thank you!

Work with PyMC Labs

If you are interested in seeing what we at PyMC Labs can do for you, then please email We work with companies at a variety of scales and with varying levels of existing modeling capacity. We also run corporate workshop training events and can provide sessions ranging from introduction to Bayes to more advanced topics.